From users at einsteintoolkit.org Mon Sep 1 15:18:01 2025 From: users at einsteintoolkit.org (users at einsteintoolkit.org) Date: Mon, 01 Sep 2025 15:18:01 -0500 Subject: [Users] Agenda for Thursday's Meeting Message-ID: <68b5ff79.p1u09SG7c/tGhgPM%users@einsteintoolkit.org> Please update the Wiki with agenda items for Thursday's meeting. Thanks! https://docs.einsteintoolkit.org/et-docs/meeting_agenda --The Maintainers From PANAGIOTIS.IOSIF at units.it Tue Sep 2 08:32:26 2025 From: PANAGIOTIS.IOSIF at units.it (IOSIF PANAGIOTIS) Date: Tue, 2 Sep 2025 13:32:26 +0000 Subject: [Users] ET build errors (Leonardo DCGP, CINECA cluster) Message-ID: Hi Einstein Toolkit users, I am reaching out regarding some errors I am encountering during my attempts to compile ET on the Leonardo, CINECA cluster (DCGP partition). Here are some details on the Leonardo cluster: * cluster specifics: https://docs.hpc.cineca.it/hpc/leonardo.html#leonardo-card * OS: Red Hat Enterprise Linux 8.7 I attach the 4 configuration files I am using (with many added comments for guidance and future reference). I adapted them from the respective config files that Bruno Giacomazzo kindly provided. I also attach the make.log output of the command: ./simfactory/bin/sim build --verbose --reconfig --machine=leonardo-dcgp1 --optionlist simfactory/mdb/optionlists/leonardo-dcgp1.cfg --allocation=CNHPC_1479290_0 --thornlist thornlists/einsteintoolkit.th 2>&1 | tee make.log >From what I understand, the build process complains about syntax errors in the Exact thorn in these two source files: * decode_pars.F * Boost_rotation_symmetric.F How can I bypass the above problems? To be honest, I am not sure that I really need this thorn. I was simply trying to compile ET using the default thornlist from a fresh download/installation. Is there any other error that the log file is pointing out that I might be missing? Best regards, Panayotis ------------------- Panagiotis Iosif postdoctoral researcher Department of Physics, University of Trieste Via Alfonso Valerio 2, Trieste 34127 Italy ------------------- -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: make.log Type: text/x-log Size: 130884 bytes Desc: make.log URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: leonardo-dcgp1.sub Type: text/x-microdvd Size: 3432 bytes Desc: leonardo-dcgp1.sub URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: leonardo-dcgp1.run Type: application/octet-stream Size: 2673 bytes Desc: leonardo-dcgp1.run URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: leonardo-dcgp1.ini Type: application/octet-stream Size: 5628 bytes Desc: leonardo-dcgp1.ini URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: leonardo-dcgp1.cfg Type: application/octet-stream Size: 9429 bytes Desc: leonardo-dcgp1.cfg URL: From rhaas at phas.ubc.ca Tue Sep 2 10:08:09 2025 From: rhaas at phas.ubc.ca (Roland Haas) Date: Tue, 2 Sep 2025 08:08:09 -0700 Subject: [Users] ET build errors (Leonardo DCGP, CINECA cluster) In-Reply-To: References: Message-ID: <20250902080809.2c5aa158@fdea4908> Hello Panayotis, Thanks for including all the output files. In your options list whenever your set FPP you also must also set FPPFLAGS: FPPFLAGS=-traditional See eg generic.cfg or (very new!) Bruno's leonardo-DCGP.cfg file (in simfactory master). Yours, Roland > [CAUTION: Non-UBC Email] > > Hi Einstein Toolkit users, > > I am reaching out regarding some errors I am encountering during my attempts to compile ET on the Leonardo, CINECA cluster (DCGP partition). > > Here are some details on the Leonardo cluster: > > * > cluster specifics: https://docs.hpc.cineca.it/hpc/leonardo.html#leonardo-card > * > OS: Red Hat Enterprise Linux 8.7 > > I attach the 4 configuration files I am using (with many added comments for guidance and future reference). > I adapted them from the respective config files that Bruno Giacomazzo kindly provided. > > I also attach the make.log output of the command: > > ./simfactory/bin/sim build --verbose --reconfig --machine=leonardo-dcgp1 --optionlist simfactory/mdb/optionlists/leonardo-dcgp1.cfg --allocation=CNHPC_1479290_0 --thornlist thornlists/einsteintoolkit.th 2>&1 | tee make.log > > From what I understand, the build process complains about syntax errors in the Exact thorn in these two source files: > > * > decode_pars.F > * > Boost_rotation_symmetric.F > > How can I bypass the above problems? > > To be honest, I am not sure that I really need this thorn. > I was simply trying to compile ET using the default thornlist from a fresh download/installation. > > Is there any other error that the log file is pointing out that I might be missing? > > Best regards, > Panayotis > > > ------------------- > Panagiotis Iosif > postdoctoral researcher > Department of Physics, University of Trieste > Via Alfonso Valerio 2, Trieste 34127 > Italy > ------------------- > -- My email is as private as my paper mail. I therefore support encrypting and signing email messages. Get my PGP key from http://pgp.mit.edu . From PANAGIOTIS.IOSIF at units.it Tue Sep 2 10:19:24 2025 From: PANAGIOTIS.IOSIF at units.it (IOSIF PANAGIOTIS) Date: Tue, 2 Sep 2025 15:19:24 +0000 Subject: [Users] ET build errors (Leonardo DCGP, CINECA cluster) In-Reply-To: <20250902080809.2c5aa158@fdea4908> References: <20250902080809.2c5aa158@fdea4908> Message-ID: Hi Roland, In the config file I attached, I do have this line included already: FPPFLAGS = -g -traditional Is there anything else I might try? Best, Panayotis ________________________________ From: Roland Haas Sent: Tuesday, September 2, 2025 5:08 PM To: IOSIF PANAGIOTIS Cc: Einstein Toolkit Users Subject: Re: [Users] ET build errors (Leonardo DCGP, CINECA cluster) Hello Panayotis, Thanks for including all the output files. In your options list whenever your set FPP you also must also set FPPFLAGS: FPPFLAGS=-traditional See eg generic.cfg or (very new!) Bruno's leonardo-DCGP.cfg file (in simfactory master). Yours, Roland > [CAUTION: Non-UBC Email] > > Hi Einstein Toolkit users, > > I am reaching out regarding some errors I am encountering during my attempts to compile ET on the Leonardo, CINECA cluster (DCGP partition). > > Here are some details on the Leonardo cluster: > > * > cluster specifics: https://docs.hpc.cineca.it/hpc/leonardo.html#leonardo-card > * > OS: Red Hat Enterprise Linux 8.7 > > I attach the 4 configuration files I am using (with many added comments for guidance and future reference). > I adapted them from the respective config files that Bruno Giacomazzo kindly provided. > > I also attach the make.log output of the command: > > ./simfactory/bin/sim build --verbose --reconfig --machine=leonardo-dcgp1 --optionlist simfactory/mdb/optionlists/leonardo-dcgp1.cfg --allocation=CNHPC_1479290_0 --thornlist thornlists/einsteintoolkit.th 2>&1 | tee make.log > > From what I understand, the build process complains about syntax errors in the Exact thorn in these two source files: > > * > decode_pars.F > * > Boost_rotation_symmetric.F > > How can I bypass the above problems? > > To be honest, I am not sure that I really need this thorn. > I was simply trying to compile ET using the default thornlist from a fresh download/installation. > > Is there any other error that the log file is pointing out that I might be missing? > > Best regards, > Panayotis > > > ------------------- > Panagiotis Iosif > postdoctoral researcher > Department of Physics, University of Trieste > Via Alfonso Valerio 2, Trieste 34127 > Italy > ------------------- > -- My email is as private as my paper mail. I therefore support encrypting and signing email messages. Get my PGP key from http://pgp.mit.edu . -------------- next part -------------- An HTML attachment was scrubbed... URL: From rhaas at phas.ubc.ca Tue Sep 2 10:43:19 2025 From: rhaas at phas.ubc.ca (Roland Haas) Date: Tue, 2 Sep 2025 08:43:19 -0700 Subject: [Users] ET build errors (Leonardo DCGP, CINECA cluster) In-Reply-To: References: <20250902080809.2c5aa158@fdea4908> Message-ID: <20250902084319.7b1f0e24@fdea4908> Hello Panayotis, Oh, I had not noticed, sorry. Hmm, that is the typical cause for these issues. The only other difference I see is `-ffixed-line-length-none` though that would also point to an issue with the Cactus Fortran file processing not breaking things into short enough lines. Yours, Roland > [CAUTION: Non-UBC Email] > > Hi Roland, > > In the config file I attached, I do have this line included already: > > FPPFLAGS = -g -traditional > > Is there anything else I might try? > > Best, > Panayotis > ________________________________ > From: Roland Haas > Sent: Tuesday, September 2, 2025 5:08 PM > To: IOSIF PANAGIOTIS > Cc: Einstein Toolkit Users > Subject: Re: [Users] ET build errors (Leonardo DCGP, CINECA cluster) > > Hello Panayotis, > > Thanks for including all the output files. > > In your options list whenever your set FPP you also must also set > FPPFLAGS: > > FPPFLAGS=-traditional > > See eg generic.cfg or (very new!) Bruno's leonardo-DCGP.cfg file (in > simfactory master). > > Yours, > Roland > > > [CAUTION: Non-UBC Email] > > > > Hi Einstein Toolkit users, > > > > I am reaching out regarding some errors I am encountering during my attempts to compile ET on the Leonardo, CINECA cluster (DCGP partition). > > > > Here are some details on the Leonardo cluster: > > > > * > > cluster specifics: https://docs.hpc.cineca.it/hpc/leonardo.html#leonardo-card > > * > > OS: Red Hat Enterprise Linux 8.7 > > > > I attach the 4 configuration files I am using (with many added comments for guidance and future reference). > > I adapted them from the respective config files that Bruno Giacomazzo kindly provided. > > > > I also attach the make.log output of the command: > > > > ./simfactory/bin/sim build --verbose --reconfig --machine=leonardo-dcgp1 --optionlist simfactory/mdb/optionlists/leonardo-dcgp1.cfg --allocation=CNHPC_1479290_0 --thornlist thornlists/einsteintoolkit.th 2>&1 | tee make.log > > > > From what I understand, the build process complains about syntax errors in the Exact thorn in these two source files: > > > > * > > decode_pars.F > > * > > Boost_rotation_symmetric.F > > > > How can I bypass the above problems? > > > > To be honest, I am not sure that I really need this thorn. > > I was simply trying to compile ET using the default thornlist from a fresh download/installation. > > > > Is there any other error that the log file is pointing out that I might be missing? > > > > Best regards, > > Panayotis > > > > > > ------------------- > > Panagiotis Iosif > > postdoctoral researcher > > Department of Physics, University of Trieste > > Via Alfonso Valerio 2, Trieste 34127 > > Italy > > ------------------- > > > > > -- > My email is as private as my paper mail. I therefore support encrypting > and signing email messages. Get my PGP key from http://pgp.mit.edu . -- My email is as private as my paper mail. I therefore support encrypting and signing email messages. Get my PGP key from http://pgp.mit.edu . From PANAGIOTIS.IOSIF at units.it Tue Sep 2 11:47:25 2025 From: PANAGIOTIS.IOSIF at units.it (IOSIF PANAGIOTIS) Date: Tue, 2 Sep 2025 16:47:25 +0000 Subject: [Users] ET build errors (Leonardo DCGP, CINECA cluster) In-Reply-To: <20250902084319.7b1f0e24@fdea4908> References: <20250902080809.2c5aa158@fdea4908> <20250902084319.7b1f0e24@fdea4908> Message-ID: Hi Roland, I amended my cfg file and added the flag -ffixed-line-length-none , in the Fortran 77 and 90 compiler options: * F77FLAGS = -g -march=native -fcray-pointer -ffixed-line-length-none * F90FLAGS = -g -march=native -fcray-pointer -ffixed-line-length-none This step solved the issue with the complaints about the "Exact" thorn. The build process continued and stopped at a different point, now complaining about the CarpetX/Arith thorn. I have attached the new make.log file. I ran the log through ChatGPT, and it suggested that this could be caused by C++ standard mismatches. Does this sound reasonable to you? Currently, the flags for the C and C++ compilers in my config file are like this: * CFLAGS = -g -march=native -std=gnu99 * CXXFLAGS = -g -march=native -std=gnu++11 I checked Bruno's file, and the respective options there are the following: * CFLAGS = -g -std=gnu99 #-std=c11 * CXXFLAGS = -g -std=gnu++0x #-std=c++14 However, I should note that in Bruno's version, all CarpetX-related thorns are disabled in the ini file. Again, I am not sure I need CarpetX. At this point, I am trying to get the build process to work and run some examples on the cluster. Returning to the compiler options, I suspect that CFLAGS can be left alone. Do you have any suggestions on how to change CXXFLAGS to bypass the CarpetX/Arith complaints? I checked the different C dialect options here, and it seems that there are a number of possibilities. Is there some -std=... option that is new enough for CarpetX and old/standard enough so that older C++ code is also OK? Best, Panayotis ________________________________ From: Roland Haas Sent: Tuesday, September 2, 2025 5:43 PM To: IOSIF PANAGIOTIS Cc: Einstein Toolkit Users Subject: Re: [Users] ET build errors (Leonardo DCGP, CINECA cluster) Hello Panayotis, Oh, I had not noticed, sorry. Hmm, that is the typical cause for these issues. The only other difference I see is `-ffixed-line-length-none` though that would also point to an issue with the Cactus Fortran file processing not breaking things into short enough lines. Yours, Roland > [CAUTION: Non-UBC Email] > > Hi Roland, > > In the config file I attached, I do have this line included already: > > FPPFLAGS = -g -traditional > > Is there anything else I might try? > > Best, > Panayotis > ________________________________ > From: Roland Haas > Sent: Tuesday, September 2, 2025 5:08 PM > To: IOSIF PANAGIOTIS > Cc: Einstein Toolkit Users > Subject: Re: [Users] ET build errors (Leonardo DCGP, CINECA cluster) > > Hello Panayotis, > > Thanks for including all the output files. > > In your options list whenever your set FPP you also must also set > FPPFLAGS: > > FPPFLAGS=-traditional > > See eg generic.cfg or (very new!) Bruno's leonardo-DCGP.cfg file (in > simfactory master). > > Yours, > Roland > > > [CAUTION: Non-UBC Email] > > > > Hi Einstein Toolkit users, > > > > I am reaching out regarding some errors I am encountering during my attempts to compile ET on the Leonardo, CINECA cluster (DCGP partition). > > > > Here are some details on the Leonardo cluster: > > > > * > > cluster specifics: https://docs.hpc.cineca.it/hpc/leonardo.html#leonardo-card > > * > > OS: Red Hat Enterprise Linux 8.7 > > > > I attach the 4 configuration files I am using (with many added comments for guidance and future reference). > > I adapted them from the respective config files that Bruno Giacomazzo kindly provided. > > > > I also attach the make.log output of the command: > > > > ./simfactory/bin/sim build --verbose --reconfig --machine=leonardo-dcgp1 --optionlist simfactory/mdb/optionlists/leonardo-dcgp1.cfg --allocation=CNHPC_1479290_0 --thornlist thornlists/einsteintoolkit.th 2>&1 | tee make.log > > > > From what I understand, the build process complains about syntax errors in the Exact thorn in these two source files: > > > > * > > decode_pars.F > > * > > Boost_rotation_symmetric.F > > > > How can I bypass the above problems? > > > > To be honest, I am not sure that I really need this thorn. > > I was simply trying to compile ET using the default thornlist from a fresh download/installation. > > > > Is there any other error that the log file is pointing out that I might be missing? > > > > Best regards, > > Panayotis > > > > > > ------------------- > > Panagiotis Iosif > > postdoctoral researcher > > Department of Physics, University of Trieste > > Via Alfonso Valerio 2, Trieste 34127 > > Italy > > ------------------- > > > > > -- > My email is as private as my paper mail. I therefore support encrypting > and signing email messages. Get my PGP key from http://pgp.mit.edu . -- My email is as private as my paper mail. I therefore support encrypting and signing email messages. Get my PGP key from http://pgp.mit.edu . -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: make.log Type: text/x-log Size: 423233 bytes Desc: make.log URL: From rhaas at phas.ubc.ca Tue Sep 2 12:05:44 2025 From: rhaas at phas.ubc.ca (Roland Haas) Date: Tue, 2 Sep 2025 10:05:44 -0700 Subject: [Users] ET build errors (Leonardo DCGP, CINECA cluster) In-Reply-To: References: <20250902080809.2c5aa158@fdea4908> <20250902084319.7b1f0e24@fdea4908> Message-ID: <20250902100544.5167e1d7@fdea4908> Hello Panayotis, oh, oops, for CarpetX (which includes Arith) code you actually need -std=c++17 . If you are not planning to develop for CarpetX then you can remove all CarpetX related thorns from your thornlist. Note that if you have a specific parameter file that you would like to run, then you can use the MakeThornList helper script to create a thornlist with just the thorns required for that parameter file, potentially speeding up compilation quite a bit. Something like: ./utils/Scripts/MakeThornList --master thornlists/einsteintoolkit.th --output thornlists/qc0.th par/qc0-mclachlan.par if you wanted to run qc0-mclachlan.par and had the full thornlist in thornlists/einsteintoolkit.th (GetComponents will do that for you). Then pass qc0.th to simfactory using its --thornlist argument (you likely also need to pass --reconfig to make sure it actually updates the thornlist it uses). Yours, Roland > [CAUTION: Non-UBC Email] > > Hi Roland, > > I amended my cfg file and added the flag -ffixed-line-length-none , > in the Fortran 77 and 90 compiler options: > > > * > F77FLAGS = -g -march=native -fcray-pointer -ffixed-line-length-none > * > F90FLAGS = -g -march=native -fcray-pointer -ffixed-line-length-none > > This step solved the issue with the complaints about the "Exact" > thorn. > > > The build process continued and stopped at a different point, now > complaining about the CarpetX/Arith thorn. > > I have attached the new make.log file. > > I ran the log through ChatGPT, and it suggested that this could be > caused by C++ standard mismatches. Does this sound reasonable to you? > > Currently, the flags for the C and C++ compilers in my config file > are like this: > > * > CFLAGS = -g -march=native -std=gnu99 > * > CXXFLAGS = -g -march=native -std=gnu++11 > > I checked Bruno's file, and the respective options there are the > following: > > * > CFLAGS = -g -std=gnu99 #-std=c11 > * > CXXFLAGS = -g -std=gnu++0x #-std=c++14 > > However, I should note that in Bruno's version, all CarpetX-related > thorns are disabled in the ini file. > > Again, I am not sure I need CarpetX. > At this point, I am trying to get the build process to work and run > some examples on the cluster. > > > Returning to the compiler options, I suspect that CFLAGS can be left > alone. Do you have any suggestions on how to change CXXFLAGS to > bypass the CarpetX/Arith complaints? > > I checked the different C dialect options > here, and > it seems that there are a number of possibilities. > > Is there some -std=... option that is new enough for CarpetX and > old/standard enough so that older C++ code is also OK? > > > Best, > Panayotis > > > > > ________________________________ > From: Roland Haas > Sent: Tuesday, September 2, 2025 5:43 PM > To: IOSIF PANAGIOTIS > Cc: Einstein Toolkit Users > Subject: Re: [Users] ET build errors (Leonardo DCGP, CINECA cluster) > > Hello Panayotis, > > Oh, I had not noticed, sorry. Hmm, that is the typical cause for these > issues. > > The only other difference I see is `-ffixed-line-length-none` though > that would also point to an issue with the Cactus Fortran file > processing not breaking things into short enough lines. > > Yours, > Roland > > > [CAUTION: Non-UBC Email] > > > > Hi Roland, > > > > In the config file I attached, I do have this line included already: > > > > FPPFLAGS = -g -traditional > > > > Is there anything else I might try? > > > > Best, > > Panayotis > > ________________________________ > > From: Roland Haas > > Sent: Tuesday, September 2, 2025 5:08 PM > > To: IOSIF PANAGIOTIS > > Cc: Einstein Toolkit Users > > Subject: Re: [Users] ET build errors (Leonardo DCGP, CINECA cluster) > > > > Hello Panayotis, > > > > Thanks for including all the output files. > > > > In your options list whenever your set FPP you also must also set > > FPPFLAGS: > > > > FPPFLAGS=-traditional > > > > See eg generic.cfg or (very new!) Bruno's leonardo-DCGP.cfg file (in > > simfactory master). > > > > Yours, > > Roland > > > > > [CAUTION: Non-UBC Email] > > > > > > Hi Einstein Toolkit users, > > > > > > I am reaching out regarding some errors I am encountering during > > > my attempts to compile ET on the Leonardo, CINECA cluster (DCGP > > > partition). > > > > > > Here are some details on the Leonardo cluster: > > > > > > * > > > cluster specifics: > > > https://docs.hpc.cineca.it/hpc/leonardo.html#leonardo-card * > > > OS: Red Hat Enterprise Linux 8.7 > > > > > > I attach the 4 configuration files I am using (with many added > > > comments for guidance and future reference). I adapted them from > > > the respective config files that Bruno Giacomazzo kindly provided. > > > > > > I also attach the make.log output of the command: > > > > > > ./simfactory/bin/sim build --verbose --reconfig > > > --machine=leonardo-dcgp1 --optionlist > > > simfactory/mdb/optionlists/leonardo-dcgp1.cfg > > > --allocation=CNHPC_1479290_0 --thornlist > > > thornlists/einsteintoolkit.th 2>&1 | tee make.log > > > > > > From what I understand, the build process complains about syntax > > > errors in the Exact thorn in these two source files: > > > > > > * > > > decode_pars.F > > > * > > > Boost_rotation_symmetric.F > > > > > > How can I bypass the above problems? > > > > > > To be honest, I am not sure that I really need this thorn. > > > I was simply trying to compile ET using the default thornlist > > > from a fresh download/installation. > > > > > > Is there any other error that the log file is pointing out that I > > > might be missing? > > > > > > Best regards, > > > Panayotis > > > > > > > > > ------------------- > > > Panagiotis Iosif > > > postdoctoral researcher > > > Department of Physics, University of Trieste > > > Via Alfonso Valerio 2, Trieste 34127 > > > Italy > > > ------------------- > > > > > > > > > -- > > My email is as private as my paper mail. I therefore support > > encrypting and signing email messages. Get my PGP key from > > http://pgp.mit.edu . > > > -- > My email is as private as my paper mail. I therefore support > encrypting and signing email messages. Get my PGP key from > http://pgp.mit.edu . Yours, Roland -- My email is as private as my paper mail. I therefore support encrypting and signing email messages. Get my PGP key from http://pgp.mit.edu . From PANAGIOTIS.IOSIF at units.it Wed Sep 3 05:23:27 2025 From: PANAGIOTIS.IOSIF at units.it (IOSIF PANAGIOTIS) Date: Wed, 3 Sep 2025 10:23:27 +0000 Subject: [Users] ET build errors (Leonardo DCGP, CINECA cluster) In-Reply-To: <20250902100544.5167e1d7@fdea4908> References: <20250902080809.2c5aa158@fdea4908> <20250902084319.7b1f0e24@fdea4908> <20250902100544.5167e1d7@fdea4908> Message-ID: Hi Roland, Got it. Here is an update: I removed all CarpetX-related thorns and tried to compile again. This time, the build complained about thorns GRHayLHDX, GRHayLIDX, and NewRadX: CST error 1: -> Thorn 'GRHayLHDX' requires the capability 'Loop'. Please add a thorn that provides 'Loop' to your ThornList or remove 'GRHayLHDX' from it ! CST error 2: -> Thorn 'GRHayLIDX' requires the capability 'Loop'. Please add a thorn that provides 'Loop' to your ThornList or remove 'GRHayLIDX' from it ! CST error 3: -> Thorn 'NewRadX' requires the capability 'Loop'. Please add a thorn that provides 'Loop' to your ThornList or remove 'NewRadX' from it ! Comparing with Bruno's original file, I saw that he had disabled GRHayLHDX and GRHayLIDX. So I went ahead and disabled these two thorns for starters. Then the build complained just about the NewRadX thorn (same error message as above). So I disabled NewRadX too (see updated .ini file attached) and the error about NewRadX went away. Now the build complains about the CCE_Export thorn. You may see the error messages in the updated log file attached. >From what I understand, the first main error seems to be this (see make_updated.log file): /leonardo/home/userexternal/piosif00/Cactus/arrangements/EinsteinAnalysis/CCE_Export/src/h5_export.cc:8:21: error: 'filesystem' is not a namespace-name; did you mean 'system'? 8 | namespace fs = std::filesystem; The text around that error message suggests that again a newer C++ dialect option might be required (-std=c++17' or '-std=gnu++17' ). In my current cfg file, following what was mentioned in the wiki page about configuring a new machine, I have an older option, namely -std=gnu+11, for CXXFLAGS. I will experiment with different C standards options and see if something else works. Questions: * Are GRHayLHDX, GRHayLIDX and NewRadX safe to disable? * For context, my goal is to start with isolated neutron star simulations. Are these thorns necessary for that? * Do we expect newer C/C++ standards, like -std=c++17, to break backwards compatibility, i.e. older code? * Please let me know if I am on the right track, or if you see some additional issues from the log file. Let me note that I did not start with all of the disabled thorns that Bruno's original ini file uses, because I didn't know exactly what was needed or not. Currently, the disabled thorns in my ini file almost align with those in Bruno's original file. There are, though, some additional thorns disabled in Bruno's original file, namely: ADIOS2, AMReX, Silo, PAPI and THCExtra/WeakRates. I do not know if I should disable them too, but since I did not encounter specific errors about them, I let them be. Let me know if they should go too. Thanks for the help so far! Best, Panayotis ________________________________ From: Roland Haas Sent: Tuesday, September 2, 2025 7:05 PM To: IOSIF PANAGIOTIS Cc: rhaas at mail.ubc.ca ; Einstein Toolkit Users Subject: Re: [Users] ET build errors (Leonardo DCGP, CINECA cluster) Hello Panayotis, oh, oops, for CarpetX (which includes Arith) code you actually need -std=c++17 . If you are not planning to develop for CarpetX then you can remove all CarpetX related thorns from your thornlist. Note that if you have a specific parameter file that you would like to run, then you can use the MakeThornList helper script to create a thornlist with just the thorns required for that parameter file, potentially speeding up compilation quite a bit. Something like: ./utils/Scripts/MakeThornList --master thornlists/einsteintoolkit.th --output thornlists/qc0.th par/qc0-mclachlan.par if you wanted to run qc0-mclachlan.par and had the full thornlist in thornlists/einsteintoolkit.th (GetComponents will do that for you). Then pass qc0.th to simfactory using its --thornlist argument (you likely also need to pass --reconfig to make sure it actually updates the thornlist it uses). Yours, Roland > [CAUTION: Non-UBC Email] > > Hi Roland, > > I amended my cfg file and added the flag -ffixed-line-length-none , > in the Fortran 77 and 90 compiler options: > > > * > F77FLAGS = -g -march=native -fcray-pointer -ffixed-line-length-none > * > F90FLAGS = -g -march=native -fcray-pointer -ffixed-line-length-none > > This step solved the issue with the complaints about the "Exact" > thorn. > > > The build process continued and stopped at a different point, now > complaining about the CarpetX/Arith thorn. > > I have attached the new make.log file. > > I ran the log through ChatGPT, and it suggested that this could be > caused by C++ standard mismatches. Does this sound reasonable to you? > > Currently, the flags for the C and C++ compilers in my config file > are like this: > > * > CFLAGS = -g -march=native -std=gnu99 > * > CXXFLAGS = -g -march=native -std=gnu++11 > > I checked Bruno's file, and the respective options there are the > following: > > * > CFLAGS = -g -std=gnu99 #-std=c11 > * > CXXFLAGS = -g -std=gnu++0x #-std=c++14 > > However, I should note that in Bruno's version, all CarpetX-related > thorns are disabled in the ini file. > > Again, I am not sure I need CarpetX. > At this point, I am trying to get the build process to work and run > some examples on the cluster. > > > Returning to the compiler options, I suspect that CFLAGS can be left > alone. Do you have any suggestions on how to change CXXFLAGS to > bypass the CarpetX/Arith complaints? > > I checked the different C dialect options > here, and > it seems that there are a number of possibilities. > > Is there some -std=... option that is new enough for CarpetX and > old/standard enough so that older C++ code is also OK? > > > Best, > Panayotis > > > > > ________________________________ > From: Roland Haas > Sent: Tuesday, September 2, 2025 5:43 PM > To: IOSIF PANAGIOTIS > Cc: Einstein Toolkit Users > Subject: Re: [Users] ET build errors (Leonardo DCGP, CINECA cluster) > > Hello Panayotis, > > Oh, I had not noticed, sorry. Hmm, that is the typical cause for these > issues. > > The only other difference I see is `-ffixed-line-length-none` though > that would also point to an issue with the Cactus Fortran file > processing not breaking things into short enough lines. > > Yours, > Roland > > > [CAUTION: Non-UBC Email] > > > > Hi Roland, > > > > In the config file I attached, I do have this line included already: > > > > FPPFLAGS = -g -traditional > > > > Is there anything else I might try? > > > > Best, > > Panayotis > > ________________________________ > > From: Roland Haas > > Sent: Tuesday, September 2, 2025 5:08 PM > > To: IOSIF PANAGIOTIS > > Cc: Einstein Toolkit Users > > Subject: Re: [Users] ET build errors (Leonardo DCGP, CINECA cluster) > > > > Hello Panayotis, > > > > Thanks for including all the output files. > > > > In your options list whenever your set FPP you also must also set > > FPPFLAGS: > > > > FPPFLAGS=-traditional > > > > See eg generic.cfg or (very new!) Bruno's leonardo-DCGP.cfg file (in > > simfactory master). > > > > Yours, > > Roland > > > > > [CAUTION: Non-UBC Email] > > > > > > Hi Einstein Toolkit users, > > > > > > I am reaching out regarding some errors I am encountering during > > > my attempts to compile ET on the Leonardo, CINECA cluster (DCGP > > > partition). > > > > > > Here are some details on the Leonardo cluster: > > > > > > * > > > cluster specifics: > > > https://docs.hpc.cineca.it/hpc/leonardo.html#leonardo-card * > > > OS: Red Hat Enterprise Linux 8.7 > > > > > > I attach the 4 configuration files I am using (with many added > > > comments for guidance and future reference). I adapted them from > > > the respective config files that Bruno Giacomazzo kindly provided. > > > > > > I also attach the make.log output of the command: > > > > > > ./simfactory/bin/sim build --verbose --reconfig > > > --machine=leonardo-dcgp1 --optionlist > > > simfactory/mdb/optionlists/leonardo-dcgp1.cfg > > > --allocation=CNHPC_1479290_0 --thornlist > > > thornlists/einsteintoolkit.th 2>&1 | tee make.log > > > > > > From what I understand, the build process complains about syntax > > > errors in the Exact thorn in these two source files: > > > > > > * > > > decode_pars.F > > > * > > > Boost_rotation_symmetric.F > > > > > > How can I bypass the above problems? > > > > > > To be honest, I am not sure that I really need this thorn. > > > I was simply trying to compile ET using the default thornlist > > > from a fresh download/installation. > > > > > > Is there any other error that the log file is pointing out that I > > > might be missing? > > > > > > Best regards, > > > Panayotis > > > > > > > > > ------------------- > > > Panagiotis Iosif > > > postdoctoral researcher > > > Department of Physics, University of Trieste > > > Via Alfonso Valerio 2, Trieste 34127 > > > Italy > > > ------------------- > > > > > > > > > -- > > My email is as private as my paper mail. I therefore support > > encrypting and signing email messages. Get my PGP key from > > http://pgp.mit.edu . > > > -- > My email is as private as my paper mail. I therefore support > encrypting and signing email messages. Get my PGP key from > http://pgp.mit.edu . Yours, Roland -- My email is as private as my paper mail. I therefore support encrypting and signing email messages. Get my PGP key from http://pgp.mit.edu . -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: make_updated.log Type: text/x-log Size: 321172 bytes Desc: make_updated.log URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: leonardo-dcgp1_updated.ini Type: application/octet-stream Size: 6470 bytes Desc: leonardo-dcgp1_updated.ini URL: From rhaas at mail.ubc.ca Wed Sep 3 10:29:06 2025 From: rhaas at mail.ubc.ca (Roland Haas) Date: Wed, 3 Sep 2025 08:29:06 -0700 Subject: [Users] ET build errors (Leonardo DCGP, CINECA cluster) In-Reply-To: References: <20250902080809.2c5aa158@fdea4908> <20250902084319.7b1f0e24@fdea4908> <20250902100544.5167e1d7@fdea4908> Message-ID: <20250903082906.03675083@haengie2.phas.ubc.ca> Hello Panayotis, > I removed all CarpetX-related thorns and tried to compile again. > This time, the build complained about thorns GRHayLHDX, GRHayLIDX, and NewRadX: They are all using CarpetX (the "X" is the giveaway :-) ). > Now the build complains about the CCE_Export thorn. Hmm, that one is very new. > You may see the error messages in the updated log file attached. > > From what I understand, the first main error seems to be this (see make_updated.log file): > /leonardo/home/userexternal/piosif00/Cactus/arrangements/EinsteinAnalysis/CCE_Export/src/h5_export.cc:8:21: error: 'filesystem' is not a namespace-name; did you mean 'system'? > 8 | namespace fs = std::filesystem; > The text around that error message suggests that again a newer C++ dialect option might be required (-std=c++17' or '-std=gnu++17' ). Hmm, for C++'s filesystem namespace is somewhat new. Some older compiler put it in experimental/filesystem . However CCE_Export already has code to handle this (in src/h5_export.cc): #if defined __cpp_lib_filesystem && __cpp_lib_filesystem < 201703L #include namespace fs = std::experimental::filesystem; #else #include namespace fs = std::filesystem; #endif Hmm, you are using gcc-12 though I'd have expected it to be new enough for this. Hmm, hmm, those __cpp_lib_filesystem ones may only haven been introduced in C++20 (https://en.cppreference.com/w/cpp/feature_test.html) and they require the header, which itself is C++20 (though it exists in gcc-12 and will have values). Unfortunately even if I add #include to src/h5_export.cc then things still fail with -std=gnu++11 since the macros are only defined when C++17 is used. So.... I'd disable the thorn (only used to export data for use with SpECTRE's CCE code) or enable C++17 support. This may require some larger reworking of the ET code to try and contain C++17 requirements to CarpetX code if possible. > In my current cfg file, following what was mentioned in the wiki page > about configuring a new > machine, > I have an older option, namely -std=gnu+11, for CXXFLAGS. gnu+11 is kind of old by now, at least I'd try gnu++14 (10 years old). So no guarantees that this would work (gcc defaults to gnu++17 as of version 11 "C++17 mode is the default since GCC 11" on https://gcc.gnu.org/projects/cxx-status.html). > I will experiment with different C standards options and see if > something else works. I'd have expected that both gnu++14 and gnu++17 should work actually (as long as CarpetX related code is disable since it and eg AMReX require C++17). > Questions: > > * > Are GRHayLHDX, GRHayLIDX and NewRadX safe to disable? Yes, they are all using CarpetX. > * > For context, my goal is to start with isolated neutron star simulations. Are these thorns necessary for that? No, they are CarpetX flavors of the GRHayLHD, GRHayLID and NewRad thorns. > * > Do we expect newer C/C++ standards, like -std=c++17, to break backwards compatibility, i.e. older code? Yes, new standards will eventually remove functionality that has been deprecated in older ones. On top of that g++ is becoming more strict in allowing non-standard constructs. This is, unfortunately, beyond our control. > * > Please let me know if I am on the right track, or if you see some > additional issues from the log file. > Let me note that I did not start with all of the disabled thorns that > Bruno's original ini > file > uses, because I didn't know exactly what was needed or not. > Currently, the disabled thorns in my ini file almost align with those > in Bruno's original file. There are, though, some additional thorns > disabled in Bruno's original file, namely: ADIOS2, AMReX, Silo, PAPI > and THCExtra/WeakRates. All except PAPI (which you'll only need if doing low-level benchmarking) are using only by CarpetX (or in WeakRates case are no even part of the Einstein Toolkit), so those can also be safely disabled. > I do not know if I should disable them too, but since I did not > encounter specific errors about them, I let them be. Let me know if > they should go too. It will make compiling faster (note the MakeThornList script I had suggested will also remove them). Yours, Roland -- My email is as private as my paper mail. I therefore support encrypting and signing email messages. Get my PGP key from http://pgp.mit.edu . From users at einsteintoolkit.org Wed Sep 3 17:15:02 2025 From: users at einsteintoolkit.org (users at einsteintoolkit.org) Date: Wed, 03 Sep 2025 17:15:02 -0500 Subject: [Users] Einstein Toolkit Meeting Reminder Message-ID: <68b8bde6.w0IYi92W+DEmY7/d%users@einsteintoolkit.org> Hello, Please consider joining the weekly Einstein Toolkit phone call at 9:00 am US central time on Thursdays. For details on how to connect and what agenda items are to be discussed, use the link below. https://docs.einsteintoolkit.org/et-docs/Main_Page#Weekly_Users_Call --The Maintainers From f.h.noori at gmail.com Thu Sep 4 10:02:54 2025 From: f.h.noori at gmail.com (Fatemeh Hossein Nouri) Date: Thu, 4 Sep 2025 17:02:54 +0200 Subject: [Users] Help with Spritz + external FUKA/Kadath Message-ID: Hi all, I?ve been trying to run Spritz using initial data generated by the latest version of FUKA for a BNS simulation, but I?ve run into issues related to the Kadath libraries. At first, I was getting a mismatch error (Assertion ndim==nbr_points.get_ndim()), which suggests an incompatibility between Spritz?s bundled Kadath and the external version of FUKA/Kadath I used to generate the initial data. To fix this, I tried to build Spritz against my own external Kadath installation (~/fuka/lib/libkadath.a) instead of the one bundled in the Einstein Toolkit. I did this by disabling these thorns in my thornlist: Fuka/kadath_pizza, Fuka/KadathImporter and Fuka/KadathThorn. I also added the following lines to my ET_2024_05/Cactus/simfactory/mdb/optionlists/.cfg KADATH_DIR = $(HOME)/fuka KADATH_INC_DIRS = $(KADATH_DIR)/include KADATH_LIB_DIRS = $(KADATH_DIR)/lib One question so far: Did I do things correctly? Then I removed the old Kadath: rm -rf configs/Spritz_LORENE/build/Kadath The code compiles fine, but when I try to run the executable with my parameter file, using the '-S' option, I get: Error: Thorn kadathimporter not found Error: Thorn kadaththorn not found Activation failed - 2 errors in activation sequence The confusing part is that my ActiveThorns line in the .par file does not explicitly list KadathThorn or KadathImporter: ActiveThorns = "volomnia bnstrackergen bnsanalysis pizzanumutils" Yet Spritz fails because those thorns are missing. From what I understand, some of the thorns I?m using (like volomnia, bnstrackergen, or bnsanalysis) depend on KadathThorn and KadathImporter, so they are indirectly required. The core problem: - I need to keep KadathThorn and KadathImporter active, since my parameter file requires them through dependencies. - But I also need them to link against my external Kadath ( ~/fuka/lib/libkadath.a) so that the FUKA initial data is compatible. Would you have suggestions on the cleanest way to make the ET KadathThorn and KadathImporter use my external Kadath library? Is there a recommended workflow for this? Thanks in advance for your help! Bests, Fatemeh Nouri -------------- next part -------------- An HTML attachment was scrubbed... URL: From PANAGIOTIS.IOSIF at units.it Thu Sep 4 12:00:39 2025 From: PANAGIOTIS.IOSIF at units.it (IOSIF PANAGIOTIS) Date: Thu, 4 Sep 2025 17:00:39 +0000 Subject: [Users] ET build errors (Leonardo DCGP, CINECA cluster) In-Reply-To: <20250903082906.03675083@haengie2.phas.ubc.ca> References: <20250902080809.2c5aa158@fdea4908> <20250902084319.7b1f0e24@fdea4908> <20250902100544.5167e1d7@fdea4908> <20250903082906.03675083@haengie2.phas.ubc.ca> Message-ID: Hi Roland, Thank you very much for your detailed explanations. Here is another follow-up that seems to have a happy ending: * I disabled the ADIOS2, AMReX and Silo thorns and recompiled. * I did not add THCExtra/WeakRates to the disabled thorn list. * >From what I understand, this must probably be related to WhiskyTHC, which I do not have installed. * The build complained about the CMake version, triggered by the ExternalLibraries/OpenPMD thorn. openPMD: Configuring... CMake Error at CMakeLists.txt:3 (cmake_minimum_required): CMake 3.22.0 or higher is required. You are running version 3.20.2 * For some reason, the build this time did not complain about the CCE_Export thorn, even though I did not update the -std=gnu+11 option under CXXFLAGS yet (I was trying to make one change at a time for clarity). * I found this behavior strange. I guess the CMake error happened first, and the build terminated before reaching the CCE_Export problem? Anyway, this error seemed easy enough to fix. * I updated envsetup in my ini file with: * module load cmake/3.27.9 * Recompiled, and the CMake version error went away, but (expectedly) the CCE_Export error reappeared. Then, I tried recompiling by adding -std=gnu++14 to CXXFLAGS. * The build failed, complaining again about the CCE_Export thorn. * At least the change from gnu+11 to gnu++14did not cause new additional errors. * at least no new errors, in the specific cluster and with the specific config files. Then I tried recompiling by adding -std=gnu++17 to CXXFLAGS. * >From the attached log file, it seems to me that the build was successful this time. * No new errors arose because of the change to -std=gnu++17 in CXXFLAGS. I will now attempt to run the TOV star example from the gallery and see if everything works as it should. I will start a new email thread in case I encounter run problems. I appreciate the help! Best, Panayotis ------------------- Panagiotis Iosif postdoctoral researcher Department of Physics, University of Trieste Via Alfonso Valerio 2, Trieste 34127 Italy ------------------- ________________________________ From: Roland Haas Sent: Wednesday, September 3, 2025 5:29 PM To: IOSIF PANAGIOTIS Cc: Einstein Toolkit Users Subject: Re: [Users] ET build errors (Leonardo DCGP, CINECA cluster) Hello Panayotis, > I removed all CarpetX-related thorns and tried to compile again. > This time, the build complained about thorns GRHayLHDX, GRHayLIDX, and NewRadX: They are all using CarpetX (the "X" is the giveaway :-) ). > Now the build complains about the CCE_Export thorn. Hmm, that one is very new. > You may see the error messages in the updated log file attached. > > From what I understand, the first main error seems to be this (see make_updated.log file): > /leonardo/home/userexternal/piosif00/Cactus/arrangements/EinsteinAnalysis/CCE_Export/src/h5_export.cc:8:21: error: 'filesystem' is not a namespace-name; did you mean 'system'? > 8 | namespace fs = std::filesystem; > The text around that error message suggests that again a newer C++ dialect option might be required (-std=c++17' or '-std=gnu++17' ). Hmm, for C++'s filesystem namespace is somewhat new. Some older compiler put it in experimental/filesystem . However CCE_Export already has code to handle this (in src/h5_export.cc): #if defined __cpp_lib_filesystem && __cpp_lib_filesystem < 201703L #include namespace fs = std::experimental::filesystem; #else #include namespace fs = std::filesystem; #endif Hmm, you are using gcc-12 though I'd have expected it to be new enough for this. Hmm, hmm, those __cpp_lib_filesystem ones may only haven been introduced in C++20 (https://en.cppreference.com/w/cpp/feature_test.html) and they require the header, which itself is C++20 (though it exists in gcc-12 and will have values). Unfortunately even if I add #include to src/h5_export.cc then things still fail with -std=gnu++11 since the macros are only defined when C++17 is used. So.... I'd disable the thorn (only used to export data for use with SpECTRE's CCE code) or enable C++17 support. This may require some larger reworking of the ET code to try and contain C++17 requirements to CarpetX code if possible. > In my current cfg file, following what was mentioned in the wiki page > about configuring a new > machine, > I have an older option, namely -std=gnu+11, for CXXFLAGS. gnu+11 is kind of old by now, at least I'd try gnu++14 (10 years old). So no guarantees that this would work (gcc defaults to gnu++17 as of version 11 "C++17 mode is the default since GCC 11" on https://gcc.gnu.org/projects/cxx-status.html). > I will experiment with different C standards options and see if > something else works. I'd have expected that both gnu++14 and gnu++17 should work actually (as long as CarpetX related code is disable since it and eg AMReX require C++17). > Questions: > > * > Are GRHayLHDX, GRHayLIDX and NewRadX safe to disable? Yes, they are all using CarpetX. > * > For context, my goal is to start with isolated neutron star simulations. Are these thorns necessary for that? No, they are CarpetX flavors of the GRHayLHD, GRHayLID and NewRad thorns. > * > Do we expect newer C/C++ standards, like -std=c++17, to break backwards compatibility, i.e. older code? Yes, new standards will eventually remove functionality that has been deprecated in older ones. On top of that g++ is becoming more strict in allowing non-standard constructs. This is, unfortunately, beyond our control. > * > Please let me know if I am on the right track, or if you see some > additional issues from the log file. > Let me note that I did not start with all of the disabled thorns that > Bruno's original ini > file > uses, because I didn't know exactly what was needed or not. > Currently, the disabled thorns in my ini file almost align with those > in Bruno's original file. There are, though, some additional thorns > disabled in Bruno's original file, namely: ADIOS2, AMReX, Silo, PAPI > and THCExtra/WeakRates. All except PAPI (which you'll only need if doing low-level benchmarking) are using only by CarpetX (or in WeakRates case are no even part of the Einstein Toolkit), so those can also be safely disabled. > I do not know if I should disable them too, but since I did not > encounter specific errors about them, I let them be. Let me know if > they should go too. It will make compiling faster (note the MakeThornList script I had suggested will also remove them). Yours, Roland -- My email is as private as my paper mail. I therefore support encrypting and signing email messages. Get my PGP key from http://pgp.mit.edu . -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: make_gnu17.log Type: text/x-log Size: 942149 bytes Desc: make_gnu17.log URL: From rhaas at mail.ubc.ca Thu Sep 4 13:20:01 2025 From: rhaas at mail.ubc.ca (Roland Haas) Date: Thu, 4 Sep 2025 11:20:01 -0700 Subject: [Users] ET build errors (Leonardo DCGP, CINECA cluster) In-Reply-To: References: <20250902080809.2c5aa158@fdea4908> <20250902084319.7b1f0e24@fdea4908> <20250902100544.5167e1d7@fdea4908> <20250903082906.03675083@haengie2.phas.ubc.ca> Message-ID: <20250904112001.6f802d77@haengie2.phas.ubc.ca> Hello Panayotis, Thanks for the update. Fingers crossed that the simulations work well. CCE_Export requires C++17 (or using the experimental branch "rhaas/no_c++17" in https://github.com/rhaas80/CCE_Export). If you do not intent to use SpECTRE's CCE code then you can disable CCE_Export. OpenPMD as of commit: commit c6ac85d8f63ec4521e77d815035c0361c025614f (HEAD -> master, origin/master, origin/HEAD, origin/ET_2025_05, ET_2025_05) Author: Roland Haas Date: Thu Aug 7 18:26:43 2025 -0700 openPMD: bump version to 0.16.1 this includes fixes for HDF5 and a newewr version of toml11 that works with CMake 4.0 bump json version to 3.12 for cmake 4.0 support should work with CMake 4.0. But it is only ever used by CarpetX so if you have disabled CarpetX you lose nothing by also disabling openPMD. Yours, Roland > [CAUTION: Non-UBC Email] > > Hi Roland, > > Thank you very much for your detailed explanations. > > Here is another follow-up that seems to have a happy ending: > > > * > I disabled the ADIOS2, AMReX and Silo thorns and recompiled. > * > I did not add THCExtra/WeakRates to the disabled thorn list. > * > From what I understand, this must probably be related to WhiskyTHC, which I do not have installed. > * > The build complained about the CMake version, triggered by the ExternalLibraries/OpenPMD thorn. > > openPMD: Configuring... > CMake Error at CMakeLists.txt:3 (cmake_minimum_required): > CMake 3.22.0 or higher is required. You are running version 3.20.2 > > * > For some reason, the build this time did not complain about the CCE_Export thorn, even though I did not update the -std=gnu+11 option under CXXFLAGS yet (I was trying to make one change at a time for clarity). > * > I found this behavior strange. I guess the CMake error happened first, and the build terminated before reaching the CCE_Export problem? > > Anyway, this error seemed easy enough to fix. > > > * > I updated envsetup in my ini file with: > * > module load cmake/3.27.9 > * > Recompiled, and the CMake version error went away, but (expectedly) the CCE_Export error reappeared. > > Then, I tried recompiling by adding -std=gnu++14 to CXXFLAGS. > > * > The build failed, complaining again about the CCE_Export thorn. > * > At least the change from gnu+11 to gnu++14did not cause new additional errors. > * > at least no new errors, in the specific cluster and with the specific config files. > > Then I tried recompiling by adding -std=gnu++17 to CXXFLAGS. > > * > From the attached log file, it seems to me that the build was successful this time. > * > No new errors arose because of the change to -std=gnu++17 in CXXFLAGS. > > I will now attempt to run the TOV star example from the gallery and see if everything works as it should. > > I will start a new email thread in case I encounter run problems. > > I appreciate the help! > > Best, > Panayotis > > > ------------------- > Panagiotis Iosif > postdoctoral researcher > Department of Physics, University of Trieste > Via Alfonso Valerio 2, Trieste 34127 > Italy > ------------------- > > ________________________________ > From: Roland Haas > Sent: Wednesday, September 3, 2025 5:29 PM > To: IOSIF PANAGIOTIS > Cc: Einstein Toolkit Users > Subject: Re: [Users] ET build errors (Leonardo DCGP, CINECA cluster) > > Hello Panayotis, > > > I removed all CarpetX-related thorns and tried to compile again. > > This time, the build complained about thorns GRHayLHDX, GRHayLIDX, and NewRadX: > > They are all using CarpetX (the "X" is the giveaway :-) ). > > > Now the build complains about the CCE_Export thorn. > > Hmm, that one is very new. > > > You may see the error messages in the updated log file attached. > > > > From what I understand, the first main error seems to be this (see make_updated.log file): > > /leonardo/home/userexternal/piosif00/Cactus/arrangements/EinsteinAnalysis/CCE_Export/src/h5_export.cc:8:21: error: 'filesystem' is not a namespace-name; did you mean 'system'? > > 8 | namespace fs = std::filesystem; > > The text around that error message suggests that again a newer C++ dialect option might be required (-std=c++17' or '-std=gnu++17' ). > > Hmm, for C++'s filesystem namespace is somewhat new. Some older > compiler put it in experimental/filesystem . However CCE_Export already > has code to handle this (in src/h5_export.cc): > > #if defined __cpp_lib_filesystem && __cpp_lib_filesystem < 201703L > #include > namespace fs = std::experimental::filesystem; > #else > #include > namespace fs = std::filesystem; > #endif > > Hmm, you are using gcc-12 though I'd have expected it to be new enough > for this. > > Hmm, hmm, those __cpp_lib_filesystem ones may only haven been introduced > in C++20 (https://en.cppreference.com/w/cpp/feature_test.html) and they > require the header, which itself is C++20 (though it exists > in gcc-12 and will have values). > > Unfortunately even if I add > > #include > > to src/h5_export.cc then things still fail with -std=gnu++11 since the > macros are only defined when C++17 is used. > > So.... I'd disable the thorn (only used to export data for use with > SpECTRE's CCE code) or enable C++17 support. > > This may require some larger reworking of the ET code to try and > contain C++17 requirements to CarpetX code if possible. > > > In my current cfg file, following what was mentioned in the wiki page > > about configuring a new > > machine, > > I have an older option, namely -std=gnu+11, for CXXFLAGS. > > gnu+11 is kind of old by now, at least I'd try gnu++14 (10 years old). > So no guarantees that this would work (gcc defaults to gnu++17 as of > version 11 "C++17 mode is the default since GCC 11" on > https://gcc.gnu.org/projects/cxx-status.html). > > > I will experiment with different C standards options and see if > > something else works. > > I'd have expected that both gnu++14 and gnu++17 should work actually > (as long as CarpetX related code is disable since it and eg AMReX > require C++17). > > > Questions: > > > > * > > Are GRHayLHDX, GRHayLIDX and NewRadX safe to disable? > > Yes, they are all using CarpetX. > > > * > > For context, my goal is to start with isolated neutron star simulations. Are these thorns necessary for that? > > No, they are CarpetX flavors of the GRHayLHD, GRHayLID and NewRad > thorns. > > > * > > Do we expect newer C/C++ standards, like -std=c++17, to break backwards compatibility, i.e. older code? > > Yes, new standards will eventually remove functionality that has been > deprecated in older ones. On top of that g++ is becoming more strict in > allowing non-standard constructs. This is, unfortunately, beyond our > control. > > > * > > Please let me know if I am on the right track, or if you see some > > additional issues from the log file. > > > Let me note that I did not start with all of the disabled thorns that > > Bruno's original ini > > file > > uses, because I didn't know exactly what was needed or not. > > Currently, the disabled thorns in my ini file almost align with those > > in Bruno's original file. There are, though, some additional thorns > > disabled in Bruno's original file, namely: ADIOS2, AMReX, Silo, PAPI > > and THCExtra/WeakRates. > > All except PAPI (which you'll only need if doing low-level > benchmarking) are using only by CarpetX (or in WeakRates case are no > even part of the Einstein Toolkit), so those can also be safely > disabled. > > > I do not know if I should disable them too, but since I did not > > encounter specific errors about them, I let them be. Let me know if > > they should go too. > > It will make compiling faster (note the MakeThornList script I had > suggested will also remove them). > > > Yours, > Roland > > -- > My email is as private as my paper mail. I therefore support encrypting > and signing email messages. Get my PGP key from http://pgp.mit.edu . Yours, Roland -- My email is as private as my paper mail. I therefore support encrypting and signing email messages. Get my PGP key from http://pgp.mit.edu . From bill.gabella at gmail.com Thu Sep 4 14:06:44 2025 From: bill.gabella at gmail.com (Bill Gabella) Date: Thu, 4 Sep 2025 15:06:44 -0400 Subject: [Users] Meeting Minutes 2025-09-04 Message-ID: <45961438-66cd-418d-8a76-ba9e590cbcbf@gmail.com> Minutes for the Einstein Toolkit Meeting, 20250904. https://docs.einsteintoolkit.org/et-docs/Main_Page#Weekly_Users_Call 9am CDT Thursdays Present:? Steve B, Deborah F, Peter D, Peyhan, Roland H, Lucas TS, Johnny T, Nikolai W, Zach E, Keith D, Bill G, Leo W Chair: Peter D?? Minutes: Bill G * BBH gallery example (Zach) -- not yet done (2024-11 also missing) Peter and Zach, no new work. * AOB ** Next release is in May 2026 and Steve is the release manager. https://docs.einsteintoolkit.org/et-docs/Release_Process * Unanswered emails https://www.einsteintoolkit.org/tools/unanswered.php ** No updates. * Open tickets sorted https://bitbucket.org/einsteintoolkit/tickets/issues?status=open&status=new&sort=-updated_on #2885, test for std::filesystem availability in Silo output likely broken Roland, this should not affect you if you are on a modern system. You have to look at a predefined variable and you have to use it to include the version of a file, and it is declared in those files.? There is a version.h that was standardized in C++20...exists in older gcc but not standardized.? In the IO code of Carpet that writes out Silo, maybe check on a file existence. Not a major issue.? Was in CC_Export also, which has a recent pull request with the fix.? Some bug with a static variable that depends on a function argument so it is never changed.? On Deborah's to do list also.? Deborah, it may not need to change. Will take a look at it. #2882, Running BBH with CarpetX with CPUs: High memory consumption and low performance Alejandra will take a look at it. #1847, FFTW3 fortran interface not working for system installation A very old ticket from Frank Loeffler.? Yosef may use FFTW in his code(s). #2858, SF files for MN5 (was: Compiling CarpetX: issues with PDESolvers) Changed the name to the current one.? Lucas has begun testing, thinks files very specific to Alejandra's work, on Marenostrum 5 (Barcelona).? Some external libraries seem to compile themselves and put them into a folder referencing the user.? Steve, not uncommon but should not had code the user directory.? Lucas, that is what is happening here.? On LSU machines there are directories hardcoded to Steve's directories; that is where the external libraries are, and this works. Steve, setup a system to have future checkouts and builds refer to previous compiled libraries/tools.? Some external libraries take forever to compile and better to refer to some already installed library.? Good to have a "benevolent user" to install these for global use. Lucas, (Marenostrum) the machine is organized into projects and allocations, and projects have an expiration date...like a DOE machine.? Steve, that is awful.? Lucas agrees.? Lucas, think singularity would be the right thing to do, if the image is put together correctly.? Peter, can you chance the setup so that they use external libraries instead of their pre-compiled ones?? Lucas, have not tested with the bundles libraries.? Believe they tried that and failed.? Roland, you might succeed, when they tried some of the libraries did a git clone of a repositories.? Lucas, Marenostrum has limitations that annoy one.? Will try the bare minimum and see if that works. #2866, Problem while building the ET on MN5: OpenCL Steve disavows any knowledge of this one---name removed from ticket.? This is for Marenostrum 5. ?? [European clusters with ET include Marenostrum 5 (Barcelona), Leonardo (Bologna), and Sunrise (Stockholm).]? ? Peter, maybe Lucas can take a look at OpenCL? Roland, we are not using OpenCL for anything now---once we had an example using it.? Lucas, will try the bundled version of OpenCL and if it fails will ask them to take it out.? Leaching off of Steve's Singularity image which works on LSU machines, might try for Marenostrum. #2867, WaveToyX examples, HDF5 output error in all three examples. Steve, we discuss this every wekk and the error message is considered harmless.? We should close this ticket. #2773, make CarpetX-ThornDoc is confused in InterLatex.pl Steve, I need to check on that. #2878, CarpetX: Add multipatch output support to Silo Lucas, sitting and waiting for review.? Not urgent. #2764, PUGH tries to free memory not allocated by malloc Beyhan, no update. #2282, gallery examples use low-order integration n Multipole Roland, part of updating. #963, Improve McLachlan accuracy Peter, will meet with Zach and work on this. #2706, Update default TwoPunctures parameters, or at least default parameters in BBH gallery example Zach, part of the BBH Gallery example and will be looked at. #2877, ET_BHaHAHA Features Ticket Zach, keep this one, I use it as a ticket to jot down ideas to improve the robustness of BHaHAHA. #2855, NoiseX: Improve noise generation in CarpetX Lucas, not worked on it for a while, intend to get back to it. #2874, NRPyElliptic lacks regeneration instructions Leo, on my list, will get back to it.? Zach, you should wait on this.? I plan to refactor NRPyElliptic in a library like BHaHAHA. The refactor will be a different NRPyElliptic. #2862, Update `SpacetimeX/Z4c` robust stability paramete file Lucas, that is just a parameter file update.? Will review desire to include.? The existing one did not work, but this one should. #2860, Don't call the optimized 4th order second derivative operators. Peter, will take a look at it. #2052, piraha assumes that assert(false) always aborts Steve, I will get to this one. * Tickets for Review https://bitbucket.org/einsteintoolkit/tickets/issues?status=new&status=open&sort=-updated_on&q=Please%20review Nothing new.? Roland, no longer a ticket to backport libraries that compile with GCC 15 to earlier or to update with cmake 4.0. Applied those changes to master and release branches.? Tickets #2880, #2883. *AOB at end of meeting Roland, have a CarpetX call next and Erik said he would give a presentation on mesh files. Zach, should we advertise the CarpetX call?? Roland, we should list it.? Will draft a description of CarpetX and give the call link.? Bill, maybe we should put it down at the ET Meeting link [wiki] and not on the front web page. CarpetX wiki https://github.com/EinsteinToolkit/CarpetX/wiki CarpetX zoom https://ubc.zoom.us/j/66033196685?pwd=zxXTpcrBY1L7nT2kNFITm71ZuoZEiO.1 Next Meeting, Thursday, 11 September 2025. https://docs.einsteintoolkit.org/et-docs/Meeting_agenda FYI, bill e.g. -- Home Page LinkedIn -------------- next part -------------- An HTML attachment was scrubbed... URL: From topolski at itp.uni-frankfurt.de Fri Sep 5 01:04:31 2025 From: topolski at itp.uni-frankfurt.de (Konrad Topolski) Date: Fri, 05 Sep 2025 08:04:31 +0200 Subject: [Users] =?utf-8?q?Help_with_Spritz_+_external_FUKA/Kadath?= In-Reply-To: References: Message-ID: <2a49b-68ba7d80-461-21e18e00@197642200> Hi Fatemeh, The thorns you've disabled in the thornlist are responsible for the import of the data in the ADMBase / Hydrobase variables and interact directly with the FUKA exporters (in the bundled/separate FUKA installation). As such, they are necessary to import FUKA ID and simultaneously independent of the thorns you use for subsequent evolution. The kadathimport thorn provides the headers for the import functions for specific initial data, whose implementation is then linked from libkadath.a.? I would recommend that you uncomment the thorns in the thornlist, recompile (kadath library linking is still governed by the .cfg file, which shouldn't change) and re-enable the thorns in the parameter file so that they're active and the ETK can schedule the import routines in the appropriate bins. I was actually unaware that bnsanalysis and related analysis thorns depend on the kadath importer thorns. Is this a custom extension or is it possible that some other thorn lists them as depencencies?? In any case, doing the above should help. Best regards Konrad W dniu: Czwartek, Wrzesie? 04, 2025 17:02 CEST, Fatemeh Hossein Nouri napisa?(a): ? Hi all,? I?ve been trying to run Spritz?using initial data generated by the latest version of FUKA for a BNS simulation, but I?ve run into issues related to the Kadath libraries. At first, I was getting a mismatch error (Assertion ndim==nbr_points.get_ndim()), which suggests an incompatibility between Spritz?s bundled Kadath and the external version of FUKA/Kadath I used to generate the initial data. To fix this, I tried to build Spritz against my own external Kadath installation (~/fuka/lib/libkadath.a) instead of the one bundled in the Einstein Toolkit. I did this by disabling these thorns in my thornlist:?Fuka/kadath_pizza,?Fuka/KadathImporter and?Fuka/KadathThorn. I also added the following lines to my?ET_2024_05/Cactus/simfactory/mdb/optionlists/.cfg KADATH_DIR = $(HOME)/fuka KADATH_INC_DIRS = $(KADATH_DIR)/include ? KADATH_LIB_DIRS =?$(KADATH_DIR)/lib ? One question so far: Did I do things correctly? ? Then I removed the old Kadath: ? ? rm -rf configs/Spritz_LORENE/build/Kadath The code compiles fine, but when I try to run the executable with my parameter file, using the '-S' option, I get:?Error: Thorn kadathimporter not found Error: Thorn kadaththorn not found Activation failed - 2 errors in activation sequence The confusing part is that my ActiveThorns line in the .par file does not explicitly list KadathThorn or KadathImporter:?ActiveThorns = "volomnia bnstrackergen bnsanalysis pizzanumutils" Yet Spritz fails because those thorns are missing. From what I understand, some of the thorns I?m using (like volomnia, bnstrackergen, or bnsanalysis) depend on KadathThorn and KadathImporter, so they are indirectly required. The core problem: - I need to keep KadathThorn and KadathImporter active, since my parameter file requires them through dependencies. - But I also need them to link against my external Kadath (~/fuka/lib/libkadath.a) so that the FUKA initial data is compatible. Would you have suggestions on the cleanest way to make the ET KadathThorn and KadathImporter use my external Kadath library? Is there a recommended workflow for this? Thanks in advance for your help! Bests, Fatemeh Nouri ? ? -------------- next part -------------- An HTML attachment was scrubbed... URL: From tootle at itp.uni-frankfurt.de Fri Sep 5 01:21:23 2025 From: tootle at itp.uni-frankfurt.de (Samuel Tootle) Date: Fri, 5 Sep 2025 08:21:23 +0200 (GMT+02:00) Subject: [Users] Help with Spritz + external FUKA/Kadath In-Reply-To: <2a49b-68ba7d80-461-21e18e00@197642200> References: <2a49b-68ba7d80-461-21e18e00@197642200> Message-ID: <4b5c7353-39e1-4b40-8206-8ba7860f6cdb@itp.uni-frankfurt.de> Hi Fatemeh, In addition to Konrad's correct notes, I want to addresd the build issues: > "At first, I was getting a mismatch error (Assertion ndim==nbr_points.get_ndim()), which suggests an incompatibility between Spritz?s bundled Kadath and the external version of FUKA/Kadath I used to generate the initial data. > > To fix this, I tried to build Spritz against my own external Kadath installation (~/fuka/lib/libkadath.a) instead of the one bundled in the Einstein Toolkit. I did this by disabling these thorns in my thornlist:?Fuka/kadath_pizza,?Fuka/KadathImporter and?Fuka/KadathThorn. I also added the following lines to my?ET_2024_05/Cactus/simfactory/mdb/optionlists/.cfg > > KADATH_DIR = $(HOME)/fuka > > KADATH_INC_DIRS = $(KADATH_DIR)/include > ? > > KADATH_LIB_DIRS =?$(KADATH_DIR)/lib" You are likely correct if there is a pre compiled version of Fuka with spritz. If the data was generated with the latest release (since June 25), there is a discrepancy with how the numerical domains are constructed and stored as noted in the Readme. Linking against the latest library should resolve the ndim error. Regarding your cfg, I believe you have to use brackets, not parentheses, e.g. ${HOME_KADATH}, for the etk config parser to recognize environment variables. All the best, Samuel Volunteer Physicist Sep 5, 2025 08:04:51 Konrad Topolski : > Hi Fatemeh, > > The thorns you've disabled in the thornlist are responsible for the import of the data in the ADMBase / Hydrobase variables and interact directly with the FUKA exporters (in the bundled/separate FUKA installation). As such, they are necessary to import FUKA ID and simultaneously independent of the thorns you use for subsequent evolution. > > The kadathimport thorn provides the headers for the import functions for specific initial data, whose implementation is then linked from libkadath.a.? > > I would recommend that you uncomment the thorns in the thornlist, recompile (kadath library linking is still governed by the .cfg file, which shouldn't change) and re-enable the thorns in the parameter file so that they're active and the ETK can schedule the import routines in the appropriate bins. > > I was actually unaware that bnsanalysis and related analysis thorns depend on the kadath importer thorns. Is this a custom extension or is it possible that some other thorn lists them as depencencies?? > > In any case, doing the above should help. > > Best regards > > Konrad > > > > W dniu: Czwartek, Wrzesie? 04, 2025 17:02 CEST, Fatemeh Hossein Nouri napisa?(a): > > ? > >> Hi all,? >> >> I?ve been trying to run Spritz?using initial data generated by the latest version of FUKA for a BNS simulation, but I?ve run into issues related to the Kadath libraries. >> >> At first, I was getting a mismatch error (*Assertion ndim==nbr_points.get_ndim()*), which suggests an incompatibility between Spritz?s bundled Kadath and the external version of FUKA/Kadath I used to generate the initial data. >> >> To fix this, I tried to build Spritz against my own external Kadath installation (*~/fuka/lib/libkadath.a*) instead of the one bundled in the Einstein Toolkit. I did this by disabling these thorns in my thornlist:?Fuka/kadath_pizza,?Fuka/KadathImporter and?Fuka/KadathThorn. I also added the following lines to my?ET_2024_05/Cactus/simfactory/mdb/optionlists/.cfg >> >> KADATH_DIR = $(HOME)/fuka >> >> KADATH_INC_DIRS = $(KADATH_DIR)/include >> ? >> >> KADATH_LIB_DIRS =?$(KADATH_DIR)/lib >> >> ? >> One question so far: Did I do things correctly? >> >> ? >> Then I removed the old Kadath: >> >> ? >> ? >> rm -rf configs/Spritz_LORENE/build/Kadath >> >> The code compiles fine, but when I try to run the executable with my parameter file, using the '-S' option, I get: >> >> ? >> *Error: Thorn kadathimporter not found Error: Thorn kadaththorn not found Activation failed - 2 errors in activation sequence* >> >> The confusing part is that my *ActiveThorns* line in the *.par* file does not explicitly list *KadathThorn* or *KadathImporter*: >> >> ? >> *ActiveThorns = "volomnia bnstrackergen bnsanalysis pizzanumutils"* >> >> Yet Spritz fails because those thorns are missing. From what I understand, some of the thorns I?m using (like *volomnia*, *bnstrackergen*, or *bnsanalysis*) depend on *KadathThorn* and *KadathImporter*, so they are indirectly required. >> >> The core problem: >> >> - I need to keep *KadathThorn* and *KadathImporter* active, since my parameter file requires them through dependencies. >> >> - But I also need them to link against my external Kadath (*~/fuka/lib/libkadath.a*) so that the FUKA initial data is compatible. >> >> Would you have suggestions on the cleanest way to make the ET *KadathThorn* and *KadathImporter* use my external Kadath library? Is there a recommended workflow for this? >> >> Thanks in advance for your help! >> >> Bests, >> >> Fatemeh Nouri >> >> >> ? >> > > > > ? > -------------- next part -------------- An HTML attachment was scrubbed... URL: From f.h.noori at gmail.com Fri Sep 5 11:11:18 2025 From: f.h.noori at gmail.com (Fatemeh Hossein Nouri) Date: Fri, 5 Sep 2025 18:11:18 +0200 Subject: [Users] Help with Spritz + external FUKA/Kadath In-Reply-To: <4b5c7353-39e1-4b40-8206-8ba7860f6cdb@itp.uni-frankfurt.de> References: <2a49b-68ba7d80-461-21e18e00@197642200> <4b5c7353-39e1-4b40-8206-8ba7860f6cdb@itp.uni-frankfurt.de> Message-ID: Thank you very much for the help! The problem is solved now. Best regards, Fatemeh On Fri, Sep 5, 2025 at 8:21?AM Samuel Tootle wrote: > Hi Fatemeh, > > In addition to Konrad's correct notes, I want to addresd the build issues: > > "At first, I was getting a mismatch error (Assertion > ndim==nbr_points.get_ndim()), which suggests an incompatibility between > Spritz?s bundled Kadath and the external version of FUKA/Kadath I used to > generate the initial data. > > To fix this, I tried to build Spritz against my own external Kadath > installation (~/fuka/lib/libkadath.a) instead of the one bundled in the > Einstein Toolkit. I did this by disabling these thorns in my > thornlist: Fuka/kadath_pizza, Fuka/KadathImporter and Fuka/KadathThorn. I > also added the following lines to > my ET_2024_05/Cactus/simfactory/mdb/optionlists/.cfg > > KADATH_DIR = $(HOME)/fuka > > KADATH_INC_DIRS = $(KADATH_DIR)/include > > > KADATH_LIB_DIRS = $(KADATH_DIR)/lib" > > > > You are likely correct if there is a pre compiled version of Fuka with > spritz. If the data was generated with the latest release (since June 25), > there is a discrepancy with how the numerical domains are constructed and > stored as noted in the Readme. Linking against the latest library should > resolve the ndim error. > > Regarding your cfg, I believe you have to use brackets, not parentheses, > e.g. ${HOME_KADATH}, for the etk config parser to recognize environment > variables. > > All the best, > Samuel > Volunteer Physicist > > Sep 5, 2025 08:04:51 Konrad Topolski : > > Hi Fatemeh, > > The thorns you've disabled in the thornlist are responsible for the import > of the data in the ADMBase / Hydrobase variables and interact directly with > the FUKA exporters (in the bundled/separate FUKA installation). As such, > they are necessary to import FUKA ID and simultaneously independent of the > thorns you use for subsequent evolution. > > The kadathimport thorn provides the headers for the import functions for > specific initial data, whose implementation is then linked from > libkadath.a. > > I would recommend that you uncomment the thorns in the thornlist, > recompile (kadath library linking is still governed by the .cfg file, which > shouldn't change) and re-enable the thorns in the parameter file so that > they're active and the ETK can schedule the import routines in the > appropriate bins. > > I was actually unaware that bnsanalysis and related analysis thorns depend > on the kadath importer thorns. Is this a custom extension or is it possible > that some other thorn lists them as depencencies? > > In any case, doing the above should help. > > Best regards > > Konrad > > > > W dniu: Czwartek, Wrzesie? 04, 2025 17:02 CEST, Fatemeh Hossein Nouri < > f.h.noori at gmail.com> napisa?(a): > > > > Hi all, > > I?ve been trying to run Spritz using initial data generated by the latest > version of FUKA for a BNS simulation, but I?ve run into issues related to > the Kadath libraries. > > At first, I was getting a mismatch error (Assertion > ndim==nbr_points.get_ndim()), which suggests an incompatibility between > Spritz?s bundled Kadath and the external version of FUKA/Kadath I used to > generate the initial data. > > To fix this, I tried to build Spritz against my own external Kadath > installation (~/fuka/lib/libkadath.a) instead of the one bundled in the > Einstein Toolkit. I did this by disabling these thorns in my thornlist: Fuka/kadath_pizza, Fuka/KadathImporter > and Fuka/KadathThorn. I also added the following lines to > my ET_2024_05/Cactus/simfactory/mdb/optionlists/.cfg > > KADATH_DIR = $(HOME)/fuka > > KADATH_INC_DIRS = $(KADATH_DIR)/include > > > > KADATH_LIB_DIRS = $(KADATH_DIR)/lib > > > > > One question so far: Did I do things correctly? > > > > > Then I removed the old Kadath: > > > > > > > rm -rf configs/Spritz_LORENE/build/Kadath > > The code compiles fine, but when I try to run the executable with my > parameter file, using the '-S' option, I get: > > Error: Thorn kadathimporter not found Error: Thorn kadaththorn not found > Activation failed - 2 errors in activation sequence > > The confusing part is that my ActiveThorns line in the .par file does not > explicitly list KadathThorn or KadathImporter: > > ActiveThorns = "volomnia bnstrackergen bnsanalysis pizzanumutils" > > Yet Spritz fails because those thorns are missing. From what I understand, > some of the thorns I?m using (like volomnia, bnstrackergen, or bnsanalysis) > depend on KadathThorn and KadathImporter, so they are indirectly required. > > The core problem: > > - I need to keep KadathThorn and KadathImporter active, since my > parameter file requires them through dependencies. > > - But I also need them to link against my external Kadath ( > ~/fuka/lib/libkadath.a) so that the FUKA initial data is compatible. > > Would you have suggestions on the cleanest way to make the ET KadathThorn > and KadathImporter use my external Kadath library? Is there a recommended > workflow for this? > > Thanks in advance for your help! > > Bests, > > Fatemeh Nouri > > > > > > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From users at einsteintoolkit.org Mon Sep 8 15:18:02 2025 From: users at einsteintoolkit.org (users at einsteintoolkit.org) Date: Mon, 08 Sep 2025 15:18:02 -0500 Subject: [Users] Agenda for Thursday's Meeting Message-ID: <68bf39fa.eruawf/8R83RmQhg%users@einsteintoolkit.org> Please update the Wiki with agenda items for Thursday's meeting. Thanks! https://docs.einsteintoolkit.org/et-docs/meeting_agenda --The Maintainers From p.moesta at uva.nl Tue Sep 9 07:38:35 2025 From: p.moesta at uva.nl (Philipp Moesta) Date: Tue, 9 Sep 2025 12:38:35 +0000 Subject: [Users] KITP program in fall 2026 Message-ID: Dear everyone, We are currently organizing a KITP workshop which is scheduled to take place in the fall next year (Oct 26-Dec 18 2026): Enigmatic Explosions: Observations, Modelling, and Microphysics of Extreme Transients. The workshop will gather researchers interested in time-domain astronomy, nuclear and neutrino physics, as well as computational and theoretical modeling of mergers, supernovae, and other extreme astrophysical events. We would like to encourage interested members of the ETK community to apply to the program. The primary application deadline is Oct 24th 2025, though KITP will continue to accept rolling applications afterwards. You can find the application form here: https://www.kitp.ucsb.edu/activities/explode26 KITP generally prioritizes longer stays and we have ~27/30 spots available at any given time in the program for participants staying for 3 weeks or more, with a few participants staying for a shorter period of time. Most participants can stay at the Munger residence near campus, and KITP can provide support to participants traveling with children and dependent adults. More information on housing and family support is available here: https://www.kitp.ucsb.edu/visitors/before-your-visit If you have any questions about the program and/or application process, do not hesitate to let us know -- we hope to see many of you next year in Santa Barbara! Best, Philipp for the organizers of the workshop: Francois Foucart, Raffaella Margutti, Philipp Moesta, and Rebecca Surman Sent from Outlook for Mac -------------- next part -------------- An HTML attachment was scrubbed... URL: From PANAGIOTIS.IOSIF at units.it Wed Sep 10 10:22:06 2025 From: PANAGIOTIS.IOSIF at units.it (IOSIF PANAGIOTIS) Date: Wed, 10 Sep 2025 15:22:06 +0000 Subject: [Users] Inconsistency warnings: cores/threads mismatch [Leonardo cluster] Message-ID: Hi all, After having installed the toolkit on Leonardo, CINECA, I wanted to try the TOV examples. However, I am getting the following warnings: Warning: Too many threads per process specified: specified num-threads=56 (ppn-used is 112) Warning: Total number of threads and number of threads per process are inconsistent: procs=16, num-threads=56 (procs*num-smt must be an integer multiple of num-threads) Warning: Total number of threads and number of cores per node are inconsistent: procs=16, ppn-used=112 (procs must be an integer multiple of ppn-used) >From those, it seems that I am doing something wrong either in my machine.ini file or in the resources request when submitting the job. Here is the command I use to run the TOV example: ./simfactory/bin/sim submit tov_ET --parfile par/tov_ET.par --cores 16 --walltime 00:20:00 And here is the ini file I am using (the "Cluster characteristics" section is identical to Bruno Giacomazzo's file: https://github.com/piosif/einstein-toolkit-config-leonardo/blob/main/leonardo-dcgp1.ini The job ran and completed fine, but from what I understand, it ran on 56 cores instead of the 16 that I requested. >From what I understand, the problem is that I am requesting just 16 cores for this simple test, but due to the cluster specs in the ini file, SimFactory expects me to request a multiple of 56. Is that so? So, my question is the following: * I want to do some tests, running a single star e.g. on a single node using 16, 32, 64 cores (to understand the timescale required). Do I need to change my ini file? * Or is there some option to specify when submitting the job that will override the cluster specs? I read the simfactory's terminology explanation on nodes, threads etc, but it is not clear to me how to request the cores I need. Best, Panayotis -------------- next part -------------- An HTML attachment was scrubbed... URL: From rhaas at mail.ubc.ca Wed Sep 10 10:59:50 2025 From: rhaas at mail.ubc.ca (Roland Haas) Date: Wed, 10 Sep 2025 08:59:50 -0700 Subject: [Users] Inconsistency warnings: cores/threads mismatch [Leonardo cluster] In-Reply-To: References: Message-ID: <20250910085950.706e31d2@haengie2.phas.ubc.ca> Hello Panayotis, "procs" as used by simfactory is (for historical reasons) a misleading term (https://simfactory.bitbucket.io/simfactory2/userguide/processterminology.html). Really "procs" is the total number of threads that will be started, which usually will be the same as total number of (logical) cpus being used, and again usually the same as the number of (phyisal) cores used. If using --cores 16 you are asking for a total of 16 threads to be created which should be assigned to 16 cores. You should then select a number of threads-per-rank using the `--num-threads` option that is good for your setup. For 16 cores most likely `--num-threads 1` (ie one thread per MPI rank) is best. This will most likely fix your issue. Note that this will leave some cores unused and a number of simfactory machine description files will not handle that case gracefully since they assume that you will always use at least one full node. For you questions: * this is usually more of an issue with the SubmitScript and RunScript rather than the ini file and the simfactory command line, though in your case the init script has `num-threads = 56` which sets the default number of for `--num-threads` to 56, which is a bit high (but is overruled by a --num-threads option on your command line) * yes, specifying --num-threads on the command line will override the value in the ini file Yours, Roland > [CAUTION: Non-UBC Email] > > Hi all, > > After having installed the toolkit on Leonardo, CINECA, I wanted to try the TOV examples. > > However, I am getting the following warnings: > Warning: Too many threads per process specified: specified num-threads=56 (ppn-used is 112) > Warning: Total number of threads and number of threads per process are inconsistent: procs=16, num-threads=56 (procs*num-smt must be an integer multiple of num-threads) > Warning: Total number of threads and number of cores per node are inconsistent: procs=16, ppn-used=112 (procs must be an integer multiple of ppn-used) > From those, it seems that I am doing something wrong either in my machine.ini file or in the resources request when submitting the job. > > Here is the command I use to run the TOV example: > ./simfactory/bin/sim submit tov_ET --parfile par/tov_ET.par --cores 16 --walltime 00:20:00 > > And here is the ini file I am using (the "Cluster characteristics" section is identical to Bruno Giacomazzo's file: > > https://github.com/piosif/einstein-toolkit-config-leonardo/blob/main/leonardo-dcgp1.ini > > The job ran and completed fine, but from what I understand, it ran on 56 cores instead of the 16 that I requested. > > From what I understand, the problem is that I am requesting just 16 cores for this simple test, but due to the cluster specs in the ini file, SimFactory expects me to request a multiple of 56. > Is that so? > > So, my question is the following: > > * > I want to do some tests, running a single star e.g. on a single node using 16, 32, 64 cores (to understand the timescale required). Do I need to change my ini file? > * > Or is there some option to specify when submitting the job that will override the cluster specs? > > I read the simfactory's terminology explanation on nodes, threads etc, but it is not clear to me how to request the cores I need. > > Best, > Panayotis -- My email is as private as my paper mail. I therefore support encrypting and signing email messages. Get my PGP key from http://pgp.mit.edu . From users at einsteintoolkit.org Wed Sep 10 17:15:02 2025 From: users at einsteintoolkit.org (users at einsteintoolkit.org) Date: Wed, 10 Sep 2025 17:15:02 -0500 Subject: [Users] Einstein Toolkit Meeting Reminder Message-ID: <68c1f866.urzrghbhf+2v3EDq%users@einsteintoolkit.org> Hello, Please consider joining the weekly Einstein Toolkit phone call at 9:00 am US central time on Thursdays. For details on how to connect and what agenda items are to be discussed, use the link below. https://docs.einsteintoolkit.org/et-docs/Main_Page#Weekly_Users_Call --The Maintainers From cranganore at ml.jku.at Wed Sep 10 12:25:49 2025 From: cranganore at ml.jku.at (Sandeep Suresh CrangaNORE) Date: Wed, 10 Sep 2025 19:25:49 +0200 Subject: [Users] Issues Running BNS and GW150914 Example Simulations + storing BSSN variables Message-ID: <2dd52b48-05bb-4830-a119-ea83945babcb@ml.jku.at> Dear EinsteinToolkit team I am currently running the Binary Neutron Star (BNS) --?(https://einsteintoolkit.org/gallery/bns/index.html)?and GW150914?(https://einsteintoolkit.org/gallery/bbh/index.html)??binary black hole example simulations on our workstation environment: * *Processors:* AMD EPYC 7542, 32 cores per socket, 2 threads per core (64 logical cores) * *Memory:* 2?TiB RAM * *Kernel:* Linux 5.14.0-570.37.1.el9_6.x86_64 * *OPENMPI version:* mpirun (Open MPI) 4.1.1 Both simulations use the standard example parameter files and initial data provided in the toolkit. I shall describe the problems while running these simulations as two different sections: *BNS: *On running the command ./simfactory/bin/sim create-submit bns --parfile bns.par --procs= --num-threads= --walltime=xx:xx:xx I encounter the error: WARNING level 1 from host panther process 0 ? in thorn ML_BSSN_Helper, file /system/user/crangano/einstein_toolkit/BNS/configs/sim/build/ML_BSSN_Helper/Parameters.c:166: ? -> Parameter ML_BSSN::my_boundary_condition is outdated; please update the parameter file. Do not use this parameter, and set up RHS boundary conditions as usual. cactus_sim: grille3d.C:125: Grille3d::Grille3d(int, int, int, int, int, int, int): Assertion `nr > 0' failed. Rank 0 with PID 3273101 received signal 6 For this I have attached the?par file,?err file, the out file (the initial data file is the one that is already there online: G2_I12vs12_D4R33T21_45km.resu.xz ) that I am currently using. I tried to find the `nr` in the par file, but didn't find it. I am not sure if this is a MPI/num procs related problem, or is it something in the .par file that one needs to be change ? *GW150914: */Error 1 -- /Here, it is much more complicated to understand for me.?This persists even after disabling |CoordinatesSymmetry::reflection_z|.?Is it likely caused by coarse grid resolution relative to the number of symmetry/ghost zones. /Error 2/ -- Another error that I often encounter is File "/system/user/crangano/simulations/GW150914_28/output-0000/GW150914.rpar", line 126, in sphere_outer_radius = int((outermost_detector + final_time)/(i*hr))*i*hr ZeroDivisionError: float division by zero Error: Error while executing parameter file script /system/user/crangano/simulations/GW150914_28/output-0000/GW150914.rpar Aborting Simfactory. -- I guess this occurs in the following line of the *GW150914.rpar* sphere_outer_radius=int((outermost_detector+final_time)/(i*hr))*i*hr sphere_outer_radius=int(sphere_outer_radius/hr) *hr+hr# round up to a multiple of hr So, I changed line 78 of the *GW150914.rpar as follows: * # Number of cells across finest grid radius n=int("@N@") if"@N@"[0] !="@"else28 i=max(int(n/4), 1) Is this a valid fix, or does this affect the simulations ? For this too, I attach the err, out and the par file such that you can inspect it. *Storing BSSN variables*:We would like to store the BSSN evolved *3-metric* (|?_ij|), *lapse* (|?|), and *shift* (|?^i|) and also the corresponding coordinates (t, x^i) of these quantities? at regular intervals (e.g., every 128 steps) to manage disk usage efficiently. Moreover, since AMR is used, is there any way we can keep track of the changes in resolution of the coordinates, since we also aim to do spatial derivatives via our FD stencils -- Jacobians and Hessians of these quantities. If there are ways to directly dump the Jacobians and Hessians of these quantities (by adding lines on the par file), w/o us implementing (since we are not aware of the AMR being used at different regions to implement our FD stencils), that would be very useful too. IOHDF5::out_vars = " ML_BSSN::ML_metric ML_BSSN::ML_lapse ML_BSSN::ML_shift Grid::Coordinates{out_every=1000000000 refinement_levels={0}} ML_BSSN::ML_log_confac WeylScal4::Psi4r WeylScal4::Psi4i WeylScal4::curvIr{refinement_levels={3 5}} WeylScal4::curvIi{refinement_levels={3 5}} WeylScal4::curvJr{refinement_levels={3 5}} WeylScal4::curvJi{refinement_levels={3 5}} " I would appreciate guidance on the proper way to quickly resolve these errors. Ideally, we don't need something very specific -- Just want to get these simulations running with minimum tweaking, given that these are tested gallery example and more importantly store the time-evolved metric, lapse and the shift for both these usecases. Thanks a lot in advance for your help! If you need anything do let us know. Best regards, Sandeep -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- # Carpet parameter file for binary Neutron star system # physical ID is LORENE dataset G2_I12vs12_D4R33T21_45km.resu # #------------------------------------------------------------------------------ # Cactus parameters: #------------------------------------------------------------------------------ Cactus::cctk_run_title = "Meudon BNS" Cactus::cctk_full_warnings = "yes" Cactus::highlight_warning_messages = "no" Cactus::terminate = "time" Cactus::cctk_final_time = 2500.0 #------------------------------------------------------------------------------ # Activate all necessary thorns: #------------------------------------------------------------------------------ ActiveThorns = "Boundary CartGrid3D CoordBase Fortran InitBase IOUtil LocalReduce SymBase Time" ActiveThorns = "AEILocalInterp" ActiveThorns = "MoL Slab SpaceMask SphericalSurface" ActiveThorns = "Carpet CarpetInterp CarpetInterp2 CarpetIOASCII CarpetIOHDF5 CarpetIOScalar CarpetLib CarpetIOBasic CarpetReduce CarpetRegrid2 CarpetSlab CarpetTracker CarpetMask LoopControl" ActiveThorns = "Formaline" ActiveThorns = "HTTPD Socket" ActiveThorns = "NaNChecker TerminationTrigger TimerReport" ActiveThorns = "ADMbase ADMcoupling ADMmacros CoordGauge StaticConformal" ActiveThorns = "RotatingSymmetry180 ReflectionSymmetry" ActiveThorns = "Constants TmunuBase HydroBase " ActiveThorns = "QuasiLocalMeasures" ActiveThorns = "EOS_Omni" ActiveThorns = "GRHydro" ActiveThorns = "SummationByParts" ActiveThorns = "GenericFD NewRad" ActiveThorns = "ML_BSSN ML_BSSN_Helper ML_ADMConstraints" ActiveThorns = "Hydro_Analysis NSTracker" ActiveThorns = "Dissipation" ActiveThorns = "SystemStatistics SystemTopology" # Wave extraction (Psi4) ActiveThorns = "WeylScal4 Multipole" #------------------------------------------------------------------------------ # Diagnostic parameters: #------------------------------------------------------------------------------ Carpet::output_timers_every = 0 Carpet::storage_verbose = "no" Carpet::verbose = "no" Carpet::veryverbose = "no" Carpet::grid_structure_filename = "carpet-grid-structure" Carpet::grid_coordinates_filename = "carpet-grid-coordinates" CarpetLib::output_bboxes = "no" CarpetMask::verbose = "no" CarpetReduce::verbose = "no" CarpetRegrid2::verbose = "no" CarpetRegrid2::veryverbose = "no" CarpetTracker::verbose = "no" TimerReport::out_every = 4096 TimerReport::out_filename = "TimerReport" TimerReport::output_all_timers = "yes" TimerReport::output_all_timers_together = "yes" TimerReport::output_all_timers_readable = "yes" TimerReport::n_top_timers = 40 QuasiLocalMeasures::verbose = "no" SphericalSurface::verbose = "no" #------------------------------------------------------------------------------ # Utility parameters: #------------------------------------------------------------------------------ NaNChecker::check_every = 128 # twice for every_coarse NaNChecker::check_vars = " ADMBase::curv ADMBase::metric ADMBase::lapse ADMBase::shift HydroBase::rho HydroBase::eps HydroBase::press HydroBase::vel " NaNChecker::action_if_found = "terminate" #NaNChecker::action_if_found = "just warn" #"terminate", "just warn", "abort" #------------------------------------------------------------------------------ # Run parameters: #------------------------------------------------------------------------------ #------ # Grid: #------ MoL::ODE_Method = "rk4" MoL::MoL_Intermediate_Steps = 4 MoL::MoL_Num_Scratch_Levels = 1 # use dt = 0.4 dx (works for core collapse) Time::dtfac = 0.4 ActiveThorns = "CoordBase" CoordBase::domainsize = "minmax" CoordBase::xmin = 0.00 CoordBase::ymin = 0.00 CoordBase::zmin = 0.00 CoordBase::xmax = +400.00 CoordBase::ymax = +400.00 CoordBase::zmax = +400.00 CoordBase::dx = 16.00 CoordBase::dy = 16.00 CoordBase::dz = 16.00 CoordBase::boundary_size_x_lower = 3 CoordBase::boundary_size_y_lower = 3 CoordBase::boundary_size_z_lower = 3 CoordBase::boundary_size_x_upper = 3 CoordBase::boundary_size_y_upper = 3 CoordBase::boundary_size_z_upper = 3 CoordBase::boundary_shiftout_x_lower = 1 CoordBase::boundary_shiftout_y_lower = 0 CoordBase::boundary_shiftout_z_lower = 1 ReflectionSymmetry::reflection_z = "yes" ReflectionSymmetry::avoid_origin_z = "no" CartGrid3D::type = "coordbase" Carpet::domain_from_coordbase = "yes" Driver::ghost_size = 3 # General Carpet parameters: Carpet::enable_all_storage = "no" Carpet::use_buffer_zones = "yes" Carpet::schedule_barriers = "no" Carpet::poison_new_timelevels = "yes" Carpet::check_for_poison = "no" Carpet::init_3_timelevels = "no" Carpet::init_fill_timelevels = "yes" CarpetLib::poison_new_memory = "yes" CarpetLib::poison_value = 114 CarpetLib::check_bboxes = "no" CarpetLib::interleave_communications = "yes" CarpetLib::combine_sends = "yes" CarpetInterp::tree_search = "yes" CarpetInterp::check_tree_search = "no" CarpetRegrid2::freeze_unaligned_levels = "yes" CarpetRegrid2::freeze_unaligned_parent_levels = "yes" CarpetRegrid2::ensure_proper_nesting = "yes" CarpetRegrid2::snap_to_coarse = "yes" CarpetRegrid2::symmetry_rotating180 = "yes" # System specific Carpet parameters: Carpet::max_refinement_levels = 9 Carpet::prolongation_order_space = 5 Carpet::prolongation_order_time = 2 Carpet::refinement_centering = "vertex" CarpetRegrid2::regrid_every = 2 # as often as required CarpetRegrid2::num_centres = 3 # box sizes are given by: # * the stars seem to puff up to about 13M during the initial phase of the evolution # * I need 12 buffer points (RK4, 3 ghost zones) # * need three coarse points for interpolation onto last fine buffer point # these boxes are minimal in this sense. The coarser grid are completely # covered by the finer grids and their buffers. # add 4 coarse grid points in between to have some leeway against roundoff # grid step sizes are for coarsest anticipated simulation dx = 1.5M CarpetRegrid2::num_levels_1 = 7 CarpetRegrid2::position_x_1 = 15.1875 CarpetRegrid2::radius_1[1] =240.0 CarpetRegrid2::radius_1[2] =120.0 CarpetRegrid2::radius_1[3] = 60.0 CarpetRegrid2::radius_1[4] = 26.125 CarpetRegrid2::radius_1[5] = 17.875 CarpetRegrid2::radius_1[6] = 13 # star seems about 9.5 intially, then puffs up CarpetRegrid2::num_levels_2 = 7 CarpetRegrid2::position_x_2 = -15.1875 CarpetRegrid2::radius_2[1] =240.0 CarpetRegrid2::radius_2[2] =120.0 CarpetRegrid2::radius_2[3] = 60.0 CarpetRegrid2::radius_2[4] = 26.125 CarpetRegrid2::radius_2[5] = 17.875 CarpetRegrid2::radius_2[6] = 13 CarpetRegrid2::num_levels_3 = 1 CarpetRegrid2::radius_3[1] =240.0 CarpetRegrid2::radius_3[2] =120.0 CarpetRegrid2::radius_3[3] = 60.0 CarpetRegrid2::radius_3[4] = 30.0 CarpetRegrid2::radius_3[5] = 15.0 CarpetRegrid2::radius_3[6] = 7.5 CarpetRegrid2::radius_3[7] = 3.75 CarpetMask::excluded_surface [0] = 2 CarpetMask::excluded_surface_factor[0] = 1.0 CarpetTracker::surface_name[0] = "Righthand NS" CarpetTracker::surface_name[1] = "Lefthand NS" #------ # MODEL: #------ ActiveThorns = "Meudon_Bin_NS" HydroBase::initial_hydro = "Meudon_Bin_NS" ADMBase::initial_data = "Meudon_Bin_NS" ADMBase::initial_lapse = "Meudon_Bin_NS" ADMBase::initial_shift = "zero" ADMBase::initial_dtlapse = "Meudon_Bin_NS" ADMBase::initial_dtshift = "zero" # change this to be the full path to he initial data file Meudon_Bin_NS::filename ="/system/user/crangano/einstein_toolkit/BNS/G2_I12vs12_D4R33T21_45km.resu" # M_ADM 3.251 # separation 45km # K 123.6 # Gamma 2 EOS_Omni::poly_K = 123.613314525753 # For other (non-Polytropic) EOSs Meudon_Bin_NS::eos_table should be labelled # e.g. ="Hybrid" for hybrid EOS. #---------- # Numerics: #---------- InitBase::initial_data_setup_method = "init_some_levels" TmunuBase::stress_energy_storage = "yes" TmunuBase::stress_energy_at_RHS = "yes" TmunuBase::timelevels = 1 TmunuBase::prolongation_type = "none" TmunuBase::support_old_CalcTmunu_mechanism = "no" HydroBase::timelevels = 3 SpaceMask::use_mask = "yes" SphericalSurface::nsurfaces = 5 SphericalSurface::maxntheta = 39 SphericalSurface::maxnphi = 76 SphericalSurface::ntheta [0] = 39 SphericalSurface::nphi [0] = 76 SphericalSurface::nghoststheta[0] = 2 SphericalSurface::nghostsphi [0] = 2 SphericalSurface::name [0] = "Righthand NS" SphericalSurface::ntheta [1] = 39 SphericalSurface::nphi [1] = 76 SphericalSurface::nghoststheta[1] = 2 SphericalSurface::nghostsphi [1] = 2 SphericalSurface::name [1] = "Lefthand NS" SphericalSurface::ntheta [3] = 39 SphericalSurface::nphi [3] = 76 SphericalSurface::nghoststheta[3] = 2 SphericalSurface::nghostsphi [3] = 2 SphericalSurface::set_spherical[3] = yes SphericalSurface::radius [3] = 100 SphericalSurface::name [3] = "waveextract surface at 100" SphericalSurface::ntheta [4] = 39 SphericalSurface::nphi [4] = 76 SphericalSurface::nghoststheta[4] = 2 SphericalSurface::nghostsphi [4] = 2 SphericalSurface::set_spherical[4] = yes SphericalSurface::radius [4] = 250 SphericalSurface::name [4] = "waveextract surface at 250" #----------- # Evolution: #----------- HydroBase::evolution_method = "GRHydro" ADMMacros::spatial_order = 4 GRHydro::sources_spatial_order = 4 GRHydro::riemann_solver = "HLLE" # Marquina is currently not supported by MP GRHydro::recon_method = "ppm" GRHydro::GRHydro_stencil = 3 GRHydro::bound = "flat" GRHydro::rho_abs_min = 1.e-11 GRHydro::GRHydro_atmo_tolerance = 0.01 GRHydro::c2p_reset_pressure = "yes" GRHydro::GRHydro_eos_type = "General" GRHydro::GRHydro_eos_table = "Ideal_Fluid" # these can save some memory since they prevent MoL from allocating unnecessary # scratch space for saveandrestore variables GRHydro::GRHydro_MaxNumSandRVars = 0 GRHydro::use_enhanced_ppm = "yes" # Parameters are defaults, which in turn are from Colella & Sekora 2008 and # McCorquodale & Colella 2011 GRHydro::sync_conserved_only = "yes" GRHydro::reconstruct_Wv = "yes" GRHydro::c2p_resort_to_bisection = "yes" GRHydro::use_cxx_code = "yes" # MacLachlan evolution parameters ADMBase::metric_type = physical ADMBase::evolution_method = ML_BSSN ADMBase::lapse_evolution_method = ML_BSSN ADMBase::shift_evolution_method = ML_BSSN ADMBase::dtlapse_evolution_method = ML_BSSN ADMBase::dtshift_evolution_method = ML_BSSN ML_BSSN::timelevels = 3 ML_BSSN::my_initial_data = "ADMBase" ML_BSSN::my_initial_boundary_condition = "extrapolate-gammas" ML_BSSN::my_boundary_condition = "none" ML_BSSN::my_rhs_boundary_condition = "NewRad" Boundary::radpower = 2 # not really needed I think but who knows what NewRad uses ML_BSSN::harmonicN = 1 # 1+log ML_BSSN::harmonicF = 2.0 # 1+log ML_BSSN::ShiftGammaCoeff = 0.75 ML_BSSN::AlphaDriver = 0.0 ML_BSSN::BetaDriver = 1.0 ML_BSSN::advectLapse = 1.0 ML_BSSN::advectShift = 1.0 ML_BSSN::MinimumLapse = 1.0e-8 ML_BSSN::ML_log_confac_bound = "none" ML_BSSN::ML_metric_bound = "none" ML_BSSN::ML_Gamma_bound = "none" ML_BSSN::ML_trace_curv_bound = "none" ML_BSSN::ML_curv_bound = "none" ML_BSSN::ML_lapse_bound = "none" ML_BSSN::ML_dtlapse_bound = "none" ML_BSSN::ML_shift_bound = "none" ML_BSSN::ML_dtshift_bound = "none" ML_BSSN::UseSpatialBetaDriver = 1 ML_BSSN::SpatialBetaDriverRadius = 50 ML_BSSN::apply_dissipation = "never" Dissipation::epsdis = 0.1 Dissipation::order = 5 Dissipation::vars = " ML_BSSN::ML_log_confac ML_BSSN::ML_metric ML_BSSN::ML_trace_curv ML_BSSN::ML_curv ML_BSSN::ML_Gamma ML_BSSN::ML_lapse ML_BSSN::ML_shift ML_BSSN::ML_dtlapse ML_BSSN::ML_dtshift " #------------------------------------------------------------------------------ # Output: #------------------------------------------------------------------------------ IO::out_dir = $parfile IOBasic::outInfo_every = 1 IOBasic::outInfo_reductions = "maximum" IOBasic::outInfo_vars = " Carpet::physical_time_per_hour HydroBase::rho ML_ADMConstraints::ML_Ham SystemStatistics::maxrss_mb GRHydro::dens{reductions = 'sum maximum'} HydroBase::w_lorentz " IOScalar::outScalar_every = 256 # every_coarse IOScalar::all_reductions_in_one_file = "yes" IOScalar::one_file_per_group = "yes" IOScalar::outScalar_reductions = "minimum maximum average norm1 norm2" IOScalar::outScalar_vars = " ADMBase::lapse ADMBase::shift ADMBase::metric ADMBase::curv HydroBase::rho HydroBase::vel HydroBase::w_lorentz GRHydro::dens{reductions = 'minimum maximum average norm1 norm2 sum'} SystemStatistics::process_memory_mb SphericalSurface::sf_radius ML_ADMConstraints::ML_Ham " IOASCII::one_file_per_group = "yes" IOASCII::compact_format = "yes" IOASCII::out0D_every = 256 # every_coarse IOASCII::out0D_vars = " Carpet::timing QuasiLocalMeasures::qlm_scalars SphericalSurface::sf_active SphericalSurface::sf_valid SphericalSurface::sf_info SphericalSurface::sf_radius SphericalSurface::sf_origin SphericalSurface::sf_coordinate_descriptors Hydro_Analysis::Hydro_Analysis_rho_max_loc Hydro_Analysis::Hydro_Analysis_rho_max_origin_distance " #Set these IOASCII options for initial data only: IOASCII::out1D_every = 0 IOASCII::out1D_d = "no" IOASCII::out1D_vars = " HydroBase::rho HydroBase::vel ADMBase::lapse ADMBase::shift ADMBase::metric ADMBase::curv ML_ADMConstraints::ML_Ham " CarpetIOHDF5::one_file_per_group = "no" # this is required by multipatch CarpetIOHDF5::open_one_input_file_at_a_time = "yes" CarpetIOHDF5::out2D_every = 1536 # 6*every coarse CarpetIOHDF5::out2D_xy = "yes" CarpetIOHDF5::out2D_xz = "no" CarpetIOHDF5::out2D_yz = "no" CarpetIOHDF5::out2D_xyplane_z = 0.0 CarpetIOHDF5::out2D_vars = " CarpetReduce::weight Grid::coordinates HydroBase::rho HydroBase::vel HydroBase::eps ADMBase::lapse ADMBase::shift ADMBase::metric ML_ADMConstraints::ML_Ham # Hydro_Analysis::Hydro_Analysis_Temperature " IOHDF5::out3D_every = 8192 # = 32*every_coarse IOHDF5::out3D_vars = " CarpetReduce::weight HydroBase::rho HydroBase::vel HydroBase::eps ADMBase::lapse ADMBase::shift ML_ADMConstraints::ML_Ham grid::coordinates " #------------------------------------------------------------------------------ # Analysis: #------------------------------------------------------------------------------ Hydro_Analysis::Hydro_Analysis_comp_rho_max = "true" Hydro_Analysis::Hydro_Analysis_rho_max_loc_only_positive_x = "true" Hydro_Analysis::Hydro_Analysis_comp_rho_max_origin_distance = "yes" Hydro_Analysis::Hydro_Analysis_average_multiple_maxima_locations = "yes" Hydro_Analysis::Hydro_Analysis_interpolator_name = "Lagrange polynomial interpolation (tensor product)" NSTracker::NSTracker_SF_Name = "Righthand NS" NSTracker::NSTracker_SF_Name_Opposite = "Lefthand NS" NSTracker::NSTracker_max_distance = 3 NSTracker::NSTracker_verbose = "yes" NSTracker::NSTracker_tracked_location = "Hydro_Analysis::Hydro_Analysis_rho_max_loc" QuasiLocalMeasures::num_surfaces = 2 QuasiLocalMeasures::spatial_order = 4 QuasiLocalMeasures::interpolator = "Lagrange polynomial interpolation" QuasiLocalMeasures::interpolator_options = "order=4" QuasiLocalMeasures::surface_name[0] = "waveextract surface at 100" QuasiLocalMeasures::surface_name[1] = "waveextract surface at 250" ################################################################################ ################################################################################ # Wave extraction ################################################################################ ################################################################################ WeylScal4::offset = 1e-8 WeylScal4::fd_order = "4th" WeylScal4::verbose = 0 Multipole::nradii = 8 Multipole::out_every = 128 Multipole::radius[0] = 45 Multipole::radius[1] = 70 Multipole::radius[2] = 100 Multipole::radius[3] = 125 Multipole::radius[4] = 150 Multipole::radius[5] = 200 Multipole::radius[6] = 250 Multipole::radius[7] = 300 Multipole::variables = "WeylScal4::Psi4r{sw=-2 cmplx='WeylScal4::Psi4i' name='Psi4'}" Multipole::l_max = 6 #------------------------------------------------------------------------------ # Checkpoint/Recovery: #------------------------------------------------------------------------------ IOHDF5::checkpoint = "yes" IO::checkpoint_dir = $parfile IO::checkpoint_ID = "yes" IO::checkpoint_on_terminate = "yes" # disable extra con2prim in Post_Recover_Variables to ensure bit-identical # recovery from checkpoints MoL::run_MoL_PostStep_in_Post_Recover_Variables = "no" IO::recover = "autoprobe" IO::recover_dir = $parfile #------------------------------------------------------------------------------ # Control #------------------------------------------------------------------------------ HTTPD::user = "cactus" HTTPD::password = "einstein" ActiveThorns = "TerminationTrigger" TerminationTrigger::max_walltime = @WALLTIME_HOURS@ # Trigger termination 30 minutes before the walltime is reached TerminationTrigger::on_remaining_walltime = 30 TerminationTrigger::termination_from_file = yes TerminationTrigger::termination_file = "terminate.txt" TerminationTrigger::create_termination_file = yes ActiveThorns = "Trigger" Trigger::Trigger_Number = 4 Trigger::Trigger_Checked_Variable[0]="Hydro_Analysis::Hydro_Analysis_rho_max_origin_distance" Trigger::Trigger_Reduction [0]="" Trigger::Trigger_Relation [0]="<" Trigger::Trigger_Checked_Value [0]=10 Trigger::Trigger_Reaction [0]="steerscalar" Trigger::Trigger_Steered_Scalar [0] = "CarpetRegrid2::num_levels[2]" # == num_levels_3 Trigger::Trigger_Steered_Scalar_Value[0] = "7" Trigger::Trigger_Checked_Variable[1]="ADMBase::alp" Trigger::Trigger_Reduction [1]="minimum" Trigger::Trigger_Relation [1]="<" Trigger::Trigger_Checked_Value [1]=0.1 Trigger::Trigger_Reaction [1]="steerscalar" Trigger::Trigger_Steered_Scalar [1] = "CarpetRegrid2::num_levels[2]" # == num_levels_3 Trigger::Trigger_Steered_Scalar_Value[1] = "8" Trigger::Trigger_Checked_Variable[2]="SphericalSurface::sf_valid[2]" Trigger::Trigger_Reduction [2]="" Trigger::Trigger_Relation [2]=">" Trigger::Trigger_Checked_Value [2]=0. Trigger::Trigger_Reaction [2]="steerscalar" Trigger::Trigger_Steered_Scalar [2] = "CarpetRegrid2::num_levels[0]" # == num_levels_1 Trigger::Trigger_Steered_Scalar_Value[2] = "1" Trigger::Trigger_Checked_Variable[3]="SphericalSurface::sf_valid[2]" Trigger::Trigger_Reduction [3]="" Trigger::Trigger_Relation [3]=">" Trigger::Trigger_Checked_Value [3]=0. Trigger::Trigger_Reaction [3]="steerscalar" Trigger::Trigger_Steered_Scalar [3] = "CarpetRegrid2::num_levels[1]" # == num_levels_2 Trigger::Trigger_Steered_Scalar_Value[3] = "1" -------------- next part -------------- + set -e + cd /system/user/crangano/simulations/bns_sim/output-0000-active + echo Checking: + pwd + hostname + date + echo Environment: + export CACTUS_NUM_PROCS=1 + CACTUS_NUM_PROCS=1 + export CACTUS_NUM_THREADS=32 + CACTUS_NUM_THREADS=32 + export GMON_OUT_PREFIX=gmon.out + GMON_OUT_PREFIX=gmon.out + export OMP_NUM_THREADS=32 + OMP_NUM_THREADS=32 + env + sort + echo Starting: ++ date +%s + export CACTUS_STARTTIME=1757517541 + CACTUS_STARTTIME=1757517541 + '[' 1 = 1 ']' + '[' 0 -eq 0 ']' + /system/user/crangano/simulations/bns_sim/SIMFACTORY/exe/cactus_sim -L 3 /system/user/crangano/simulations/bns_sim/output-0000/bns.par WARNING level 1 from host panther process 0 in thorn ML_BSSN_Helper, file /system/user/crangano/einstein_toolkit/BNS/configs/sim/build/ML_BSSN_Helper/Parameters.c:10: -> Forcing ML_BSSN::initial_boundary_condition="extrapolate-gammas" because ML_BSSN::my_initial_boundary_condition="extrapolate-gammas" WARNING level 1 from host panther process 0 in thorn ML_BSSN_Helper, file /system/user/crangano/einstein_toolkit/BNS/configs/sim/build/ML_BSSN_Helper/Parameters.c:25: -> Forcing ML_BSSN::rhs_boundary_condition="NewRad" because ML_BSSN::my_rhs_boundary_condition="NewRad" WARNING level 1 from host panther process 0 in thorn ML_BSSN_Helper, file /system/user/crangano/einstein_toolkit/BNS/configs/sim/build/ML_BSSN_Helper/Parameters.c:45: -> Forcing ML_BSSN::epsDiss=0.0 because ML_BSSN::apply_dissipation="never" WARNING level 1 from host panther process 0 in thorn ML_BSSN_Helper, file /system/user/crangano/einstein_toolkit/BNS/configs/sim/build/ML_BSSN_Helper/Parameters.c:145: -> Parameter ML_BSSN::my_initial_data is outdated; please update the parameter file. Do not use this parameter, and set up initial conditions via ADMBase as usual. WARNING level 1 from host panther process 0 in thorn ML_BSSN_Helper, file /system/user/crangano/einstein_toolkit/BNS/configs/sim/build/ML_BSSN_Helper/Parameters.c:152: -> Parameter ML_BSSN::my_initial_boundary_condition is outdated; please update the parameter file. Do not use this parameter, and set up initial boundary conditions as usual. WARNING level 1 from host panther process 0 in thorn ML_BSSN_Helper, file /system/user/crangano/einstein_toolkit/BNS/configs/sim/build/ML_BSSN_Helper/Parameters.c:159: -> Parameter ML_BSSN::my_rhs_boundary_condition is outdated; please update the parameter file. Do not use this parameter, and set up RHS boundary conditions as usual. WARNING level 1 from host panther process 0 in thorn ML_BSSN_Helper, file /system/user/crangano/einstein_toolkit/BNS/configs/sim/build/ML_BSSN_Helper/Parameters.c:166: -> Parameter ML_BSSN::my_boundary_condition is outdated; please update the parameter file. Do not use this parameter, and set up RHS boundary conditions as usual. cactus_sim: grille3d.C:125: Grille3d::Grille3d(int, int, int, int, int, int, int): Assertion `nr > 0' failed. Rank 0 with PID 3273101 received signal 6 Writing backtrace to bns/backtrace.0.txt /system/user/crangano/simulations/bns_sim/output-0000/SIMFACTORY/RunScript: line 36: 3273101 Aborted (core dumped) /system/user/crangano/simulations/bns_sim/SIMFACTORY/exe/cactus_sim -L 3 /system/user/crangano/simulations/bns_sim/output-0000/bns.par -------------- next part -------------- Running simulation bns_sim Preparing: Checking: /system/user/crangano/simulations/bns_sim/output-0000-active panther Wed Sep 10 05:19:01 PM CEST 2025 Environment: Starting: INFO (Cactus): Increased logging level from 0 to 3 -------------------------------------------------------------------------------- 10 1 0101 ************************ 01 1010 10 The Cactus Code V4.18.0 1010 1101 011 www.cactuscode.org 1001 100101 ************************ 00010101 100011 (c) Copyright The Authors 0100 GNU Licensed. No Warranty 0101 -------------------------------------------------------------------------------- Cactus version: 4.18.0 Compile date: Sep 09 2025 (16:36:24) Run date: Sep 10 2025 (17:19:02+0200) Run host: panther (pid=3273101) Working directory: /system/user/crangano/simulations/bns_sim/output-0000 Executable: /system/user/crangano/simulations/bns_sim/SIMFACTORY/exe/cactus_sim Parameter file: /system/user/crangano/simulations/bns_sim/output-0000/bns.par -------------------------------------------------------------------------------- Activating thorn Cactus...Success -> active implementation Cactus Activation requested for --->Boundary CartGrid3D CoordBase Fortran InitBase IOUtil LocalReduce SymBase Time AEILocalInterp MoL Slab SpaceMask SphericalSurface Carpet CarpetInterp CarpetInterp2 CarpetIOASCII CarpetIOHDF5 CarpetIOScalar CarpetLib CarpetIOBasic CarpetReduce CarpetRegrid2 CarpetSlab CarpetTracker CarpetMask LoopControl Formaline HTTPD Socket NaNChecker TerminationTrigger TimerReport ADMbase ADMcoupling ADMmacros CoordGauge StaticConformal RotatingSymmetry180 ReflectionSymmetry Constants TmunuBase HydroBase QuasiLocalMeasures EOS_Omni GRHydro SummationByParts GenericFD NewRad ML_BSSN ML_BSSN_Helper ML_ADMConstraints Hydro_Analysis NSTracker Dissipation SystemStatistics SystemTopology WeylScal4 Multipole CoordBase Meudon_Bin_NS TerminationTrigger Trigger<--- Warning: thorn CoordBase already scheduled for activation Warning: thorn TerminationTrigger already scheduled for activation Thorn Carpet requests automatic activation of MPI Thorn Carpet requests automatic activation of Timers Thorn CarpetIOHDF5 requests automatic activation of HDF5 Thorn CarpetLib requests automatic activation of Vectors Thorn CarpetLib requests automatic activation of CycleClock Thorn Formaline requests automatic activation of pthreads Thorn GRHydro requests automatic activation of EOS_Polytrope Thorn LoopControl requests automatic activation of hwloc Thorn Meudon_Bin_NS requests automatic activation of LORENE Thorn QuasiLocalMeasures requests automatic activation of LAPACK Thorn QuasiLocalMeasures requests automatic activation of TGRtensor Thorn EOS_Polytrope requests automatic activation of EOS_Base Thorn HDF5 requests automatic activation of zlib Thorn LAPACK requests automatic activation of BLAS Thorn LORENE requests automatic activation of GSL Activating thorn ADMbase...Success -> active implementation ADMBase Activating thorn ADMcoupling...Success -> active implementation ADMCoupling Activating thorn ADMmacros...Success -> active implementation ADMMacros Activating thorn AEILocalInterp...Success -> active implementation AEILocalInterp Activating thorn BLAS...Success -> active implementation BLAS Activating thorn Boundary...Success -> active implementation boundary Activating thorn Carpet...Success -> active implementation Driver Activating thorn CarpetInterp...Success -> active implementation interp Activating thorn CarpetInterp2...Success -> active implementation interp2 Activating thorn CarpetIOASCII...Success -> active implementation IOASCII Activating thorn CarpetIOBasic...Success -> active implementation IOBasic Activating thorn CarpetIOHDF5...Success -> active implementation IOHDF5 Activating thorn CarpetIOScalar...Success -> active implementation IOScalar Activating thorn CarpetLib...Success -> active implementation CarpetLib Activating thorn CarpetMask...Success -> active implementation CarpetMask Activating thorn CarpetReduce...Success -> active implementation reduce Activating thorn CarpetRegrid2...Success -> active implementation CarpetRegrid2 Activating thorn CarpetSlab...Success -> active implementation Hyperslab Activating thorn CarpetTracker...Success -> active implementation CarpetTracker Activating thorn CartGrid3D...Success -> active implementation grid Activating thorn Constants...Success -> active implementation Constants Activating thorn CoordBase...Success -> active implementation CoordBase Activating thorn CoordGauge...Success -> active implementation CoordGauge Activating thorn CycleClock...Success -> active implementation CycleClock Activating thorn Dissipation...Success -> active implementation Dissipation Activating thorn EOS_Base...Success -> active implementation EOS_Base Activating thorn EOS_Omni...Success -> active implementation EOS_Omni Activating thorn EOS_Polytrope...Success -> active implementation EOS_2d_Polytrope Activating thorn Formaline...Success -> active implementation Formaline Activating thorn Fortran...Success -> active implementation Fortran Activating thorn GenericFD...Success -> active implementation GenericFD Activating thorn GRHydro...Success -> active implementation GRHydro Activating thorn GSL...Success -> active implementation GSL Activating thorn HDF5...Success -> active implementation HDF5 Activating thorn HTTPD...Success -> active implementation HTTPD Activating thorn hwloc...Success -> active implementation hwloc Activating thorn Hydro_Analysis...Success -> active implementation Hydro_Analysis Activating thorn HydroBase...Success -> active implementation HydroBase Activating thorn InitBase...Success -> active implementation InitBase Activating thorn IOUtil...Success -> active implementation IO Activating thorn LAPACK...Success -> active implementation LAPACK Activating thorn LocalReduce...Success -> active implementation LocalReduce Activating thorn LoopControl...Success -> active implementation LoopControl Activating thorn LORENE...Success -> active implementation LORENE Activating thorn Meudon_Bin_NS...Success -> active implementation Meudon_Bin_NS Activating thorn ML_ADMConstraints...Success -> active implementation ML_ADMConstraints Activating thorn ML_BSSN...Success -> active implementation ML_BSSN Activating thorn ML_BSSN_Helper...Success -> active implementation ML_BSSN_Helper Activating thorn MoL...Success -> active implementation MethodOfLines Activating thorn MPI...Success -> active implementation MPI Activating thorn Multipole...Success -> active implementation multipole Activating thorn NaNChecker...Success -> active implementation NaNChecker Activating thorn NewRad...Success -> active implementation NewRad Activating thorn NSTracker...Success -> active implementation NSTracker Activating thorn pthreads...Success -> active implementation PTHREADS Activating thorn QuasiLocalMeasures...Success -> active implementation QuasiLocalMeasures Activating thorn ReflectionSymmetry...Success -> active implementation ReflectionSymmetry Activating thorn RotatingSymmetry180...Success -> active implementation RotatingSymmetry180 Activating thorn Slab...Success -> active implementation Slab Activating thorn Socket...Success -> active implementation Socket Activating thorn SpaceMask...Success -> active implementation SpaceMask Activating thorn SphericalSurface...Success -> active implementation SphericalSurface Activating thorn StaticConformal...Success -> active implementation StaticConformal Activating thorn SummationByParts...Success -> active implementation SummationByParts Activating thorn SymBase...Success -> active implementation SymBase Activating thorn SystemStatistics...Success -> active implementation SystemStatistics Activating thorn SystemTopology...Success -> active implementation SystemTopology Activating thorn TerminationTrigger...Success -> active implementation TerminationTrigger Activating thorn TGRtensor...Success -> active implementation TGRtensor Activating thorn Time...Success -> active implementation time Activating thorn TimerReport...Success -> active implementation timerreport Activating thorn Timers...Success -> active implementation Timers Activating thorn TmunuBase...Success -> active implementation TmunuBase Activating thorn Trigger...Success -> active implementation trigger Activating thorn Vectors...Success -> active implementation Vectors Activating thorn WeylScal4...Success -> active implementation WeylScal4 Activating thorn zlib...Success -> active implementation zlib WARNING level 3 from host panther process 0 in thorn IOUtil, file /system/user/crangano/einstein_toolkit/BNS/configs/sim/build/IOUtil/CheckpointRecovery.cc:748: -> No HDF5 checkpoint files with basefilename 'checkpoint.chkpt' and file extension '.h5' found in recovery directory 'bns' -------------------------------------------------------------------------------- if (recover initial data) Recover parameters endif Startup routines [CCTK_STARTUP] Carpet::MultiModel_Startup: Multi-model Startup routine CycleClock::CycleClock_Setup: Set up CycleClock LoopControl::LC_setup: Set up LoopControl ML_BSSN_Helper::ML_BSSN_SetGroupTags: [meta] Set checkpointing and prolongation group tags GROUP hwloc_startup: hwloc startup group hwloc::hwloc_version: Output hwloc version Timers::Timer_Startup: Prepare hierarchical timers SystemTopology::ST_system_topology: Output and/or modify system topology and hardware locality CarpetInterp::CarpetInterpStartup: Startup routine CarpetReduce::CarpetReduceStartup: Startup routine CartGrid3D::SymmetryStartup: Register GH Extension for GridSymmetry CoordBase::CoordBase_Startup: Register a GH extension to store the coordinate system handles Carpet::Driver_Startup: Startup routine EOS_Omni::EOS_Omni_Startup: [global] Set up conversion factors and other fun stuff EOS_Polytrope::EOS_Polytrope_Startup: Setup the polytropic EOS Formaline::Formaline_PrintIDs: [meta] Print the build and simulation ids GRHydro::GRHydro_Startup: Startup banner GRHydro::GRHydro_RegisterMask: Register the hydro masks GROUP HTTP_Startup: HTTP daemon startup group HTTPD::HTTP_StartServer: [global] Start HTTP server GROUP HTTP_SetupPages: Group to setup stuff which needs to be done between starting the server and the first time it serves pages HTTPD::HTTP_FirstServ: [global] Serve first pages at startup HydroBase::HydroBase_StartUp: Startup banner IOUtil::IOUtil_Startup: Startup routine LocalReduce::LocalReduce_Startup: Startup routine CarpetIOBasic::CarpetIOBasicStartup: [global] Startup routine ML_ADMConstraints::ML_ADMConstraints_Startup: [meta] create banner ML_BSSN::ML_BSSN_Startup: [meta] create banner ML_BSSN_Helper::ML_BSSN_RegisterSlicing: [meta] Register slicing CarpetIOHDF5::CarpetIOHDF5_Startup: Startup routine MoL::MoL_Startup: Startup banner Slab::Slab_InitMPIDatatypes: Create MPI datatypes for complex variables in C SymBase::SymBase_Startup: Register GH Extension for SymBase CarpetIOScalar::CarpetIOScalarStartup: [global] Startup routine TerminationTrigger::TerminationTrigger_StartSignalHandler: Start signal handler CarpetIOASCII::CarpetIOASCIIStartup: [global] Startup routine Trigger::Trigger_Startup: Startup Routine Vectors::Vectors_Startup: Print startup message WeylScal4::WeylScal4_Startup: [meta] create banner AEILocalInterp::AEILocalInterp_U_Startup: register CCTK_InterpLocalUniform() interpolation operators Startup routines which need an existing grid hierarchy [CCTK_WRAGH] ADMBase::Einstein_InitSymBound: [global] Set up GF symmetries Boundary::Boundary_RegisterBCs: [global] Register boundary conditions that this thorn provides CarpetRegrid2::CarpetRegrid2_Initialise: [global] Initialise locations of refined regions CartGrid3D::RegisterCartGrid3DCoords: [meta] Register coordinates for the Cartesian grid CoordGauge::Einstein_ActivateSlicing: Initialize slicing, setup priorities for mixed slicings CoordGauge::Einstein_SetNextSlicing: Identify the slicing for the next iteration Formaline::Formaline_OutputSource: [meta] Output Cactus source tree Formaline::Formaline_RegisterWarnings: [meta] Register to receive warnings and info messages from the flesh Formaline::Formaline_AnnounceInitial: [global] Put some meta information about the current run into permanent storage GRHydro::GRHydro_ClearLastMoLPostStep: [global] Initialize InLastMoLPostStep to zero ML_BSSN_Helper::ML_BSSN_ParamCompat: [meta] Handle parameter backward compatibility MoL::MoL_SetupIndexArrays: Set up the MoL bookkeeping index arrays MoL::MoL_SetScheduleStatus: [global] Set the flag so it is ok to register with MoL TmunuBase::TmunuBase_SetStressEnergyState: [global] Set the stress_energy_state variable GROUP MoL_Register: The group where physics thorns register variables with MoL GRHydro::GRHydro_Register: Register variables for MoL ML_ADMConstraints::ML_ADMConstraints_RegisterVars: [meta] Register Variables for MoL ML_BSSN::ML_BSSN_RegisterVars: [meta] Register Variables for MoL ML_BSSN_Helper::ML_BSSN_RegisterConstrained: [meta] Register ADMBase variables as constrained WeylScal4::WeylScal4_RegisterVars: [meta] Register Variables for MoL Slab::Slab_InitTimers: Initialise timers SpaceMask::MaskSym: [global] Set grid symmetries for mask SpaceMask::MaskSym_emask: [global] Set grid symmetries for emask (compatibility mode) GROUP SymBase_Wrapper: Wrapper group for SymBase GROUP SymmetryRegister: Register your symmetries here CartGrid3D::CartGrid3D_RegisterSymmetryBoundaries: [meta] Register symmetry boundaries ML_ADMConstraints::ML_ADMConstraints_RegisterSymmetries: [meta] register symmetries ML_BSSN::ML_BSSN_RegisterSymmetries: [meta] register symmetries ReflectionSymmetry::ReflectionSymmetry_Register: Register reflection symmetry boundaries RotatingSymmetry180::Rot180_RegisterSymmetry: Register symmetry boundaries WeylScal4::WeylScal4_RegisterSymmetries: [meta] register symmetries SymBase::SymBase_Statistics: Print symmetry boundary face descriptions MoL::MoL_ReportNumberVariables: [meta] Report how many of each type of variable there are Parameter checking routines [CCTK_PARAMCHECK] ADMBase::ADMBase_ParamCheck: [global] Check consistency of parameters Boundary::Boundary_Check: Check dimension of grid variables Carpet::CarpetParamCheck: Parameter checking routine CarpetLib::CarpetLib_test_prolongate_3d_rf2: [global] Test prolongation operators CarpetMask::CarpetSurfaceParamCheck: [global] Check parameters CarpetRegrid2::CarpetRegrid2_ParamCheck: Check parameters CartGrid3D::ParamCheck_CartGrid3D: Check coordinates for CartGrid3D Dissipation::dissipation_paramcheck: Check dissipation parameters for consistency Fortran::CheckFortranParameters: Test whether Fortran parameters work correctly GRHydro::GRHydro_ParamCheck: Check parameters ML_BSSN_Helper::ML_BSSN_ParamCheck: [meta] Check parameters MoL::MoL_ParamCheck: Basic parameter checking Multipole::Multipole_ParamCheck: [global] Check Multipole parameters QuasiLocalMeasures::qlm_paramcheck: [global] Check quasi-local parameter settings RotatingSymmetry180::Rot180_CheckTensorTypes: Check tensor type definitions for consistency SphericalSurface::SphericalSurface_ParamCheck: [global] Check that all surface names are unique TerminationTrigger::TerminationTrigger_ParamCheck: Check consitency of parameters TmunuBase::TmunuBase_ParamCheck: [global] Check that no deprecated parameters are used. Trigger::Trigger_ParamCheck: Parameter checking Vectors::Vectors_Test: Run correctness tests. Initialisation if (NOT (recover initial data AND recovery_mode is 'strict')) [CCTK_PREREGRIDINITIAL] Set up grid hierarchy [CCTK_POSTREGRIDINITIAL] CartGrid3D::SpatialCoordinates: Set Coordinates after regridding GROUP MaskBase_SetupMask: Set up the weight function GROUP MaskBase_SetupMaskAll: Set up the weight function CarpetReduce::MaskBase_AllocateMask: [global] Allocate the weight function CarpetReduce::MaskBase_InitMask: [global] [loop-local] Initialise the weight function GROUP SetupIMaskInternal: Set up the integer weight function (schedule other routines in here) CarpetReduce::CoordBase_SetupMask: [global] [loop-local] Set up the outer boundaries of the weight function CarpetReduce::CarpetMaskSetup: [global] [loop-singlemap] Set up the weight function for the restriction regions GROUP SetupIMask: Set up the integer weight function (schedule other routines in here) CarpetMask::CarpetExcludedSetup: [global] [loop-local] Set up the weight function for the excluded regions CarpetMask::CarpetSurfaceSetup: [global] [loop-local] Set up the weight function for the excluded spherical surfaces CarpetReduce::MaskBase_SetMask: [global] [loop-local] Set the weight function GROUP SetupMask: Set up the real weight function (schedule other routines in here) CarpetReduce::MaskBase_TestMask: [global] Test the weight function Dissipation::setup_epsdis: Setup spatially varying dissipation SpaceMask::MaskZero: Initialise mask to zero GRHydro::GRHydro_RefinementLevel: Calculate current refinement level GROUP GRHydro_PrimitiveBoundaries: Apply boundary conditions to all primitives GRHydro::GRHydro_SelectPrimitiveBoundaries: [level] Select primitive variables for boundary conditions GROUP GRHydro_ApplyPrimitiveBCs: Apply boundary conditions to all primitive variables GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP HydroBase_ExcisionMaskSetup: Set up hydro excision mask HydroBase::HydroBase_InitExcisionMask: Initialize hydro excision mask to 'no excision everywhere' GROUP MoL_PseudoEvolutionBoundaries: Apply boundary conditions to pseudo-evolved quantities GROUP ML_ADMConstraints_evaluate_bc_group: ML_ADMConstraints_evaluate ML_ADMConstraints::ML_ADMConstraints_evaluate_SelectBCs: [level] ML_ADMConstraints_evaluate_SelectBCs GROUP ML_ADMConstraints_evaluate_ApplyBCs: Apply BCs for groups set in ML_ADMConstraints_evaluate GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP ML_BSSN_ConstraintsEverywhere_bc_group: ML_BSSN_ConstraintsEverywhere ML_BSSN::ML_BSSN_ConstraintsEverywhere_SelectBCs: [level] ML_BSSN_ConstraintsEverywhere_SelectBCs GROUP ML_BSSN_ConstraintsEverywhere_ApplyBCs: Apply BCs for groups set in ML_BSSN_ConstraintsEverywhere GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP ML_BSSN_ConstraintsInterior_bc_group: ML_BSSN_ConstraintsInterior ML_BSSN::ML_BSSN_ConstraintsInterior_SelectBCs: [level] ML_BSSN_ConstraintsInterior_SelectBCs GROUP ML_BSSN_ConstraintsInterior_ApplyBCs: Apply BCs for groups set in ML_BSSN_ConstraintsInterior GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP WeylScal4_psi4_calc_4th_bc_group: WeylScal4_psi4_calc_4th WeylScal4::WeylScal4_psi4_calc_4th_SelectBCs: [level] WeylScal4_psi4_calc_4th_SelectBCs GROUP WeylScal4_psi4_calc_4th_ApplyBCs: Apply BCs for groups set in WeylScal4_psi4_calc_4th GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions SpaceMask::MaskOne: Set mask to one GRHydro::GRHydro_SetupMask: Initialize the atmosphere mask [CCTK_BASEGRID] ADMBase::ADMBase_SetShiftStateOn: Set the shift_state variable to 1 ADMBase::ADMBase_SetDtLapseStateOn: Set the dtlapse_state variable to 1 ADMBase::ADMBase_SetDtShiftStateOn: Set the dtshift_state variable to 1 ADMMacros::ADMMacros_SetLocalSpatialOrder: Initialize the local_spatial_order CartGrid3D::SpatialSpacings: Set up ranges for spatial 3D Cartesian coordinates (on all grids) CartGrid3D::SpatialCoordinates: Set up spatial 3D Cartesian coordinates on the GH SphericalSurface::SphericalSurface_SetupRes: [global] [loop-local] Set surface resolution automatically Dissipation::dissipation_basegrid: Ensure that there are enough ghost zones GRHydro::GRHydro_Reset_Execution_Flags: [global] Initially set execution flags to 'YEAH, Execute'! GRHydro::GRHydro_InitSymBound: Schedule symmetries and check shift state GRHydro::reset_GRHydro_C2P_failed: Initialise the mask function that contains the points where C2P has failed (at BASEGRID) Hydro_Analysis::Hydro_Analysis_Init: [global] Initialize variables ML_ADMConstraints::ML_ADMConstraints_CheckBoundaries: [meta] check boundaries treatment ML_BSSN::ML_BSSN_CheckBoundaries: [meta] check boundaries treatment NaNChecker::NaNChecker_ResetCounter: [global] Reset the NaNChecker::NaNsFound counter SpaceMask::MaskZero: Initialise mask to zero SpaceMask::MaskOne: Set old style mask to one SphericalSurface::SphericalSurface_Setup: [global] Calculate surface coordinate descriptors GROUP MaskBase_SetupMask: Set up the weight function GROUP MaskBase_SetupMaskAll: Set up the weight function CarpetReduce::MaskBase_AllocateMask: [global] Allocate the weight function CarpetReduce::MaskBase_InitMask: [global] [loop-local] Initialise the weight function GROUP SetupIMaskInternal: Set up the integer weight function (schedule other routines in here) CarpetReduce::CoordBase_SetupMask: [global] [loop-local] Set up the outer boundaries of the weight function CarpetReduce::CarpetMaskSetup: [global] [loop-singlemap] Set up the weight function for the restriction regions GROUP SetupIMask: Set up the integer weight function (schedule other routines in here) CarpetMask::CarpetExcludedSetup: [global] [loop-local] Set up the weight function for the excluded regions CarpetMask::CarpetSurfaceSetup: [global] [loop-local] Set up the weight function for the excluded spherical surfaces CarpetReduce::MaskBase_SetMask: [global] [loop-local] Set the weight function GROUP SetupMask: Set up the real weight function (schedule other routines in here) CarpetReduce::MaskBase_TestMask: [global] Test the weight function SphericalSurface::SphericalSurface_Set: [global] Set surface radii to be used for initial setup in other thorns GROUP SphericalSurface_HasBeenSet: Set the spherical surfaces before this group, and use it afterwards SphericalSurface::SphericalSurface_CheckState: [global] Test the state of the spherical surfaces SummationByParts::SBP_SetNormMask: Setup the mask for the calculation of the norm SymBase::SymBase_Check: Check whether the driver set up the grid consistently TerminationTrigger::TerminationTrigger_ResetTrigger: Clear trigger state TerminationTrigger::TerminationTrigger_StartTimer: Start timer TerminationTrigger::TerminationTrigger_CreateFile: Create termination file Time::Time_Initialise: [global] Initialise Time variables Time::TemporalSpacings: [singlemap] Set timestep based on Courant condition (courant_static) WeylScal4::WeylScal4_CheckBoundaries: [meta] check boundaries treatment [CCTK_INITIAL] StaticConformal::StaticConformal_InitialiseState: Set the conformal_state variable to 0 GROUP ADMBase_InitialData: Schedule group for calculating ADM initial data GRHydro::GRHydro_EOSHandle: [global] Set the EOS number CarpetIOASCII::CarpetIOASCIIInit: [global] Initialisation routine CarpetIOBasic::CarpetIOBasicInit: [global] Initialisation routine CarpetIOHDF5::CarpetIOHDF5_Init: [global] Initialisation routine CarpetIOScalar::CarpetIOScalarInit: [global] Initialisation routine GRHydro::GRHydro_Rho_Minima_Setup: Set up minimum for the rest-mass density in the atmosphere (before intial data) GRHydro::GRHydro_SetupMask: Initialize the atmosphere mask GRHydro::GRHydro_RefinementLevel: Calculate current refinement level GROUP ADMBase_InitialGauge: Schedule group for the ADM initial gauge condition ADMBase::ADMBase_ShiftZero: Set the shift to 0 at all points ADMBase::ADMBase_DtShiftZero: Set the dtshift to 0 at all points GROUP HydroBase_Initial: HydroBase initial data group GROUP GRHydro_Initial: GRHydro initial data group GROUP HydroBase_ExcisionMaskSetup: Set up hydro excision mask HydroBase::HydroBase_InitExcisionMask: Initialize hydro excision mask to 'no excision everywhere' Meudon_Bin_NS::Meudon_Bin_NS_initialise: Set up binary neutron star initial data GROUP ADMBase_PostInitial: Schedule group for modifying the ADM initial data, such as e.g. adding noise GRHydro::GRHydro_SqrtSpatialDeterminant: Calculate sdetg GRHydro::GRHydro_InitialAtmosphereReset: Use mask to enforce atmosphere at initial time ML_BSSN::ML_BSSN_InitialADMBase1Everywhere: ML_BSSN_InitialADMBase1Everywhere ML_BSSN::ML_BSSN_InitialADMBase2Interior: ML_BSSN_InitialADMBase2Interior ML_BSSN::ML_BSSN_InitialADMBase2BoundaryScalar: ML_BSSN_InitialADMBase2BoundaryScalar ML_BSSN_Helper::ML_BSSN_ExtrapolateGammas: Extrapolate Gammas and time derivatives of lapse and shift MoL::MoL_StartLoop: [level] Initialise the step size control QuasiLocalMeasures::qlm_init: [global] Initialise quasi-local calculations GROUP HydroBase_Prim2ConInitial: Recover the conservative variables from the primitive variables GRHydro::Primitive2ConservativeCells: Convert initial data given in primive variables to conserved variables [CCTK_POSTINITIAL] CarpetIOHDF5::CarpetIOHDF5_CloseFiles: [global] Close all filereader input files GRHydro::GRHydro_Scalar_Setup: Set up and check scalars for efficiency GROUP MoL_PostStepModify: The group for physics thorns to schedule enforcing constraints ML_BSSN::ML_BSSN_EnforceEverywhere: ML_BSSN_EnforceEverywhere GROUP MoL_PostStep: Ensure that everything is correct after the initial data have been set up ML_BSSN::ML_BSSN_SelectBoundConds: [level] select boundary conditions GRHydro::GRHydro_RefinementLevel: Calculate current refinement level GRHydro::GRHydro_SetLastMoLPostStep: [level] Set grid scalar InLastMoLPostStep if this is the last MoL PostStep call GROUP ML_BSSN_ApplyBCs: Apply boundary conditions controlled by thorn Boundary GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions ML_ADMConstraints::ML_ADMConstraints_SelectBoundConds: [level] select boundary conditions GROUP ML_ADMConstraints_ApplyBCs: Apply boundary conditions controlled by thorn Boundary GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions ML_BSSN::ML_BSSN_ADMBaseInterior: ML_BSSN_ADMBaseInterior ML_BSSN::ML_BSSN_ADMBaseBoundaryScalar: ML_BSSN_ADMBaseBoundaryScalar ML_BSSN::ML_BSSN_ADMBaseEverywhere: ML_BSSN_ADMBaseEverywhere ML_BSSN_Helper::ML_BSSN_ADMBase_SelectBCs: [level] Select boundary conditions for ADMBase variables GROUP ML_BSSN_ADMBase_ApplyBCs: Apply boundary conditions to ADMBase variables GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP ADMBase_SetADMVars: Set the ADM variables before this group, and use them afterwards GROUP HydroBase_PostStep: Post step tasks for hydro thorns GROUP GRHydro_PostStep: Post step tasks for GRHydro GROUP GRHydro_AtmosphereMaskBoundaries: Apply boundary conditions to primitives GRHydro::GRHydro_SelectAtmosphereMaskBoundaries: [level] Select atmosphere mask for boundary conditions GROUP GRHydro_ApplyAtmosphereMaskBCs: Apply boundary conditions to real-valued atmosphere mask GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GRHydro::GRHydroPostSyncAtmosphereMask: Set integer atmosphere mask from synchronized real atmosphere mask if (GRHydro::InLastMoLPostStep) GRHydro::GRHydro_AtmosphereReset: Reset the atmosphere if (GRHydro::InLastMoLPostStep) GROUP GRHydro_PrimitiveInitialGuessesBoundaries: Apply boundary conditions to those primitives used as initial guesses GRHydro::GRHydro_SelectPrimitiveInitialGuessesBoundaries: [level] Select initial guess primitive variables for boudary conditions GROUP GRHydro_ApplyPrimitiveInitialGuessBCs: Apply boundary conditions to initial guess primitive variables GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP HydroBase_Boundaries: HydroBase-internal Boundary conditions group GROUP Do_GRHydro_Boundaries: GRHydro Boundary conditions group GROUP HydroBase_Select_Boundaries: Group to schedule the boundary condition functions if (GRHydro::execute_MoL_PostStep) GRHydro::GRHydro_Bound: [level] Select GRHydro boundary conditions GROUP HydroBase_ApplyBCs: Apply the boundary conditions of HydroBase GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP HydroBase_Con2Prim: Convert from conservative to primitive variables if (GRHydro::execute_MoL_Step) GRHydro::GRHydro_SqrtSpatialDeterminant: Calculate sdetg if (GRHydro::execute_MoL_PostStep) GRHydro::Con2Prim: Convert back to primitive variables (general) GROUP SetTmunu: Group for calculating the stress-energy tensor TmunuBase::TmunuBase_ZeroTmunu: Initialise the stress-energy tensor to zero GROUP AddToTmunu: Add to the stress-energy tensor here GRHydro::GRHydro_Tmunu: Compute the energy-momentum tensor WeylScal4::WeylScal4_SelectBoundConds: [level] select boundary conditions GROUP WeylScal4_ApplyBCs: Apply boundary conditions controlled by thorn Boundary GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP MoL_PseudoEvolution: Calculate pseudo-evolved quantities GROUP ADMBase_SetADMVars: Set the ADM variables before this group, and use them afterwards GROUP ML_ADMConstraints_evaluate_group: ML_ADMConstraints_evaluate ML_ADMConstraints::ML_ADMConstraints_evaluate: ML_ADMConstraints_evaluate GROUP ML_ADMConstraints_evaluate_bc_group: ML_ADMConstraints_evaluate ML_ADMConstraints::ML_ADMConstraints_evaluate_SelectBCs: [level] ML_ADMConstraints_evaluate_SelectBCs GROUP ML_ADMConstraints_evaluate_ApplyBCs: Apply BCs for groups set in ML_ADMConstraints_evaluate GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP ML_BSSN_ConstraintsEverywhere_group: ML_BSSN_ConstraintsEverywhere ML_BSSN::ML_BSSN_ConstraintsEverywhere: ML_BSSN_ConstraintsEverywhere GROUP ML_BSSN_ConstraintsEverywhere_bc_group: ML_BSSN_ConstraintsEverywhere ML_BSSN::ML_BSSN_ConstraintsEverywhere_SelectBCs: [level] ML_BSSN_ConstraintsEverywhere_SelectBCs GROUP ML_BSSN_ConstraintsEverywhere_ApplyBCs: Apply BCs for groups set in ML_BSSN_ConstraintsEverywhere GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP ML_BSSN_ConstraintsInterior_group: ML_BSSN_ConstraintsInterior ML_BSSN::ML_BSSN_ConstraintsInterior: ML_BSSN_ConstraintsInterior GROUP ML_BSSN_ConstraintsInterior_bc_group: ML_BSSN_ConstraintsInterior ML_BSSN::ML_BSSN_ConstraintsInterior_SelectBCs: [level] ML_BSSN_ConstraintsInterior_SelectBCs GROUP ML_BSSN_ConstraintsInterior_ApplyBCs: Apply BCs for groups set in ML_BSSN_ConstraintsInterior GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP WeylScal4_psi4_calc_4th_group: WeylScal4_psi4_calc_4th WeylScal4::WeylScal4_psi4_calc_4th: WeylScal4_psi4_calc_4th GROUP WeylScal4_psi4_calc_4th_bc_group: WeylScal4_psi4_calc_4th WeylScal4::WeylScal4_psi4_calc_4th_SelectBCs: [level] WeylScal4_psi4_calc_4th_SelectBCs GROUP WeylScal4_psi4_calc_4th_ApplyBCs: Apply BCs for groups set in WeylScal4_psi4_calc_4th GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions SummationByParts::SBP_CheckGridSizes: Check grid sizes and ghost zones Initialise finer grids recursively Restrict from finer grids [CCTK_POSTRESTRICTINITIAL] GROUP MoL_PostStep: Ensure that everything is correct after restriction ML_BSSN::ML_BSSN_SelectBoundConds: [level] select boundary conditions GRHydro::GRHydro_RefinementLevel: Calculate current refinement level GRHydro::GRHydro_SetLastMoLPostStep: [level] Set grid scalar InLastMoLPostStep if this is the last MoL PostStep call GROUP ML_BSSN_ApplyBCs: Apply boundary conditions controlled by thorn Boundary GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions ML_ADMConstraints::ML_ADMConstraints_SelectBoundConds: [level] select boundary conditions GROUP ML_ADMConstraints_ApplyBCs: Apply boundary conditions controlled by thorn Boundary GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions ML_BSSN::ML_BSSN_ADMBaseInterior: ML_BSSN_ADMBaseInterior ML_BSSN::ML_BSSN_ADMBaseBoundaryScalar: ML_BSSN_ADMBaseBoundaryScalar ML_BSSN::ML_BSSN_ADMBaseEverywhere: ML_BSSN_ADMBaseEverywhere ML_BSSN_Helper::ML_BSSN_ADMBase_SelectBCs: [level] Select boundary conditions for ADMBase variables GROUP ML_BSSN_ADMBase_ApplyBCs: Apply boundary conditions to ADMBase variables GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP ADMBase_SetADMVars: Set the ADM variables before this group, and use them afterwards GROUP HydroBase_PostStep: Post step tasks for hydro thorns GROUP GRHydro_PostStep: Post step tasks for GRHydro GROUP GRHydro_AtmosphereMaskBoundaries: Apply boundary conditions to primitives GRHydro::GRHydro_SelectAtmosphereMaskBoundaries: [level] Select atmosphere mask for boundary conditions GROUP GRHydro_ApplyAtmosphereMaskBCs: Apply boundary conditions to real-valued atmosphere mask GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GRHydro::GRHydroPostSyncAtmosphereMask: Set integer atmosphere mask from synchronized real atmosphere mask if (GRHydro::InLastMoLPostStep) GRHydro::GRHydro_AtmosphereReset: Reset the atmosphere if (GRHydro::InLastMoLPostStep) GROUP GRHydro_PrimitiveInitialGuessesBoundaries: Apply boundary conditions to those primitives used as initial guesses GRHydro::GRHydro_SelectPrimitiveInitialGuessesBoundaries: [level] Select initial guess primitive variables for boudary conditions GROUP GRHydro_ApplyPrimitiveInitialGuessBCs: Apply boundary conditions to initial guess primitive variables GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP HydroBase_Boundaries: HydroBase-internal Boundary conditions group GROUP Do_GRHydro_Boundaries: GRHydro Boundary conditions group GROUP HydroBase_Select_Boundaries: Group to schedule the boundary condition functions if (GRHydro::execute_MoL_PostStep) GRHydro::GRHydro_Bound: [level] Select GRHydro boundary conditions GROUP HydroBase_ApplyBCs: Apply the boundary conditions of HydroBase GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP HydroBase_Con2Prim: Convert from conservative to primitive variables if (GRHydro::execute_MoL_Step) GRHydro::GRHydro_SqrtSpatialDeterminant: Calculate sdetg if (GRHydro::execute_MoL_PostStep) GRHydro::Con2Prim: Convert back to primitive variables (general) GROUP SetTmunu: Group for calculating the stress-energy tensor TmunuBase::TmunuBase_ZeroTmunu: Initialise the stress-energy tensor to zero GROUP AddToTmunu: Add to the stress-energy tensor here GRHydro::GRHydro_Tmunu: Compute the energy-momentum tensor WeylScal4::WeylScal4_SelectBoundConds: [level] select boundary conditions GROUP WeylScal4_ApplyBCs: Apply boundary conditions controlled by thorn Boundary GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP MoL_PseudoEvolutionBoundaries: Apply boundary conditions to pseudo-evolved quantities GROUP ML_ADMConstraints_evaluate_bc_group: ML_ADMConstraints_evaluate ML_ADMConstraints::ML_ADMConstraints_evaluate_SelectBCs: [level] ML_ADMConstraints_evaluate_SelectBCs GROUP ML_ADMConstraints_evaluate_ApplyBCs: Apply BCs for groups set in ML_ADMConstraints_evaluate GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP ML_BSSN_ConstraintsEverywhere_bc_group: ML_BSSN_ConstraintsEverywhere ML_BSSN::ML_BSSN_ConstraintsEverywhere_SelectBCs: [level] ML_BSSN_ConstraintsEverywhere_SelectBCs GROUP ML_BSSN_ConstraintsEverywhere_ApplyBCs: Apply BCs for groups set in ML_BSSN_ConstraintsEverywhere GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP ML_BSSN_ConstraintsInterior_bc_group: ML_BSSN_ConstraintsInterior ML_BSSN::ML_BSSN_ConstraintsInterior_SelectBCs: [level] ML_BSSN_ConstraintsInterior_SelectBCs GROUP ML_BSSN_ConstraintsInterior_ApplyBCs: Apply BCs for groups set in ML_BSSN_ConstraintsInterior GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP WeylScal4_psi4_calc_4th_bc_group: WeylScal4_psi4_calc_4th WeylScal4::WeylScal4_psi4_calc_4th_SelectBCs: [level] WeylScal4_psi4_calc_4th_SelectBCs GROUP WeylScal4_psi4_calc_4th_ApplyBCs: Apply BCs for groups set in WeylScal4_psi4_calc_4th GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions [CCTK_POSTPOSTINITIAL] GRHydro::GRHydro_Rho_Minima_Setup_Final: Set the value of the rest-mass density of the atmosphere which will be used during the evolution GRHydro::GRHydro_InitialAtmosphereReset: Use mask to enforce atmosphere at initial time GROUP Con2Prim: Convert from conservative to primitive variables (might be redundant) if (GRHydro::execute_MoL_Step) GRHydro::GRHydro_SqrtSpatialDeterminant: Calculate sdetg if (GRHydro::execute_MoL_PostStep) GRHydro::Con2Prim: Convert back to primitive variables (general) GROUP SetTmunu: Calculate the stress-energy tensor TmunuBase::TmunuBase_ZeroTmunu: Initialise the stress-energy tensor to zero GROUP AddToTmunu: Add to the stress-energy tensor here GRHydro::GRHydro_Tmunu: Compute the energy-momentum tensor GROUP ADMConstraintsGroup: Evaluate ADM constraints, and perform symmetry boundary conditions [CCTK_POSTSTEP] SphericalSurface::SphericalSurface_Set: [global] Set surface radii GROUP SphericalSurface_HasBeenSet: Set the spherical surfaces before this group, and use it afterwards SphericalSurface::SphericalSurface_CheckState: [global] Test the state of the spherical surfaces GRHydro::GRHydro_RefinementLevel: Calculate current refinement level (for the check of the C2P mask) GRHydro::check_GRHydro_C2P_failed: Check the mask function that contains the points where C2P has failed and report an error in case a failure is found HTTPD::HTTP_Work: [global] Working routine HTTPD::HTTP_ContentWork: [global] Content Working routine GROUP HydroBase_ExcisionHasBeenSet: Group to schedule thorns changing the mask before and thorns using the mask after GROUP Hydro_Analysis: Group for Hydro_Analysis routines Hydro_Analysis::Hydro_Analysis_PrepareReduction: [global] [loop-local] Compute the local reduction results Hydro_Analysis::Hydro_Analysis_Reduction: [global] Compute the global reduction results GROUP Hydro_Analysis_LocationSearch: Look for the location of the maximum density Hydro_Analysis::Hydro_Analysis_LocationSearch_Setup: [global] Prepare data structures for search Hydro_Analysis::Hydro_Analysis_LocationSearch_Search: [global] [loop-local] Search for the location of the maximum density Hydro_Analysis::Hydro_Analysis_LocationSearch_Combine: [global] Communicate and verify the location of the maximum density Hydro_Analysis::Hydro_Analysis_FindSeparation: [global] Compute the proper distance between the maximum of the density and the origin (along a straight coordinate line) NSTracker::NSTracker: [global] Look for the star and set grid accordingly GROUP zzz_NaNChecker_NaNCheck: Check for NaNs and count them in NaNChecker::NaNsFound NaNChecker::NaNChecker_NaNCheck_Prepare: [level] Prepare data structures to check for NaNs NaNChecker::NaNChecker_NaNCheck_Check: [local] Check for NaNs NaNChecker::NaNChecker_NaNCheck_Finish: [level] Count NaNs in NaNChecker::NaNsFound NaNChecker::NaNChecker_TakeAction: [global] [loop-level] Output NaNChecker::NaNmask and take action according to NaNChecker::action_if_found SpaceMask::CheckMask: Ensure that all mask values are legal Dissipation::setup_epsdis: Setup spatially varying dissipation GROUP MaskBase_SetupMask: Set up the weight function GROUP MaskBase_SetupMaskAll: Set up the weight function CarpetReduce::MaskBase_AllocateMask: [global] Allocate the weight function CarpetReduce::MaskBase_InitMask: [global] [loop-local] Initialise the weight function GROUP SetupIMaskInternal: Set up the integer weight function (schedule other routines in here) CarpetReduce::CoordBase_SetupMask: [global] [loop-local] Set up the outer boundaries of the weight function CarpetReduce::CarpetMaskSetup: [global] [loop-singlemap] Set up the weight function for the restriction regions GROUP SetupIMask: Set up the integer weight function (schedule other routines in here) CarpetMask::CarpetExcludedSetup: [global] [loop-local] Set up the weight function for the excluded regions CarpetMask::CarpetSurfaceSetup: [global] [loop-local] Set up the weight function for the excluded spherical surfaces CarpetReduce::MaskBase_SetMask: [global] [loop-local] Set the weight function GROUP SetupMask: Set up the real weight function (schedule other routines in here) CarpetReduce::MaskBase_TestMask: [global] Test the weight function endif if (recover initial data) [CCTK_BASEGRID] ADMBase::ADMBase_SetShiftStateOn: Set the shift_state variable to 1 ADMBase::ADMBase_SetDtLapseStateOn: Set the dtlapse_state variable to 1 ADMBase::ADMBase_SetDtShiftStateOn: Set the dtshift_state variable to 1 ADMMacros::ADMMacros_SetLocalSpatialOrder: Initialize the local_spatial_order CartGrid3D::SpatialSpacings: Set up ranges for spatial 3D Cartesian coordinates (on all grids) CartGrid3D::SpatialCoordinates: Set up spatial 3D Cartesian coordinates on the GH SphericalSurface::SphericalSurface_SetupRes: [global] [loop-local] Set surface resolution automatically Dissipation::dissipation_basegrid: Ensure that there are enough ghost zones GRHydro::GRHydro_Reset_Execution_Flags: [global] Initially set execution flags to 'YEAH, Execute'! GRHydro::GRHydro_InitSymBound: Schedule symmetries and check shift state GRHydro::reset_GRHydro_C2P_failed: Initialise the mask function that contains the points where C2P has failed (at BASEGRID) Hydro_Analysis::Hydro_Analysis_Init: [global] Initialize variables ML_ADMConstraints::ML_ADMConstraints_CheckBoundaries: [meta] check boundaries treatment ML_BSSN::ML_BSSN_CheckBoundaries: [meta] check boundaries treatment NaNChecker::NaNChecker_ResetCounter: [global] Reset the NaNChecker::NaNsFound counter SpaceMask::MaskZero: Initialise mask to zero SpaceMask::MaskOne: Set old style mask to one SphericalSurface::SphericalSurface_Setup: [global] Calculate surface coordinate descriptors GROUP MaskBase_SetupMask: Set up the weight function GROUP MaskBase_SetupMaskAll: Set up the weight function CarpetReduce::MaskBase_AllocateMask: [global] Allocate the weight function CarpetReduce::MaskBase_InitMask: [global] [loop-local] Initialise the weight function GROUP SetupIMaskInternal: Set up the integer weight function (schedule other routines in here) CarpetReduce::CoordBase_SetupMask: [global] [loop-local] Set up the outer boundaries of the weight function CarpetReduce::CarpetMaskSetup: [global] [loop-singlemap] Set up the weight function for the restriction regions GROUP SetupIMask: Set up the integer weight function (schedule other routines in here) CarpetMask::CarpetExcludedSetup: [global] [loop-local] Set up the weight function for the excluded regions CarpetMask::CarpetSurfaceSetup: [global] [loop-local] Set up the weight function for the excluded spherical surfaces CarpetReduce::MaskBase_SetMask: [global] [loop-local] Set the weight function GROUP SetupMask: Set up the real weight function (schedule other routines in here) CarpetReduce::MaskBase_TestMask: [global] Test the weight function SphericalSurface::SphericalSurface_Set: [global] Set surface radii to be used for initial setup in other thorns GROUP SphericalSurface_HasBeenSet: Set the spherical surfaces before this group, and use it afterwards SphericalSurface::SphericalSurface_CheckState: [global] Test the state of the spherical surfaces SummationByParts::SBP_SetNormMask: Setup the mask for the calculation of the norm SymBase::SymBase_Check: Check whether the driver set up the grid consistently TerminationTrigger::TerminationTrigger_ResetTrigger: Clear trigger state TerminationTrigger::TerminationTrigger_StartTimer: Start timer TerminationTrigger::TerminationTrigger_CreateFile: Create termination file Time::Time_Initialise: [global] Initialise Time variables Time::TemporalSpacings: [singlemap] Set timestep based on Courant condition (courant_static) WeylScal4::WeylScal4_CheckBoundaries: [meta] check boundaries treatment [CCTK_RECOVER_VARIABLES] IOUtil::IOUtil_RecoverGH: [level] Checkpoint recovery routine [CCTK_POST_RECOVER_VARIABLES] CarpetIOHDF5::CarpetIOHDF5_InitCheckpointingIntervals: [global] Initialisation of checkpointing intervals after recovery CarpetIOHDF5::CarpetIOHDF5_CloseFiles: [meta] Close all initial data checkpoint files after recovery GROUP MaskBase_SetupMask: Set up the weight function GROUP MaskBase_SetupMaskAll: Set up the weight function CarpetReduce::MaskBase_AllocateMask: [global] Allocate the weight function CarpetReduce::MaskBase_InitMask: [global] [loop-local] Initialise the weight function GROUP SetupIMaskInternal: Set up the integer weight function (schedule other routines in here) CarpetReduce::CoordBase_SetupMask: [global] [loop-local] Set up the outer boundaries of the weight function CarpetReduce::CarpetMaskSetup: [global] [loop-singlemap] Set up the weight function for the restriction regions GROUP SetupIMask: Set up the integer weight function (schedule other routines in here) CarpetMask::CarpetExcludedSetup: [global] [loop-local] Set up the weight function for the excluded regions CarpetMask::CarpetSurfaceSetup: [global] [loop-local] Set up the weight function for the excluded spherical surfaces CarpetReduce::MaskBase_SetMask: [global] [loop-local] Set the weight function GROUP SetupMask: Set up the real weight function (schedule other routines in here) CarpetReduce::MaskBase_TestMask: [global] Test the weight function GRHydro::GRHydro_EOSHandle: [global] Set the EOS number GRHydro::GRHydroCopyIntegerMask: Initialize the real valued atmosphere mask after checkpoint recovery GRHydro::GRHydro_SqrtSpatialDeterminant: Calculate sdetg GROUP HydroBase_ExcisionMaskSetup: Set up hydro excision mask HydroBase::HydroBase_InitExcisionMask: Initialize hydro excision mask to 'no excision everywhere' GROUP zzz_NaNChecker_NaNCheck: Check for NaNs and count them in NaNChecker::NaNsFound NaNChecker::NaNChecker_NaNCheck_Prepare: [level] Prepare data structures to check for NaNs NaNChecker::NaNChecker_NaNCheck_Check: [local] Check for NaNs NaNChecker::NaNChecker_NaNCheck_Finish: [level] Count NaNs in NaNChecker::NaNsFound NaNChecker::NaNChecker_TakeAction: [global] [loop-level] Output NaNChecker::NaNmask and take action according to NaNChecker::action_if_found TerminationTrigger::TerminationTrigger_ResetMinutes: [global] Reset Watchtime endif if (checkpoint initial data) [CCTK_CPINITIAL] CarpetIOHDF5::CarpetIOHDF5_InitialDataCheckpoint: [meta] Initial data checkpoint routine endif if (analysis) [CCTK_ANALYSIS] CarpetLib::CarpetLib_printtimestats: [global] Print timing statistics if desired CarpetLib::CarpetLib_printmemstats: [global] Print memory statistics if desired Formaline::Formaline_AnnounceUpdate: [global] Put some meta information about the current run into permanent storage LoopControl::LC_statistics_analysis: [meta] Output LoopControl statistics GROUP ML_BSSN_EvolutionAnalysis: Calculate RHS at analysis ML_BSSN::ML_BSSN_EvolutionAnalysisInit: ML_BSSN_EvolutionAnalysisInit ML_BSSN::ML_BSSN_EvolutionAnalysisInterior: ML_BSSN_EvolutionAnalysisInterior ML_BSSN_Helper::ML_BSSN_NewRad: Apply NewRad boundary conditions to RHS Multipole::Multipole_Calc: [global] Calculate Multipoles QuasiLocalMeasures::qlm_calculate: [global] Calculate quasi-local quantities SystemStatistics::SystemStatistics_Collect: [global] Collect system statistics TerminationTrigger::TerminationTrigger_CheckWalltime: Check elapsed job walltime TerminationTrigger::TerminationTrigger_CheckSignal: Check if we received a termination signal TerminationTrigger::TerminationTrigger_CheckFile: Check termination file TimerReport::zzz_TimerReport_Output: [global] Print the timer report Trigger::Trigger_Check: [global] test endif Output grid variables do loop over timesteps [CCTK_PREREGRID] CarpetTracker::CarpetTracker_SetPositions: [global] Set positions of refined regions Change grid hierarchy [CCTK_POSTREGRID] CartGrid3D::SpatialCoordinates: Set Coordinates after regridding GROUP MaskBase_SetupMask: Set up the weight function GROUP MaskBase_SetupMaskAll: Set up the weight function CarpetReduce::MaskBase_AllocateMask: [global] Allocate the weight function CarpetReduce::MaskBase_InitMask: [global] [loop-local] Initialise the weight function GROUP SetupIMaskInternal: Set up the integer weight function (schedule other routines in here) CarpetReduce::CoordBase_SetupMask: [global] [loop-local] Set up the outer boundaries of the weight function CarpetReduce::CarpetMaskSetup: [global] [loop-singlemap] Set up the weight function for the restriction regions GROUP SetupIMask: Set up the integer weight function (schedule other routines in here) CarpetMask::CarpetExcludedSetup: [global] [loop-local] Set up the weight function for the excluded regions CarpetMask::CarpetSurfaceSetup: [global] [loop-local] Set up the weight function for the excluded spherical surfaces CarpetReduce::MaskBase_SetMask: [global] [loop-local] Set the weight function GROUP SetupMask: Set up the real weight function (schedule other routines in here) CarpetReduce::MaskBase_TestMask: [global] Test the weight function Dissipation::setup_epsdis: Setup spatially varying dissipation SpaceMask::MaskZero: Initialise mask to zero GROUP GRHydro_PrimitiveBoundaries: Apply boundary conditions to all primitives GRHydro::GRHydro_SelectPrimitiveBoundaries: [level] Select primitive variables for boundary conditions GROUP GRHydro_ApplyPrimitiveBCs: Apply boundary conditions to all primitive variables GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP HydroBase_ExcisionMaskSetup: Set up hydro excision mask HydroBase::HydroBase_InitExcisionMask: Initialize hydro excision mask to 'no excision everywhere' SpaceMask::MaskOne: Set mask to one GRHydro::GRHydro_SetupMask: Initialize the atmosphere mask GROUP MoL_PostStep: Ensure that everything is correct after regridding ML_BSSN::ML_BSSN_SelectBoundConds: [level] select boundary conditions GRHydro::GRHydro_RefinementLevel: Calculate current refinement level GRHydro::GRHydro_SetLastMoLPostStep: [level] Set grid scalar InLastMoLPostStep if this is the last MoL PostStep call GROUP ML_BSSN_ApplyBCs: Apply boundary conditions controlled by thorn Boundary GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions ML_ADMConstraints::ML_ADMConstraints_SelectBoundConds: [level] select boundary conditions GROUP ML_ADMConstraints_ApplyBCs: Apply boundary conditions controlled by thorn Boundary GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions ML_BSSN::ML_BSSN_ADMBaseInterior: ML_BSSN_ADMBaseInterior ML_BSSN::ML_BSSN_ADMBaseBoundaryScalar: ML_BSSN_ADMBaseBoundaryScalar ML_BSSN::ML_BSSN_ADMBaseEverywhere: ML_BSSN_ADMBaseEverywhere ML_BSSN_Helper::ML_BSSN_ADMBase_SelectBCs: [level] Select boundary conditions for ADMBase variables GROUP ML_BSSN_ADMBase_ApplyBCs: Apply boundary conditions to ADMBase variables GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP ADMBase_SetADMVars: Set the ADM variables before this group, and use them afterwards GROUP HydroBase_PostStep: Post step tasks for hydro thorns GROUP GRHydro_PostStep: Post step tasks for GRHydro GROUP GRHydro_AtmosphereMaskBoundaries: Apply boundary conditions to primitives GRHydro::GRHydro_SelectAtmosphereMaskBoundaries: [level] Select atmosphere mask for boundary conditions GROUP GRHydro_ApplyAtmosphereMaskBCs: Apply boundary conditions to real-valued atmosphere mask GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GRHydro::GRHydroPostSyncAtmosphereMask: Set integer atmosphere mask from synchronized real atmosphere mask if (GRHydro::InLastMoLPostStep) GRHydro::GRHydro_AtmosphereReset: Reset the atmosphere if (GRHydro::InLastMoLPostStep) GROUP GRHydro_PrimitiveInitialGuessesBoundaries: Apply boundary conditions to those primitives used as initial guesses GRHydro::GRHydro_SelectPrimitiveInitialGuessesBoundaries: [level] Select initial guess primitive variables for boudary conditions GROUP GRHydro_ApplyPrimitiveInitialGuessBCs: Apply boundary conditions to initial guess primitive variables GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP HydroBase_Boundaries: HydroBase-internal Boundary conditions group GROUP Do_GRHydro_Boundaries: GRHydro Boundary conditions group GROUP HydroBase_Select_Boundaries: Group to schedule the boundary condition functions if (GRHydro::execute_MoL_PostStep) GRHydro::GRHydro_Bound: [level] Select GRHydro boundary conditions GROUP HydroBase_ApplyBCs: Apply the boundary conditions of HydroBase GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP HydroBase_Con2Prim: Convert from conservative to primitive variables if (GRHydro::execute_MoL_Step) GRHydro::GRHydro_SqrtSpatialDeterminant: Calculate sdetg if (GRHydro::execute_MoL_PostStep) GRHydro::Con2Prim: Convert back to primitive variables (general) GROUP SetTmunu: Group for calculating the stress-energy tensor TmunuBase::TmunuBase_ZeroTmunu: Initialise the stress-energy tensor to zero GROUP AddToTmunu: Add to the stress-energy tensor here GRHydro::GRHydro_Tmunu: Compute the energy-momentum tensor WeylScal4::WeylScal4_SelectBoundConds: [level] select boundary conditions GROUP WeylScal4_ApplyBCs: Apply boundary conditions controlled by thorn Boundary GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP MoL_PseudoEvolutionBoundaries: Apply boundary conditions to pseudo-evolved quantities GROUP ML_ADMConstraints_evaluate_bc_group: ML_ADMConstraints_evaluate ML_ADMConstraints::ML_ADMConstraints_evaluate_SelectBCs: [level] ML_ADMConstraints_evaluate_SelectBCs GROUP ML_ADMConstraints_evaluate_ApplyBCs: Apply BCs for groups set in ML_ADMConstraints_evaluate GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP ML_BSSN_ConstraintsEverywhere_bc_group: ML_BSSN_ConstraintsEverywhere ML_BSSN::ML_BSSN_ConstraintsEverywhere_SelectBCs: [level] ML_BSSN_ConstraintsEverywhere_SelectBCs GROUP ML_BSSN_ConstraintsEverywhere_ApplyBCs: Apply BCs for groups set in ML_BSSN_ConstraintsEverywhere GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP ML_BSSN_ConstraintsInterior_bc_group: ML_BSSN_ConstraintsInterior ML_BSSN::ML_BSSN_ConstraintsInterior_SelectBCs: [level] ML_BSSN_ConstraintsInterior_SelectBCs GROUP ML_BSSN_ConstraintsInterior_ApplyBCs: Apply BCs for groups set in ML_BSSN_ConstraintsInterior GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP WeylScal4_psi4_calc_4th_bc_group: WeylScal4_psi4_calc_4th WeylScal4::WeylScal4_psi4_calc_4th_SelectBCs: [level] WeylScal4_psi4_calc_4th_SelectBCs GROUP WeylScal4_psi4_calc_4th_ApplyBCs: Apply BCs for groups set in WeylScal4_psi4_calc_4th GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions Rotate timelevels iteration = iteration+1 t = t+dt [CCTK_PRESTEP] CoordGauge::Einstein_SetNextSlicing: Identify the slicing for the next iteration GRHydro::reset_GRHydro_C2P_failed: Reset the mask function that contains the points where C2P has failed (at PRESTEP) LoopControl::LC_steer: [meta] Update LoopControl algorithm preferences NaNChecker::NaNChecker_ResetCounter: [global] Reset the NaNChecker::NaNsFound counter [CCTK_EVOL] MoL::MoL_StartLoop: [level] Initialise the step size control while (MoL::MoL_Stepsize_Bad) GROUP MoL_Evolution: A single Cactus evolution step using MoL GROUP MoL_StartStep: MoL internal setup for the evolution step MoL::MoL_SetCounter: [level] Set the counter for the ODE method to loop over MoL::MoL_SetTime: [level] Ensure the correct time and timestep are used MoL::MoL_AllocateScratchSpace: [level] Allocate storage for scratch levels GROUP MoL_PreStep: Physics thorns can schedule preloop setup routines in here GRHydro::GRHydro_Scalar_Setup: Set up and check scalars for efficiency MoL::MoL_AllocateScratch: Allocate sufficient space for array scratch variables MoL::MoL_InitialCopy: Ensure the data is in the correct timelevel while (MoL::MoL_Intermediate_Step) GROUP MoL_Step: The loop over the intermediate steps for the ODE integrator MoL::MoL_InitRHS: Initialise the RHS functions GROUP MoL_CalcRHS: Physics thorns schedule the calculation of the discrete spatial operator in here GROUP HydroBase_RHS: Groups for scheduling tasks for calculating RHS of hydro variables if (GRHydro::execute_MoL_Step) GROUP GRHydroRHS: Calculate the update terms GRHydro::SourceTerms: Source term calculation GRHydro::GRHydroStartLoop: [level] Set the flux_direction variable while (GRHydro::flux_direction) GROUP FluxTerms: Calculation of intercell fluxes GRHydro::GRHydro_RefinementLevel: Calculate current refinement level GRHydro::Reconstruct: Reconstruct the functions at the cell boundaries GRHydro::Riemann: Solve the local Riemann problems GRHydro::UpdateCalcul: Calculate the update term from the fluxes GRHydro::GRHydroAdvanceLoop: [level] Decrement the flux_direction variable end while GRHydro::GRHydroUpdateAtmosphereMask: Alter the update terms if inside the atmosphere region ML_BSSN::ML_BSSN_EvolutionBoundaryScalar: ML_BSSN_EvolutionBoundaryScalar GROUP ML_BSSN_EvolutionInteriorSplitBy: ML_BSSN::ML_BSSN_EvolutionInteriorSplitBy1: ML_BSSN_EvolutionInteriorSplitBy1 ML_BSSN::ML_BSSN_EvolutionInteriorSplitBy2: ML_BSSN_EvolutionInteriorSplitBy2 ML_BSSN::ML_BSSN_EvolutionInteriorSplitBy3: ML_BSSN_EvolutionInteriorSplitBy3 ML_BSSN_Helper::ML_BSSN_NewRad: Apply NewRad boundary conditions to RHS GROUP MoL_PostRHS: Modify RHS functions Dissipation::dissipation_add: Add Kreiss-Oliger dissipation to the right hand sides GROUP MoL_RHSBoundaries: Any 'final' modifications to the RHS functions (boundaries etc.) MoL::MoL_Add: Updates calculated with the efficient Runge-Kutta 4 method MoL::MoL_DecrementCounter: [level] Alter the counter number MoL::MoL_ResetTime: [level] If necessary, change the time GROUP MoL_PostStepModify: The group for physics thorns to schedule enforcing constraints ML_BSSN::ML_BSSN_EnforceEverywhere: ML_BSSN_EnforceEverywhere GROUP MoL_PostStep: The group for physics thorns to schedule boundary calls etc. ML_BSSN::ML_BSSN_SelectBoundConds: [level] select boundary conditions GRHydro::GRHydro_RefinementLevel: Calculate current refinement level GRHydro::GRHydro_SetLastMoLPostStep: [level] Set grid scalar InLastMoLPostStep if this is the last MoL PostStep call GROUP ML_BSSN_ApplyBCs: Apply boundary conditions controlled by thorn Boundary GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions ML_ADMConstraints::ML_ADMConstraints_SelectBoundConds: [level] select boundary conditions GROUP ML_ADMConstraints_ApplyBCs: Apply boundary conditions controlled by thorn Boundary GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions ML_BSSN::ML_BSSN_ADMBaseInterior: ML_BSSN_ADMBaseInterior ML_BSSN::ML_BSSN_ADMBaseBoundaryScalar: ML_BSSN_ADMBaseBoundaryScalar ML_BSSN::ML_BSSN_ADMBaseEverywhere: ML_BSSN_ADMBaseEverywhere ML_BSSN_Helper::ML_BSSN_ADMBase_SelectBCs: [level] Select boundary conditions for ADMBase variables GROUP ML_BSSN_ADMBase_ApplyBCs: Apply boundary conditions to ADMBase variables GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP ADMBase_SetADMVars: Set the ADM variables before this group, and use them afterwards GROUP HydroBase_PostStep: Post step tasks for hydro thorns GROUP GRHydro_PostStep: Post step tasks for GRHydro GROUP GRHydro_AtmosphereMaskBoundaries: Apply boundary conditions to primitives GRHydro::GRHydro_SelectAtmosphereMaskBoundaries: [level] Select atmosphere mask for boundary conditions GROUP GRHydro_ApplyAtmosphereMaskBCs: Apply boundary conditions to real-valued atmosphere mask GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GRHydro::GRHydroPostSyncAtmosphereMask: Set integer atmosphere mask from synchronized real atmosphere mask if (GRHydro::InLastMoLPostStep) GRHydro::GRHydro_AtmosphereReset: Reset the atmosphere if (GRHydro::InLastMoLPostStep) GROUP GRHydro_PrimitiveInitialGuessesBoundaries: Apply boundary conditions to those primitives used as initial guesses GRHydro::GRHydro_SelectPrimitiveInitialGuessesBoundaries: [level] Select initial guess primitive variables for boudary conditions GROUP GRHydro_ApplyPrimitiveInitialGuessBCs: Apply boundary conditions to initial guess primitive variables GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP HydroBase_Boundaries: HydroBase-internal Boundary conditions group GROUP Do_GRHydro_Boundaries: GRHydro Boundary conditions group GROUP HydroBase_Select_Boundaries: Group to schedule the boundary condition functions if (GRHydro::execute_MoL_PostStep) GRHydro::GRHydro_Bound: [level] Select GRHydro boundary conditions GROUP HydroBase_ApplyBCs: Apply the boundary conditions of HydroBase GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP HydroBase_Con2Prim: Convert from conservative to primitive variables if (GRHydro::execute_MoL_Step) GRHydro::GRHydro_SqrtSpatialDeterminant: Calculate sdetg if (GRHydro::execute_MoL_PostStep) GRHydro::Con2Prim: Convert back to primitive variables (general) GROUP SetTmunu: Group for calculating the stress-energy tensor TmunuBase::TmunuBase_ZeroTmunu: Initialise the stress-energy tensor to zero GROUP AddToTmunu: Add to the stress-energy tensor here GRHydro::GRHydro_Tmunu: Compute the energy-momentum tensor WeylScal4::WeylScal4_SelectBoundConds: [level] select boundary conditions GROUP WeylScal4_ApplyBCs: Apply boundary conditions controlled by thorn Boundary GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GRHydro::GRHydro_ClearLastMoLPostStep: [level] Reset InLastMoLPostStep to zero MoL::MoL_ResetDeltaTime: [level] If necessary, change the timestep end while MoL::MoL_FinishLoop: [level] Control the step size MoL::MoL_RestoreSandR: Restoring the Save and Restore variables to the original state MoL::MoL_FreeScratchSpace: [level] Free storage for scratch levels end while GRHydro::sync_GRHydro_C2P_failed: Syncronise the mask function that contains the points where C2P has failed GROUP MoL_PseudoEvolution: Calculate pseudo-evolved quantities GROUP ADMBase_SetADMVars: Set the ADM variables before this group, and use them afterwards GROUP ML_ADMConstraints_evaluate_group: ML_ADMConstraints_evaluate ML_ADMConstraints::ML_ADMConstraints_evaluate: ML_ADMConstraints_evaluate GROUP ML_ADMConstraints_evaluate_bc_group: ML_ADMConstraints_evaluate ML_ADMConstraints::ML_ADMConstraints_evaluate_SelectBCs: [level] ML_ADMConstraints_evaluate_SelectBCs GROUP ML_ADMConstraints_evaluate_ApplyBCs: Apply BCs for groups set in ML_ADMConstraints_evaluate GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP ML_BSSN_ConstraintsEverywhere_group: ML_BSSN_ConstraintsEverywhere ML_BSSN::ML_BSSN_ConstraintsEverywhere: ML_BSSN_ConstraintsEverywhere GROUP ML_BSSN_ConstraintsEverywhere_bc_group: ML_BSSN_ConstraintsEverywhere ML_BSSN::ML_BSSN_ConstraintsEverywhere_SelectBCs: [level] ML_BSSN_ConstraintsEverywhere_SelectBCs GROUP ML_BSSN_ConstraintsEverywhere_ApplyBCs: Apply BCs for groups set in ML_BSSN_ConstraintsEverywhere GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP ML_BSSN_ConstraintsInterior_group: ML_BSSN_ConstraintsInterior ML_BSSN::ML_BSSN_ConstraintsInterior: ML_BSSN_ConstraintsInterior GROUP ML_BSSN_ConstraintsInterior_bc_group: ML_BSSN_ConstraintsInterior ML_BSSN::ML_BSSN_ConstraintsInterior_SelectBCs: [level] ML_BSSN_ConstraintsInterior_SelectBCs GROUP ML_BSSN_ConstraintsInterior_ApplyBCs: Apply BCs for groups set in ML_BSSN_ConstraintsInterior GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP WeylScal4_psi4_calc_4th_group: WeylScal4_psi4_calc_4th WeylScal4::WeylScal4_psi4_calc_4th: WeylScal4_psi4_calc_4th GROUP WeylScal4_psi4_calc_4th_bc_group: WeylScal4_psi4_calc_4th WeylScal4::WeylScal4_psi4_calc_4th_SelectBCs: [level] WeylScal4_psi4_calc_4th_SelectBCs GROUP WeylScal4_psi4_calc_4th_ApplyBCs: Apply BCs for groups set in WeylScal4_psi4_calc_4th GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions Evolve finer grids recursively Restrict from finer grids [CCTK_POSTRESTRICT] GROUP MoL_PostStep: Ensure that everything is correct after restriction ML_BSSN::ML_BSSN_SelectBoundConds: [level] select boundary conditions GRHydro::GRHydro_RefinementLevel: Calculate current refinement level GRHydro::GRHydro_SetLastMoLPostStep: [level] Set grid scalar InLastMoLPostStep if this is the last MoL PostStep call GROUP ML_BSSN_ApplyBCs: Apply boundary conditions controlled by thorn Boundary GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions ML_ADMConstraints::ML_ADMConstraints_SelectBoundConds: [level] select boundary conditions GROUP ML_ADMConstraints_ApplyBCs: Apply boundary conditions controlled by thorn Boundary GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions ML_BSSN::ML_BSSN_ADMBaseInterior: ML_BSSN_ADMBaseInterior ML_BSSN::ML_BSSN_ADMBaseBoundaryScalar: ML_BSSN_ADMBaseBoundaryScalar ML_BSSN::ML_BSSN_ADMBaseEverywhere: ML_BSSN_ADMBaseEverywhere ML_BSSN_Helper::ML_BSSN_ADMBase_SelectBCs: [level] Select boundary conditions for ADMBase variables GROUP ML_BSSN_ADMBase_ApplyBCs: Apply boundary conditions to ADMBase variables GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP ADMBase_SetADMVars: Set the ADM variables before this group, and use them afterwards GROUP HydroBase_PostStep: Post step tasks for hydro thorns GROUP GRHydro_PostStep: Post step tasks for GRHydro GROUP GRHydro_AtmosphereMaskBoundaries: Apply boundary conditions to primitives GRHydro::GRHydro_SelectAtmosphereMaskBoundaries: [level] Select atmosphere mask for boundary conditions GROUP GRHydro_ApplyAtmosphereMaskBCs: Apply boundary conditions to real-valued atmosphere mask GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GRHydro::GRHydroPostSyncAtmosphereMask: Set integer atmosphere mask from synchronized real atmosphere mask if (GRHydro::InLastMoLPostStep) GRHydro::GRHydro_AtmosphereReset: Reset the atmosphere if (GRHydro::InLastMoLPostStep) GROUP GRHydro_PrimitiveInitialGuessesBoundaries: Apply boundary conditions to those primitives used as initial guesses GRHydro::GRHydro_SelectPrimitiveInitialGuessesBoundaries: [level] Select initial guess primitive variables for boudary conditions GROUP GRHydro_ApplyPrimitiveInitialGuessBCs: Apply boundary conditions to initial guess primitive variables GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP HydroBase_Boundaries: HydroBase-internal Boundary conditions group GROUP Do_GRHydro_Boundaries: GRHydro Boundary conditions group GROUP HydroBase_Select_Boundaries: Group to schedule the boundary condition functions if (GRHydro::execute_MoL_PostStep) GRHydro::GRHydro_Bound: [level] Select GRHydro boundary conditions GROUP HydroBase_ApplyBCs: Apply the boundary conditions of HydroBase GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP HydroBase_Con2Prim: Convert from conservative to primitive variables if (GRHydro::execute_MoL_Step) GRHydro::GRHydro_SqrtSpatialDeterminant: Calculate sdetg if (GRHydro::execute_MoL_PostStep) GRHydro::Con2Prim: Convert back to primitive variables (general) GROUP SetTmunu: Group for calculating the stress-energy tensor TmunuBase::TmunuBase_ZeroTmunu: Initialise the stress-energy tensor to zero GROUP AddToTmunu: Add to the stress-energy tensor here GRHydro::GRHydro_Tmunu: Compute the energy-momentum tensor WeylScal4::WeylScal4_SelectBoundConds: [level] select boundary conditions GROUP WeylScal4_ApplyBCs: Apply boundary conditions controlled by thorn Boundary GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP MoL_PseudoEvolutionBoundaries: Apply boundary conditions to pseudo-evolved quantities GROUP ML_ADMConstraints_evaluate_bc_group: ML_ADMConstraints_evaluate ML_ADMConstraints::ML_ADMConstraints_evaluate_SelectBCs: [level] ML_ADMConstraints_evaluate_SelectBCs GROUP ML_ADMConstraints_evaluate_ApplyBCs: Apply BCs for groups set in ML_ADMConstraints_evaluate GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP ML_BSSN_ConstraintsEverywhere_bc_group: ML_BSSN_ConstraintsEverywhere ML_BSSN::ML_BSSN_ConstraintsEverywhere_SelectBCs: [level] ML_BSSN_ConstraintsEverywhere_SelectBCs GROUP ML_BSSN_ConstraintsEverywhere_ApplyBCs: Apply BCs for groups set in ML_BSSN_ConstraintsEverywhere GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP ML_BSSN_ConstraintsInterior_bc_group: ML_BSSN_ConstraintsInterior ML_BSSN::ML_BSSN_ConstraintsInterior_SelectBCs: [level] ML_BSSN_ConstraintsInterior_SelectBCs GROUP ML_BSSN_ConstraintsInterior_ApplyBCs: Apply BCs for groups set in ML_BSSN_ConstraintsInterior GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP WeylScal4_psi4_calc_4th_bc_group: WeylScal4_psi4_calc_4th WeylScal4::WeylScal4_psi4_calc_4th_SelectBCs: [level] WeylScal4_psi4_calc_4th_SelectBCs GROUP WeylScal4_psi4_calc_4th_ApplyBCs: Apply BCs for groups set in WeylScal4_psi4_calc_4th GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions [CCTK_POSTSTEP] SphericalSurface::SphericalSurface_Set: [global] Set surface radii GROUP SphericalSurface_HasBeenSet: Set the spherical surfaces before this group, and use it afterwards SphericalSurface::SphericalSurface_CheckState: [global] Test the state of the spherical surfaces GRHydro::GRHydro_RefinementLevel: Calculate current refinement level (for the check of the C2P mask) GRHydro::check_GRHydro_C2P_failed: Check the mask function that contains the points where C2P has failed and report an error in case a failure is found HTTPD::HTTP_Work: [global] Working routine HTTPD::HTTP_ContentWork: [global] Content Working routine GROUP HydroBase_ExcisionHasBeenSet: Group to schedule thorns changing the mask before and thorns using the mask after GROUP Hydro_Analysis: Group for Hydro_Analysis routines Hydro_Analysis::Hydro_Analysis_PrepareReduction: [global] [loop-local] Compute the local reduction results Hydro_Analysis::Hydro_Analysis_Reduction: [global] Compute the global reduction results GROUP Hydro_Analysis_LocationSearch: Look for the location of the maximum density Hydro_Analysis::Hydro_Analysis_LocationSearch_Setup: [global] Prepare data structures for search Hydro_Analysis::Hydro_Analysis_LocationSearch_Search: [global] [loop-local] Search for the location of the maximum density Hydro_Analysis::Hydro_Analysis_LocationSearch_Combine: [global] Communicate and verify the location of the maximum density Hydro_Analysis::Hydro_Analysis_FindSeparation: [global] Compute the proper distance between the maximum of the density and the origin (along a straight coordinate line) NSTracker::NSTracker: [global] Look for the star and set grid accordingly GROUP zzz_NaNChecker_NaNCheck: Check for NaNs and count them in NaNChecker::NaNsFound NaNChecker::NaNChecker_NaNCheck_Prepare: [level] Prepare data structures to check for NaNs NaNChecker::NaNChecker_NaNCheck_Check: [local] Check for NaNs NaNChecker::NaNChecker_NaNCheck_Finish: [level] Count NaNs in NaNChecker::NaNsFound NaNChecker::NaNChecker_TakeAction: [global] [loop-level] Output NaNChecker::NaNmask and take action according to NaNChecker::action_if_found SpaceMask::CheckMask: Ensure that all mask values are legal Dissipation::setup_epsdis: Setup spatially varying dissipation GROUP MaskBase_SetupMask: Set up the weight function GROUP MaskBase_SetupMaskAll: Set up the weight function CarpetReduce::MaskBase_AllocateMask: [global] Allocate the weight function CarpetReduce::MaskBase_InitMask: [global] [loop-local] Initialise the weight function GROUP SetupIMaskInternal: Set up the integer weight function (schedule other routines in here) CarpetReduce::CoordBase_SetupMask: [global] [loop-local] Set up the outer boundaries of the weight function CarpetReduce::CarpetMaskSetup: [global] [loop-singlemap] Set up the weight function for the restriction regions GROUP SetupIMask: Set up the integer weight function (schedule other routines in here) CarpetMask::CarpetExcludedSetup: [global] [loop-local] Set up the weight function for the excluded regions CarpetMask::CarpetSurfaceSetup: [global] [loop-local] Set up the weight function for the excluded spherical surfaces CarpetReduce::MaskBase_SetMask: [global] [loop-local] Set the weight function GROUP SetupMask: Set up the real weight function (schedule other routines in here) CarpetReduce::MaskBase_TestMask: [global] Test the weight function if (checkpoint) [CCTK_CHECKPOINT] CarpetIOHDF5::CarpetIOHDF5_EvolutionCheckpoint: [meta] Evolution checkpoint routine TimerReport::zzz_TimerReport_Checkpoint: [global] Print the timer report endif if (analysis) [CCTK_ANALYSIS] CarpetLib::CarpetLib_printtimestats: [global] Print timing statistics if desired CarpetLib::CarpetLib_printmemstats: [global] Print memory statistics if desired Formaline::Formaline_AnnounceUpdate: [global] Put some meta information about the current run into permanent storage LoopControl::LC_statistics_analysis: [meta] Output LoopControl statistics GROUP ML_BSSN_EvolutionAnalysis: Calculate RHS at analysis ML_BSSN::ML_BSSN_EvolutionAnalysisInit: ML_BSSN_EvolutionAnalysisInit ML_BSSN::ML_BSSN_EvolutionAnalysisInterior: ML_BSSN_EvolutionAnalysisInterior ML_BSSN_Helper::ML_BSSN_NewRad: Apply NewRad boundary conditions to RHS Multipole::Multipole_Calc: [global] Calculate Multipoles QuasiLocalMeasures::qlm_calculate: [global] Calculate quasi-local quantities SystemStatistics::SystemStatistics_Collect: [global] Collect system statistics TerminationTrigger::TerminationTrigger_CheckWalltime: Check elapsed job walltime TerminationTrigger::TerminationTrigger_CheckSignal: Check if we received a termination signal TerminationTrigger::TerminationTrigger_CheckFile: Check termination file TimerReport::zzz_TimerReport_Output: [global] Print the timer report Trigger::Trigger_Check: [global] test endif Output grid variables enddo Termination routines [CCTK_TERMINATE] CarpetIOHDF5::CarpetIOHDF5_TerminationCheckpoint: [meta] Termination checkpoint routine Formaline::Formaline_AnnounceFinal: [global] Put some meta information about the current run into permanent storage LoopControl::LC_statistics_terminate: [meta] Output LoopControl statistics MoL::MoL_FreeIndexArrays: Free the MoL bookkeeping index arrays TimerReport::zzz_TimerReport_Output: [global] Print the timer report Shutdown routines [CCTK_SHUTDOWN] HTTPD::HTTP_Shutdown: [global] HTTP daemon shutdown Timers::Timer_Shutdown: Prepare hierarchical timers Routines run after changing the grid hierarchy: [CCTK_POSTREGRID] CartGrid3D::SpatialCoordinates: Set Coordinates after regridding GROUP MaskBase_SetupMask: Set up the weight function GROUP MaskBase_SetupMaskAll: Set up the weight function CarpetReduce::MaskBase_AllocateMask: [global] Allocate the weight function CarpetReduce::MaskBase_InitMask: [global] [loop-local] Initialise the weight function GROUP SetupIMaskInternal: Set up the integer weight function (schedule other routines in here) CarpetReduce::CoordBase_SetupMask: [global] [loop-local] Set up the outer boundaries of the weight function CarpetReduce::CarpetMaskSetup: [global] [loop-singlemap] Set up the weight function for the restriction regions GROUP SetupIMask: Set up the integer weight function (schedule other routines in here) CarpetMask::CarpetExcludedSetup: [global] [loop-local] Set up the weight function for the excluded regions CarpetMask::CarpetSurfaceSetup: [global] [loop-local] Set up the weight function for the excluded spherical surfaces CarpetReduce::MaskBase_SetMask: [global] [loop-local] Set the weight function GROUP SetupMask: Set up the real weight function (schedule other routines in here) CarpetReduce::MaskBase_TestMask: [global] Test the weight function Dissipation::setup_epsdis: Setup spatially varying dissipation SpaceMask::MaskZero: Initialise mask to zero GROUP GRHydro_PrimitiveBoundaries: Apply boundary conditions to all primitives GRHydro::GRHydro_SelectPrimitiveBoundaries: [level] Select primitive variables for boundary conditions GROUP GRHydro_ApplyPrimitiveBCs: Apply boundary conditions to all primitive variables GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP HydroBase_ExcisionMaskSetup: Set up hydro excision mask HydroBase::HydroBase_InitExcisionMask: Initialize hydro excision mask to 'no excision everywhere' SpaceMask::MaskOne: Set mask to one GRHydro::GRHydro_SetupMask: Initialize the atmosphere mask GROUP MoL_PostStep: Ensure that everything is correct after regridding ML_BSSN::ML_BSSN_SelectBoundConds: [level] select boundary conditions GRHydro::GRHydro_RefinementLevel: Calculate current refinement level GRHydro::GRHydro_SetLastMoLPostStep: [level] Set grid scalar InLastMoLPostStep if this is the last MoL PostStep call GROUP ML_BSSN_ApplyBCs: Apply boundary conditions controlled by thorn Boundary GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions ML_ADMConstraints::ML_ADMConstraints_SelectBoundConds: [level] select boundary conditions GROUP ML_ADMConstraints_ApplyBCs: Apply boundary conditions controlled by thorn Boundary GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions ML_BSSN::ML_BSSN_ADMBaseInterior: ML_BSSN_ADMBaseInterior ML_BSSN::ML_BSSN_ADMBaseBoundaryScalar: ML_BSSN_ADMBaseBoundaryScalar ML_BSSN::ML_BSSN_ADMBaseEverywhere: ML_BSSN_ADMBaseEverywhere ML_BSSN_Helper::ML_BSSN_ADMBase_SelectBCs: [level] Select boundary conditions for ADMBase variables GROUP ML_BSSN_ADMBase_ApplyBCs: Apply boundary conditions to ADMBase variables GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP ADMBase_SetADMVars: Set the ADM variables before this group, and use them afterwards GROUP HydroBase_PostStep: Post step tasks for hydro thorns GROUP GRHydro_PostStep: Post step tasks for GRHydro GROUP GRHydro_AtmosphereMaskBoundaries: Apply boundary conditions to primitives GRHydro::GRHydro_SelectAtmosphereMaskBoundaries: [level] Select atmosphere mask for boundary conditions GROUP GRHydro_ApplyAtmosphereMaskBCs: Apply boundary conditions to real-valued atmosphere mask GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GRHydro::GRHydroPostSyncAtmosphereMask: Set integer atmosphere mask from synchronized real atmosphere mask if (GRHydro::InLastMoLPostStep) GRHydro::GRHydro_AtmosphereReset: Reset the atmosphere if (GRHydro::InLastMoLPostStep) GROUP GRHydro_PrimitiveInitialGuessesBoundaries: Apply boundary conditions to those primitives used as initial guesses GRHydro::GRHydro_SelectPrimitiveInitialGuessesBoundaries: [level] Select initial guess primitive variables for boudary conditions GROUP GRHydro_ApplyPrimitiveInitialGuessBCs: Apply boundary conditions to initial guess primitive variables GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP HydroBase_Boundaries: HydroBase-internal Boundary conditions group GROUP Do_GRHydro_Boundaries: GRHydro Boundary conditions group GROUP HydroBase_Select_Boundaries: Group to schedule the boundary condition functions if (GRHydro::execute_MoL_PostStep) GRHydro::GRHydro_Bound: [level] Select GRHydro boundary conditions GROUP HydroBase_ApplyBCs: Apply the boundary conditions of HydroBase GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP HydroBase_Con2Prim: Convert from conservative to primitive variables if (GRHydro::execute_MoL_Step) GRHydro::GRHydro_SqrtSpatialDeterminant: Calculate sdetg if (GRHydro::execute_MoL_PostStep) GRHydro::Con2Prim: Convert back to primitive variables (general) GROUP SetTmunu: Group for calculating the stress-energy tensor TmunuBase::TmunuBase_ZeroTmunu: Initialise the stress-energy tensor to zero GROUP AddToTmunu: Add to the stress-energy tensor here GRHydro::GRHydro_Tmunu: Compute the energy-momentum tensor WeylScal4::WeylScal4_SelectBoundConds: [level] select boundary conditions GROUP WeylScal4_ApplyBCs: Apply boundary conditions controlled by thorn Boundary GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP MoL_PseudoEvolutionBoundaries: Apply boundary conditions to pseudo-evolved quantities GROUP ML_ADMConstraints_evaluate_bc_group: ML_ADMConstraints_evaluate ML_ADMConstraints::ML_ADMConstraints_evaluate_SelectBCs: [level] ML_ADMConstraints_evaluate_SelectBCs GROUP ML_ADMConstraints_evaluate_ApplyBCs: Apply BCs for groups set in ML_ADMConstraints_evaluate GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP ML_BSSN_ConstraintsEverywhere_bc_group: ML_BSSN_ConstraintsEverywhere ML_BSSN::ML_BSSN_ConstraintsEverywhere_SelectBCs: [level] ML_BSSN_ConstraintsEverywhere_SelectBCs GROUP ML_BSSN_ConstraintsEverywhere_ApplyBCs: Apply BCs for groups set in ML_BSSN_ConstraintsEverywhere GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP ML_BSSN_ConstraintsInterior_bc_group: ML_BSSN_ConstraintsInterior ML_BSSN::ML_BSSN_ConstraintsInterior_SelectBCs: [level] ML_BSSN_ConstraintsInterior_SelectBCs GROUP ML_BSSN_ConstraintsInterior_ApplyBCs: Apply BCs for groups set in ML_BSSN_ConstraintsInterior GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP WeylScal4_psi4_calc_4th_bc_group: WeylScal4_psi4_calc_4th WeylScal4::WeylScal4_psi4_calc_4th_SelectBCs: [level] WeylScal4_psi4_calc_4th_SelectBCs GROUP WeylScal4_psi4_calc_4th_ApplyBCs: Apply BCs for groups set in WeylScal4_psi4_calc_4th GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions -------------------------------------------------------------------------------- INFO (Carpet): Multi-Model listing: model 0: "world" INFO (Carpet): Multi-Model process distribution: process 0: model 0 "world" INFO (Carpet): Multi-Model: This is process 0, model 0 "world" INFO (CarpetLib): Process startup time was 1.76 seconds Current core file size limit: hard=[unlimited], soft=[unlimited] Current addres space size limit: hard=[unlimited], soft=[unlimited] Current data segment size limit: hard=[unlimited], soft=[unlimited] Current resident set size limit: hard=[unlimited], soft=[unlimited] INFO (CycleClock): Measuring CycleClock tick via OpenMP... INFO (CycleClock): Calibrated CycleClock: 0.345488 ns per clock tick (2.89446 GHz) INFO (hwloc): library version 2.0.4, API version 0x20000 INFO (SystemTopology): MPI process-to-host mapping: This is MPI process 0 of 1 MPI hosts: 0: panther This MPI process runs on host 0 of 1 On this host, this is MPI process 0 of 1 INFO (SystemTopology): Topology support: Discovery support: discovery->pu : yes CPU binding support: cpubind->set_thisproc_cpubind : yes cpubind->get_thisproc_cpubind : yes cpubind->set_proc_cpubind : yes cpubind->get_proc_cpubind : yes cpubind->set_thisthread_cpubind : yes cpubind->get_thisthread_cpubind : yes cpubind->set_thread_cpubind : yes cpubind->get_thread_cpubind : yes cpubind->get_thisproc_last_cpu_location : yes cpubind->get_proc_last_cpu_location : yes cpubind->get_thisthread_last_cpu_location: yes Memory binding support: membind->set_thisproc_membind : no membind->get_thisproc_membind : no membind->set_proc_membind : no membind->get_proc_membind : no membind->set_thisthread_membind : yes membind->get_thisthread_membind : yes membind->set_area_membind : yes membind->get_area_membind : yes membind->alloc_membind : yes membind->firsttouch_membind : yes membind->bind_membind : yes membind->interleave_membind : yes membind->nexttouch_membind : no membind->migrate_membind : yes INFO (SystemTopology): Hardware objects in this node: Machine L#0: (P#0, total=1043114796KB, DMIProductName="PowerEdge R7515", DMIProductVersion=, DMIBoardVendor="Dell Inc.", DMIBoardName=04F3CJ, DMIBoardVersion=A02, DMIChassisVendor="Dell Inc.", DMIChassisType=23, DMIChassisVersion=, DMIChassisAssetTag=, DMIBIOSVendor="Dell Inc.", DMIBIOSVersion=2.18.1, DMIBIOSDate=02/03/2025, DMISysVendor="Dell Inc.", Backend=Linux, OSName=Linux, OSRelease=5.14.0-570.37.1.el9_6.x86_64, OSVersion="#1 SMP PREEMPT_DYNAMIC Thu Aug 28 10:41:06 UTC 2025", HostName=panther, Architecture=x86_64, hwlocVersion=2.0.4, ProcessName=cactus_sim) Package L#0: (P#0, total=1043114796KB, CPUVendor=AuthenticAMD, CPUFamilyNumber=23, CPUModelNumber=49, CPUModel="AMD EPYC 7542 32-Core Processor ", CPUStepping=0) L3Cache L#0: (P#-1, size=16384KB, linesize=64, ways=16, Inclusive=0) L2Cache L#0: (P#-1, size=512KB, linesize=64, ways=8, Inclusive=1) L1dCache L#0: (P#-1, size=32KB, linesize=64, ways=8, Inclusive=0) Core L#0: (P#0) PU L#0: (P#0) PU L#1: (P#32) L2Cache L#1: (P#-1, size=512KB, linesize=64, ways=8, Inclusive=1) L1dCache L#1: (P#-1, size=32KB, linesize=64, ways=8, Inclusive=0) Core L#1: (P#1) PU L#2: (P#1) PU L#3: (P#33) L2Cache L#2: (P#-1, size=512KB, linesize=64, ways=8, Inclusive=1) L1dCache L#2: (P#-1, size=32KB, linesize=64, ways=8, Inclusive=0) Core L#2: (P#2) PU L#4: (P#2) PU L#5: (P#34) L2Cache L#3: (P#-1, size=512KB, linesize=64, ways=8, Inclusive=1) L1dCache L#3: (P#-1, size=32KB, linesize=64, ways=8, Inclusive=0) Core L#3: (P#3) PU L#6: (P#3) PU L#7: (P#35) L3Cache L#1: (P#-1, size=16384KB, linesize=64, ways=16, Inclusive=0) L2Cache L#4: (P#-1, size=512KB, linesize=64, ways=8, Inclusive=1) L1dCache L#4: (P#-1, size=32KB, linesize=64, ways=8, Inclusive=0) Core L#4: (P#4) PU L#8: (P#4) PU L#9: (P#36) L2Cache L#5: (P#-1, size=512KB, linesize=64, ways=8, Inclusive=1) L1dCache L#5: (P#-1, size=32KB, linesize=64, ways=8, Inclusive=0) Core L#5: (P#5) PU L#10: (P#5) PU L#11: (P#37) L2Cache L#6: (P#-1, size=512KB, linesize=64, ways=8, Inclusive=1) L1dCache L#6: (P#-1, size=32KB, linesize=64, ways=8, Inclusive=0) Core L#6: (P#6) PU L#12: (P#6) PU L#13: (P#38) L2Cache L#7: (P#-1, size=512KB, linesize=64, ways=8, Inclusive=1) L1dCache L#7: (P#-1, size=32KB, linesize=64, ways=8, Inclusive=0) Core L#7: (P#7) PU L#14: (P#7) PU L#15: (P#39) L3Cache L#2: (P#-1, size=16384KB, linesize=64, ways=16, Inclusive=0) L2Cache L#8: (P#-1, size=512KB, linesize=64, ways=8, Inclusive=1) L1dCache L#8: (P#-1, size=32KB, linesize=64, ways=8, Inclusive=0) Core L#8: (P#8) PU L#16: (P#8) PU L#17: (P#40) L2Cache L#9: (P#-1, size=512KB, linesize=64, ways=8, Inclusive=1) L1dCache L#9: (P#-1, size=32KB, linesize=64, ways=8, Inclusive=0) Core L#9: (P#9) PU L#18: (P#9) PU L#19: (P#41) L2Cache L#10: (P#-1, size=512KB, linesize=64, ways=8, Inclusive=1) L1dCache L#10: (P#-1, size=32KB, linesize=64, ways=8, Inclusive=0) Core L#10: (P#10) PU L#20: (P#10) PU L#21: (P#42) L2Cache L#11: (P#-1, size=512KB, linesize=64, ways=8, Inclusive=1) L1dCache L#11: (P#-1, size=32KB, linesize=64, ways=8, Inclusive=0) Core L#11: (P#11) PU L#22: (P#11) PU L#23: (P#43) L3Cache L#3: (P#-1, size=16384KB, linesize=64, ways=16, Inclusive=0) L2Cache L#12: (P#-1, size=512KB, linesize=64, ways=8, Inclusive=1) L1dCache L#12: (P#-1, size=32KB, linesize=64, ways=8, Inclusive=0) Core L#12: (P#12) PU L#24: (P#12) PU L#25: (P#44) L2Cache L#13: (P#-1, size=512KB, linesize=64, ways=8, Inclusive=1) L1dCache L#13: (P#-1, size=32KB, linesize=64, ways=8, Inclusive=0) Core L#13: (P#13) PU L#26: (P#13) PU L#27: (P#45) L2Cache L#14: (P#-1, size=512KB, linesize=64, ways=8, Inclusive=1) L1dCache L#14: (P#-1, size=32KB, linesize=64, ways=8, Inclusive=0) Core L#14: (P#14) PU L#28: (P#14) PU L#29: (P#46) L2Cache L#15: (P#-1, size=512KB, linesize=64, ways=8, Inclusive=1) L1dCache L#15: (P#-1, size=32KB, linesize=64, ways=8, Inclusive=0) Core L#15: (P#15) PU L#30: (P#15) PU L#31: (P#47) L3Cache L#4: (P#-1, size=16384KB, linesize=64, ways=16, Inclusive=0) L2Cache L#16: (P#-1, size=512KB, linesize=64, ways=8, Inclusive=1) L1dCache L#16: (P#-1, size=32KB, linesize=64, ways=8, Inclusive=0) Core L#16: (P#16) PU L#32: (P#16) PU L#33: (P#48) L2Cache L#17: (P#-1, size=512KB, linesize=64, ways=8, Inclusive=1) L1dCache L#17: (P#-1, size=32KB, linesize=64, ways=8, Inclusive=0) Core L#17: (P#17) PU L#34: (P#17) PU L#35: (P#49) L2Cache L#18: (P#-1, size=512KB, linesize=64, ways=8, Inclusive=1) L1dCache L#18: (P#-1, size=32KB, linesize=64, ways=8, Inclusive=0) Core L#18: (P#18) PU L#36: (P#18) PU L#37: (P#50) L2Cache L#19: (P#-1, size=512KB, linesize=64, ways=8, Inclusive=1) L1dCache L#19: (P#-1, size=32KB, linesize=64, ways=8, Inclusive=0) Core L#19: (P#19) PU L#38: (P#19) PU L#39: (P#51) L3Cache L#5: (P#-1, size=16384KB, linesize=64, ways=16, Inclusive=0) L2Cache L#20: (P#-1, size=512KB, linesize=64, ways=8, Inclusive=1) L1dCache L#20: (P#-1, size=32KB, linesize=64, ways=8, Inclusive=0) Core L#20: (P#20) PU L#40: (P#20) PU L#41: (P#52) L2Cache L#21: (P#-1, size=512KB, linesize=64, ways=8, Inclusive=1) L1dCache L#21: (P#-1, size=32KB, linesize=64, ways=8, Inclusive=0) Core L#21: (P#21) PU L#42: (P#21) PU L#43: (P#53) L2Cache L#22: (P#-1, size=512KB, linesize=64, ways=8, Inclusive=1) L1dCache L#22: (P#-1, size=32KB, linesize=64, ways=8, Inclusive=0) Core L#22: (P#22) PU L#44: (P#22) PU L#45: (P#54) L2Cache L#23: (P#-1, size=512KB, linesize=64, ways=8, Inclusive=1) L1dCache L#23: (P#-1, size=32KB, linesize=64, ways=8, Inclusive=0) Core L#23: (P#23) PU L#46: (P#23) PU L#47: (P#55) L3Cache L#6: (P#-1, size=16384KB, linesize=64, ways=16, Inclusive=0) L2Cache L#24: (P#-1, size=512KB, linesize=64, ways=8, Inclusive=1) L1dCache L#24: (P#-1, size=32KB, linesize=64, ways=8, Inclusive=0) Core L#24: (P#24) PU L#48: (P#24) PU L#49: (P#56) L2Cache L#25: (P#-1, size=512KB, linesize=64, ways=8, Inclusive=1) L1dCache L#25: (P#-1, size=32KB, linesize=64, ways=8, Inclusive=0) Core L#25: (P#25) PU L#50: (P#25) PU L#51: (P#57) L2Cache L#26: (P#-1, size=512KB, linesize=64, ways=8, Inclusive=1) L1dCache L#26: (P#-1, size=32KB, linesize=64, ways=8, Inclusive=0) Core L#26: (P#26) PU L#52: (P#26) PU L#53: (P#58) L2Cache L#27: (P#-1, size=512KB, linesize=64, ways=8, Inclusive=1) L1dCache L#27: (P#-1, size=32KB, linesize=64, ways=8, Inclusive=0) Core L#27: (P#27) PU L#54: (P#27) PU L#55: (P#59) L3Cache L#7: (P#-1, size=16384KB, linesize=64, ways=16, Inclusive=0) L2Cache L#28: (P#-1, size=512KB, linesize=64, ways=8, Inclusive=1) L1dCache L#28: (P#-1, size=32KB, linesize=64, ways=8, Inclusive=0) Core L#28: (P#28) PU L#56: (P#28) PU L#57: (P#60) L2Cache L#29: (P#-1, size=512KB, linesize=64, ways=8, Inclusive=1) L1dCache L#29: (P#-1, size=32KB, linesize=64, ways=8, Inclusive=0) Core L#29: (P#29) PU L#58: (P#29) PU L#59: (P#61) L2Cache L#30: (P#-1, size=512KB, linesize=64, ways=8, Inclusive=1) L1dCache L#30: (P#-1, size=32KB, linesize=64, ways=8, Inclusive=0) Core L#30: (P#30) PU L#60: (P#30) PU L#61: (P#62) L2Cache L#31: (P#-1, size=512KB, linesize=64, ways=8, Inclusive=1) L1dCache L#31: (P#-1, size=32KB, linesize=64, ways=8, Inclusive=0) Core L#31: (P#31) PU L#62: (P#31) PU L#63: (P#63) INFO (SystemTopology): Thread CPU bindings: MPI process 0 on host 0 (process 0 of 1 on this host) OpenMP thread 0: PU set L#{0-63} P#{0-63} OpenMP thread 1: PU set L#{0-63} P#{0-63} OpenMP thread 2: PU set L#{0-63} P#{0-63} OpenMP thread 3: PU set L#{0-63} P#{0-63} OpenMP thread 4: PU set L#{0-63} P#{0-63} OpenMP thread 5: PU set L#{0-63} P#{0-63} OpenMP thread 6: PU set L#{0-63} P#{0-63} OpenMP thread 7: PU set L#{0-63} P#{0-63} OpenMP thread 8: PU set L#{0-63} P#{0-63} OpenMP thread 9: PU set L#{0-63} P#{0-63} OpenMP thread 10: PU set L#{0-63} P#{0-63} OpenMP thread 11: PU set L#{0-63} P#{0-63} OpenMP thread 12: PU set L#{0-63} P#{0-63} OpenMP thread 13: PU set L#{0-63} P#{0-63} OpenMP thread 14: PU set L#{0-63} P#{0-63} OpenMP thread 15: PU set L#{0-63} P#{0-63} OpenMP thread 16: PU set L#{0-63} P#{0-63} OpenMP thread 17: PU set L#{0-63} P#{0-63} OpenMP thread 18: PU set L#{0-63} P#{0-63} OpenMP thread 19: PU set L#{0-63} P#{0-63} OpenMP thread 20: PU set L#{0-63} P#{0-63} OpenMP thread 21: PU set L#{0-63} P#{0-63} OpenMP thread 22: PU set L#{0-63} P#{0-63} OpenMP thread 23: PU set L#{0-63} P#{0-63} OpenMP thread 24: PU set L#{0-63} P#{0-63} OpenMP thread 25: PU set L#{0-63} P#{0-63} OpenMP thread 26: PU set L#{0-63} P#{0-63} OpenMP thread 27: PU set L#{0-63} P#{0-63} OpenMP thread 28: PU set L#{0-63} P#{0-63} OpenMP thread 29: PU set L#{0-63} P#{0-63} OpenMP thread 30: PU set L#{0-63} P#{0-63} OpenMP thread 31: PU set L#{0-63} P#{0-63} INFO (SystemTopology): Setting thread CPU bindings: INFO (SystemTopology): Thread CPU bindings: MPI process 0 on host 0 (process 0 of 1 on this host) OpenMP thread 0: PU set L#{0} P#{0} OpenMP thread 1: PU set L#{2} P#{1} OpenMP thread 2: PU set L#{4} P#{2} OpenMP thread 3: PU set L#{6} P#{3} OpenMP thread 4: PU set L#{8} P#{4} OpenMP thread 5: PU set L#{10} P#{5} OpenMP thread 6: PU set L#{12} P#{6} OpenMP thread 7: PU set L#{14} P#{7} OpenMP thread 8: PU set L#{16} P#{8} OpenMP thread 9: PU set L#{18} P#{9} OpenMP thread 10: PU set L#{20} P#{10} OpenMP thread 11: PU set L#{22} P#{11} OpenMP thread 12: PU set L#{24} P#{12} OpenMP thread 13: PU set L#{26} P#{13} OpenMP thread 14: PU set L#{28} P#{14} OpenMP thread 15: PU set L#{30} P#{15} OpenMP thread 16: PU set L#{32} P#{16} OpenMP thread 17: PU set L#{34} P#{17} OpenMP thread 18: PU set L#{36} P#{18} OpenMP thread 19: PU set L#{38} P#{19} OpenMP thread 20: PU set L#{40} P#{20} OpenMP thread 21: PU set L#{42} P#{21} OpenMP thread 22: PU set L#{44} P#{22} OpenMP thread 23: PU set L#{46} P#{23} OpenMP thread 24: PU set L#{48} P#{24} OpenMP thread 25: PU set L#{50} P#{25} OpenMP thread 26: PU set L#{52} P#{26} OpenMP thread 27: PU set L#{54} P#{27} OpenMP thread 28: PU set L#{56} P#{28} OpenMP thread 29: PU set L#{58} P#{29} OpenMP thread 30: PU set L#{60} P#{30} OpenMP thread 31: PU set L#{62} P#{31} INFO (SystemTopology): Extracting CPU/cache/memory properties: There are 2 PUs per core (aka hardware SMT threads) There are 1 threads per core (aka SMT threads used) Cache (unknown name) has type "data" depth 1 size 32768 linesize 64 associativity 8 stride 4096, for 2 PUs Cache (unknown name) has type "unified" depth 2 size 524288 linesize 64 associativity 8 stride 65536, for 2 PUs Cache (unknown name) has type "unified" depth 3 size 16777216 linesize 64 associativity 16 stride 1048576, for 8 PUs INFO (Formaline): Configuration id: config-sim-panther-system-user-crangano-einstein_toolkit-BNS INFO (Formaline): Build id: build-sim-panther-crangano-2025.09.09-14.35.11-2994181 INFO (Formaline): Simulation id: run-bns-panther-crangano-2025.09.10-15.19.02-3273101 INFO (Formaline): Run id: run-bns-panther-crangano-2025.09.10-15.19.02-3273101 Server started on http://panther:5555/ Not announcing location via Twitter. INFO (Vectors): Using vector size 2 for architecture SSE2+SSE4.1 (64-bit precision) -------------------------------------------------------------------------------- AMR driver provided by Carpet -------------------------------------------------------------------------------- GRHydro: relativistic hydrodynamics, no ice. -------------------------------------------------------------------------------- HydroBase: Let it flow. -------------------------------------------------------------------------------- AMR info I/O provided by CarpetIOBasic -------------------------------------------------------------------------------- ML_ADMConstraints -------------------------------------------------------------------------------- ML_BSSN -------------------------------------------------------------------------------- AMR HDF5 I/O provided by CarpetIOHDF5 -------------------------------------------------------------------------------- AMR 0D HDF5 I/O provided by CarpetIOHDF5 -------------------------------------------------------------------------------- AMR 1D HDF5 I/O provided by CarpetIOHDF5 -------------------------------------------------------------------------------- AMR 2D HDF5 I/O provided by CarpetIOHDF5 -------------------------------------------------------------------------------- AMR 3D HDF5 I/O provided by CarpetIOHDF5 -------------------------------------------------------------------------------- MoL: Generalized time integration. -------------------------------------------------------------------------------- AMR scalar I/O provided by CarpetIOScalar -------------------------------------------------------------------------------- AMR 0D ASCII I/O provided by CarpetIOASCII -------------------------------------------------------------------------------- AMR 1D ASCII I/O provided by CarpetIOASCII -------------------------------------------------------------------------------- AMR 2D ASCII I/O provided by CarpetIOASCII -------------------------------------------------------------------------------- AMR 3D ASCII I/O provided by CarpetIOASCII -------------------------------------------------------------------------------- WeylScal4 -------------------------------------------------------------------------------- INFO (Carpet): MPI is enabled INFO (Carpet): Carpet is running on 1 processes INFO (Carpet): This is process 0 INFO (Carpet): OpenMP is enabled INFO (Carpet): This process contains 32 threads, this is thread 0 INFO (Carpet): There are 32 threads in total INFO (Carpet): There are 32 threads per process INFO (Carpet): This process runs on host panther, pid=3273101 INFO (Carpet): This process runs on 32 cores: 0-31 INFO (Carpet): Thread 0 runs on 1 core: 0 INFO (Carpet): Thread 1 runs on 1 core: 1 INFO (Carpet): Thread 2 runs on 1 core: 2 INFO (Carpet): Thread 3 runs on 1 core: 3 INFO (Carpet): Thread 4 runs on 1 core: 4 INFO (Carpet): Thread 5 runs on 1 core: 5 INFO (Carpet): Thread 6 runs on 1 core: 6 INFO (Carpet): Thread 7 runs on 1 core: 7 INFO (Carpet): Thread 8 runs on 1 core: 8 INFO (Carpet): Thread 9 runs on 1 core: 9 INFO (Carpet): Thread 10 runs on 1 core: 10 INFO (Carpet): Thread 11 runs on 1 core: 11 INFO (Carpet): Thread 12 runs on 1 core: 12 INFO (Carpet): Thread 13 runs on 1 core: 13 INFO (Carpet): Thread 14 runs on 1 core: 14 INFO (Carpet): Thread 15 runs on 1 core: 15 INFO (Carpet): Thread 16 runs on 1 core: 16 INFO (Carpet): Thread 17 runs on 1 core: 17 INFO (Carpet): Thread 18 runs on 1 core: 18 INFO (Carpet): Thread 19 runs on 1 core: 19 INFO (Carpet): Thread 20 runs on 1 core: 20 INFO (Carpet): Thread 21 runs on 1 core: 21 INFO (Carpet): Thread 22 runs on 1 core: 22 INFO (Carpet): Thread 23 runs on 1 core: 23 INFO (Carpet): Thread 24 runs on 1 core: 24 INFO (Carpet): Thread 25 runs on 1 core: 25 INFO (Carpet): Thread 26 runs on 1 core: 26 INFO (Carpet): Thread 27 runs on 1 core: 27 INFO (Carpet): Thread 28 runs on 1 core: 28 INFO (Carpet): Thread 29 runs on 1 core: 29 INFO (Carpet): Thread 30 runs on 1 core: 30 INFO (Carpet): Thread 31 runs on 1 core: 31 INFO (Carpet): This simulation is running in 3 dimensions INFO (Carpet): Boundary specification for map 0: nboundaryzones: [[3,3,3],[3,3,3]] is_internal : [[0,0,0],[0,0,0]] is_staggered : [[0,0,0],[0,0,0]] shiftout : [[1,0,1],[0,0,0]] INFO (Carpet): CoordBase domain specification for map 0: physical extent: [0,0,0] : [400,400,400] ([400,400,400]) interior extent: [0,16,0] : [384,384,384] ([384,368,384]) exterior extent: [-48,-32,-48] : [432,432,432] ([480,464,480]) base_spacing : [16,16,16] INFO (Carpet): Adapted domain specification for map 0: convergence factor: 2 convergence level : 0 physical extent : [0,0,0] : [400,400,400] ([400,400,400]) interior extent : [0,16,0] : [384,384,384] ([384,368,384]) exterior extent : [-48,-32,-48] : [432,432,432] ([480,464,480]) spacing : [16,16,16] INFO (Carpet): Base grid specification for map 0: number of grid points : [31,30,31] number of coarse grid ghost points: [[3,3,3],[3,3,3]] INFO (Carpet): Buffer zone counts (excluding ghosts): [0]: [[0,0,0],[0,0,0]] [1]: [[9,9,9],[9,9,9]] [2]: [[9,9,9],[9,9,9]] [3]: [[9,9,9],[9,9,9]] [4]: [[9,9,9],[9,9,9]] [5]: [[9,9,9],[9,9,9]] [6]: [[9,9,9],[9,9,9]] [7]: [[9,9,9],[9,9,9]] [8]: [[9,9,9],[9,9,9]] INFO (Carpet): Overlap zone counts: [0]: [[0,0,0],[0,0,0]] [1]: [[0,0,0],[0,0,0]] [2]: [[0,0,0],[0,0,0]] [3]: [[0,0,0],[0,0,0]] [4]: [[0,0,0],[0,0,0]] [5]: [[0,0,0],[0,0,0]] [6]: [[0,0,0],[0,0,0]] [7]: [[0,0,0],[0,0,0]] [8]: [[0,0,0],[0,0,0]] INFO (Carpet): Group and variable statistics: INFO (Carpet): There are 1162 grid functions in 159 groups INFO (Carpet): There are 604 grid scalars in 96 groups INFO (Carpet): There are 130 1-dimensional grid arrays in 13 groups INFO (Carpet): There are 143 2-dimensional grid arrays in 18 groups INFO (Carpet): There are 0 3-dimensional grid arrays in 0 groups INFO (Carpet): (The number of variables counts all time levels) INFO (CarpetIOHDF5): I/O Method 'IOHDF5' registered: AMR output of grid variables to HDF5 files INFO (CarpetIOHDF5): I/O Method 'IOHDF5_0D' registered: 0D AMR output of grid variables to HDF5 files INFO (CarpetIOHDF5): I/O Method 'IOHDF5_1D' registered: 1D AMR output of grid variables to HDF5 files INFO (CarpetIOHDF5): I/O Method 'IOHDF5_2D' registered: 2D AMR output of grid variables to HDF5 files INFO (CarpetIOHDF5): Periodic 2D AMR output requested for: ADMBASE::gxx ADMBASE::gxy ADMBASE::gxz ADMBASE::gyy ADMBASE::gyz ADMBASE::gzz ADMBASE::alp ADMBASE::betax ADMBASE::betay ADMBASE::betaz CARPETREDUCE::weight GRID::x GRID::y GRID::z GRID::r HYDROBASE::rho HYDROBASE::eps HYDROBASE::vel[0] HYDROBASE::vel[1] HYDROBASE::vel[2] ML_ADMCONSTRAINTS::H INFO (CarpetIOHDF5): I/O Method 'IOHDF5_3D' registered: 3D AMR output of grid variables to HDF5 files INFO (CarpetIOHDF5): Periodic 3D AMR output requested for: ADMBASE::alp ADMBASE::betax ADMBASE::betay ADMBASE::betaz CARPETREDUCE::weight GRID::x GRID::y GRID::z GRID::r HYDROBASE::rho HYDROBASE::eps HYDROBASE::vel[0] HYDROBASE::vel[1] HYDROBASE::vel[2] ML_ADMCONSTRAINTS::H INFO (CarpetIOScalar): Periodic scalar output requested for: ADMBASE::gxx ADMBASE::gxy ADMBASE::gxz ADMBASE::gyy ADMBASE::gyz ADMBASE::gzz ADMBASE::kxx ADMBASE::kxy ADMBASE::kxz ADMBASE::kyy ADMBASE::kyz ADMBASE::kzz ADMBASE::alp ADMBASE::betax ADMBASE::betay ADMBASE::betaz GRHYDRO::dens HYDROBASE::rho HYDROBASE::vel[0] HYDROBASE::vel[1] HYDROBASE::vel[2] HYDROBASE::w_lorentz ML_ADMCONSTRAINTS::H SPHERICALSURFACE::sf_radius[0] SPHERICALSURFACE::sf_radius[1] SPHERICALSURFACE::sf_radius[2] SPHERICALSURFACE::sf_radius[3] SPHERICALSURFACE::sf_radius[4] SYSTEMSTATISTICS::maxrss_mb SYSTEMSTATISTICS::majflt_mb SYSTEMSTATISTICS::arena_mb SYSTEMSTATISTICS::ordblks_mb SYSTEMSTATISTICS::hblks_mb SYSTEMSTATISTICS::hblkhd_mb SYSTEMSTATISTICS::uordblks_mb SYSTEMSTATISTICS::fordblks_mb SYSTEMSTATISTICS::keepcost_mb SYSTEMSTATISTICS::swap_used_mb INFO (CarpetIOASCII): I/O Method 'IOASCII_0D' registered: 0D AMR output of grid variables to ASCII files INFO (CarpetIOASCII): Periodic 0D AMR output requested for: CARPET::physical_time_per_hour CARPET::current_physical_time_per_hour CARPET::time_total CARPET::time_evolution CARPET::time_computing CARPET::time_communicating CARPET::time_io CARPET::evolution_steps_count CARPET::local_grid_points_per_second CARPET::total_grid_points_per_second CARPET::local_grid_point_updates_count CARPET::total_grid_point_updates_count CARPET::local_interior_points_per_second CARPET::total_interior_points_per_second CARPET::local_interior_point_updates_count CARPET::total_interior_point_updates_count CARPET::io_per_second CARPET::io_bytes_per_second CARPET::io_bytes_ascii_per_second CARPET::io_bytes_binary_per_second CARPET::io_count CARPET::io_bytes_count CARPET::io_bytes_ascii_count CARPET::io_bytes_binary_count CARPET::comm_per_second CARPET::comm_bytes_per_second CARPET::comm_count CARPET::comm_bytes_count CARPET::time_levels CARPET::current_walltime CARPET::syncs_count HYDRO_ANALYSIS::Hydro_Analysis_rho_max_loc[0] HYDRO_ANALYSIS::Hydro_Analysis_rho_max_loc[1] HYDRO_ANALYSIS::Hydro_Analysis_rho_max_loc[2] HYDRO_ANALYSIS::Hydro_Analysis_rho_max_origin_distance QUASILOCALMEASURES::qlm_time[0] QUASILOCALMEASURES::qlm_time[1] QUASILOCALMEASURES::qlm_equatorial_circumference[0] QUASILOCALMEASURES::qlm_equatorial_circumference[1] QUASILOCALMEASURES::qlm_polar_circumference_0[0] QUASILOCALMEASURES::qlm_polar_circumference_0[1] QUASILOCALMEASURES::qlm_polar_circumference_pi_2[0] QUASILOCALMEASURES::qlm_polar_circumference_pi_2[1] QUASILOCALMEASURES::qlm_area[0] QUASILOCALMEASURES::qlm_area[1] QUASILOCALMEASURES::qlm_irreducible_mass[0] QUASILOCALMEASURES::qlm_irreducible_mass[1] QUASILOCALMEASURES::qlm_radius[0] QUASILOCALMEASURES::qlm_radius[1] QUASILOCALMEASURES::qlm_spin_guess[0] QUASILOCALMEASURES::qlm_spin_guess[1] QUASILOCALMEASURES::qlm_mass_guess[0] QUASILOCALMEASURES::qlm_mass_guess[1] QUASILOCALMEASURES::qlm_killing_eigenvalue_re[0] QUASILOCALMEASURES::qlm_killing_eigenvalue_re[1] QUASILOCALMEASURES::qlm_killing_eigenvalue_im[0] QUASILOCALMEASURES::qlm_killing_eigenvalue_im[1] QUASILOCALMEASURES::qlm_spin[0] QUASILOCALMEASURES::qlm_spin[1] QUASILOCALMEASURES::qlm_npspin[0] QUASILOCALMEASURES::qlm_npspin[1] QUASILOCALMEASURES::qlm_wsspin[0] QUASILOCALMEASURES::qlm_wsspin[1] QUASILOCALMEASURES::qlm_cvspin[0] QUASILOCALMEASURES::qlm_cvspin[1] QUASILOCALMEASURES::qlm_coordspinx[0] QUASILOCALMEASURES::qlm_coordspinx[1] QUASILOCALMEASURES::qlm_coordspiny[0] QUASILOCALMEASURES::qlm_coordspiny[1] QUASILOCALMEASURES::qlm_coordspinz[0] QUASILOCALMEASURES::qlm_coordspinz[1] QUASILOCALMEASURES::qlm_mass[0] QUASILOCALMEASURES::qlm_mass[1] QUASILOCALMEASURES::qlm_adm_energy[0] QUASILOCALMEASURES::qlm_adm_energy[1] QUASILOCALMEASURES::qlm_adm_momentum_x[0] QUASILOCALMEASURES::qlm_adm_momentum_x[1] QUASILOCALMEASURES::qlm_adm_momentum_y[0] QUASILOCALMEASURES::qlm_adm_momentum_y[1] QUASILOCALMEASURES::qlm_adm_momentum_z[0] QUASILOCALMEASURES::qlm_adm_momentum_z[1] QUASILOCALMEASURES::qlm_adm_angular_momentum_x[0] QUASILOCALMEASURES::qlm_adm_angular_momentum_x[1] QUASILOCALMEASURES::qlm_adm_angular_momentum_y[0] QUASILOCALMEASURES::qlm_adm_angular_momentum_y[1] QUASILOCALMEASURES::qlm_adm_angular_momentum_z[0] QUASILOCALMEASURES::qlm_adm_angular_momentum_z[1] QUASILOCALMEASURES::qlm_w_energy[0] QUASILOCALMEASURES::qlm_w_energy[1] QUASILOCALMEASURES::qlm_w_momentum_x[0] QUASILOCALMEASURES::qlm_w_momentum_x[1] QUASILOCALMEASURES::qlm_w_momentum_y[0] QUASILOCALMEASURES::qlm_w_momentum_y[1] QUASILOCALMEASURES::qlm_w_momentum_z[0] QUASILOCALMEASURES::qlm_w_momentum_z[1] QUASILOCALMEASURES::qlm_w_angular_momentum_x[0] QUASILOCALMEASURES::qlm_w_angular_momentum_x[1] QUASILOCALMEASURES::qlm_w_angular_momentum_y[0] QUASILOCALMEASURES::qlm_w_angular_momentum_y[1] QUASILOCALMEASURES::qlm_w_angular_momentum_z[0] QUASILOCALMEASURES::qlm_w_angular_momentum_z[1] SPHERICALSURFACE::sf_active[0] SPHERICALSURFACE::sf_active[1] SPHERICALSURFACE::sf_active[2] SPHERICALSURFACE::sf_active[3] SPHERICALSURFACE::sf_active[4] SPHERICALSURFACE::sf_valid[0] SPHERICALSURFACE::sf_valid[1] SPHERICALSURFACE::sf_valid[2] SPHERICALSURFACE::sf_valid[3] SPHERICALSURFACE::sf_valid[4] SPHERICALSURFACE::sf_area[0] SPHERICALSURFACE::sf_area[1] SPHERICALSURFACE::sf_area[2] SPHERICALSURFACE::sf_area[3] SPHERICALSURFACE::sf_area[4] SPHERICALSURFACE::sf_mean_radius[0] SPHERICALSURFACE::sf_mean_radius[1] SPHERICALSURFACE::sf_mean_radius[2] SPHERICALSURFACE::sf_mean_radius[3] SPHERICALSURFACE::sf_mean_radius[4] SPHERICALSURFACE::sf_centroid_x[0] SPHERICALSURFACE::sf_centroid_x[1] SPHERICALSURFACE::sf_centroid_x[2] SPHERICALSURFACE::sf_centroid_x[3] SPHERICALSURFACE::sf_centroid_x[4] SPHERICALSURFACE::sf_centroid_y[0] SPHERICALSURFACE::sf_centroid_y[1] SPHERICALSURFACE::sf_centroid_y[2] SPHERICALSURFACE::sf_centroid_y[3] SPHERICALSURFACE::sf_centroid_y[4] SPHERICALSURFACE::sf_centroid_z[0] SPHERICALSURFACE::sf_centroid_z[1] SPHERICALSURFACE::sf_centroid_z[2] SPHERICALSURFACE::sf_centroid_z[3] SPHERICALSURFACE::sf_centroid_z[4] SPHERICALSURFACE::sf_quadrupole_xx[0] SPHERICALSURFACE::sf_quadrupole_xx[1] SPHERICALSURFACE::sf_quadrupole_xx[2] SPHERICALSURFACE::sf_quadrupole_xx[3] SPHERICALSURFACE::sf_quadrupole_xx[4] SPHERICALSURFACE::sf_quadrupole_xy[0] SPHERICALSURFACE::sf_quadrupole_xy[1] SPHERICALSURFACE::sf_quadrupole_xy[2] SPHERICALSURFACE::sf_quadrupole_xy[3] SPHERICALSURFACE::sf_quadrupole_xy[4] SPHERICALSURFACE::sf_quadrupole_xz[0] SPHERICALSURFACE::sf_quadrupole_xz[1] SPHERICALSURFACE::sf_quadrupole_xz[2] SPHERICALSURFACE::sf_quadrupole_xz[3] SPHERICALSURFACE::sf_quadrupole_xz[4] SPHERICALSURFACE::sf_quadrupole_yy[0] SPHERICALSURFACE::sf_quadrupole_yy[1] SPHERICALSURFACE::sf_quadrupole_yy[2] SPHERICALSURFACE::sf_quadrupole_yy[3] SPHERICALSURFACE::sf_quadrupole_yy[4] SPHERICALSURFACE::sf_quadrupole_yz[0] SPHERICALSURFACE::sf_quadrupole_yz[1] SPHERICALSURFACE::sf_quadrupole_yz[2] SPHERICALSURFACE::sf_quadrupole_yz[3] SPHERICALSURFACE::sf_quadrupole_yz[4] SPHERICALSURFACE::sf_quadrupole_zz[0] SPHERICALSURFACE::sf_quadrupole_zz[1] SPHERICALSURFACE::sf_quadrupole_zz[2] SPHERICALSURFACE::sf_quadrupole_zz[3] SPHERICALSURFACE::sf_quadrupole_zz[4] SPHERICALSURFACE::sf_min_radius[0] SPHERICALSURFACE::sf_min_radius[1] SPHERICALSURFACE::sf_min_radius[2] SPHERICALSURFACE::sf_min_radius[3] SPHERICALSURFACE::sf_min_radius[4] SPHERICALSURFACE::sf_max_radius[0] SPHERICALSURFACE::sf_max_radius[1] SPHERICALSURFACE::sf_max_radius[2] SPHERICALSURFACE::sf_max_radius[3] SPHERICALSURFACE::sf_max_radius[4] SPHERICALSURFACE::sf_min_x[0] SPHERICALSURFACE::sf_min_x[1] SPHERICALSURFACE::sf_min_x[2] SPHERICALSURFACE::sf_min_x[3] SPHERICALSURFACE::sf_min_x[4] SPHERICALSURFACE::sf_min_y[0] SPHERICALSURFACE::sf_min_y[1] SPHERICALSURFACE::sf_min_y[2] SPHERICALSURFACE::sf_min_y[3] SPHERICALSURFACE::sf_min_y[4] SPHERICALSURFACE::sf_min_z[0] SPHERICALSURFACE::sf_min_z[1] SPHERICALSURFACE::sf_min_z[2] SPHERICALSURFACE::sf_min_z[3] SPHERICALSURFACE::sf_min_z[4] SPHERICALSURFACE::sf_max_x[0] SPHERICALSURFACE::sf_max_x[1] SPHERICALSURFACE::sf_max_x[2] SPHERICALSURFACE::sf_max_x[3] SPHERICALSURFACE::sf_max_x[4] SPHERICALSURFACE::sf_max_y[0] SPHERICALSURFACE::sf_max_y[1] SPHERICALSURFACE::sf_max_y[2] SPHERICALSURFACE::sf_max_y[3] SPHERICALSURFACE::sf_max_y[4] SPHERICALSURFACE::sf_max_z[0] SPHERICALSURFACE::sf_max_z[1] SPHERICALSURFACE::sf_max_z[2] SPHERICALSURFACE::sf_max_z[3] SPHERICALSURFACE::sf_max_z[4] SPHERICALSURFACE::sf_radius[0] SPHERICALSURFACE::sf_radius[1] SPHERICALSURFACE::sf_radius[2] SPHERICALSURFACE::sf_radius[3] SPHERICALSURFACE::sf_radius[4] SPHERICALSURFACE::sf_origin_x[0] SPHERICALSURFACE::sf_origin_x[1] SPHERICALSURFACE::sf_origin_x[2] SPHERICALSURFACE::sf_origin_x[3] SPHERICALSURFACE::sf_origin_x[4] SPHERICALSURFACE::sf_origin_y[0] SPHERICALSURFACE::sf_origin_y[1] SPHERICALSURFACE::sf_origin_y[2] SPHERICALSURFACE::sf_origin_y[3] SPHERICALSURFACE::sf_origin_y[4] SPHERICALSURFACE::sf_origin_z[0] SPHERICALSURFACE::sf_origin_z[1] SPHERICALSURFACE::sf_origin_z[2] SPHERICALSURFACE::sf_origin_z[3] SPHERICALSURFACE::sf_origin_z[4] SPHERICALSURFACE::sf_origin_theta[0] SPHERICALSURFACE::sf_origin_theta[1] SPHERICALSURFACE::sf_origin_theta[2] SPHERICALSURFACE::sf_origin_theta[3] SPHERICALSURFACE::sf_origin_theta[4] SPHERICALSURFACE::sf_origin_phi[0] SPHERICALSURFACE::sf_origin_phi[1] SPHERICALSURFACE::sf_origin_phi[2] SPHERICALSURFACE::sf_origin_phi[3] SPHERICALSURFACE::sf_origin_phi[4] SPHERICALSURFACE::sf_delta_theta[0] SPHERICALSURFACE::sf_delta_theta[1] SPHERICALSURFACE::sf_delta_theta[2] SPHERICALSURFACE::sf_delta_theta[3] SPHERICALSURFACE::sf_delta_theta[4] SPHERICALSURFACE::sf_delta_phi[0] SPHERICALSURFACE::sf_delta_phi[1] SPHERICALSURFACE::sf_delta_phi[2] SPHERICALSURFACE::sf_delta_phi[3] SPHERICALSURFACE::sf_delta_phi[4] INFO (CarpetIOASCII): I/O Method 'IOASCII_1D' registered: 1D AMR output of grid variables to ASCII files INFO (CarpetIOASCII): Periodic 1D AMR output requested for: ADMBASE::gxx ADMBASE::gxy ADMBASE::gxz ADMBASE::gyy ADMBASE::gyz ADMBASE::gzz ADMBASE::kxx ADMBASE::kxy ADMBASE::kxz ADMBASE::kyy ADMBASE::kyz ADMBASE::kzz ADMBASE::alp ADMBASE::betax ADMBASE::betay ADMBASE::betaz HYDROBASE::rho HYDROBASE::vel[0] HYDROBASE::vel[1] HYDROBASE::vel[2] ML_ADMCONSTRAINTS::H INFO (CarpetIOASCII): I/O Method 'IOASCII_2D' registered: 2D AMR output of grid variables to ASCII files INFO (CarpetIOASCII): I/O Method 'IOASCII_3D' registered: 3D AMR output of grid variables to ASCII files -------------------------------------------------------------------------------- INFO (Formaline): Writing tarballs with the Cactus sources into the directory "bns/cactus-source" WARNING level 1 from host panther process 0 in thorn ML_BSSN_Helper, file /system/user/crangano/einstein_toolkit/BNS/configs/sim/build/ML_BSSN_Helper/Parameters.c:10: -> Forcing ML_BSSN::initial_boundary_condition="extrapolate-gammas" because ML_BSSN::my_initial_boundary_condition="extrapolate-gammas" WARNING level 1 from host panther process 0 in thorn ML_BSSN_Helper, file /system/user/crangano/einstein_toolkit/BNS/configs/sim/build/ML_BSSN_Helper/Parameters.c:25: -> Forcing ML_BSSN::rhs_boundary_condition="NewRad" because ML_BSSN::my_rhs_boundary_condition="NewRad" WARNING level 1 from host panther process 0 in thorn ML_BSSN_Helper, file /system/user/crangano/einstein_toolkit/BNS/configs/sim/build/ML_BSSN_Helper/Parameters.c:45: -> Forcing ML_BSSN::epsDiss=0.0 because ML_BSSN::apply_dissipation="never" INFO (MoL): Using Runge-Kutta 4 as the time integrator. INFO (SymBase): Symmetry on lower x-face: rotating_symmetry_180 INFO (SymBase): Symmetry on lower z-face: reflection_symmetry INFO (MoL): The maximum number of evolved variables is 1246. 30 are registered. INFO (MoL): The maximum number of slow evolved variables is 1246. 0 are registered. INFO (MoL): The maximum number of constrained variables is 1246. 37 are registered. INFO (MoL): The maximum number of SandR variables is 1246. 0 are registered. INFO (MoL): The maximum number of evolved array variables is 1246. 0 are registered. INFO (MoL): The maximum number of constrained array variables is 1246. 0 are registered. INFO (MoL): The maximum number of SandR array variables is 1246. 0 are registered. INFO (MoL): The maximum size of any array variables is 0. WARNING level 1 from host panther process 0 in thorn ML_BSSN_Helper, file /system/user/crangano/einstein_toolkit/BNS/configs/sim/build/ML_BSSN_Helper/Parameters.c:145: -> Parameter ML_BSSN::my_initial_data is outdated; please update the parameter file. Do not use this parameter, and set up initial conditions via ADMBase as usual. WARNING level 1 from host panther process 0 in thorn ML_BSSN_Helper, file /system/user/crangano/einstein_toolkit/BNS/configs/sim/build/ML_BSSN_Helper/Parameters.c:152: -> Parameter ML_BSSN::my_initial_boundary_condition is outdated; please update the parameter file. Do not use this parameter, and set up initial boundary conditions as usual. WARNING level 1 from host panther process 0 in thorn ML_BSSN_Helper, file /system/user/crangano/einstein_toolkit/BNS/configs/sim/build/ML_BSSN_Helper/Parameters.c:159: -> Parameter ML_BSSN::my_rhs_boundary_condition is outdated; please update the parameter file. Do not use this parameter, and set up RHS boundary conditions as usual. WARNING level 1 from host panther process 0 in thorn ML_BSSN_Helper, file /system/user/crangano/einstein_toolkit/BNS/configs/sim/build/ML_BSSN_Helper/Parameters.c:166: -> Parameter ML_BSSN::my_boundary_condition is outdated; please update the parameter file. Do not use this parameter, and set up RHS boundary conditions as usual. INFO (Vectors): Testing vectorisation... [errors may result in segfaults] INFO (Vectors): 375/375 tests passed INFO (CarpetRegrid2): Enforcing grid structure properties, iteration 0 INFO (CarpetRegrid2): Enforcing grid structure properties, iteration 1 INFO (CarpetRegrid2): Enforcing grid structure properties, iteration 0 INFO (CarpetRegrid2): Enforcing grid structure properties, iteration 1 INFO (CarpetRegrid2): Enforcing grid structure properties, iteration 0 INFO (CarpetRegrid2): Enforcing grid structure properties, iteration 1 INFO (CarpetRegrid2): Enforcing grid structure properties, iteration 0 INFO (CarpetRegrid2): Enforcing grid structure properties, iteration 1 INFO (CarpetRegrid2): Enforcing grid structure properties, iteration 0 INFO (CarpetRegrid2): Enforcing grid structure properties, iteration 1 INFO (CarpetRegrid2): Enforcing grid structure properties, iteration 0 INFO (CarpetRegrid2): Enforcing grid structure properties, iteration 1 INFO (Carpet): Grid structure (superregions, grid points): [0][0][0] exterior: [0,0,0] : [30,29,30] ([31,30,31] + PADDING) 28830 [1][0][0] exterior: [3,2,3] : [47,56,45] ([45,55,43] + PADDING) 106425 [2][0][0] exterior: [9,6,9] : [55,47,51] ([47,42,43] + PADDING) 84882 [2][0][1] exterior: [9,69,9] : [55,110,51] ([47,42,43] + PADDING) 84882 [3][0][0] exterior: [21,14,21] : [71,55,63] ([51,42,43] + PADDING) 92106 [3][0][1] exterior: [21,177,21] : [71,218,63] ([51,42,43] + PADDING) 92106 [4][0][0] exterior: [45,30,45] : [105,73,89] ([61,44,45] + PADDING) 120780 [4][0][1] exterior: [45,391,45] : [105,434,89] ([61,44,45] + PADDING) 120780 [5][0][0] exterior: [93,62,93] : [175,111,143] ([83,50,51] + PADDING) 211650 [5][0][1] exterior: [93,817,93] : [175,866,143] ([83,50,51] + PADDING) 211650 [6][0][0] exterior: [189,126,189] : [315,189,253] ([127,64,65] + PADDING) 528320 [6][0][1] exterior: [189,1667,189] : [315,1730,253] ([127,64,65] + PADDING) 528320 INFO (Carpet): Grid structure (superregions, coordinates): [0][0][0] exterior: [-48,-32,-48] : [432,432,432] : [16,16,16] [1][0][0] exterior: [-24,-16,-24] : [328,416,312] : [8,8,8] [2][0][0] exterior: [-12,-8,-12] : [172,156,156] : [4,4,4] [2][0][1] exterior: [-12,244,-12] : [172,408,156] : [4,4,4] [3][0][0] exterior: [-6,-4,-6] : [94,78,78] : [2,2,2] [3][0][1] exterior: [-6,322,-6] : [94,404,78] : [2,2,2] [4][0][0] exterior: [-3,-2,-3] : [57,41,41] : [1,1,1] [4][0][1] exterior: [-3,359,-3] : [57,402,41] : [1,1,1] [5][0][0] exterior: [-1.5,-1,-1.5] : [39.5,23.5,23.5] : [0.5,0.5,0.5] [5][0][1] exterior: [-1.5,376.5,-1.5] : [39.5,401,23.5] : [0.5,0.5,0.5] [6][0][0] exterior: [-0.75,-0.5,-0.75] : [30.75,15.25,15.25] : [0.25,0.25,0.25] [6][0][1] exterior: [-0.75,384.75,-0.75] : [30.75,400.5,15.25] : [0.25,0.25,0.25] INFO (Carpet): Global grid structure statistics: INFO (Carpet): GF: rhs: 393k active, 604k owned (+54%), 780k total (+29%), 127 steps/time INFO (Carpet): GF: vars: 319, pts: 367M active, 601M owned (+64%), 812M total (+35%), 1.0 comp/proc INFO (Carpet): GA: vars: 808, pts: 0M active, 0M total (+0%) INFO (Carpet): Total required memory: 6.462 GByte (for GAs and currently active GFs) INFO (Carpet): Load balance: min avg max sdv max/avg-1 INFO (Carpet): Level 0: 5M 5M 5M 0M owned 0% INFO (Carpet): Level 1: 26M 26M 26M 0M owned 0% INFO (Carpet): Level 2: 44M 44M 44M 0M owned 0% INFO (Carpet): Level 3: 48M 48M 48M 0M owned 0% INFO (Carpet): Level 4: 64M 64M 64M 0M owned 0% INFO (Carpet): Level 5: 115M 115M 115M 0M owned 0% INFO (Carpet): Level 6: 299M 299M 299M 0M owned 0% INFO (CartGrid3D): Grid Spacings: INFO (CartGrid3D): dx=>1.6000000e+01 dy=>1.6000000e+01 dz=>1.6000000e+01 INFO (CartGrid3D): Computational Coordinates: INFO (CartGrid3D): x=>[-48.000,432.000] y=>[-32.000,432.000] z=>[-48.000,432.000] INFO (CartGrid3D): Indices of Physical Coordinates: INFO (CartGrid3D): x=>[0,30] y=>[0,29] z=>[0,30] INFO (TerminationTrigger): Reminding you every 60 minutes about remaining walltime INFO (Time): Timestep set to 6.4 (courant_static) INFO (GRHydro): Trying to get EOS handles INFO (GRHydro): Trying to get EOS handles INFO (GRHydro): GRHydro will use the Ideal_Fluid equation of state. INFO (GRHydro): Setting up the atmosphere mask: all points are not_atmosphere INFO (Meudon_Bin_NS): Setting up LORENE Bin_NS initial data INFO (Meudon_Bin_NS): Setting up coordinates INFO (Meudon_Bin_NS): Meudon_Bin_NS will use the 2D_Polytrope equation of state. INFO (Meudon_Bin_NS): Meudon_Bin_NS will use the 1 eos handle INFO (Meudon_Bin_NS): Reading from file "/system/user/crangano/einstein_toolkit/BNS/G2_I12vs12_D4R33T21_45km.resu" Simfactory Done at date: Wed 10 Sep 2025 05:19:25 PM CEST -------------- next part -------------- ################################################################################ # Script variables ################################################################################ # D = 10.0 # M = 1.0 # Pmx = 0.00084541526517121 # Pmy = -0.09530152296974252 # Pmz = 0 # Pphi = 0.09530152296974252 # Ppx = -0.00084541526517121 # Ppy = 0.09530152296974252 # Ppz = 0 # Pr = -0.00084541526517121 # Smx = 0.0 # Smy = 0.0 # Smz = -0.09156449704142013 # Spx = 0.0 # Spy = 0.0 # Spz = 0.09509112426035504 # __file__ = /system/user/crangano/einstein_toolkit/Cactus/repos/einsteinexamples/par/GW150914/GW150914.rpar # __name__ = __main__ # ahrm = 0.4461538461538462 # ahrp = 0.5538461538461539 # center_offset = -0.5384615384615383 # chim = -0.46 # chip = 0.31 # col_width = 19 # dt0 = 0.2753406593406593 # dt_it = 0.002151098901098901 # dtdx = 0.45 # dtdx0 = 0.225 # e = 2.718281828459045 # expected_merger = 1000.0 # final_time = 1700.0 # find_cah = 371902 # h0 = 1.2237362637362637 # h0_min = 1.4276923076923076 # half_D = 5.0 # hfm_min = 0.022307692307692306 # horizon_every = 256 # hr = 1.2237362637362637 # hr_min = 1.92 # i = 7 # inf = inf # key = xp # levelsm = [0,17.132308,8.566154,4.283077,2.141538,1.070769,0.535385] # levelsp = [0,21.267692,10.633846,5.316923,2.658462,1.329231,0.664615] # maxrls = 9 # mm = 0.4461538461538462 # mp = 0.5538461538461539 # n = 28 # n_angular = 28 # n_min = 24 # nan = nan # out2d_every = 1024 # out3d_every = 0 # out_every = 128 # outermost_detector = 500.0 # pi = 3.141592653589793 # q = 1.2413793103448276 # rl0_every = 128 # rl1_every = 128 # rl_max = 8 # rlsm = 7 # rlsp = 7 # rm = 0.5353846153846153 # rp = 0.6646153846153846 # sphere_inner_radius = 51.39692307692307 # sphere_outer_radius = 2194.1591208791206 # tau = 6.283185307179586 # time_after_merger = 200.0 # val = 4.461538461538462 # wave_extract_every = 256 # waveform_length = 1200.0 # xm = -5.538461538461538 # xp = 4.461538461538462 ################################################################################ # Active thorns ################################################################################ ActiveThorns = " ADMBase ML_ADMConstraints ADMCoupling ADMMacros AEILocalInterp AHFinderDirect Boundary Carpet CarpetIOASCII CarpetIOBasic CarpetIOHDF5 CarpetIOScalar CarpetInterp CarpetInterp2 CarpetLib CarpetReduce CarpetRegrid2 CarpetTracker CartGrid3D CoordBase CoordGauge Coordinates CoordinatesSymmetry Formaline GlobalDerivative hwloc IOUtil InitBase Interpolate2 QuasiLocalMeasures LocalInterp LoopControl MoL NaNChecker PunctureTracker Slab SpaceMask SphericalSurface StaticConformal SummationByParts SymBase SystemStatistics SystemTopology TerminationTrigger TensorTypes Time TmunuBase TwoPunctures Vectors ML_BSSN ML_BSSN_Helper NewRad GenericFD WeylScal4 Multipole # WaveExtractCPM ADMDerivatives " ################################################################################ # Grid structure ################################################################################ Carpet::domain_from_multipatch = yes CartGrid3D::type = "multipatch" CartGrid3D::set_coordinate_ranges_on = "all maps" Coordinates::coordinate_system = "Thornburg04" Coordinates::h_cartesian = 1.2237362637362637 Coordinates::h_radial = 1.2237362637362637 Coordinates::sphere_inner_radius = 51.39692307692307 Coordinates::sphere_outer_radius = 2194.1591208791206 Coordinates::n_angular = 28 Driver::ghost_size = 5 Coordinates::patch_boundary_size = 5 Coordinates::additional_overlap_size = 3 Coordinates::outer_boundary_size = 5 # change these (to their defaults) to disable z-zymmetry CoordinatesSymmetry::reflection_z = no CoordinatesSymmetry::stagger = no Coordinates::symmetry = "+z bitant" Coordinates::additional_symmetry_size = 1 Coordinates::verbose = no Time::timestep_method = "given" Time::timestep = 0.2753406593406593 Carpet::time_refinement_factors = "[1,1,2,4,8,16,32,64,128,256]" ################################################################################ # Mesh refinement ################################################################################ Carpet::max_refinement_levels = 9 CarpetRegrid2::num_centres = 2 CarpetRegrid2::num_levels_1 = 7 CarpetRegrid2::position_x_1 = 4.461538461538462 CarpetRegrid2::radius_1 = [0,21.267692,10.633846,5.316923,2.658462,1.329231,0.664615] CarpetRegrid2::num_levels_2 = 7 CarpetRegrid2::position_x_2 = -5.538461538461538 CarpetRegrid2::radius_2 = [0,17.132308,8.566154,4.283077,2.141538,1.070769,0.535385] Carpet::use_buffer_zones = yes Carpet::prolongation_order_space = 5 Carpet::prolongation_order_time = 2 CarpetRegrid2::regrid_every = 128 CarpetRegrid2::verbose = no Carpet::grid_coordinates_filename = "carpet-grid.asc" ################################################################################ # Initial data ################################################################################ ADMBase::initial_data = "twopunctures" ADMBase::initial_lapse = "twopunctures-averaged" ADMBase::initial_shift = "zero" ADMBase::initial_dtlapse = "zero" ADMBase::initial_dtshift = "zero" TwoPunctures::target_M_plus = 0.5538461538461539 TwoPunctures::target_M_minus = 0.4461538461538462 # Initial guesses for iterative solve TwoPunctures::par_m_plus = 0.5538461538461539 TwoPunctures::par_m_minus = 0.4461538461538462 TwoPunctures::par_b = 5.0 TwoPunctures::center_offset[0] = -0.5384615384615383 TwoPunctures::par_P_plus[0] = -0.00084541526517121 TwoPunctures::par_P_plus[1] = 0.09530152296974252 TwoPunctures::par_P_plus[2] = 0 TwoPunctures::par_P_minus[0] = 0.00084541526517121 TwoPunctures::par_P_minus[1] = -0.09530152296974252 TwoPunctures::par_P_minus[2] = 0 TwoPunctures::par_S_plus[0] = 0.0 TwoPunctures::par_S_plus[1] = 0.0 TwoPunctures::par_S_plus[2] = 0.09509112426035504 TwoPunctures::par_S_minus[0] = 0.0 TwoPunctures::par_S_minus[1] = 0.0 TwoPunctures::par_S_minus[2] = -0.09156449704142013 TwoPunctures::grid_setup_method = "evaluation" TwoPunctures::give_bare_mass = no TwoPunctures::TP_epsilon = 1e-6 Carpet::init_fill_timelevels = yes InitBase::initial_data_setup_method = "init_single_level" ################################################################################ # Evolution and boundary ################################################################################ GenericFD::jacobian_group = "Coordinates::jacobian" GenericFD::jacobian_derivative_group = "Coordinates::jacobian2" GenericFD::jacobian_identity_map = 0 ADMBase::evolution_method = "ML_BSSN" ADMBase::lapse_evolution_method = "ML_BSSN" ADMBase::shift_evolution_method = "ML_BSSN" ADMBase::dtlapse_evolution_method = "ML_BSSN" ADMBase::dtshift_evolution_method = "ML_BSSN" ML_BSSN::fdOrder = 8 # 1+log slicing requires harmonicN = 1 and harmonicF = 2.0 ML_BSSN::harmonicN = 1 ML_BSSN::harmonicF = 2.0 ML_BSSN::ShiftGammaCoeff = 0.75 ML_BSSN::BetaDriver = 1.0 ML_BSSN::advectLapse = 1 ML_BSSN::advectShift = 1 ML_BSSN::MinimumLapse = 1.0e-8 # conformalaMethod = 1 for W, 0 for phi ML_BSSN::conformalMethod = 1 # We apply dissipation using GlobalDerivaitive so disable it here ML_BSSN::epsDiss = 0.0 ML_BSSN::dt_lapse_shift_method = "noLapseShiftAdvection" ML_BSSN::initial_boundary_condition = "extrapolate-gammas" ML_BSSN::rhs_boundary_condition = "scalar" Boundary::radpower = 2 ################################################################################ # BH tracking ################################################################################ CarpetTracker::surface [0] = 0 CarpetTracker::surface [1] = 1 PunctureTracker::track [0] = yes PunctureTracker::initial_x [0] = 4.461538461538462 PunctureTracker::which_surface_to_store_info[0] = 0 PunctureTracker::track [1] = yes PunctureTracker::initial_x [1] = -5.538461538461538 PunctureTracker::which_surface_to_store_info[1] = 1 ################################################################################ # Spatial finite differencing ################################################################################ SummationByParts::order = 8 # Drop order instead of using upwinded stencils, only for advection derivatives SummationByParts::sbp_upwind_deriv = no SummationByParts::sbp_1st_deriv = yes SummationByParts::sbp_2nd_deriv = no SummationByParts::onesided_interpatch_boundaries = no SummationByParts::onesided_outer_boundaries = yes SummationByParts::use_dissipation = no GlobalDerivative::use_dissipation = yes SummationByParts::scale_with_h = yes SummationByParts::dissipation_type = "Kreiss-Oliger" SummationByParts::epsdis = 0.15 # Because dt/dx is half that of the other levels we also need half the dissipation GlobalDerivative::epsdis_for_level [0] = 0.075 # Variables for dissipation SummationByParts::vars = " ML_BSSN::ML_log_confac ML_BSSN::ML_metric ML_BSSN::ML_trace_curv ML_BSSN::ML_curv ML_BSSN::ML_Gamma ML_BSSN::ML_lapse ML_BSSN::ML_shift ML_BSSN::ML_dtlapse ML_BSSN::ML_dtshift " ################################################################################ # Time integration ################################################################################ MoL::ODE_Method = "rk4" MoL::MoL_Intermediate_Steps = 4 MoL::MoL_Num_Scratch_Levels = 1 ################################################################################ # Interpolation ################################################################################ CarpetInterp::check_tree_search = no CarpetInterp::tree_search = yes # Use 5-th order interpatch interpolation on the Llama grid Interpolate::interpolator_order = 5 ################################################################################ # Psi4 computation ################################################################################ WeylScal4::fdOrder = 8 WeylScal4::calc_scalars = "psis" WeylScal4::calc_invariants = "always" ################################################################################ # Psi4 mode decomposition ################################################################################ # Radii are chosen to be evenly spaced in 1/r as that is the variable # extrapolation is performed in Multipole::nradii = 7 Multipole::radius[0] = 100 Multipole::radius[1] = 115 Multipole::radius[2] = 136 Multipole::radius[3] = 167 Multipole::radius[4] = 214 Multipole::radius[5] = 300 Multipole::radius[6] = 500 Multipole::ntheta = 120 Multipole::nphi = 240 Multipole::variables = "WeylScal4::Psi4r{sw=-2 cmplx='WeylScal4::Psi4i' name='psi4'}" Multipole::out_every = 256 Multipole::l_max = 8 Multipole::output_hdf5 = yes # Disable ASCII output to avoid creating a large number of files Multipole::output_ascii = no ################################################################################ # Gauge invariant perturbations of Schwarzschild (ZM-CPM variables) ################################################################################ # WaveExtractCPM::out_every = 256 # WaveExtractCPM::use_carpetinterp2 = no # WaveExtractCPM::calc_when_necessary = no # WaveExtractCPM::verbose = 0 # WaveExtractCPM::maximum_detector_number = 7 # WaveExtractCPM::switch_output_format = 100 # WaveExtractCPM::rsch2_computation = "average Schwarzschild metric" # WaveExtractCPM::l_mode = 8 # WaveExtractCPM::m_mode = 8 # WaveExtractCPM::detector_radius [0] = 100 # WaveExtractCPM::detector_radius [1] = 115 # WaveExtractCPM::detector_radius [2] = 136 # WaveExtractCPM::detector_radius [3] = 167 # WaveExtractCPM::detector_radius [4] = 214 # WaveExtractCPM::detector_radius [5] = 300 # WaveExtractCPM::detector_radius [6] = 500 # WaveExtractCPM::maxntheta = 120 # WaveExtractCPM::maxnphi = 240 # WaveExtractCPM::output_hdf5 = yes # WaveExtractCPM::output_ascii = no # WaveExtractCPM::output_h = yes # WaveExtractCPM::output_Psi = yes ################################################################################ # Apparent Horizons ################################################################################ AHFinderDirect::N_horizons = 3 AHFinderDirect::find_every = 256 AHFinderDirect::output_h_every = 0 AHFinderDirect::max_Newton_iterations__initial = 50 AHFinderDirect::max_Newton_iterations__subsequent = 50 AHFinderDirect::max_allowable_Theta_growth_iterations = 10 AHFinderDirect::max_allowable_Theta_nonshrink_iterations = 10 AHFinderDirect::geometry_interpolator_name = "Lagrange polynomial interpolation" AHFinderDirect::geometry_interpolator_pars = "order=4" AHFinderDirect::surface_interpolator_name = "Lagrange polynomial interpolation" AHFinderDirect::surface_interpolator_pars = "order=4" AHFinderDirect::verbose_level = "physics details" AHFinderDirect::move_origins = yes AHFinderDirect::origin_x [1] = 4.461538461538462 AHFinderDirect::initial_guess__coord_sphere__x_center[1] = 4.461538461538462 AHFinderDirect::initial_guess__coord_sphere__radius [1] = 0.6646153846153846 AHFinderDirect::which_surface_to_store_info [1] = 2 AHFinderDirect::set_mask_for_individual_horizon [1] = no AHFinderDirect::reset_horizon_after_not_finding [1] = no AHFinderDirect::track_origin_from_grid_scalar [1] = yes AHFinderDirect::track_origin_source_x [1] = "PunctureTracker::pt_loc_x[0]" AHFinderDirect::track_origin_source_y [1] = "PunctureTracker::pt_loc_y[0]" AHFinderDirect::track_origin_source_z [1] = "PunctureTracker::pt_loc_z[0]" AHFinderDirect::max_allowable_horizon_radius [1] = 3 AHFinderDirect::origin_x [2] = -5.538461538461538 AHFinderDirect::initial_guess__coord_sphere__x_center[2] = -5.538461538461538 AHFinderDirect::initial_guess__coord_sphere__radius [2] = 0.5353846153846153 AHFinderDirect::which_surface_to_store_info [2] = 3 AHFinderDirect::set_mask_for_individual_horizon [2] = no AHFinderDirect::reset_horizon_after_not_finding [2] = no AHFinderDirect::track_origin_from_grid_scalar [2] = yes AHFinderDirect::track_origin_source_x [2] = "PunctureTracker::pt_loc_x[1]" AHFinderDirect::track_origin_source_y [2] = "PunctureTracker::pt_loc_y[1]" AHFinderDirect::track_origin_source_z [2] = "PunctureTracker::pt_loc_z[1]" AHFinderDirect::max_allowable_horizon_radius [2] = 3 AHFinderDirect::origin_x [3] = 0 AHFinderDirect::find_after_individual [3] = 371902 AHFinderDirect::initial_guess__coord_sphere__x_center[3] = 0 AHFinderDirect::initial_guess__coord_sphere__radius [3] = 1.0 AHFinderDirect::which_surface_to_store_info [3] = 4 AHFinderDirect::set_mask_for_individual_horizon [3] = no AHFinderDirect::max_allowable_horizon_radius [3] = 6 ################################################################################ # Spherical surfaces ################################################################################ SphericalSurface::nsurfaces = 5 SphericalSurface::maxntheta = 66 SphericalSurface::maxnphi = 124 SphericalSurface::verbose = no # Surfaces 0 and 1 are used by PunctureTracker # Horizon 1 SphericalSurface::ntheta [2] = 41 SphericalSurface::nphi [2] = 80 SphericalSurface::nghoststheta [2] = 2 SphericalSurface::nghostsphi [2] = 2 # Horizon 2 SphericalSurface::ntheta [3] = 41 SphericalSurface::nphi [3] = 80 SphericalSurface::nghoststheta [3] = 2 SphericalSurface::nghostsphi [3] = 2 # Common horizon SphericalSurface::ntheta [4] = 41 SphericalSurface::nphi [4] = 80 SphericalSurface::nghoststheta [4] = 2 SphericalSurface::nghostsphi [4] = 2 ################################################################################ # Isolated Horizons ################################################################################ QuasiLocalMeasures::verbose = no QuasiLocalMeasures::veryverbose = no QuasiLocalMeasures::interpolator = "Lagrange polynomial interpolation" QuasiLocalMeasures::interpolator_options = "order=4" QuasiLocalMeasures::spatial_order = 4 QuasiLocalMeasures::num_surfaces = 3 QuasiLocalMeasures::surface_index [0] = 2 QuasiLocalMeasures::surface_index [1] = 3 QuasiLocalMeasures::surface_index [2] = 4 QuasiLocalMeasures::output_vtk_every = 0 ################################################################################ # Correctness checking ################################################################################ Carpet::poison_new_timelevels = no Carpet::check_for_poison = no NaNChecker::check_every = 256 NanChecker::check_after = 0 NaNChecker::report_max = 10 NaNChecker::verbose = "all" NaNChecker::action_if_found = terminate NaNChecker::out_NaNmask = yes NaNChecker::check_vars = " ML_BSSN::ML_log_confac " ################################################################################ # Timers ################################################################################ Carpet::output_timer_tree_every = 1024 Carpet::output_initialise_timer_tree = yes ################################################################################ # Output ################################################################################ IO::out_dir = "@SIMULATION_NAME@" IOScalar::one_file_per_group = yes IOASCII::one_file_per_group = yes IOBasic::outInfo_every = 1 IOBasic::outInfo_reductions = "minimum maximum" IOBasic::outInfo_vars = " ML_BSSN::ML_log_confac Carpet::physical_time_per_hour SystemStatistics::maxrss_mb SystemStatistics::swap_used_mb " IOScalar::outScalar_every = 256 IOScalar::outScalar_reductions = "minimum maximum average" IOScalar::outScalar_vars = "SystemStatistics::process_memory_mb" IOASCII::out0D_every = 256 IOASCII::out0D_vars = " Carpet::timing PunctureTracker::pt_loc QuasiLocalMeasures::qlm_scalars{out_every = 256} " IOASCII::out1D_every = 0 IOASCII::out1D_d = no IOASCII::out1D_x = yes IOASCII::out1D_y = no IOASCII::out1D_z = yes IOASCII::out1D_vars = " ML_BSSN::ML_log_confac ML_BSSN::ML_trace_curv WeylScal4::Psi4r " IOASCII::out2D_every = 0 IOASCII::out2D_vars = " " IOHDF5::out_every = 0 IOHDF5::out_vars = " Grid::Coordinates{out_every=1000000000 refinement_levels={0}} ML_BSSN::ML_log_confac WeylScal4::Psi4r WeylScal4::Psi4i WeylScal4::curvIr{refinement_levels={3 5}} WeylScal4::curvIi{refinement_levels={3 5}} WeylScal4::curvJr{refinement_levels={3 5}} WeylScal4::curvJi{refinement_levels={3 5}} " IOHDF5::out2D_every = 1024 IOHDF5::out2D_vars = " Grid::Coordinates{out_every=1000000000 refinement_levels={0}} ML_BSSN::alpha ML_BSSN::phi ML_BSSN::trK WeylScal4::Psi4r WeylScal4::Psi4i " ################################################################################ # Checkpointing and recovery ################################################################################ CarpetIOHDF5::checkpoint = yes IO::checkpoint_ID = no IO::recover = "autoprobe" IO::out_proc_every = 2 IO::checkpoint_on_terminate = yes IO::checkpoint_dir = "../checkpoints" IO::recover_dir = "../checkpoints" IO::abort_on_io_errors = yes CarpetIOHDF5::open_one_input_file_at_a_time = yes CarpetIOHDF5::compression_level = 0 ################################################################################ # Run termination ################################################################################ TerminationTrigger::max_walltime = @WALLTIME_HOURS@ # Trigger termination 30 minutes before the walltime is reached TerminationTrigger::on_remaining_walltime = 30 TerminationTrigger::output_remtime_every_minutes = 30 TerminationTrigger::termination_from_file = yes TerminationTrigger::termination_file = "terminate.txt" TerminationTrigger::create_termination_file = yes Cactus::terminate = time Cactus::cctk_final_time = 1700.0 -------------- next part -------------- + set -e + cd /system/user/crangano/simulations/GW150914_28/output-0000-active + echo Checking: + pwd + hostname + date + echo Environment: + export CACTUS_NUM_PROCS=1 + CACTUS_NUM_PROCS=1 + export CACTUS_NUM_THREADS=32 + CACTUS_NUM_THREADS=32 + export GMON_OUT_PREFIX=gmon.out + GMON_OUT_PREFIX=gmon.out + export OMP_NUM_THREADS=32 + OMP_NUM_THREADS=32 + sort + env + echo Starting: ++ date +%s + export CACTUS_STARTTIME=1757517004 + CACTUS_STARTTIME=1757517004 + '[' 1 = 1 ']' + '[' 0 -eq 0 ']' + /system/user/crangano/simulations/GW150914_28/SIMFACTORY/exe/cactus_sim -L 3 /system/user/crangano/simulations/GW150914_28/output-0000/GW150914.par -------------------------------------------------------------------------- No OpenFabrics connection schemes reported that they were able to be used on a specific port. As such, the openib BTL (OpenFabrics support) will be disabled for this port. Local host: panther Local device: bnxt_re0 Local port: 1 CPCs attempted: rdmacm, udcm -------------------------------------------------------------------------- -------------------------------------------------------------------------- Open MPI failed an OFI Libfabric library call (fi_endpoint). This is highly unusual; your job may behave unpredictably (and/or abort) after this. Local host: panther Location: mtl_ofi_component.c:513 Error: Invalid argument (22) -------------------------------------------------------------------------- WARNING level 0 from host panther process 0 in thorn CoordinatesSymmetry, file /system/user/crangano/einstein_toolkit/Cactus/configs/sim/build/CoordinatesSymmetry/apply.c:665: -> Cannot apply symmetry boundary zones in the lower x direction, since there seem to be more symmetry zones than interior zones cactus_sim: /system/user/crangano/einstein_toolkit/Cactus/configs/sim/build/Carpet/helpers.cc:275: int Carpet::Abort(const cGH*, int): Assertion `0' failed. Rank 0 with PID 3271897 received signal 6 Writing backtrace to GW150914_28/backtrace.0.txt [panther:3271901] 1 more process has sent help message help-mpi-btl-openib-cpc-base.txt / no cpcs for port [panther:3271901] Set MCA parameter "orte_base_help_aggregate" to 0 to see all help / error messages /system/user/crangano/simulations/GW150914_28/output-0000/SIMFACTORY/RunScript: line 36: 3271897 Aborted (core dumped) /system/user/crangano/simulations/GW150914_28/SIMFACTORY/exe/cactus_sim -L 3 /system/user/crangano/simulations/GW150914_28/output-0000/GW150914.par -------------- next part -------------- Simulation name: GW150914_28 Running simulation GW150914_28 Preparing: Checking: /system/user/crangano/simulations/GW150914_28/output-0000-active panther Wed Sep 10 05:10:04 PM CEST 2025 Environment: Starting: [1757517005.372585993] panther:pid3271897.cactus_sim: Unable to alloc recv buffer MR on bnxt_re0: Cannot allocate memory [1757517005.372630518] panther:pid3271897.cactus_sim: Unable to allocate UD recv buffer pool [1757517005.376325443] panther:pid3271897.cactus_sim: Unable to alloc recv buffer MR on bnxt_re0: Cannot allocate memory [1757517005.376328448] panther:pid3271897.cactus_sim: Unable to allocate UD recv buffer pool [1757517005.378921438] panther:pid3271897.cactus_sim: Unable to alloc recv buffer MR on bnxt_re0: Cannot allocate memory [1757517005.378924854] panther:pid3271897.cactus_sim: Unable to allocate UD recv buffer pool [1757517005.439461572] panther:pid3271897.cactus_sim: Unable to alloc recv buffer MR on bnxt_re0: Cannot allocate memory [1757517005.439464628] panther:pid3271897.cactus_sim: Unable to allocate UD recv buffer pool INFO (Cactus): Increased logging level from 0 to 3 -------------------------------------------------------------------------------- 10 1 0101 ************************ 01 1010 10 The Cactus Code V4.18.0 1010 1101 011 www.cactuscode.org 1001 100101 ************************ 00010101 100011 (c) Copyright The Authors 0100 GNU Licensed. No Warranty 0101 -------------------------------------------------------------------------------- Cactus version: 4.18.0 Compile date: Sep 10 2025 (16:44:10) Run date: Sep 10 2025 (17:10:05+0200) Run host: panther (pid=3271897) Working directory: /system/user/crangano/simulations/GW150914_28/output-0000 Executable: /system/user/crangano/simulations/GW150914_28/SIMFACTORY/exe/cactus_sim Parameter file: /system/user/crangano/simulations/GW150914_28/output-0000/GW150914.par -------------------------------------------------------------------------------- Activating thorn Cactus...Success -> active implementation Cactus Activation requested for --->ADMBase ML_ADMConstraints ADMCoupling ADMMacros AEILocalInterp AHFinderDirect Boundary Carpet CarpetIOASCII CarpetIOBasic CarpetIOHDF5 CarpetIOScalar CarpetInterp CarpetInterp2 CarpetLib CarpetReduce CarpetRegrid2 CarpetTracker CartGrid3D CoordBase CoordGauge Coordinates CoordinatesSymmetry Formaline GlobalDerivative hwloc IOUtil InitBase Interpolate2 QuasiLocalMeasures LocalInterp LoopControl MoL NaNChecker PunctureTracker Slab SpaceMask SphericalSurface StaticConformal SummationByParts SymBase SystemStatistics SystemTopology TerminationTrigger TensorTypes Time TmunuBase TwoPunctures Vectors ML_BSSN ML_BSSN_Helper NewRad GenericFD WeylScal4 Multipole ADMDerivatives<--- Thorn AHFinderDirect requests automatic activation of HDF5 Thorn Carpet requests automatic activation of MPI Thorn Carpet requests automatic activation of Timers Thorn CarpetLib requests automatic activation of CycleClock Thorn Coordinates requests automatic activation of TGRtensor Thorn Formaline requests automatic activation of pthreads Thorn hwloc requests automatic activation of zlib Thorn Interpolate2 requests automatic activation of LAPACK Thorn QuasiLocalMeasures requests automatic activation of Fortran Thorn TwoPunctures requests automatic activation of GSL Thorn LAPACK requests automatic activation of BLAS Activating thorn ADMBase...Success -> active implementation ADMBase Activating thorn ADMCoupling...Success -> active implementation ADMCoupling Activating thorn ADMDerivatives...Success -> active implementation ADMDerivatives Activating thorn ADMMacros...Success -> active implementation ADMMacros Activating thorn AEILocalInterp...Success -> active implementation AEILocalInterp Activating thorn AHFinderDirect...Success -> active implementation AHFinderDirect Activating thorn BLAS...Success -> active implementation BLAS Activating thorn Boundary...Success -> active implementation boundary Activating thorn Carpet...Success -> active implementation Driver Activating thorn CarpetInterp...Success -> active implementation interp Activating thorn CarpetInterp2...Success -> active implementation interp2 Activating thorn CarpetIOASCII...Success -> active implementation IOASCII Activating thorn CarpetIOBasic...Success -> active implementation IOBasic Activating thorn CarpetIOHDF5...Success -> active implementation IOHDF5 Activating thorn CarpetIOScalar...Success -> active implementation IOScalar Activating thorn CarpetLib...Success -> active implementation CarpetLib Activating thorn CarpetReduce...Success -> active implementation reduce Activating thorn CarpetRegrid2...Success -> active implementation CarpetRegrid2 Activating thorn CarpetTracker...Success -> active implementation CarpetTracker Activating thorn CartGrid3D...Success -> active implementation grid Activating thorn CoordBase...Success -> active implementation CoordBase Activating thorn CoordGauge...Success -> active implementation CoordGauge Activating thorn Coordinates...Success -> active implementation Coordinates Activating thorn CoordinatesSymmetry...Success -> active implementation CoordinatesSymmetry Activating thorn CycleClock...Success -> active implementation CycleClock Activating thorn Formaline...Success -> active implementation Formaline Activating thorn Fortran...Success -> active implementation Fortran Activating thorn GenericFD...Success -> active implementation GenericFD Activating thorn GlobalDerivative...Success -> active implementation GlobalDerivative Activating thorn GSL...Success -> active implementation GSL Activating thorn HDF5...Success -> active implementation HDF5 Activating thorn hwloc...Success -> active implementation hwloc Activating thorn InitBase...Success -> active implementation InitBase Activating thorn Interpolate2...Success -> active implementation Interpolate Activating thorn IOUtil...Success -> active implementation IO Activating thorn LAPACK...Success -> active implementation LAPACK Activating thorn LocalInterp...Success -> active implementation LocalInterp Activating thorn LoopControl...Success -> active implementation LoopControl Activating thorn ML_ADMConstraints...Success -> active implementation ML_ADMConstraints Activating thorn ML_BSSN...Success -> active implementation ML_BSSN Activating thorn ML_BSSN_Helper...Success -> active implementation ML_BSSN_Helper Activating thorn MoL...Success -> active implementation MethodOfLines Activating thorn MPI...Success -> active implementation MPI Activating thorn Multipole...Success -> active implementation multipole Activating thorn NaNChecker...Success -> active implementation NaNChecker Activating thorn NewRad...Success -> active implementation NewRad Activating thorn pthreads...Success -> active implementation PTHREADS Activating thorn PunctureTracker...Success -> active implementation PunctureTracker Activating thorn QuasiLocalMeasures...Success -> active implementation QuasiLocalMeasures Activating thorn Slab...Success -> active implementation Slab Activating thorn SpaceMask...Success -> active implementation SpaceMask Activating thorn SphericalSurface...Success -> active implementation SphericalSurface Activating thorn StaticConformal...Success -> active implementation StaticConformal Activating thorn SummationByParts...Success -> active implementation SummationByParts Activating thorn SymBase...Success -> active implementation SymBase Activating thorn SystemStatistics...Success -> active implementation SystemStatistics Activating thorn SystemTopology...Success -> active implementation SystemTopology Activating thorn TensorTypes...Success -> active implementation TensorTypes Activating thorn TerminationTrigger...Success -> active implementation TerminationTrigger Activating thorn TGRtensor...Success -> active implementation TGRtensor Activating thorn Time...Success -> active implementation time Activating thorn Timers...Success -> active implementation Timers Activating thorn TmunuBase...Success -> active implementation TmunuBase Activating thorn TwoPunctures...Success -> active implementation TwoPunctures Activating thorn Vectors...Success -> active implementation Vectors Activating thorn WeylScal4...Success -> active implementation WeylScal4 Activating thorn zlib...Success -> active implementation zlib INFO (IOUtil): Recovery directory '../checkpoints' doesn't exist -------------------------------------------------------------------------------- if (recover initial data) Recover parameters endif Startup routines [CCTK_STARTUP] Carpet::MultiModel_Startup: Multi-model Startup routine Coordinates::Coordinates_ChoosePatchSystem: [meta] Choose patch system CycleClock::CycleClock_Setup: Set up CycleClock LoopControl::LC_setup: Set up LoopControl ML_BSSN_Helper::ML_BSSN_SetGroupTags: [meta] Set checkpointing and prolongation group tags GROUP hwloc_startup: hwloc startup group hwloc::hwloc_version: Output hwloc version Timers::Timer_Startup: Prepare hierarchical timers CarpetInterp::CarpetInterpStartup: Startup routine CarpetReduce::CarpetReduceStartup: Startup routine CartGrid3D::SymmetryStartup: Register GH Extension for GridSymmetry CoordBase::CoordBase_Startup: Register a GH extension to store the coordinate system handles SystemTopology::ST_system_topology: Output and/or modify system topology and hardware locality Carpet::Driver_Startup: Startup routine Formaline::Formaline_PrintIDs: [meta] Print the build and simulation ids IOUtil::IOUtil_Startup: Startup routine LocalInterp::LocalInterp_Startup: register LocalInterp's interpolation operators CarpetIOHDF5::CarpetIOHDF5_Startup: Startup routine ML_ADMConstraints::ML_ADMConstraints_Startup: [meta] create banner ML_BSSN::ML_BSSN_Startup: [meta] create banner ML_BSSN_Helper::ML_BSSN_RegisterSlicing: [meta] Register slicing CarpetIOScalar::CarpetIOScalarStartup: [global] Startup routine MoL::MoL_Startup: Startup banner Slab::Slab_InitMPIDatatypes: Create MPI datatypes for complex variables in C SymBase::SymBase_Startup: Register GH Extension for SymBase CarpetIOBasic::CarpetIOBasicStartup: [global] Startup routine TerminationTrigger::TerminationTrigger_StartSignalHandler: Start signal handler AEILocalInterp::AEILocalInterp_U_Startup: register CCTK_InterpLocalUniform() interpolation operators Vectors::Vectors_Startup: Print startup message WeylScal4::WeylScal4_Startup: [meta] create banner CarpetIOASCII::CarpetIOASCIIStartup: [global] Startup routine Startup routines which need an existing grid hierarchy [CCTK_WRAGH] ADMBase::Einstein_InitSymBound: [global] Set up GF symmetries Boundary::Boundary_RegisterBCs: [global] Register boundary conditions that this thorn provides CarpetRegrid2::CarpetRegrid2_Initialise: [global] Initialise locations of refined regions CartGrid3D::RegisterCartGrid3DCoords: [meta] Register coordinates for the Cartesian grid CoordGauge::Einstein_ActivateSlicing: Initialize slicing, setup priorities for mixed slicings CoordGauge::Einstein_SetNextSlicing: Identify the slicing for the next iteration Formaline::Formaline_OutputSource: [meta] Output Cactus source tree Formaline::Formaline_RegisterWarnings: [meta] Register to receive warnings and info messages from the flesh Formaline::Formaline_AnnounceInitial: [global] Put some meta information about the current run into permanent storage ML_BSSN_Helper::ML_BSSN_ParamCompat: [meta] Handle parameter backward compatibility MoL::MoL_SetupIndexArrays: Set up the MoL bookkeeping index arrays MoL::MoL_SetScheduleStatus: [global] Set the flag so it is ok to register with MoL TmunuBase::TmunuBase_SetStressEnergyState: [global] Set the stress_energy_state variable GROUP MoL_Register: The group where physics thorns register variables with MoL ML_ADMConstraints::ML_ADMConstraints_RegisterVars: [meta] Register Variables for MoL ML_BSSN::ML_BSSN_RegisterVars: [meta] Register Variables for MoL ML_BSSN_Helper::ML_BSSN_RegisterConstrained: [meta] Register ADMBase variables as constrained WeylScal4::WeylScal4_RegisterVars: [meta] Register Variables for MoL Slab::Slab_InitTimers: Initialise timers GROUP SymBase_Wrapper: Wrapper group for SymBase GROUP SymmetryRegister: Register your symmetries here CartGrid3D::CartGrid3D_RegisterSymmetryBoundaries: [meta] Register symmetry boundaries Coordinates::Coordinates_RegisterSymmetry: [meta] Register inter-patch boundaries as symmetries ML_ADMConstraints::ML_ADMConstraints_RegisterSymmetries: [meta] register symmetries ML_BSSN::ML_BSSN_RegisterSymmetries: [meta] register symmetries WeylScal4::WeylScal4_RegisterSymmetries: [meta] register symmetries SymBase::SymBase_Statistics: Print symmetry boundary face descriptions MoL::MoL_ReportNumberVariables: [meta] Report how many of each type of variable there are Parameter checking routines [CCTK_PARAMCHECK] ADMBase::ADMBase_ParamCheck: [global] Check consistency of parameters Boundary::Boundary_Check: Check dimension of grid variables Carpet::CarpetParamCheck: Parameter checking routine CarpetLib::CarpetLib_test_prolongate_3d_rf2: [global] Test prolongation operators CarpetRegrid2::CarpetRegrid2_ParamCheck: Check parameters CartGrid3D::ParamCheck_CartGrid3D: Check coordinates for CartGrid3D Coordinates::Coordinates_ParamCheck: Check thorn parameters for consistency. Fortran::CheckFortranParameters: Test whether Fortran parameters work correctly GlobalDerivative::GlobalDeriv_ParamCheck: [global] Check parameters ML_BSSN_Helper::ML_BSSN_ParamCheck: [meta] Check parameters MoL::MoL_ParamCheck: Basic parameter checking Multipole::Multipole_ParamCheck: [global] Check Multipole parameters QuasiLocalMeasures::qlm_paramcheck: [global] Check quasi-local parameter settings SphericalSurface::SphericalSurface_ParamCheck: [global] Check that all surface names are unique TerminationTrigger::TerminationTrigger_ParamCheck: Check consitency of parameters TmunuBase::TmunuBase_ParamCheck: [global] Check that no deprecated parameters are used. TwoPunctures::TwoPunctures_ParamCheck: Check parameters and thorn needs Vectors::Vectors_Test: Run correctness tests. Initialisation if (NOT (recover initial data AND recovery_mode is 'strict')) [CCTK_PREREGRIDINITIAL] Set up grid hierarchy [CCTK_POSTREGRIDINITIAL] AHFinderDirect::AHFinderDirect_maybe_do_masks: set mask(s) based on apparent horizon position(s) CartGrid3D::SpatialCoordinates: Set Coordinates after regridding GROUP MaskBase_SetupMask: Set up the weight function GROUP MaskBase_SetupMaskAll: Set up the weight function CarpetReduce::MaskBase_AllocateMask: [global] Allocate the weight function CarpetReduce::MaskBase_InitMask: [global] [loop-local] Initialise the weight function GROUP SetupIMaskInternal: Set up the integer weight function (schedule other routines in here) CarpetReduce::CoordBase_SetupMask: [global] [loop-local] Set up the outer boundaries of the weight function CarpetReduce::CarpetMaskSetup: [global] [loop-singlemap] Set up the weight function for the restriction regions GROUP SetupIMask: Set up the integer weight function (schedule other routines in here) CarpetReduce::MaskBase_SetMask: [global] [loop-local] Set the weight function GROUP SetupMask: Set up the real weight function (schedule other routines in here) CarpetReduce::MaskBase_TestMask: [global] Test the weight function GROUP MultiPatch_SpatialCoordinates: Determine the global coordinates of each gridpoint. Coordinates::Coordinates_SetGlobalCoords: [local] Determine the global coordinates of each gridpoint. Coordinates::Coordinates_SetJacobian: [local] Fill in the Jacobian matrices for Thornburg04 coordinates. Coordinates::Coordinates_SetVolumeForm: [local] Set weight mask (for Carpet reduction operators) GROUP MoL_PseudoEvolutionBoundaries: Apply boundary conditions to pseudo-evolved quantities GROUP ML_ADMConstraints_evaluate_bc_group: ML_ADMConstraints_evaluate ML_ADMConstraints::ML_ADMConstraints_evaluate_SelectBCs: [level] ML_ADMConstraints_evaluate_SelectBCs GROUP ML_ADMConstraints_evaluate_ApplyBCs: Apply BCs for groups set in ML_ADMConstraints_evaluate GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions CoordinatesSymmetry::CoordinatesSymmetry_Apply_FirstPass: Apply multipatch symmetries Interpolate2::Interpolate2ApplyBC: [level] Apply interpolating inter-patch boundaries CoordinatesSymmetry::CoordinatesSymmetry_Apply_SecondPass: Apply multipatch symmetries for remaining points not filled by interpatch interpolation Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP ML_BSSN_ConstraintsEverywhere_bc_group: ML_BSSN_ConstraintsEverywhere ML_BSSN::ML_BSSN_ConstraintsEverywhere_SelectBCs: [level] ML_BSSN_ConstraintsEverywhere_SelectBCs GROUP ML_BSSN_ConstraintsEverywhere_ApplyBCs: Apply BCs for groups set in ML_BSSN_ConstraintsEverywhere GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions CoordinatesSymmetry::CoordinatesSymmetry_Apply_FirstPass: Apply multipatch symmetries Interpolate2::Interpolate2ApplyBC: [level] Apply interpolating inter-patch boundaries CoordinatesSymmetry::CoordinatesSymmetry_Apply_SecondPass: Apply multipatch symmetries for remaining points not filled by interpatch interpolation Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP ML_BSSN_ConstraintsInterior_bc_group: ML_BSSN_ConstraintsInterior ML_BSSN::ML_BSSN_ConstraintsInterior_SelectBCs: [level] ML_BSSN_ConstraintsInterior_SelectBCs GROUP ML_BSSN_ConstraintsInterior_ApplyBCs: Apply BCs for groups set in ML_BSSN_ConstraintsInterior GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions CoordinatesSymmetry::CoordinatesSymmetry_Apply_FirstPass: Apply multipatch symmetries Interpolate2::Interpolate2ApplyBC: [level] Apply interpolating inter-patch boundaries CoordinatesSymmetry::CoordinatesSymmetry_Apply_SecondPass: Apply multipatch symmetries for remaining points not filled by interpatch interpolation Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP WeylScal4_psis_calc_Nth_bc_group: WeylScal4_psis_calc_Nth WeylScal4::WeylScal4_psis_calc_Nth_SelectBCs: [level] WeylScal4_psis_calc_Nth_SelectBCs GROUP WeylScal4_psis_calc_Nth_ApplyBCs: Apply BCs for groups set in WeylScal4_psis_calc_Nth GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions CoordinatesSymmetry::CoordinatesSymmetry_Apply_FirstPass: Apply multipatch symmetries Interpolate2::Interpolate2ApplyBC: [level] Apply interpolating inter-patch boundaries CoordinatesSymmetry::CoordinatesSymmetry_Apply_SecondPass: Apply multipatch symmetries for remaining points not filled by interpatch interpolation Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions [CCTK_BASEGRID] ADMBase::ADMBase_SetShiftStateOn: Set the shift_state variable to 1 ADMBase::ADMBase_SetDtLapseStateOn: Set the dtlapse_state variable to 1 ADMBase::ADMBase_SetDtShiftStateOn: Set the dtshift_state variable to 1 ADMMacros::ADMMacros_SetLocalSpatialOrder: Initialize the local_spatial_order CartGrid3D::SpatialSpacings: [singlemap] Set up ranges for spatial 3D Cartesian coordinates (on all maps) CartGrid3D::SpatialCoordinates: Set up spatial 3D Cartesian coordinates on the GH SphericalSurface::SphericalSurface_SetupRes: [global] [loop-local] Set surface resolution automatically AHFinderDirect::AHFinderDirect_setup: [global] setup data structures GROUP MultiPatch_SpatialCoordinates: Determine the global coordinates of each gridpoint. Coordinates::Coordinates_SetGlobalCoords: [local] Determine the global coordinates of each gridpoint. Coordinates::Coordinates_SetJacobian: [local] Fill in the Jacobian matrices for Thornburg04 coordinates. Coordinates::Coordinates_SetVolumeForm: [local] Set weight mask (for Carpet reduction operators) Interpolate2::Interpolate2Init: Initialise interpolating inter-patch boundaries GROUP Interpolate2Test: Test interpolating inter-patch boundaries Interpolate2::Interpolate2TestInit: Test: Initialise test grid function Interpolate2::Interpolate2TestSelectBCs: [level] Test: Interpolate test grid function GROUP Interpolate2TestApplyBCs: Test: Interpolate test grid function GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions CoordinatesSymmetry::CoordinatesSymmetry_Apply_FirstPass: Apply multipatch symmetries Interpolate2::Interpolate2ApplyBC: [level] Apply interpolating inter-patch boundaries CoordinatesSymmetry::CoordinatesSymmetry_Apply_SecondPass: Apply multipatch symmetries for remaining points not filled by interpatch interpolation Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions Interpolate2::Interpolate2TestCheck: Test: Check test grid function ML_ADMConstraints::ML_ADMConstraints_CheckBoundaries: [meta] check boundaries treatment ML_BSSN::ML_BSSN_CheckBoundaries: [meta] check boundaries treatment NaNChecker::NaNChecker_ResetCounter: [global] Reset the NaNChecker::NaNsFound counter SphericalSurface::SphericalSurface_Setup: [global] Calculate surface coordinate descriptors GROUP MaskBase_SetupMask: Set up the weight function GROUP MaskBase_SetupMaskAll: Set up the weight function CarpetReduce::MaskBase_AllocateMask: [global] Allocate the weight function CarpetReduce::MaskBase_InitMask: [global] [loop-local] Initialise the weight function GROUP SetupIMaskInternal: Set up the integer weight function (schedule other routines in here) CarpetReduce::CoordBase_SetupMask: [global] [loop-local] Set up the outer boundaries of the weight function CarpetReduce::CarpetMaskSetup: [global] [loop-singlemap] Set up the weight function for the restriction regions GROUP SetupIMask: Set up the integer weight function (schedule other routines in here) CarpetReduce::MaskBase_SetMask: [global] [loop-local] Set the weight function GROUP SetupMask: Set up the real weight function (schedule other routines in here) CarpetReduce::MaskBase_TestMask: [global] Test the weight function SphericalSurface::SphericalSurface_Set: [global] Set surface radii to be used for initial setup in other thorns GROUP SphericalSurface_HasBeenSet: Set the spherical surfaces before this group, and use it afterwards SphericalSurface::SphericalSurface_CheckState: [global] Test the state of the spherical surfaces SummationByParts::SBP_SetNormMask: Setup the mask for the calculation of the norm SymBase::SymBase_Check: Check whether the driver set up the grid consistently TerminationTrigger::TerminationTrigger_ResetTrigger: Clear trigger state TerminationTrigger::TerminationTrigger_StartTimer: Start timer TerminationTrigger::TerminationTrigger_CreateFile: Create termination file Time::Time_Initialise: [global] Initialise Time variables Time::TemporalSpacings: [singlemap] Set fixed timestep WeylScal4::WeylScal4_CheckBoundaries: [meta] check boundaries treatment [CCTK_INITIAL] StaticConformal::StaticConformal_InitialiseState: Set the conformal_state variable to 0 GROUP ADMBase_InitialData: Schedule group for calculating ADM initial data GROUP TwoPunctures_Group: TwoPunctures initial data group TwoPunctures::TwoPunctures: Create puncture black hole initial data TwoPunctures::TwoPunctures_Metadata: [global] Output TwoPunctures metadata GROUP ADMBase_InitialGauge: Schedule group for the ADM initial gauge condition ADMBase::ADMBase_ShiftZero: Set the shift to 0 at all points ADMBase::ADMBase_DtLapseZero: Set the dtlapse to 0 at all points ADMBase::ADMBase_DtShiftZero: Set the dtshift to 0 at all points CarpetIOASCII::CarpetIOASCIIInit: [global] Initialisation routine CarpetIOBasic::CarpetIOBasicInit: [global] Initialisation routine CarpetIOHDF5::CarpetIOHDF5_Init: [global] Initialisation routine CarpetIOScalar::CarpetIOScalarInit: [global] Initialisation routine GROUP ADMBase_PostInitial: Schedule group for modifying the ADM initial data, such as e.g. adding noise ML_BSSN::ML_BSSN_InitialADMBase1Everywhere: ML_BSSN_InitialADMBase1Everywhere ML_BSSN::ML_BSSN_InitialADMBase2Interior: ML_BSSN_InitialADMBase2Interior MoL::MoL_StartLoop: [level] Initialise the step size control PunctureTracker::PunctureTracker_Init: [global] Calculate initial location of punctures QuasiLocalMeasures::qlm_init: [global] Initialise quasi-local calculations ML_BSSN_Helper::ML_BSSN_ExtrapolateGammas: Extrapolate Gammas and time derivatives of lapse and shift [CCTK_POSTINITIAL] CarpetIOHDF5::CarpetIOHDF5_CloseFiles: [global] Close all filereader input files GROUP MoL_PostStepModify: The group for physics thorns to schedule enforcing constraints ML_BSSN::ML_BSSN_EnforceEverywhere: ML_BSSN_EnforceEverywhere GROUP MoL_PostStep: Ensure that everything is correct after the initial data have been set up ML_BSSN::ML_BSSN_SelectBoundConds: [level] select boundary conditions ML_ADMConstraints::ML_ADMConstraints_SelectBoundConds: [level] select boundary conditions GROUP ML_ADMConstraints_ApplyBCs: Apply boundary conditions controlled by thorn Boundary GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions CoordinatesSymmetry::CoordinatesSymmetry_Apply_FirstPass: Apply multipatch symmetries Interpolate2::Interpolate2ApplyBC: [level] Apply interpolating inter-patch boundaries CoordinatesSymmetry::CoordinatesSymmetry_Apply_SecondPass: Apply multipatch symmetries for remaining points not filled by interpatch interpolation Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP ML_BSSN_ApplyBCs: Apply boundary conditions controlled by thorn Boundary GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions CoordinatesSymmetry::CoordinatesSymmetry_Apply_FirstPass: Apply multipatch symmetries Interpolate2::Interpolate2ApplyBC: [level] Apply interpolating inter-patch boundaries CoordinatesSymmetry::CoordinatesSymmetry_Apply_SecondPass: Apply multipatch symmetries for remaining points not filled by interpatch interpolation Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions ML_BSSN::ML_BSSN_ADMBaseInterior: ML_BSSN_ADMBaseInterior ML_BSSN::ML_BSSN_ADMBaseBoundaryScalar: ML_BSSN_ADMBaseBoundaryScalar ML_BSSN::ML_BSSN_ADMBaseEverywhere: ML_BSSN_ADMBaseEverywhere ML_BSSN_Helper::ML_BSSN_ADMBase_SelectBCs: [level] Select boundary conditions for ADMBase variables GROUP ML_BSSN_ADMBase_ApplyBCs: Apply boundary conditions to ADMBase variables GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions CoordinatesSymmetry::CoordinatesSymmetry_Apply_FirstPass: Apply multipatch symmetries Interpolate2::Interpolate2ApplyBC: [level] Apply interpolating inter-patch boundaries CoordinatesSymmetry::CoordinatesSymmetry_Apply_SecondPass: Apply multipatch symmetries for remaining points not filled by interpatch interpolation Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP ADMBase_SetADMVars: Set the ADM variables before this group, and use them afterwards WeylScal4::WeylScal4_SelectBoundConds: [level] select boundary conditions GROUP WeylScal4_ApplyBCs: Apply boundary conditions controlled by thorn Boundary GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions CoordinatesSymmetry::CoordinatesSymmetry_Apply_FirstPass: Apply multipatch symmetries Interpolate2::Interpolate2ApplyBC: [level] Apply interpolating inter-patch boundaries CoordinatesSymmetry::CoordinatesSymmetry_Apply_SecondPass: Apply multipatch symmetries for remaining points not filled by interpatch interpolation Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP MoL_PseudoEvolution: Calculate pseudo-evolved quantities GROUP ADMBase_SetADMVars: Set the ADM variables before this group, and use them afterwards GROUP ML_ADMConstraints_evaluate_group: ML_ADMConstraints_evaluate ML_ADMConstraints::ML_ADMConstraints_evaluate: ML_ADMConstraints_evaluate GROUP ML_ADMConstraints_evaluate_bc_group: ML_ADMConstraints_evaluate ML_ADMConstraints::ML_ADMConstraints_evaluate_SelectBCs: [level] ML_ADMConstraints_evaluate_SelectBCs GROUP ML_ADMConstraints_evaluate_ApplyBCs: Apply BCs for groups set in ML_ADMConstraints_evaluate GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions CoordinatesSymmetry::CoordinatesSymmetry_Apply_FirstPass: Apply multipatch symmetries Interpolate2::Interpolate2ApplyBC: [level] Apply interpolating inter-patch boundaries CoordinatesSymmetry::CoordinatesSymmetry_Apply_SecondPass: Apply multipatch symmetries for remaining points not filled by interpatch interpolation Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP ML_BSSN_ConstraintsEverywhere_group: ML_BSSN_ConstraintsEverywhere ML_BSSN::ML_BSSN_ConstraintsEverywhere: ML_BSSN_ConstraintsEverywhere GROUP ML_BSSN_ConstraintsEverywhere_bc_group: ML_BSSN_ConstraintsEverywhere ML_BSSN::ML_BSSN_ConstraintsEverywhere_SelectBCs: [level] ML_BSSN_ConstraintsEverywhere_SelectBCs GROUP ML_BSSN_ConstraintsEverywhere_ApplyBCs: Apply BCs for groups set in ML_BSSN_ConstraintsEverywhere GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions CoordinatesSymmetry::CoordinatesSymmetry_Apply_FirstPass: Apply multipatch symmetries Interpolate2::Interpolate2ApplyBC: [level] Apply interpolating inter-patch boundaries CoordinatesSymmetry::CoordinatesSymmetry_Apply_SecondPass: Apply multipatch symmetries for remaining points not filled by interpatch interpolation Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP ML_BSSN_ConstraintsInterior_group: ML_BSSN_ConstraintsInterior ML_BSSN::ML_BSSN_ConstraintsInterior: ML_BSSN_ConstraintsInterior GROUP ML_BSSN_ConstraintsInterior_bc_group: ML_BSSN_ConstraintsInterior ML_BSSN::ML_BSSN_ConstraintsInterior_SelectBCs: [level] ML_BSSN_ConstraintsInterior_SelectBCs GROUP ML_BSSN_ConstraintsInterior_ApplyBCs: Apply BCs for groups set in ML_BSSN_ConstraintsInterior GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions CoordinatesSymmetry::CoordinatesSymmetry_Apply_FirstPass: Apply multipatch symmetries Interpolate2::Interpolate2ApplyBC: [level] Apply interpolating inter-patch boundaries CoordinatesSymmetry::CoordinatesSymmetry_Apply_SecondPass: Apply multipatch symmetries for remaining points not filled by interpatch interpolation Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP WeylScal4_psis_calc_Nth_group: WeylScal4_psis_calc_Nth WeylScal4::WeylScal4_psis_calc_Nth: WeylScal4_psis_calc_Nth GROUP WeylScal4_psis_calc_Nth_bc_group: WeylScal4_psis_calc_Nth WeylScal4::WeylScal4_psis_calc_Nth_SelectBCs: [level] WeylScal4_psis_calc_Nth_SelectBCs GROUP WeylScal4_psis_calc_Nth_ApplyBCs: Apply BCs for groups set in WeylScal4_psis_calc_Nth GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions CoordinatesSymmetry::CoordinatesSymmetry_Apply_FirstPass: Apply multipatch symmetries Interpolate2::Interpolate2ApplyBC: [level] Apply interpolating inter-patch boundaries CoordinatesSymmetry::CoordinatesSymmetry_Apply_SecondPass: Apply multipatch symmetries for remaining points not filled by interpatch interpolation Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP WeylScal4_invars_calc_Nth_group: WeylScal4_invars_calc_Nth WeylScal4::WeylScal4_invars_calc_Nth: WeylScal4_invars_calc_Nth SummationByParts::SBP_CheckGridSizes: Check grid sizes and ghost zones Initialise finer grids recursively Restrict from finer grids [CCTK_POSTRESTRICTINITIAL] GROUP MoL_PostStep: Ensure that everything is correct after restriction ML_BSSN::ML_BSSN_SelectBoundConds: [level] select boundary conditions ML_ADMConstraints::ML_ADMConstraints_SelectBoundConds: [level] select boundary conditions GROUP ML_ADMConstraints_ApplyBCs: Apply boundary conditions controlled by thorn Boundary GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions CoordinatesSymmetry::CoordinatesSymmetry_Apply_FirstPass: Apply multipatch symmetries Interpolate2::Interpolate2ApplyBC: [level] Apply interpolating inter-patch boundaries CoordinatesSymmetry::CoordinatesSymmetry_Apply_SecondPass: Apply multipatch symmetries for remaining points not filled by interpatch interpolation Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP ML_BSSN_ApplyBCs: Apply boundary conditions controlled by thorn Boundary GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions CoordinatesSymmetry::CoordinatesSymmetry_Apply_FirstPass: Apply multipatch symmetries Interpolate2::Interpolate2ApplyBC: [level] Apply interpolating inter-patch boundaries CoordinatesSymmetry::CoordinatesSymmetry_Apply_SecondPass: Apply multipatch symmetries for remaining points not filled by interpatch interpolation Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions ML_BSSN::ML_BSSN_ADMBaseInterior: ML_BSSN_ADMBaseInterior ML_BSSN::ML_BSSN_ADMBaseBoundaryScalar: ML_BSSN_ADMBaseBoundaryScalar ML_BSSN::ML_BSSN_ADMBaseEverywhere: ML_BSSN_ADMBaseEverywhere ML_BSSN_Helper::ML_BSSN_ADMBase_SelectBCs: [level] Select boundary conditions for ADMBase variables GROUP ML_BSSN_ADMBase_ApplyBCs: Apply boundary conditions to ADMBase variables GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions CoordinatesSymmetry::CoordinatesSymmetry_Apply_FirstPass: Apply multipatch symmetries Interpolate2::Interpolate2ApplyBC: [level] Apply interpolating inter-patch boundaries CoordinatesSymmetry::CoordinatesSymmetry_Apply_SecondPass: Apply multipatch symmetries for remaining points not filled by interpatch interpolation Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP ADMBase_SetADMVars: Set the ADM variables before this group, and use them afterwards WeylScal4::WeylScal4_SelectBoundConds: [level] select boundary conditions GROUP WeylScal4_ApplyBCs: Apply boundary conditions controlled by thorn Boundary GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions CoordinatesSymmetry::CoordinatesSymmetry_Apply_FirstPass: Apply multipatch symmetries Interpolate2::Interpolate2ApplyBC: [level] Apply interpolating inter-patch boundaries CoordinatesSymmetry::CoordinatesSymmetry_Apply_SecondPass: Apply multipatch symmetries for remaining points not filled by interpatch interpolation Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP MoL_PseudoEvolutionBoundaries: Apply boundary conditions to pseudo-evolved quantities GROUP ML_ADMConstraints_evaluate_bc_group: ML_ADMConstraints_evaluate ML_ADMConstraints::ML_ADMConstraints_evaluate_SelectBCs: [level] ML_ADMConstraints_evaluate_SelectBCs GROUP ML_ADMConstraints_evaluate_ApplyBCs: Apply BCs for groups set in ML_ADMConstraints_evaluate GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions CoordinatesSymmetry::CoordinatesSymmetry_Apply_FirstPass: Apply multipatch symmetries Interpolate2::Interpolate2ApplyBC: [level] Apply interpolating inter-patch boundaries CoordinatesSymmetry::CoordinatesSymmetry_Apply_SecondPass: Apply multipatch symmetries for remaining points not filled by interpatch interpolation Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP ML_BSSN_ConstraintsEverywhere_bc_group: ML_BSSN_ConstraintsEverywhere ML_BSSN::ML_BSSN_ConstraintsEverywhere_SelectBCs: [level] ML_BSSN_ConstraintsEverywhere_SelectBCs GROUP ML_BSSN_ConstraintsEverywhere_ApplyBCs: Apply BCs for groups set in ML_BSSN_ConstraintsEverywhere GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions CoordinatesSymmetry::CoordinatesSymmetry_Apply_FirstPass: Apply multipatch symmetries Interpolate2::Interpolate2ApplyBC: [level] Apply interpolating inter-patch boundaries CoordinatesSymmetry::CoordinatesSymmetry_Apply_SecondPass: Apply multipatch symmetries for remaining points not filled by interpatch interpolation Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP ML_BSSN_ConstraintsInterior_bc_group: ML_BSSN_ConstraintsInterior ML_BSSN::ML_BSSN_ConstraintsInterior_SelectBCs: [level] ML_BSSN_ConstraintsInterior_SelectBCs GROUP ML_BSSN_ConstraintsInterior_ApplyBCs: Apply BCs for groups set in ML_BSSN_ConstraintsInterior GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions CoordinatesSymmetry::CoordinatesSymmetry_Apply_FirstPass: Apply multipatch symmetries Interpolate2::Interpolate2ApplyBC: [level] Apply interpolating inter-patch boundaries CoordinatesSymmetry::CoordinatesSymmetry_Apply_SecondPass: Apply multipatch symmetries for remaining points not filled by interpatch interpolation Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP WeylScal4_psis_calc_Nth_bc_group: WeylScal4_psis_calc_Nth WeylScal4::WeylScal4_psis_calc_Nth_SelectBCs: [level] WeylScal4_psis_calc_Nth_SelectBCs GROUP WeylScal4_psis_calc_Nth_ApplyBCs: Apply BCs for groups set in WeylScal4_psis_calc_Nth GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions CoordinatesSymmetry::CoordinatesSymmetry_Apply_FirstPass: Apply multipatch symmetries Interpolate2::Interpolate2ApplyBC: [level] Apply interpolating inter-patch boundaries CoordinatesSymmetry::CoordinatesSymmetry_Apply_SecondPass: Apply multipatch symmetries for remaining points not filled by interpatch interpolation Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions [CCTK_POSTPOSTINITIAL] [CCTK_POSTSTEP] GROUP ADMDerivatives_Group: Group for ADMDerivatives computations. ADMDerivatives::ADMDerivatives_CalcDerivatives: [local] Calculate requested derivatives of ADM lapse, shift, 3-metric ADMDerivatives::ADMDerivatives_radial_SelectBC: [level] Selecting BC for metric radial derivatives ADMDerivatives::ADMDerivatives_time_SelectBC: [level] Selecting BC for metric time derivatives GROUP ADMDerivatives_ApplyBCs: Apply boundary conditions controlled by thorn Boundary GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions CoordinatesSymmetry::CoordinatesSymmetry_Apply_FirstPass: Apply multipatch symmetries Interpolate2::Interpolate2ApplyBC: [level] Apply interpolating inter-patch boundaries CoordinatesSymmetry::CoordinatesSymmetry_Apply_SecondPass: Apply multipatch symmetries for remaining points not filled by interpatch interpolation Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions AHFinderDirect::AHFinderDirect_import_mask: [global] [loop-local] import the excision mask AHFinderDirect::AHFinderDirect_find_horizons: [global] find apparent horizon(s) after this time step AHFinderDirect::AHFinderDirect_store: [global] store apparent horizon(s) into spherical surface(s) AHFinderDirect::AHFinderDirect_save: [global] save apparent horizon(s) into Cactus variables AHFinderDirect::AHFinderDirect_maybe_do_masks: set mask(s) based on apparent horizon position(s) GROUP zzz_NaNChecker_NaNCheck: Check for NaNs and count them in NaNChecker::NaNsFound NaNChecker::NaNChecker_NaNCheck_Prepare: [level] Prepare data structures to check for NaNs NaNChecker::NaNChecker_NaNCheck_Check: [local] Check for NaNs NaNChecker::NaNChecker_NaNCheck_Finish: [level] Count NaNs in NaNChecker::NaNsFound NaNChecker::NaNChecker_TakeAction: [global] [loop-level] Output NaNChecker::NaNmask and take action according to NaNChecker::action_if_found PunctureTracker::PunctureTracker_SetPositions: [global] Copy puncture positions to spherical surfaces SphericalSurface::SphericalSurface_Set: [global] Set surface radii GROUP SphericalSurface_HasBeenSet: Set the spherical surfaces before this group, and use it afterwards SphericalSurface::SphericalSurface_CheckState: [global] Test the state of the spherical surfaces endif if (recover initial data) [CCTK_BASEGRID] ADMBase::ADMBase_SetShiftStateOn: Set the shift_state variable to 1 ADMBase::ADMBase_SetDtLapseStateOn: Set the dtlapse_state variable to 1 ADMBase::ADMBase_SetDtShiftStateOn: Set the dtshift_state variable to 1 ADMMacros::ADMMacros_SetLocalSpatialOrder: Initialize the local_spatial_order CartGrid3D::SpatialSpacings: [singlemap] Set up ranges for spatial 3D Cartesian coordinates (on all maps) CartGrid3D::SpatialCoordinates: Set up spatial 3D Cartesian coordinates on the GH SphericalSurface::SphericalSurface_SetupRes: [global] [loop-local] Set surface resolution automatically AHFinderDirect::AHFinderDirect_setup: [global] setup data structures GROUP MultiPatch_SpatialCoordinates: Determine the global coordinates of each gridpoint. Coordinates::Coordinates_SetGlobalCoords: [local] Determine the global coordinates of each gridpoint. Coordinates::Coordinates_SetJacobian: [local] Fill in the Jacobian matrices for Thornburg04 coordinates. Coordinates::Coordinates_SetVolumeForm: [local] Set weight mask (for Carpet reduction operators) Interpolate2::Interpolate2Init: Initialise interpolating inter-patch boundaries GROUP Interpolate2Test: Test interpolating inter-patch boundaries Interpolate2::Interpolate2TestInit: Test: Initialise test grid function Interpolate2::Interpolate2TestSelectBCs: [level] Test: Interpolate test grid function GROUP Interpolate2TestApplyBCs: Test: Interpolate test grid function GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions CoordinatesSymmetry::CoordinatesSymmetry_Apply_FirstPass: Apply multipatch symmetries Interpolate2::Interpolate2ApplyBC: [level] Apply interpolating inter-patch boundaries CoordinatesSymmetry::CoordinatesSymmetry_Apply_SecondPass: Apply multipatch symmetries for remaining points not filled by interpatch interpolation Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions Interpolate2::Interpolate2TestCheck: Test: Check test grid function ML_ADMConstraints::ML_ADMConstraints_CheckBoundaries: [meta] check boundaries treatment ML_BSSN::ML_BSSN_CheckBoundaries: [meta] check boundaries treatment NaNChecker::NaNChecker_ResetCounter: [global] Reset the NaNChecker::NaNsFound counter SphericalSurface::SphericalSurface_Setup: [global] Calculate surface coordinate descriptors GROUP MaskBase_SetupMask: Set up the weight function GROUP MaskBase_SetupMaskAll: Set up the weight function CarpetReduce::MaskBase_AllocateMask: [global] Allocate the weight function CarpetReduce::MaskBase_InitMask: [global] [loop-local] Initialise the weight function GROUP SetupIMaskInternal: Set up the integer weight function (schedule other routines in here) CarpetReduce::CoordBase_SetupMask: [global] [loop-local] Set up the outer boundaries of the weight function CarpetReduce::CarpetMaskSetup: [global] [loop-singlemap] Set up the weight function for the restriction regions GROUP SetupIMask: Set up the integer weight function (schedule other routines in here) CarpetReduce::MaskBase_SetMask: [global] [loop-local] Set the weight function GROUP SetupMask: Set up the real weight function (schedule other routines in here) CarpetReduce::MaskBase_TestMask: [global] Test the weight function SphericalSurface::SphericalSurface_Set: [global] Set surface radii to be used for initial setup in other thorns GROUP SphericalSurface_HasBeenSet: Set the spherical surfaces before this group, and use it afterwards SphericalSurface::SphericalSurface_CheckState: [global] Test the state of the spherical surfaces SummationByParts::SBP_SetNormMask: Setup the mask for the calculation of the norm SymBase::SymBase_Check: Check whether the driver set up the grid consistently TerminationTrigger::TerminationTrigger_ResetTrigger: Clear trigger state TerminationTrigger::TerminationTrigger_StartTimer: Start timer TerminationTrigger::TerminationTrigger_CreateFile: Create termination file Time::Time_Initialise: [global] Initialise Time variables Time::TemporalSpacings: [singlemap] Set fixed timestep WeylScal4::WeylScal4_CheckBoundaries: [meta] check boundaries treatment [CCTK_RECOVER_VARIABLES] IOUtil::IOUtil_RecoverGH: [level] Checkpoint recovery routine [CCTK_POST_RECOVER_VARIABLES] GROUP ADMDerivatives_Group: Group for ADMDerivatives computations. ADMDerivatives::ADMDerivatives_CalcDerivatives: [local] Calculate requested derivatives of ADM lapse, shift, 3-metric ADMDerivatives::ADMDerivatives_radial_SelectBC: [level] Selecting BC for metric radial derivatives ADMDerivatives::ADMDerivatives_time_SelectBC: [level] Selecting BC for metric time derivatives GROUP ADMDerivatives_ApplyBCs: Apply boundary conditions controlled by thorn Boundary GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions CoordinatesSymmetry::CoordinatesSymmetry_Apply_FirstPass: Apply multipatch symmetries Interpolate2::Interpolate2ApplyBC: [level] Apply interpolating inter-patch boundaries CoordinatesSymmetry::CoordinatesSymmetry_Apply_SecondPass: Apply multipatch symmetries for remaining points not filled by interpatch interpolation Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions AHFinderDirect::AHFinderDirect_recover: [global] import horizon data from Cactus variables CarpetIOHDF5::CarpetIOHDF5_InitCheckpointingIntervals: [global] Initialisation of checkpointing intervals after recovery CarpetIOHDF5::CarpetIOHDF5_CloseFiles: [meta] Close all initial data checkpoint files after recovery GROUP MaskBase_SetupMask: Set up the weight function GROUP MaskBase_SetupMaskAll: Set up the weight function CarpetReduce::MaskBase_AllocateMask: [global] Allocate the weight function CarpetReduce::MaskBase_InitMask: [global] [loop-local] Initialise the weight function GROUP SetupIMaskInternal: Set up the integer weight function (schedule other routines in here) CarpetReduce::CoordBase_SetupMask: [global] [loop-local] Set up the outer boundaries of the weight function CarpetReduce::CarpetMaskSetup: [global] [loop-singlemap] Set up the weight function for the restriction regions GROUP SetupIMask: Set up the integer weight function (schedule other routines in here) CarpetReduce::MaskBase_SetMask: [global] [loop-local] Set the weight function GROUP SetupMask: Set up the real weight function (schedule other routines in here) CarpetReduce::MaskBase_TestMask: [global] Test the weight function GROUP MoL_PostStep: Ensure that everything is correct after recovery ML_BSSN::ML_BSSN_SelectBoundConds: [level] select boundary conditions ML_ADMConstraints::ML_ADMConstraints_SelectBoundConds: [level] select boundary conditions GROUP ML_ADMConstraints_ApplyBCs: Apply boundary conditions controlled by thorn Boundary GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions CoordinatesSymmetry::CoordinatesSymmetry_Apply_FirstPass: Apply multipatch symmetries Interpolate2::Interpolate2ApplyBC: [level] Apply interpolating inter-patch boundaries CoordinatesSymmetry::CoordinatesSymmetry_Apply_SecondPass: Apply multipatch symmetries for remaining points not filled by interpatch interpolation Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP ML_BSSN_ApplyBCs: Apply boundary conditions controlled by thorn Boundary GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions CoordinatesSymmetry::CoordinatesSymmetry_Apply_FirstPass: Apply multipatch symmetries Interpolate2::Interpolate2ApplyBC: [level] Apply interpolating inter-patch boundaries CoordinatesSymmetry::CoordinatesSymmetry_Apply_SecondPass: Apply multipatch symmetries for remaining points not filled by interpatch interpolation Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions ML_BSSN::ML_BSSN_ADMBaseInterior: ML_BSSN_ADMBaseInterior ML_BSSN::ML_BSSN_ADMBaseBoundaryScalar: ML_BSSN_ADMBaseBoundaryScalar ML_BSSN::ML_BSSN_ADMBaseEverywhere: ML_BSSN_ADMBaseEverywhere ML_BSSN_Helper::ML_BSSN_ADMBase_SelectBCs: [level] Select boundary conditions for ADMBase variables GROUP ML_BSSN_ADMBase_ApplyBCs: Apply boundary conditions to ADMBase variables GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions CoordinatesSymmetry::CoordinatesSymmetry_Apply_FirstPass: Apply multipatch symmetries Interpolate2::Interpolate2ApplyBC: [level] Apply interpolating inter-patch boundaries CoordinatesSymmetry::CoordinatesSymmetry_Apply_SecondPass: Apply multipatch symmetries for remaining points not filled by interpatch interpolation Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP ADMBase_SetADMVars: Set the ADM variables before this group, and use them afterwards WeylScal4::WeylScal4_SelectBoundConds: [level] select boundary conditions GROUP WeylScal4_ApplyBCs: Apply boundary conditions controlled by thorn Boundary GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions CoordinatesSymmetry::CoordinatesSymmetry_Apply_FirstPass: Apply multipatch symmetries Interpolate2::Interpolate2ApplyBC: [level] Apply interpolating inter-patch boundaries CoordinatesSymmetry::CoordinatesSymmetry_Apply_SecondPass: Apply multipatch symmetries for remaining points not filled by interpatch interpolation Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP zzz_NaNChecker_NaNCheck: Check for NaNs and count them in NaNChecker::NaNsFound NaNChecker::NaNChecker_NaNCheck_Prepare: [level] Prepare data structures to check for NaNs NaNChecker::NaNChecker_NaNCheck_Check: [local] Check for NaNs NaNChecker::NaNChecker_NaNCheck_Finish: [level] Count NaNs in NaNChecker::NaNsFound NaNChecker::NaNChecker_TakeAction: [global] [loop-level] Output NaNChecker::NaNmask and take action according to NaNChecker::action_if_found TerminationTrigger::TerminationTrigger_ResetMinutes: [global] Reset Watchtime endif if (checkpoint initial data) [CCTK_CPINITIAL] CarpetIOHDF5::CarpetIOHDF5_InitialDataCheckpoint: [meta] Initial data checkpoint routine endif if (analysis) [CCTK_ANALYSIS] CarpetLib::CarpetLib_printtimestats: [global] Print timing statistics if desired CarpetLib::CarpetLib_printmemstats: [global] Print memory statistics if desired Formaline::Formaline_AnnounceUpdate: [global] Put some meta information about the current run into permanent storage LoopControl::LC_statistics_analysis: [meta] Output LoopControl statistics GROUP ML_BSSN_EvolutionAnalysis: Calculate RHS at analysis ML_BSSN::ML_BSSN_EvolutionAnalysisInit: ML_BSSN_EvolutionAnalysisInit ML_BSSN::ML_BSSN_EvolutionAnalysisInterior: ML_BSSN_EvolutionAnalysisInterior Multipole::Multipole_Calc: [global] Calculate Multipoles QuasiLocalMeasures::qlm_calculate: [global] Calculate quasi-local quantities SystemStatistics::SystemStatistics_Collect: [global] Collect system statistics TerminationTrigger::TerminationTrigger_CheckWalltime: Check elapsed job walltime TerminationTrigger::TerminationTrigger_CheckSignal: Check if we received a termination signal TerminationTrigger::TerminationTrigger_CheckFile: Check termination file endif Output grid variables do loop over timesteps [CCTK_PREREGRID] CarpetTracker::CarpetTracker_SetPositions: [global] Set positions of refined regions Change grid hierarchy [CCTK_POSTREGRID] AHFinderDirect::AHFinderDirect_maybe_do_masks: set mask(s) based on apparent horizon position(s) CartGrid3D::SpatialCoordinates: Set Coordinates after regridding GROUP MaskBase_SetupMask: Set up the weight function GROUP MaskBase_SetupMaskAll: Set up the weight function CarpetReduce::MaskBase_AllocateMask: [global] Allocate the weight function CarpetReduce::MaskBase_InitMask: [global] [loop-local] Initialise the weight function GROUP SetupIMaskInternal: Set up the integer weight function (schedule other routines in here) CarpetReduce::CoordBase_SetupMask: [global] [loop-local] Set up the outer boundaries of the weight function CarpetReduce::CarpetMaskSetup: [global] [loop-singlemap] Set up the weight function for the restriction regions GROUP SetupIMask: Set up the integer weight function (schedule other routines in here) CarpetReduce::MaskBase_SetMask: [global] [loop-local] Set the weight function GROUP SetupMask: Set up the real weight function (schedule other routines in here) CarpetReduce::MaskBase_TestMask: [global] Test the weight function GROUP MultiPatch_SpatialCoordinates: Determine the global coordinates of each gridpoint. Coordinates::Coordinates_SetGlobalCoords: [local] Determine the global coordinates of each gridpoint. Coordinates::Coordinates_SetJacobian: [local] Fill in the Jacobian matrices for Thornburg04 coordinates. Coordinates::Coordinates_SetVolumeForm: [local] Set weight mask (for Carpet reduction operators) GROUP MoL_PostStep: Ensure that everything is correct after regridding ML_BSSN::ML_BSSN_SelectBoundConds: [level] select boundary conditions ML_ADMConstraints::ML_ADMConstraints_SelectBoundConds: [level] select boundary conditions GROUP ML_ADMConstraints_ApplyBCs: Apply boundary conditions controlled by thorn Boundary GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions CoordinatesSymmetry::CoordinatesSymmetry_Apply_FirstPass: Apply multipatch symmetries Interpolate2::Interpolate2ApplyBC: [level] Apply interpolating inter-patch boundaries CoordinatesSymmetry::CoordinatesSymmetry_Apply_SecondPass: Apply multipatch symmetries for remaining points not filled by interpatch interpolation Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP ML_BSSN_ApplyBCs: Apply boundary conditions controlled by thorn Boundary GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions CoordinatesSymmetry::CoordinatesSymmetry_Apply_FirstPass: Apply multipatch symmetries Interpolate2::Interpolate2ApplyBC: [level] Apply interpolating inter-patch boundaries CoordinatesSymmetry::CoordinatesSymmetry_Apply_SecondPass: Apply multipatch symmetries for remaining points not filled by interpatch interpolation Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions ML_BSSN::ML_BSSN_ADMBaseInterior: ML_BSSN_ADMBaseInterior ML_BSSN::ML_BSSN_ADMBaseBoundaryScalar: ML_BSSN_ADMBaseBoundaryScalar ML_BSSN::ML_BSSN_ADMBaseEverywhere: ML_BSSN_ADMBaseEverywhere ML_BSSN_Helper::ML_BSSN_ADMBase_SelectBCs: [level] Select boundary conditions for ADMBase variables GROUP ML_BSSN_ADMBase_ApplyBCs: Apply boundary conditions to ADMBase variables GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions CoordinatesSymmetry::CoordinatesSymmetry_Apply_FirstPass: Apply multipatch symmetries Interpolate2::Interpolate2ApplyBC: [level] Apply interpolating inter-patch boundaries CoordinatesSymmetry::CoordinatesSymmetry_Apply_SecondPass: Apply multipatch symmetries for remaining points not filled by interpatch interpolation Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP ADMBase_SetADMVars: Set the ADM variables before this group, and use them afterwards WeylScal4::WeylScal4_SelectBoundConds: [level] select boundary conditions GROUP WeylScal4_ApplyBCs: Apply boundary conditions controlled by thorn Boundary GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions CoordinatesSymmetry::CoordinatesSymmetry_Apply_FirstPass: Apply multipatch symmetries Interpolate2::Interpolate2ApplyBC: [level] Apply interpolating inter-patch boundaries CoordinatesSymmetry::CoordinatesSymmetry_Apply_SecondPass: Apply multipatch symmetries for remaining points not filled by interpatch interpolation Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP MoL_PseudoEvolutionBoundaries: Apply boundary conditions to pseudo-evolved quantities GROUP ML_ADMConstraints_evaluate_bc_group: ML_ADMConstraints_evaluate ML_ADMConstraints::ML_ADMConstraints_evaluate_SelectBCs: [level] ML_ADMConstraints_evaluate_SelectBCs GROUP ML_ADMConstraints_evaluate_ApplyBCs: Apply BCs for groups set in ML_ADMConstraints_evaluate GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions CoordinatesSymmetry::CoordinatesSymmetry_Apply_FirstPass: Apply multipatch symmetries Interpolate2::Interpolate2ApplyBC: [level] Apply interpolating inter-patch boundaries CoordinatesSymmetry::CoordinatesSymmetry_Apply_SecondPass: Apply multipatch symmetries for remaining points not filled by interpatch interpolation Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP ML_BSSN_ConstraintsEverywhere_bc_group: ML_BSSN_ConstraintsEverywhere ML_BSSN::ML_BSSN_ConstraintsEverywhere_SelectBCs: [level] ML_BSSN_ConstraintsEverywhere_SelectBCs GROUP ML_BSSN_ConstraintsEverywhere_ApplyBCs: Apply BCs for groups set in ML_BSSN_ConstraintsEverywhere GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions CoordinatesSymmetry::CoordinatesSymmetry_Apply_FirstPass: Apply multipatch symmetries Interpolate2::Interpolate2ApplyBC: [level] Apply interpolating inter-patch boundaries CoordinatesSymmetry::CoordinatesSymmetry_Apply_SecondPass: Apply multipatch symmetries for remaining points not filled by interpatch interpolation Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP ML_BSSN_ConstraintsInterior_bc_group: ML_BSSN_ConstraintsInterior ML_BSSN::ML_BSSN_ConstraintsInterior_SelectBCs: [level] ML_BSSN_ConstraintsInterior_SelectBCs GROUP ML_BSSN_ConstraintsInterior_ApplyBCs: Apply BCs for groups set in ML_BSSN_ConstraintsInterior GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions CoordinatesSymmetry::CoordinatesSymmetry_Apply_FirstPass: Apply multipatch symmetries Interpolate2::Interpolate2ApplyBC: [level] Apply interpolating inter-patch boundaries CoordinatesSymmetry::CoordinatesSymmetry_Apply_SecondPass: Apply multipatch symmetries for remaining points not filled by interpatch interpolation Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP WeylScal4_psis_calc_Nth_bc_group: WeylScal4_psis_calc_Nth WeylScal4::WeylScal4_psis_calc_Nth_SelectBCs: [level] WeylScal4_psis_calc_Nth_SelectBCs GROUP WeylScal4_psis_calc_Nth_ApplyBCs: Apply BCs for groups set in WeylScal4_psis_calc_Nth GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions CoordinatesSymmetry::CoordinatesSymmetry_Apply_FirstPass: Apply multipatch symmetries Interpolate2::Interpolate2ApplyBC: [level] Apply interpolating inter-patch boundaries CoordinatesSymmetry::CoordinatesSymmetry_Apply_SecondPass: Apply multipatch symmetries for remaining points not filled by interpatch interpolation Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions Rotate timelevels iteration = iteration+1 t = t+dt [CCTK_PRESTEP] CoordGauge::Einstein_SetNextSlicing: Identify the slicing for the next iteration LoopControl::LC_steer: [meta] Update LoopControl algorithm preferences NaNChecker::NaNChecker_ResetCounter: [global] Reset the NaNChecker::NaNsFound counter [CCTK_EVOL] MoL::MoL_StartLoop: [level] Initialise the step size control while (MoL::MoL_Stepsize_Bad) GROUP MoL_Evolution: A single Cactus evolution step using MoL GROUP MoL_StartStep: MoL internal setup for the evolution step MoL::MoL_SetCounter: [level] Set the counter for the ODE method to loop over MoL::MoL_SetTime: [level] Ensure the correct time and timestep are used MoL::MoL_AllocateScratchSpace: [level] Allocate storage for scratch levels GROUP MoL_PreStep: Physics thorns can schedule preloop setup routines in here MoL::MoL_AllocateScratch: Allocate sufficient space for array scratch variables MoL::MoL_InitialCopy: Ensure the data is in the correct timelevel while (MoL::MoL_Intermediate_Step) GROUP MoL_Step: The loop over the intermediate steps for the ODE integrator MoL::MoL_InitRHS: Initialise the RHS functions GROUP MoL_CalcRHS: Physics thorns schedule the calculation of the discrete spatial operator in here ML_BSSN::ML_BSSN_EvolutionBoundaryScalar: ML_BSSN_EvolutionBoundaryScalar GROUP ML_BSSN_EvolutionInteriorSplitBy: ML_BSSN::ML_BSSN_EvolutionInteriorSplitBy1: ML_BSSN_EvolutionInteriorSplitBy1 ML_BSSN::ML_BSSN_EvolutionInteriorSplitBy2: ML_BSSN_EvolutionInteriorSplitBy2 ML_BSSN::ML_BSSN_EvolutionInteriorSplitBy3: ML_BSSN_EvolutionInteriorSplitBy3 GROUP MoL_PostRHS: Modify RHS functions GlobalDerivative::GlobalDeriv_Dissipation: [local] Apply global dissipation to registered variables GROUP MoL_RHSBoundaries: Any 'final' modifications to the RHS functions (boundaries etc.) MoL::MoL_Add: Updates calculated with the efficient Runge-Kutta 4 method MoL::MoL_DecrementCounter: [level] Alter the counter number MoL::MoL_ResetTime: [level] If necessary, change the time GROUP MoL_PostStepModify: The group for physics thorns to schedule enforcing constraints ML_BSSN::ML_BSSN_EnforceEverywhere: ML_BSSN_EnforceEverywhere GROUP MoL_PostStep: The group for physics thorns to schedule boundary calls etc. ML_BSSN::ML_BSSN_SelectBoundConds: [level] select boundary conditions ML_ADMConstraints::ML_ADMConstraints_SelectBoundConds: [level] select boundary conditions GROUP ML_ADMConstraints_ApplyBCs: Apply boundary conditions controlled by thorn Boundary GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions CoordinatesSymmetry::CoordinatesSymmetry_Apply_FirstPass: Apply multipatch symmetries Interpolate2::Interpolate2ApplyBC: [level] Apply interpolating inter-patch boundaries CoordinatesSymmetry::CoordinatesSymmetry_Apply_SecondPass: Apply multipatch symmetries for remaining points not filled by interpatch interpolation Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP ML_BSSN_ApplyBCs: Apply boundary conditions controlled by thorn Boundary GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions CoordinatesSymmetry::CoordinatesSymmetry_Apply_FirstPass: Apply multipatch symmetries Interpolate2::Interpolate2ApplyBC: [level] Apply interpolating inter-patch boundaries CoordinatesSymmetry::CoordinatesSymmetry_Apply_SecondPass: Apply multipatch symmetries for remaining points not filled by interpatch interpolation Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions ML_BSSN::ML_BSSN_ADMBaseInterior: ML_BSSN_ADMBaseInterior ML_BSSN::ML_BSSN_ADMBaseBoundaryScalar: ML_BSSN_ADMBaseBoundaryScalar ML_BSSN::ML_BSSN_ADMBaseEverywhere: ML_BSSN_ADMBaseEverywhere ML_BSSN_Helper::ML_BSSN_ADMBase_SelectBCs: [level] Select boundary conditions for ADMBase variables GROUP ML_BSSN_ADMBase_ApplyBCs: Apply boundary conditions to ADMBase variables GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions CoordinatesSymmetry::CoordinatesSymmetry_Apply_FirstPass: Apply multipatch symmetries Interpolate2::Interpolate2ApplyBC: [level] Apply interpolating inter-patch boundaries CoordinatesSymmetry::CoordinatesSymmetry_Apply_SecondPass: Apply multipatch symmetries for remaining points not filled by interpatch interpolation Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP ADMBase_SetADMVars: Set the ADM variables before this group, and use them afterwards WeylScal4::WeylScal4_SelectBoundConds: [level] select boundary conditions GROUP WeylScal4_ApplyBCs: Apply boundary conditions controlled by thorn Boundary GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions CoordinatesSymmetry::CoordinatesSymmetry_Apply_FirstPass: Apply multipatch symmetries Interpolate2::Interpolate2ApplyBC: [level] Apply interpolating inter-patch boundaries CoordinatesSymmetry::CoordinatesSymmetry_Apply_SecondPass: Apply multipatch symmetries for remaining points not filled by interpatch interpolation Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions MoL::MoL_ResetDeltaTime: [level] If necessary, change the timestep end while MoL::MoL_FinishLoop: [level] Control the step size MoL::MoL_RestoreSandR: Restoring the Save and Restore variables to the original state MoL::MoL_FreeScratchSpace: [level] Free storage for scratch levels end while GROUP MoL_PseudoEvolution: Calculate pseudo-evolved quantities GROUP ADMBase_SetADMVars: Set the ADM variables before this group, and use them afterwards GROUP ML_ADMConstraints_evaluate_group: ML_ADMConstraints_evaluate ML_ADMConstraints::ML_ADMConstraints_evaluate: ML_ADMConstraints_evaluate GROUP ML_ADMConstraints_evaluate_bc_group: ML_ADMConstraints_evaluate ML_ADMConstraints::ML_ADMConstraints_evaluate_SelectBCs: [level] ML_ADMConstraints_evaluate_SelectBCs GROUP ML_ADMConstraints_evaluate_ApplyBCs: Apply BCs for groups set in ML_ADMConstraints_evaluate GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions CoordinatesSymmetry::CoordinatesSymmetry_Apply_FirstPass: Apply multipatch symmetries Interpolate2::Interpolate2ApplyBC: [level] Apply interpolating inter-patch boundaries CoordinatesSymmetry::CoordinatesSymmetry_Apply_SecondPass: Apply multipatch symmetries for remaining points not filled by interpatch interpolation Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP ML_BSSN_ConstraintsEverywhere_group: ML_BSSN_ConstraintsEverywhere ML_BSSN::ML_BSSN_ConstraintsEverywhere: ML_BSSN_ConstraintsEverywhere GROUP ML_BSSN_ConstraintsEverywhere_bc_group: ML_BSSN_ConstraintsEverywhere ML_BSSN::ML_BSSN_ConstraintsEverywhere_SelectBCs: [level] ML_BSSN_ConstraintsEverywhere_SelectBCs GROUP ML_BSSN_ConstraintsEverywhere_ApplyBCs: Apply BCs for groups set in ML_BSSN_ConstraintsEverywhere GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions CoordinatesSymmetry::CoordinatesSymmetry_Apply_FirstPass: Apply multipatch symmetries Interpolate2::Interpolate2ApplyBC: [level] Apply interpolating inter-patch boundaries CoordinatesSymmetry::CoordinatesSymmetry_Apply_SecondPass: Apply multipatch symmetries for remaining points not filled by interpatch interpolation Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP ML_BSSN_ConstraintsInterior_group: ML_BSSN_ConstraintsInterior ML_BSSN::ML_BSSN_ConstraintsInterior: ML_BSSN_ConstraintsInterior GROUP ML_BSSN_ConstraintsInterior_bc_group: ML_BSSN_ConstraintsInterior ML_BSSN::ML_BSSN_ConstraintsInterior_SelectBCs: [level] ML_BSSN_ConstraintsInterior_SelectBCs GROUP ML_BSSN_ConstraintsInterior_ApplyBCs: Apply BCs for groups set in ML_BSSN_ConstraintsInterior GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions CoordinatesSymmetry::CoordinatesSymmetry_Apply_FirstPass: Apply multipatch symmetries Interpolate2::Interpolate2ApplyBC: [level] Apply interpolating inter-patch boundaries CoordinatesSymmetry::CoordinatesSymmetry_Apply_SecondPass: Apply multipatch symmetries for remaining points not filled by interpatch interpolation Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP WeylScal4_psis_calc_Nth_group: WeylScal4_psis_calc_Nth WeylScal4::WeylScal4_psis_calc_Nth: WeylScal4_psis_calc_Nth GROUP WeylScal4_psis_calc_Nth_bc_group: WeylScal4_psis_calc_Nth WeylScal4::WeylScal4_psis_calc_Nth_SelectBCs: [level] WeylScal4_psis_calc_Nth_SelectBCs GROUP WeylScal4_psis_calc_Nth_ApplyBCs: Apply BCs for groups set in WeylScal4_psis_calc_Nth GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions CoordinatesSymmetry::CoordinatesSymmetry_Apply_FirstPass: Apply multipatch symmetries Interpolate2::Interpolate2ApplyBC: [level] Apply interpolating inter-patch boundaries CoordinatesSymmetry::CoordinatesSymmetry_Apply_SecondPass: Apply multipatch symmetries for remaining points not filled by interpatch interpolation Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP WeylScal4_invars_calc_Nth_group: WeylScal4_invars_calc_Nth WeylScal4::WeylScal4_invars_calc_Nth: WeylScal4_invars_calc_Nth PunctureTracker::PunctureTracker_Track: [global] Calculate new location of punctures Evolve finer grids recursively Restrict from finer grids [CCTK_POSTRESTRICT] GROUP MoL_PostStep: Ensure that everything is correct after restriction ML_BSSN::ML_BSSN_SelectBoundConds: [level] select boundary conditions ML_ADMConstraints::ML_ADMConstraints_SelectBoundConds: [level] select boundary conditions GROUP ML_ADMConstraints_ApplyBCs: Apply boundary conditions controlled by thorn Boundary GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions CoordinatesSymmetry::CoordinatesSymmetry_Apply_FirstPass: Apply multipatch symmetries Interpolate2::Interpolate2ApplyBC: [level] Apply interpolating inter-patch boundaries CoordinatesSymmetry::CoordinatesSymmetry_Apply_SecondPass: Apply multipatch symmetries for remaining points not filled by interpatch interpolation Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP ML_BSSN_ApplyBCs: Apply boundary conditions controlled by thorn Boundary GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions CoordinatesSymmetry::CoordinatesSymmetry_Apply_FirstPass: Apply multipatch symmetries Interpolate2::Interpolate2ApplyBC: [level] Apply interpolating inter-patch boundaries CoordinatesSymmetry::CoordinatesSymmetry_Apply_SecondPass: Apply multipatch symmetries for remaining points not filled by interpatch interpolation Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions ML_BSSN::ML_BSSN_ADMBaseInterior: ML_BSSN_ADMBaseInterior ML_BSSN::ML_BSSN_ADMBaseBoundaryScalar: ML_BSSN_ADMBaseBoundaryScalar ML_BSSN::ML_BSSN_ADMBaseEverywhere: ML_BSSN_ADMBaseEverywhere ML_BSSN_Helper::ML_BSSN_ADMBase_SelectBCs: [level] Select boundary conditions for ADMBase variables GROUP ML_BSSN_ADMBase_ApplyBCs: Apply boundary conditions to ADMBase variables GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions CoordinatesSymmetry::CoordinatesSymmetry_Apply_FirstPass: Apply multipatch symmetries Interpolate2::Interpolate2ApplyBC: [level] Apply interpolating inter-patch boundaries CoordinatesSymmetry::CoordinatesSymmetry_Apply_SecondPass: Apply multipatch symmetries for remaining points not filled by interpatch interpolation Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP ADMBase_SetADMVars: Set the ADM variables before this group, and use them afterwards WeylScal4::WeylScal4_SelectBoundConds: [level] select boundary conditions GROUP WeylScal4_ApplyBCs: Apply boundary conditions controlled by thorn Boundary GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions CoordinatesSymmetry::CoordinatesSymmetry_Apply_FirstPass: Apply multipatch symmetries Interpolate2::Interpolate2ApplyBC: [level] Apply interpolating inter-patch boundaries CoordinatesSymmetry::CoordinatesSymmetry_Apply_SecondPass: Apply multipatch symmetries for remaining points not filled by interpatch interpolation Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP MoL_PseudoEvolutionBoundaries: Apply boundary conditions to pseudo-evolved quantities GROUP ML_ADMConstraints_evaluate_bc_group: ML_ADMConstraints_evaluate ML_ADMConstraints::ML_ADMConstraints_evaluate_SelectBCs: [level] ML_ADMConstraints_evaluate_SelectBCs GROUP ML_ADMConstraints_evaluate_ApplyBCs: Apply BCs for groups set in ML_ADMConstraints_evaluate GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions CoordinatesSymmetry::CoordinatesSymmetry_Apply_FirstPass: Apply multipatch symmetries Interpolate2::Interpolate2ApplyBC: [level] Apply interpolating inter-patch boundaries CoordinatesSymmetry::CoordinatesSymmetry_Apply_SecondPass: Apply multipatch symmetries for remaining points not filled by interpatch interpolation Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP ML_BSSN_ConstraintsEverywhere_bc_group: ML_BSSN_ConstraintsEverywhere ML_BSSN::ML_BSSN_ConstraintsEverywhere_SelectBCs: [level] ML_BSSN_ConstraintsEverywhere_SelectBCs GROUP ML_BSSN_ConstraintsEverywhere_ApplyBCs: Apply BCs for groups set in ML_BSSN_ConstraintsEverywhere GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions CoordinatesSymmetry::CoordinatesSymmetry_Apply_FirstPass: Apply multipatch symmetries Interpolate2::Interpolate2ApplyBC: [level] Apply interpolating inter-patch boundaries CoordinatesSymmetry::CoordinatesSymmetry_Apply_SecondPass: Apply multipatch symmetries for remaining points not filled by interpatch interpolation Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP ML_BSSN_ConstraintsInterior_bc_group: ML_BSSN_ConstraintsInterior ML_BSSN::ML_BSSN_ConstraintsInterior_SelectBCs: [level] ML_BSSN_ConstraintsInterior_SelectBCs GROUP ML_BSSN_ConstraintsInterior_ApplyBCs: Apply BCs for groups set in ML_BSSN_ConstraintsInterior GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions CoordinatesSymmetry::CoordinatesSymmetry_Apply_FirstPass: Apply multipatch symmetries Interpolate2::Interpolate2ApplyBC: [level] Apply interpolating inter-patch boundaries CoordinatesSymmetry::CoordinatesSymmetry_Apply_SecondPass: Apply multipatch symmetries for remaining points not filled by interpatch interpolation Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP WeylScal4_psis_calc_Nth_bc_group: WeylScal4_psis_calc_Nth WeylScal4::WeylScal4_psis_calc_Nth_SelectBCs: [level] WeylScal4_psis_calc_Nth_SelectBCs GROUP WeylScal4_psis_calc_Nth_ApplyBCs: Apply BCs for groups set in WeylScal4_psis_calc_Nth GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions CoordinatesSymmetry::CoordinatesSymmetry_Apply_FirstPass: Apply multipatch symmetries Interpolate2::Interpolate2ApplyBC: [level] Apply interpolating inter-patch boundaries CoordinatesSymmetry::CoordinatesSymmetry_Apply_SecondPass: Apply multipatch symmetries for remaining points not filled by interpatch interpolation Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions [CCTK_POSTSTEP] GROUP ADMDerivatives_Group: Group for ADMDerivatives computations. ADMDerivatives::ADMDerivatives_CalcDerivatives: [local] Calculate requested derivatives of ADM lapse, shift, 3-metric ADMDerivatives::ADMDerivatives_radial_SelectBC: [level] Selecting BC for metric radial derivatives ADMDerivatives::ADMDerivatives_time_SelectBC: [level] Selecting BC for metric time derivatives GROUP ADMDerivatives_ApplyBCs: Apply boundary conditions controlled by thorn Boundary GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions CoordinatesSymmetry::CoordinatesSymmetry_Apply_FirstPass: Apply multipatch symmetries Interpolate2::Interpolate2ApplyBC: [level] Apply interpolating inter-patch boundaries CoordinatesSymmetry::CoordinatesSymmetry_Apply_SecondPass: Apply multipatch symmetries for remaining points not filled by interpatch interpolation Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions AHFinderDirect::AHFinderDirect_import_mask: [global] [loop-local] import the excision mask AHFinderDirect::AHFinderDirect_find_horizons: [global] find apparent horizon(s) after this time step AHFinderDirect::AHFinderDirect_store: [global] store apparent horizon(s) into spherical surface(s) AHFinderDirect::AHFinderDirect_save: [global] save apparent horizon(s) into Cactus variables AHFinderDirect::AHFinderDirect_maybe_do_masks: set mask(s) based on apparent horizon position(s) GROUP zzz_NaNChecker_NaNCheck: Check for NaNs and count them in NaNChecker::NaNsFound NaNChecker::NaNChecker_NaNCheck_Prepare: [level] Prepare data structures to check for NaNs NaNChecker::NaNChecker_NaNCheck_Check: [local] Check for NaNs NaNChecker::NaNChecker_NaNCheck_Finish: [level] Count NaNs in NaNChecker::NaNsFound NaNChecker::NaNChecker_TakeAction: [global] [loop-level] Output NaNChecker::NaNmask and take action according to NaNChecker::action_if_found PunctureTracker::PunctureTracker_SetPositions: [global] Copy puncture positions to spherical surfaces SphericalSurface::SphericalSurface_Set: [global] Set surface radii GROUP SphericalSurface_HasBeenSet: Set the spherical surfaces before this group, and use it afterwards SphericalSurface::SphericalSurface_CheckState: [global] Test the state of the spherical surfaces if (checkpoint) [CCTK_CHECKPOINT] CarpetIOHDF5::CarpetIOHDF5_EvolutionCheckpoint: [meta] Evolution checkpoint routine endif if (analysis) [CCTK_ANALYSIS] CarpetLib::CarpetLib_printtimestats: [global] Print timing statistics if desired CarpetLib::CarpetLib_printmemstats: [global] Print memory statistics if desired Formaline::Formaline_AnnounceUpdate: [global] Put some meta information about the current run into permanent storage LoopControl::LC_statistics_analysis: [meta] Output LoopControl statistics GROUP ML_BSSN_EvolutionAnalysis: Calculate RHS at analysis ML_BSSN::ML_BSSN_EvolutionAnalysisInit: ML_BSSN_EvolutionAnalysisInit ML_BSSN::ML_BSSN_EvolutionAnalysisInterior: ML_BSSN_EvolutionAnalysisInterior Multipole::Multipole_Calc: [global] Calculate Multipoles QuasiLocalMeasures::qlm_calculate: [global] Calculate quasi-local quantities SystemStatistics::SystemStatistics_Collect: [global] Collect system statistics TerminationTrigger::TerminationTrigger_CheckWalltime: Check elapsed job walltime TerminationTrigger::TerminationTrigger_CheckSignal: Check if we received a termination signal TerminationTrigger::TerminationTrigger_CheckFile: Check termination file endif Output grid variables enddo Termination routines [CCTK_TERMINATE] CarpetIOHDF5::CarpetIOHDF5_TerminationCheckpoint: [meta] Termination checkpoint routine Formaline::Formaline_AnnounceFinal: [global] Put some meta information about the current run into permanent storage LoopControl::LC_statistics_terminate: [meta] Output LoopControl statistics MoL::MoL_FreeIndexArrays: Free the MoL bookkeeping index arrays Shutdown routines [CCTK_SHUTDOWN] Timers::Timer_Shutdown: Prepare hierarchical timers Routines run after changing the grid hierarchy: [CCTK_POSTREGRID] AHFinderDirect::AHFinderDirect_maybe_do_masks: set mask(s) based on apparent horizon position(s) CartGrid3D::SpatialCoordinates: Set Coordinates after regridding GROUP MaskBase_SetupMask: Set up the weight function GROUP MaskBase_SetupMaskAll: Set up the weight function CarpetReduce::MaskBase_AllocateMask: [global] Allocate the weight function CarpetReduce::MaskBase_InitMask: [global] [loop-local] Initialise the weight function GROUP SetupIMaskInternal: Set up the integer weight function (schedule other routines in here) CarpetReduce::CoordBase_SetupMask: [global] [loop-local] Set up the outer boundaries of the weight function CarpetReduce::CarpetMaskSetup: [global] [loop-singlemap] Set up the weight function for the restriction regions GROUP SetupIMask: Set up the integer weight function (schedule other routines in here) CarpetReduce::MaskBase_SetMask: [global] [loop-local] Set the weight function GROUP SetupMask: Set up the real weight function (schedule other routines in here) CarpetReduce::MaskBase_TestMask: [global] Test the weight function GROUP MultiPatch_SpatialCoordinates: Determine the global coordinates of each gridpoint. Coordinates::Coordinates_SetGlobalCoords: [local] Determine the global coordinates of each gridpoint. Coordinates::Coordinates_SetJacobian: [local] Fill in the Jacobian matrices for Thornburg04 coordinates. Coordinates::Coordinates_SetVolumeForm: [local] Set weight mask (for Carpet reduction operators) GROUP MoL_PostStep: Ensure that everything is correct after regridding ML_BSSN::ML_BSSN_SelectBoundConds: [level] select boundary conditions ML_ADMConstraints::ML_ADMConstraints_SelectBoundConds: [level] select boundary conditions GROUP ML_ADMConstraints_ApplyBCs: Apply boundary conditions controlled by thorn Boundary GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions CoordinatesSymmetry::CoordinatesSymmetry_Apply_FirstPass: Apply multipatch symmetries Interpolate2::Interpolate2ApplyBC: [level] Apply interpolating inter-patch boundaries CoordinatesSymmetry::CoordinatesSymmetry_Apply_SecondPass: Apply multipatch symmetries for remaining points not filled by interpatch interpolation Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP ML_BSSN_ApplyBCs: Apply boundary conditions controlled by thorn Boundary GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions CoordinatesSymmetry::CoordinatesSymmetry_Apply_FirstPass: Apply multipatch symmetries Interpolate2::Interpolate2ApplyBC: [level] Apply interpolating inter-patch boundaries CoordinatesSymmetry::CoordinatesSymmetry_Apply_SecondPass: Apply multipatch symmetries for remaining points not filled by interpatch interpolation Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions ML_BSSN::ML_BSSN_ADMBaseInterior: ML_BSSN_ADMBaseInterior ML_BSSN::ML_BSSN_ADMBaseBoundaryScalar: ML_BSSN_ADMBaseBoundaryScalar ML_BSSN::ML_BSSN_ADMBaseEverywhere: ML_BSSN_ADMBaseEverywhere ML_BSSN_Helper::ML_BSSN_ADMBase_SelectBCs: [level] Select boundary conditions for ADMBase variables GROUP ML_BSSN_ADMBase_ApplyBCs: Apply boundary conditions to ADMBase variables GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions CoordinatesSymmetry::CoordinatesSymmetry_Apply_FirstPass: Apply multipatch symmetries Interpolate2::Interpolate2ApplyBC: [level] Apply interpolating inter-patch boundaries CoordinatesSymmetry::CoordinatesSymmetry_Apply_SecondPass: Apply multipatch symmetries for remaining points not filled by interpatch interpolation Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP ADMBase_SetADMVars: Set the ADM variables before this group, and use them afterwards WeylScal4::WeylScal4_SelectBoundConds: [level] select boundary conditions GROUP WeylScal4_ApplyBCs: Apply boundary conditions controlled by thorn Boundary GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions CoordinatesSymmetry::CoordinatesSymmetry_Apply_FirstPass: Apply multipatch symmetries Interpolate2::Interpolate2ApplyBC: [level] Apply interpolating inter-patch boundaries CoordinatesSymmetry::CoordinatesSymmetry_Apply_SecondPass: Apply multipatch symmetries for remaining points not filled by interpatch interpolation Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP MoL_PseudoEvolutionBoundaries: Apply boundary conditions to pseudo-evolved quantities GROUP ML_ADMConstraints_evaluate_bc_group: ML_ADMConstraints_evaluate ML_ADMConstraints::ML_ADMConstraints_evaluate_SelectBCs: [level] ML_ADMConstraints_evaluate_SelectBCs GROUP ML_ADMConstraints_evaluate_ApplyBCs: Apply BCs for groups set in ML_ADMConstraints_evaluate GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions CoordinatesSymmetry::CoordinatesSymmetry_Apply_FirstPass: Apply multipatch symmetries Interpolate2::Interpolate2ApplyBC: [level] Apply interpolating inter-patch boundaries CoordinatesSymmetry::CoordinatesSymmetry_Apply_SecondPass: Apply multipatch symmetries for remaining points not filled by interpatch interpolation Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP ML_BSSN_ConstraintsEverywhere_bc_group: ML_BSSN_ConstraintsEverywhere ML_BSSN::ML_BSSN_ConstraintsEverywhere_SelectBCs: [level] ML_BSSN_ConstraintsEverywhere_SelectBCs GROUP ML_BSSN_ConstraintsEverywhere_ApplyBCs: Apply BCs for groups set in ML_BSSN_ConstraintsEverywhere GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions CoordinatesSymmetry::CoordinatesSymmetry_Apply_FirstPass: Apply multipatch symmetries Interpolate2::Interpolate2ApplyBC: [level] Apply interpolating inter-patch boundaries CoordinatesSymmetry::CoordinatesSymmetry_Apply_SecondPass: Apply multipatch symmetries for remaining points not filled by interpatch interpolation Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP ML_BSSN_ConstraintsInterior_bc_group: ML_BSSN_ConstraintsInterior ML_BSSN::ML_BSSN_ConstraintsInterior_SelectBCs: [level] ML_BSSN_ConstraintsInterior_SelectBCs GROUP ML_BSSN_ConstraintsInterior_ApplyBCs: Apply BCs for groups set in ML_BSSN_ConstraintsInterior GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions CoordinatesSymmetry::CoordinatesSymmetry_Apply_FirstPass: Apply multipatch symmetries Interpolate2::Interpolate2ApplyBC: [level] Apply interpolating inter-patch boundaries CoordinatesSymmetry::CoordinatesSymmetry_Apply_SecondPass: Apply multipatch symmetries for remaining points not filled by interpatch interpolation Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP WeylScal4_psis_calc_Nth_bc_group: WeylScal4_psis_calc_Nth WeylScal4::WeylScal4_psis_calc_Nth_SelectBCs: [level] WeylScal4_psis_calc_Nth_SelectBCs GROUP WeylScal4_psis_calc_Nth_ApplyBCs: Apply BCs for groups set in WeylScal4_psis_calc_Nth GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions CoordinatesSymmetry::CoordinatesSymmetry_Apply_FirstPass: Apply multipatch symmetries Interpolate2::Interpolate2ApplyBC: [level] Apply interpolating inter-patch boundaries CoordinatesSymmetry::CoordinatesSymmetry_Apply_SecondPass: Apply multipatch symmetries for remaining points not filled by interpatch interpolation Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions -------------------------------------------------------------------------------- INFO (Carpet): Multi-Model listing: model 0: "world" INFO (Carpet): Multi-Model process distribution: process 0: model 0 "world" INFO (Carpet): Multi-Model: This is process 0, model 0 "world" INFO (CarpetLib): Process startup time was 1.49 seconds Current core file size limit: hard=[unlimited], soft=[unlimited] Current addres space size limit: hard=[unlimited], soft=[unlimited] Current data segment size limit: hard=[unlimited], soft=[unlimited] Current resident set size limit: hard=[unlimited], soft=[unlimited] INFO (CycleClock): Measuring CycleClock tick via OpenMP... INFO (CycleClock): Calibrated CycleClock: 0.345487 ns per clock tick (2.89446 GHz) INFO (hwloc): library version 2.0.4, API version 0x20000 INFO (SystemTopology): MPI process-to-host mapping: This is MPI process 0 of 1 MPI hosts: 0: panther This MPI process runs on host 0 of 1 On this host, this is MPI process 0 of 1 INFO (SystemTopology): Topology support: Discovery support: discovery->pu : yes CPU binding support: cpubind->set_thisproc_cpubind : yes cpubind->get_thisproc_cpubind : yes cpubind->set_proc_cpubind : yes cpubind->get_proc_cpubind : yes cpubind->set_thisthread_cpubind : yes cpubind->get_thisthread_cpubind : yes cpubind->set_thread_cpubind : yes cpubind->get_thread_cpubind : yes cpubind->get_thisproc_last_cpu_location : yes cpubind->get_proc_last_cpu_location : yes cpubind->get_thisthread_last_cpu_location: yes Memory binding support: membind->set_thisproc_membind : no membind->get_thisproc_membind : no membind->set_proc_membind : no membind->get_proc_membind : no membind->set_thisthread_membind : yes membind->get_thisthread_membind : yes membind->set_area_membind : yes membind->get_area_membind : yes membind->alloc_membind : yes membind->firsttouch_membind : yes membind->bind_membind : yes membind->interleave_membind : yes membind->nexttouch_membind : no membind->migrate_membind : yes INFO (SystemTopology): Hardware objects in this node: Machine L#0: (P#0, total=1043114796KB, DMIProductName="PowerEdge R7515", DMIProductVersion=, DMIBoardVendor="Dell Inc.", DMIBoardName=04F3CJ, DMIBoardVersion=A02, DMIChassisVendor="Dell Inc.", DMIChassisType=23, DMIChassisVersion=, DMIChassisAssetTag=, DMIBIOSVendor="Dell Inc.", DMIBIOSVersion=2.18.1, DMIBIOSDate=02/03/2025, DMISysVendor="Dell Inc.", Backend=Linux, OSName=Linux, OSRelease=5.14.0-570.37.1.el9_6.x86_64, OSVersion="#1 SMP PREEMPT_DYNAMIC Thu Aug 28 10:41:06 UTC 2025", HostName=panther, Architecture=x86_64, hwlocVersion=2.0.4, ProcessName=cactus_sim) Package L#0: (P#0, total=1043114796KB, CPUVendor=AuthenticAMD, CPUFamilyNumber=23, CPUModelNumber=49, CPUModel="AMD EPYC 7542 32-Core Processor ", CPUStepping=0) L3Cache L#0: (P#-1, size=16384KB, linesize=64, ways=16, Inclusive=0) L2Cache L#0: (P#-1, size=512KB, linesize=64, ways=8, Inclusive=1) L1dCache L#0: (P#-1, size=32KB, linesize=64, ways=8, Inclusive=0) Core L#0: (P#0) PU L#0: (P#0) PU L#1: (P#32) L2Cache L#1: (P#-1, size=512KB, linesize=64, ways=8, Inclusive=1) L1dCache L#1: (P#-1, size=32KB, linesize=64, ways=8, Inclusive=0) Core L#1: (P#1) PU L#2: (P#1) PU L#3: (P#33) L2Cache L#2: (P#-1, size=512KB, linesize=64, ways=8, Inclusive=1) L1dCache L#2: (P#-1, size=32KB, linesize=64, ways=8, Inclusive=0) Core L#2: (P#2) PU L#4: (P#2) PU L#5: (P#34) L2Cache L#3: (P#-1, size=512KB, linesize=64, ways=8, Inclusive=1) L1dCache L#3: (P#-1, size=32KB, linesize=64, ways=8, Inclusive=0) Core L#3: (P#3) PU L#6: (P#3) PU L#7: (P#35) L3Cache L#1: (P#-1, size=16384KB, linesize=64, ways=16, Inclusive=0) L2Cache L#4: (P#-1, size=512KB, linesize=64, ways=8, Inclusive=1) L1dCache L#4: (P#-1, size=32KB, linesize=64, ways=8, Inclusive=0) Core L#4: (P#4) PU L#8: (P#4) PU L#9: (P#36) L2Cache L#5: (P#-1, size=512KB, linesize=64, ways=8, Inclusive=1) L1dCache L#5: (P#-1, size=32KB, linesize=64, ways=8, Inclusive=0) Core L#5: (P#5) PU L#10: (P#5) PU L#11: (P#37) L2Cache L#6: (P#-1, size=512KB, linesize=64, ways=8, Inclusive=1) L1dCache L#6: (P#-1, size=32KB, linesize=64, ways=8, Inclusive=0) Core L#6: (P#6) PU L#12: (P#6) PU L#13: (P#38) L2Cache L#7: (P#-1, size=512KB, linesize=64, ways=8, Inclusive=1) L1dCache L#7: (P#-1, size=32KB, linesize=64, ways=8, Inclusive=0) Core L#7: (P#7) PU L#14: (P#7) PU L#15: (P#39) L3Cache L#2: (P#-1, size=16384KB, linesize=64, ways=16, Inclusive=0) L2Cache L#8: (P#-1, size=512KB, linesize=64, ways=8, Inclusive=1) L1dCache L#8: (P#-1, size=32KB, linesize=64, ways=8, Inclusive=0) Core L#8: (P#8) PU L#16: (P#8) PU L#17: (P#40) L2Cache L#9: (P#-1, size=512KB, linesize=64, ways=8, Inclusive=1) L1dCache L#9: (P#-1, size=32KB, linesize=64, ways=8, Inclusive=0) Core L#9: (P#9) PU L#18: (P#9) PU L#19: (P#41) L2Cache L#10: (P#-1, size=512KB, linesize=64, ways=8, Inclusive=1) L1dCache L#10: (P#-1, size=32KB, linesize=64, ways=8, Inclusive=0) Core L#10: (P#10) PU L#20: (P#10) PU L#21: (P#42) L2Cache L#11: (P#-1, size=512KB, linesize=64, ways=8, Inclusive=1) L1dCache L#11: (P#-1, size=32KB, linesize=64, ways=8, Inclusive=0) Core L#11: (P#11) PU L#22: (P#11) PU L#23: (P#43) L3Cache L#3: (P#-1, size=16384KB, linesize=64, ways=16, Inclusive=0) L2Cache L#12: (P#-1, size=512KB, linesize=64, ways=8, Inclusive=1) L1dCache L#12: (P#-1, size=32KB, linesize=64, ways=8, Inclusive=0) Core L#12: (P#12) PU L#24: (P#12) PU L#25: (P#44) L2Cache L#13: (P#-1, size=512KB, linesize=64, ways=8, Inclusive=1) L1dCache L#13: (P#-1, size=32KB, linesize=64, ways=8, Inclusive=0) Core L#13: (P#13) PU L#26: (P#13) PU L#27: (P#45) L2Cache L#14: (P#-1, size=512KB, linesize=64, ways=8, Inclusive=1) L1dCache L#14: (P#-1, size=32KB, linesize=64, ways=8, Inclusive=0) Core L#14: (P#14) PU L#28: (P#14) PU L#29: (P#46) L2Cache L#15: (P#-1, size=512KB, linesize=64, ways=8, Inclusive=1) L1dCache L#15: (P#-1, size=32KB, linesize=64, ways=8, Inclusive=0) Core L#15: (P#15) PU L#30: (P#15) PU L#31: (P#47) L3Cache L#4: (P#-1, size=16384KB, linesize=64, ways=16, Inclusive=0) L2Cache L#16: (P#-1, size=512KB, linesize=64, ways=8, Inclusive=1) L1dCache L#16: (P#-1, size=32KB, linesize=64, ways=8, Inclusive=0) Core L#16: (P#16) PU L#32: (P#16) PU L#33: (P#48) L2Cache L#17: (P#-1, size=512KB, linesize=64, ways=8, Inclusive=1) L1dCache L#17: (P#-1, size=32KB, linesize=64, ways=8, Inclusive=0) Core L#17: (P#17) PU L#34: (P#17) PU L#35: (P#49) L2Cache L#18: (P#-1, size=512KB, linesize=64, ways=8, Inclusive=1) L1dCache L#18: (P#-1, size=32KB, linesize=64, ways=8, Inclusive=0) Core L#18: (P#18) PU L#36: (P#18) PU L#37: (P#50) L2Cache L#19: (P#-1, size=512KB, linesize=64, ways=8, Inclusive=1) L1dCache L#19: (P#-1, size=32KB, linesize=64, ways=8, Inclusive=0) Core L#19: (P#19) PU L#38: (P#19) PU L#39: (P#51) L3Cache L#5: (P#-1, size=16384KB, linesize=64, ways=16, Inclusive=0) L2Cache L#20: (P#-1, size=512KB, linesize=64, ways=8, Inclusive=1) L1dCache L#20: (P#-1, size=32KB, linesize=64, ways=8, Inclusive=0) Core L#20: (P#20) PU L#40: (P#20) PU L#41: (P#52) L2Cache L#21: (P#-1, size=512KB, linesize=64, ways=8, Inclusive=1) L1dCache L#21: (P#-1, size=32KB, linesize=64, ways=8, Inclusive=0) Core L#21: (P#21) PU L#42: (P#21) PU L#43: (P#53) L2Cache L#22: (P#-1, size=512KB, linesize=64, ways=8, Inclusive=1) L1dCache L#22: (P#-1, size=32KB, linesize=64, ways=8, Inclusive=0) Core L#22: (P#22) PU L#44: (P#22) PU L#45: (P#54) L2Cache L#23: (P#-1, size=512KB, linesize=64, ways=8, Inclusive=1) L1dCache L#23: (P#-1, size=32KB, linesize=64, ways=8, Inclusive=0) Core L#23: (P#23) PU L#46: (P#23) PU L#47: (P#55) L3Cache L#6: (P#-1, size=16384KB, linesize=64, ways=16, Inclusive=0) L2Cache L#24: (P#-1, size=512KB, linesize=64, ways=8, Inclusive=1) L1dCache L#24: (P#-1, size=32KB, linesize=64, ways=8, Inclusive=0) Core L#24: (P#24) PU L#48: (P#24) PU L#49: (P#56) L2Cache L#25: (P#-1, size=512KB, linesize=64, ways=8, Inclusive=1) L1dCache L#25: (P#-1, size=32KB, linesize=64, ways=8, Inclusive=0) Core L#25: (P#25) PU L#50: (P#25) PU L#51: (P#57) L2Cache L#26: (P#-1, size=512KB, linesize=64, ways=8, Inclusive=1) L1dCache L#26: (P#-1, size=32KB, linesize=64, ways=8, Inclusive=0) Core L#26: (P#26) PU L#52: (P#26) PU L#53: (P#58) L2Cache L#27: (P#-1, size=512KB, linesize=64, ways=8, Inclusive=1) L1dCache L#27: (P#-1, size=32KB, linesize=64, ways=8, Inclusive=0) Core L#27: (P#27) PU L#54: (P#27) PU L#55: (P#59) L3Cache L#7: (P#-1, size=16384KB, linesize=64, ways=16, Inclusive=0) L2Cache L#28: (P#-1, size=512KB, linesize=64, ways=8, Inclusive=1) L1dCache L#28: (P#-1, size=32KB, linesize=64, ways=8, Inclusive=0) Core L#28: (P#28) PU L#56: (P#28) PU L#57: (P#60) L2Cache L#29: (P#-1, size=512KB, linesize=64, ways=8, Inclusive=1) L1dCache L#29: (P#-1, size=32KB, linesize=64, ways=8, Inclusive=0) Core L#29: (P#29) PU L#58: (P#29) PU L#59: (P#61) L2Cache L#30: (P#-1, size=512KB, linesize=64, ways=8, Inclusive=1) L1dCache L#30: (P#-1, size=32KB, linesize=64, ways=8, Inclusive=0) Core L#30: (P#30) PU L#60: (P#30) PU L#61: (P#62) L2Cache L#31: (P#-1, size=512KB, linesize=64, ways=8, Inclusive=1) L1dCache L#31: (P#-1, size=32KB, linesize=64, ways=8, Inclusive=0) Core L#31: (P#31) PU L#62: (P#31) PU L#63: (P#63) INFO (SystemTopology): Thread CPU bindings: MPI process 0 on host 0 (process 0 of 1 on this host) OpenMP thread 0: PU set L#{0-63} P#{0-63} OpenMP thread 1: PU set L#{0-63} P#{0-63} OpenMP thread 2: PU set L#{0-63} P#{0-63} OpenMP thread 3: PU set L#{0-63} P#{0-63} OpenMP thread 4: PU set L#{0-63} P#{0-63} OpenMP thread 5: PU set L#{0-63} P#{0-63} OpenMP thread 6: PU set L#{0-63} P#{0-63} OpenMP thread 7: PU set L#{0-63} P#{0-63} OpenMP thread 8: PU set L#{0-63} P#{0-63} OpenMP thread 9: PU set L#{0-63} P#{0-63} OpenMP thread 10: PU set L#{0-63} P#{0-63} OpenMP thread 11: PU set L#{0-63} P#{0-63} OpenMP thread 12: PU set L#{0-63} P#{0-63} OpenMP thread 13: PU set L#{0-63} P#{0-63} OpenMP thread 14: PU set L#{0-63} P#{0-63} OpenMP thread 15: PU set L#{0-63} P#{0-63} OpenMP thread 16: PU set L#{0-63} P#{0-63} OpenMP thread 17: PU set L#{0-63} P#{0-63} OpenMP thread 18: PU set L#{0-63} P#{0-63} OpenMP thread 19: PU set L#{0-63} P#{0-63} OpenMP thread 20: PU set L#{0-63} P#{0-63} OpenMP thread 21: PU set L#{0-63} P#{0-63} OpenMP thread 22: PU set L#{0-63} P#{0-63} OpenMP thread 23: PU set L#{0-63} P#{0-63} OpenMP thread 24: PU set L#{0-63} P#{0-63} OpenMP thread 25: PU set L#{0-63} P#{0-63} OpenMP thread 26: PU set L#{0-63} P#{0-63} OpenMP thread 27: PU set L#{0-63} P#{0-63} OpenMP thread 28: PU set L#{0-63} P#{0-63} OpenMP thread 29: PU set L#{0-63} P#{0-63} OpenMP thread 30: PU set L#{0-63} P#{0-63} OpenMP thread 31: PU set L#{0-63} P#{0-63} INFO (SystemTopology): Setting thread CPU bindings: INFO (SystemTopology): Thread CPU bindings: MPI process 0 on host 0 (process 0 of 1 on this host) OpenMP thread 0: PU set L#{0} P#{0} OpenMP thread 1: PU set L#{2} P#{1} OpenMP thread 2: PU set L#{4} P#{2} OpenMP thread 3: PU set L#{6} P#{3} OpenMP thread 4: PU set L#{8} P#{4} OpenMP thread 5: PU set L#{10} P#{5} OpenMP thread 6: PU set L#{12} P#{6} OpenMP thread 7: PU set L#{14} P#{7} OpenMP thread 8: PU set L#{16} P#{8} OpenMP thread 9: PU set L#{18} P#{9} OpenMP thread 10: PU set L#{20} P#{10} OpenMP thread 11: PU set L#{22} P#{11} OpenMP thread 12: PU set L#{24} P#{12} OpenMP thread 13: PU set L#{26} P#{13} OpenMP thread 14: PU set L#{28} P#{14} OpenMP thread 15: PU set L#{30} P#{15} OpenMP thread 16: PU set L#{32} P#{16} OpenMP thread 17: PU set L#{34} P#{17} OpenMP thread 18: PU set L#{36} P#{18} OpenMP thread 19: PU set L#{38} P#{19} OpenMP thread 20: PU set L#{40} P#{20} OpenMP thread 21: PU set L#{42} P#{21} OpenMP thread 22: PU set L#{44} P#{22} OpenMP thread 23: PU set L#{46} P#{23} OpenMP thread 24: PU set L#{48} P#{24} OpenMP thread 25: PU set L#{50} P#{25} OpenMP thread 26: PU set L#{52} P#{26} OpenMP thread 27: PU set L#{54} P#{27} OpenMP thread 28: PU set L#{56} P#{28} OpenMP thread 29: PU set L#{58} P#{29} OpenMP thread 30: PU set L#{60} P#{30} OpenMP thread 31: PU set L#{62} P#{31} INFO (SystemTopology): Extracting CPU/cache/memory properties: There are 2 PUs per core (aka hardware SMT threads) There are 1 threads per core (aka SMT threads used) Cache (unknown name) has type "data" depth 1 size 32768 linesize 64 associativity 8 stride 4096, for 2 PUs Cache (unknown name) has type "unified" depth 2 size 524288 linesize 64 associativity 8 stride 65536, for 2 PUs Cache (unknown name) has type "unified" depth 3 size 16777216 linesize 64 associativity 16 stride 1048576, for 8 PUs INFO (Formaline): Configuration id: config-sim-panther-system-user-crangano-einstein_toolkit-Cactus INFO (Formaline): Build id: build-sim-panther-crangano-2025.09.10-14.43.00-3265191 INFO (Formaline): Simulation id: run-GW150914-panther-crangano-2025.09.10-15.10.05-3271897 INFO (Formaline): Run id: run-GW150914-panther-crangano-2025.09.10-15.10.05-3271897 INFO (Vectors): Using vector size 2 for architecture SSE2+SSE4.1 (64-bit precision) -------------------------------------------------------------------------------- AMR driver provided by Carpet -------------------------------------------------------------------------------- AMR HDF5 I/O provided by CarpetIOHDF5 -------------------------------------------------------------------------------- AMR 0D HDF5 I/O provided by CarpetIOHDF5 -------------------------------------------------------------------------------- AMR 1D HDF5 I/O provided by CarpetIOHDF5 -------------------------------------------------------------------------------- AMR 2D HDF5 I/O provided by CarpetIOHDF5 -------------------------------------------------------------------------------- AMR 3D HDF5 I/O provided by CarpetIOHDF5 -------------------------------------------------------------------------------- ML_ADMConstraints -------------------------------------------------------------------------------- ML_BSSN -------------------------------------------------------------------------------- AMR scalar I/O provided by CarpetIOScalar -------------------------------------------------------------------------------- MoL: Generalized time integration. -------------------------------------------------------------------------------- AMR info I/O provided by CarpetIOBasic -------------------------------------------------------------------------------- WeylScal4 -------------------------------------------------------------------------------- AMR 0D ASCII I/O provided by CarpetIOASCII -------------------------------------------------------------------------------- AMR 1D ASCII I/O provided by CarpetIOASCII -------------------------------------------------------------------------------- AMR 2D ASCII I/O provided by CarpetIOASCII -------------------------------------------------------------------------------- AMR 3D ASCII I/O provided by CarpetIOASCII -------------------------------------------------------------------------------- INFO (Carpet): MPI is enabled INFO (Carpet): Carpet is running on 1 processes INFO (Carpet): This is process 0 INFO (Carpet): OpenMP is enabled INFO (Carpet): This process contains 32 threads, this is thread 0 INFO (Carpet): There are 32 threads in total INFO (Carpet): There are 32 threads per process INFO (Carpet): This process runs on host panther, pid=3271897 INFO (Carpet): This process runs on 32 cores: 0-31 INFO (Carpet): Thread 0 runs on 1 core: 0 INFO (Carpet): Thread 1 runs on 1 core: 1 INFO (Carpet): Thread 2 runs on 1 core: 2 INFO (Carpet): Thread 3 runs on 1 core: 3 INFO (Carpet): Thread 4 runs on 1 core: 4 INFO (Carpet): Thread 5 runs on 1 core: 5 INFO (Carpet): Thread 6 runs on 1 core: 6 INFO (Carpet): Thread 7 runs on 1 core: 7 INFO (Carpet): Thread 8 runs on 1 core: 8 INFO (Carpet): Thread 9 runs on 1 core: 9 INFO (Carpet): Thread 10 runs on 1 core: 10 INFO (Carpet): Thread 11 runs on 1 core: 11 INFO (Carpet): Thread 12 runs on 1 core: 12 INFO (Carpet): Thread 13 runs on 1 core: 13 INFO (Carpet): Thread 14 runs on 1 core: 14 INFO (Carpet): Thread 15 runs on 1 core: 15 INFO (Carpet): Thread 16 runs on 1 core: 16 INFO (Carpet): Thread 17 runs on 1 core: 17 INFO (Carpet): Thread 18 runs on 1 core: 18 INFO (Carpet): Thread 19 runs on 1 core: 19 INFO (Carpet): Thread 20 runs on 1 core: 20 INFO (Carpet): Thread 21 runs on 1 core: 21 INFO (Carpet): Thread 22 runs on 1 core: 22 INFO (Carpet): Thread 23 runs on 1 core: 23 INFO (Carpet): Thread 24 runs on 1 core: 24 INFO (Carpet): Thread 25 runs on 1 core: 25 INFO (Carpet): Thread 26 runs on 1 core: 26 INFO (Carpet): Thread 27 runs on 1 core: 27 INFO (Carpet): Thread 28 runs on 1 core: 28 INFO (Carpet): Thread 29 runs on 1 core: 29 INFO (Carpet): Thread 30 runs on 1 core: 30 INFO (Carpet): Thread 31 runs on 1 core: 31 INFO (Carpet): This simulation is running in 3 dimensions INFO (Carpet): Boundary specification for map 0: nboundaryzones: [[5,5,5],[5,5,5]] is_internal : [[0,0,0],[0,0,0]] is_staggered : [[0,0,0],[0,0,0]] shiftout : [[4,4,1],[4,4,4]] INFO (Carpet): CoordBase domain specification for map 0: physical extent: [-51.3969,-51.3969,0] : [51.3969,51.3969,51.3969] ([102.794,102.794,51.3969]) interior extent: [-102.794,-102.794,0] : [102.794,102.794,102.794] ([205.588,205.588,102.794]) exterior extent: [-188.455,-188.455,-85.6615] : [188.455,188.455,188.455] ([376.911,376.911,274.117]) base_spacing : [17.1323,17.1323,17.1323] INFO (Carpet): Adapted domain specification for map 0: convergence factor: 2 convergence level : 0 physical extent : [-51.3969,-51.3969,0] : [51.3969,51.3969,51.3969] ([102.794,102.794,51.3969]) interior extent : [-102.794,-102.794,0] : [102.794,102.794,102.794] ([205.588,205.588,102.794]) exterior extent : [-188.455,-188.455,-85.6615] : [188.455,188.455,188.455] ([376.911,376.911,274.117]) spacing : [17.1323,17.1323,17.1323] INFO (Carpet): Base grid specification for map 0: number of grid points : [23,23,17] number of coarse grid ghost points: [[5,5,5],[5,5,5]] INFO (Carpet): Buffer zone counts (excluding ghosts): [0]: [[0,0,0],[0,0,0]] [1]: [[15,15,15],[15,15,15]] [2]: [[15,15,15],[15,15,15]] [3]: [[15,15,15],[15,15,15]] [4]: [[15,15,15],[15,15,15]] [5]: [[15,15,15],[15,15,15]] [6]: [[15,15,15],[15,15,15]] [7]: [[15,15,15],[15,15,15]] [8]: [[15,15,15],[15,15,15]] INFO (Carpet): Overlap zone counts: [0]: [[0,0,0],[0,0,0]] [1]: [[0,0,0],[0,0,0]] [2]: [[0,0,0],[0,0,0]] [3]: [[0,0,0],[0,0,0]] [4]: [[0,0,0],[0,0,0]] [5]: [[0,0,0],[0,0,0]] [6]: [[0,0,0],[0,0,0]] [7]: [[0,0,0],[0,0,0]] [8]: [[0,0,0],[0,0,0]] INFO (Carpet): Boundary specification for map 1: nboundaryzones: [[5,5,5],[5,5,5]] is_internal : [[0,0,0],[0,0,0]] is_staggered : [[0,0,0],[0,0,0]] shiftout : [[1,4,4],[4,4,0]] INFO (Carpet): CoordBase domain specification for map 1: physical extent: [0,-0.785398,51.3969] : [0.785398,0.785398,2210.07] ([0.785398,1.5708,2158.67]) interior extent: [0,-1.9635,0] : [1.9635,1.9635,2192.94] ([1.9635,3.92699,2192.94]) exterior extent: [-1.9635,-3.92699,-85.6615] : [3.92699,3.92699,2278.6] ([5.89049,7.85398,2364.26]) base_spacing : [0.392699,0.392699,17.1323] INFO (Carpet): Adapted domain specification for map 1: convergence factor: 2 convergence level : 0 physical extent : [0,-0.785398,51.3969] : [0.785398,0.785398,2210.07] ([0.785398,1.5708,2158.67]) interior extent : [0,-1.9635,0] : [1.9635,1.9635,2192.94] ([1.9635,3.92699,2192.94]) exterior extent : [-1.9635,-3.92699,-85.6615] : [3.92699,3.92699,2278.6] ([5.89049,7.85398,2364.26]) spacing : [0.392699,0.392699,17.1323] INFO (Carpet): Base grid specification for map 1: number of grid points : [16,21,139] number of coarse grid ghost points: [[5,5,5],[5,5,5]] INFO (Carpet): Buffer zone counts (excluding ghosts): [0]: [[0,0,0],[0,0,0]] [1]: [[15,15,15],[15,15,15]] [2]: [[15,15,15],[15,15,15]] [3]: [[15,15,15],[15,15,15]] [4]: [[15,15,15],[15,15,15]] [5]: [[15,15,15],[15,15,15]] [6]: [[15,15,15],[15,15,15]] [7]: [[15,15,15],[15,15,15]] [8]: [[15,15,15],[15,15,15]] INFO (Carpet): Overlap zone counts: [0]: [[0,0,0],[0,0,0]] [1]: [[0,0,0],[0,0,0]] [2]: [[0,0,0],[0,0,0]] [3]: [[0,0,0],[0,0,0]] [4]: [[0,0,0],[0,0,0]] [5]: [[0,0,0],[0,0,0]] [6]: [[0,0,0],[0,0,0]] [7]: [[0,0,0],[0,0,0]] [8]: [[0,0,0],[0,0,0]] INFO (Carpet): Boundary specification for map 2: nboundaryzones: [[5,5,5],[5,5,5]] is_internal : [[0,0,0],[0,0,0]] is_staggered : [[0,0,0],[0,0,0]] shiftout : [[4,4,4],[1,4,0]] INFO (Carpet): CoordBase domain specification for map 2: physical extent: [-0.785398,-0.785398,51.3969] : [0,0.785398,2210.07] ([0.785398,1.5708,2158.67]) interior extent: [-1.9635,-1.9635,0] : [0,1.9635,2192.94] ([1.9635,3.92699,2192.94]) exterior extent: [-3.92699,-3.92699,-85.6615] : [1.9635,3.92699,2278.6] ([5.89049,7.85398,2364.26]) base_spacing : [0.392699,0.392699,17.1323] INFO (Carpet): Adapted domain specification for map 2: convergence factor: 2 convergence level : 0 physical extent : [-0.785398,-0.785398,51.3969] : [0,0.785398,2210.07] ([0.785398,1.5708,2158.67]) interior extent : [-1.9635,-1.9635,0] : [0,1.9635,2192.94] ([1.9635,3.92699,2192.94]) exterior extent : [-3.92699,-3.92699,-85.6615] : [1.9635,3.92699,2278.6] ([5.89049,7.85398,2364.26]) spacing : [0.392699,0.392699,17.1323] INFO (Carpet): Base grid specification for map 2: number of grid points : [16,21,139] number of coarse grid ghost points: [[5,5,5],[5,5,5]] INFO (Carpet): Buffer zone counts (excluding ghosts): [0]: [[0,0,0],[0,0,0]] [1]: [[15,15,15],[15,15,15]] [2]: [[15,15,15],[15,15,15]] [3]: [[15,15,15],[15,15,15]] [4]: [[15,15,15],[15,15,15]] [5]: [[15,15,15],[15,15,15]] [6]: [[15,15,15],[15,15,15]] [7]: [[15,15,15],[15,15,15]] [8]: [[15,15,15],[15,15,15]] INFO (Carpet): Overlap zone counts: [0]: [[0,0,0],[0,0,0]] [1]: [[0,0,0],[0,0,0]] [2]: [[0,0,0],[0,0,0]] [3]: [[0,0,0],[0,0,0]] [4]: [[0,0,0],[0,0,0]] [5]: [[0,0,0],[0,0,0]] [6]: [[0,0,0],[0,0,0]] [7]: [[0,0,0],[0,0,0]] [8]: [[0,0,0],[0,0,0]] INFO (Carpet): Boundary specification for map 3: nboundaryzones: [[5,5,5],[5,5,5]] is_internal : [[0,0,0],[0,0,0]] is_staggered : [[0,0,0],[0,0,0]] shiftout : [[1,4,4],[4,4,0]] INFO (Carpet): CoordBase domain specification for map 3: physical extent: [0,-0.785398,51.3969] : [0.785398,0.785398,2210.07] ([0.785398,1.5708,2158.67]) interior extent: [0,-1.9635,0] : [1.9635,1.9635,2192.94] ([1.9635,3.92699,2192.94]) exterior extent: [-1.9635,-3.92699,-85.6615] : [3.92699,3.92699,2278.6] ([5.89049,7.85398,2364.26]) base_spacing : [0.392699,0.392699,17.1323] INFO (Carpet): Adapted domain specification for map 3: convergence factor: 2 convergence level : 0 physical extent : [0,-0.785398,51.3969] : [0.785398,0.785398,2210.07] ([0.785398,1.5708,2158.67]) interior extent : [0,-1.9635,0] : [1.9635,1.9635,2192.94] ([1.9635,3.92699,2192.94]) exterior extent : [-1.9635,-3.92699,-85.6615] : [3.92699,3.92699,2278.6] ([5.89049,7.85398,2364.26]) spacing : [0.392699,0.392699,17.1323] INFO (Carpet): Base grid specification for map 3: number of grid points : [16,21,139] number of coarse grid ghost points: [[5,5,5],[5,5,5]] INFO (Carpet): Buffer zone counts (excluding ghosts): [0]: [[0,0,0],[0,0,0]] [1]: [[15,15,15],[15,15,15]] [2]: [[15,15,15],[15,15,15]] [3]: [[15,15,15],[15,15,15]] [4]: [[15,15,15],[15,15,15]] [5]: [[15,15,15],[15,15,15]] [6]: [[15,15,15],[15,15,15]] [7]: [[15,15,15],[15,15,15]] [8]: [[15,15,15],[15,15,15]] INFO (Carpet): Overlap zone counts: [0]: [[0,0,0],[0,0,0]] [1]: [[0,0,0],[0,0,0]] [2]: [[0,0,0],[0,0,0]] [3]: [[0,0,0],[0,0,0]] [4]: [[0,0,0],[0,0,0]] [5]: [[0,0,0],[0,0,0]] [6]: [[0,0,0],[0,0,0]] [7]: [[0,0,0],[0,0,0]] [8]: [[0,0,0],[0,0,0]] INFO (Carpet): Boundary specification for map 4: nboundaryzones: [[5,5,5],[5,5,5]] is_internal : [[0,0,0],[0,0,0]] is_staggered : [[0,0,0],[0,0,0]] shiftout : [[4,4,4],[1,4,0]] INFO (Carpet): CoordBase domain specification for map 4: physical extent: [-0.785398,-0.785398,51.3969] : [0,0.785398,2210.07] ([0.785398,1.5708,2158.67]) interior extent: [-1.9635,-1.9635,0] : [0,1.9635,2192.94] ([1.9635,3.92699,2192.94]) exterior extent: [-3.92699,-3.92699,-85.6615] : [1.9635,3.92699,2278.6] ([5.89049,7.85398,2364.26]) base_spacing : [0.392699,0.392699,17.1323] INFO (Carpet): Adapted domain specification for map 4: convergence factor: 2 convergence level : 0 physical extent : [-0.785398,-0.785398,51.3969] : [0,0.785398,2210.07] ([0.785398,1.5708,2158.67]) interior extent : [-1.9635,-1.9635,0] : [0,1.9635,2192.94] ([1.9635,3.92699,2192.94]) exterior extent : [-3.92699,-3.92699,-85.6615] : [1.9635,3.92699,2278.6] ([5.89049,7.85398,2364.26]) spacing : [0.392699,0.392699,17.1323] INFO (Carpet): Base grid specification for map 4: number of grid points : [16,21,139] number of coarse grid ghost points: [[5,5,5],[5,5,5]] INFO (Carpet): Buffer zone counts (excluding ghosts): [0]: [[0,0,0],[0,0,0]] [1]: [[15,15,15],[15,15,15]] [2]: [[15,15,15],[15,15,15]] [3]: [[15,15,15],[15,15,15]] [4]: [[15,15,15],[15,15,15]] [5]: [[15,15,15],[15,15,15]] [6]: [[15,15,15],[15,15,15]] [7]: [[15,15,15],[15,15,15]] [8]: [[15,15,15],[15,15,15]] INFO (Carpet): Overlap zone counts: [0]: [[0,0,0],[0,0,0]] [1]: [[0,0,0],[0,0,0]] [2]: [[0,0,0],[0,0,0]] [3]: [[0,0,0],[0,0,0]] [4]: [[0,0,0],[0,0,0]] [5]: [[0,0,0],[0,0,0]] [6]: [[0,0,0],[0,0,0]] [7]: [[0,0,0],[0,0,0]] [8]: [[0,0,0],[0,0,0]] INFO (Carpet): Boundary specification for map 5: nboundaryzones: [[5,5,5],[5,5,5]] is_internal : [[0,0,0],[0,0,0]] is_staggered : [[0,0,0],[0,0,0]] shiftout : [[4,4,4],[4,4,0]] INFO (Carpet): CoordBase domain specification for map 5: physical extent: [-0.785398,-0.785398,51.3969] : [0.785398,0.785398,2210.07] ([1.5708,1.5708,2158.67]) interior extent: [-1.9635,-1.9635,0] : [1.9635,1.9635,2192.94] ([3.92699,3.92699,2192.94]) exterior extent: [-3.92699,-3.92699,-85.6615] : [3.92699,3.92699,2278.6] ([7.85398,7.85398,2364.26]) base_spacing : [0.392699,0.392699,17.1323] INFO (Carpet): Adapted domain specification for map 5: convergence factor: 2 convergence level : 0 physical extent : [-0.785398,-0.785398,51.3969] : [0.785398,0.785398,2210.07] ([1.5708,1.5708,2158.67]) interior extent : [-1.9635,-1.9635,0] : [1.9635,1.9635,2192.94] ([3.92699,3.92699,2192.94]) exterior extent : [-3.92699,-3.92699,-85.6615] : [3.92699,3.92699,2278.6] ([7.85398,7.85398,2364.26]) spacing : [0.392699,0.392699,17.1323] INFO (Carpet): Base grid specification for map 5: number of grid points : [21,21,139] number of coarse grid ghost points: [[5,5,5],[5,5,5]] INFO (Carpet): Buffer zone counts (excluding ghosts): [0]: [[0,0,0],[0,0,0]] [1]: [[15,15,15],[15,15,15]] [2]: [[15,15,15],[15,15,15]] [3]: [[15,15,15],[15,15,15]] [4]: [[15,15,15],[15,15,15]] [5]: [[15,15,15],[15,15,15]] [6]: [[15,15,15],[15,15,15]] [7]: [[15,15,15],[15,15,15]] [8]: [[15,15,15],[15,15,15]] INFO (Carpet): Overlap zone counts: [0]: [[0,0,0],[0,0,0]] [1]: [[0,0,0],[0,0,0]] [2]: [[0,0,0],[0,0,0]] [3]: [[0,0,0],[0,0,0]] [4]: [[0,0,0],[0,0,0]] [5]: [[0,0,0],[0,0,0]] [6]: [[0,0,0],[0,0,0]] [7]: [[0,0,0],[0,0,0]] [8]: [[0,0,0],[0,0,0]] INFO (Carpet): Group and variable statistics: INFO (Carpet): There are 966 grid functions in 88 groups INFO (Carpet): There are 820 grid scalars in 78 groups INFO (Carpet): There are 111 1-dimensional grid arrays in 10 groups INFO (Carpet): There are 211 2-dimensional grid arrays in 18 groups INFO (Carpet): There are 3 3-dimensional grid arrays in 1 groups INFO (Carpet): (The number of variables counts all time levels) INFO (CarpetIOHDF5): I/O Method 'IOHDF5' registered: AMR output of grid variables to HDF5 files INFO (CarpetIOHDF5): Periodic AMR output requested for: GRID::x GRID::y GRID::z GRID::r ML_BSSN::phi WEYLSCAL4::Psi4r WEYLSCAL4::Psi4i WEYLSCAL4::curvIr WEYLSCAL4::curvIi WEYLSCAL4::curvJr WEYLSCAL4::curvJi INFO (CarpetIOHDF5): I/O Method 'IOHDF5_0D' registered: 0D AMR output of grid variables to HDF5 files INFO (CarpetIOHDF5): I/O Method 'IOHDF5_1D' registered: 1D AMR output of grid variables to HDF5 files INFO (CarpetIOHDF5): I/O Method 'IOHDF5_2D' registered: 2D AMR output of grid variables to HDF5 files INFO (CarpetIOHDF5): Periodic 2D AMR output requested for: GRID::x GRID::y GRID::z GRID::r ML_BSSN::phi ML_BSSN::trK ML_BSSN::alpha WEYLSCAL4::Psi4r WEYLSCAL4::Psi4i INFO (CarpetIOHDF5): I/O Method 'IOHDF5_3D' registered: 3D AMR output of grid variables to HDF5 files INFO (CarpetIOScalar): Periodic scalar output requested for: SYSTEMSTATISTICS::maxrss_mb SYSTEMSTATISTICS::majflt_mb SYSTEMSTATISTICS::arena_mb SYSTEMSTATISTICS::ordblks_mb SYSTEMSTATISTICS::hblks_mb SYSTEMSTATISTICS::hblkhd_mb SYSTEMSTATISTICS::uordblks_mb SYSTEMSTATISTICS::fordblks_mb SYSTEMSTATISTICS::keepcost_mb SYSTEMSTATISTICS::swap_used_mb INFO (CarpetIOASCII): I/O Method 'IOASCII_0D' registered: 0D AMR output of grid variables to ASCII files INFO (CarpetIOASCII): Periodic 0D AMR output requested for: CARPET::physical_time_per_hour CARPET::current_physical_time_per_hour CARPET::time_total CARPET::time_evolution CARPET::time_computing CARPET::time_communicating CARPET::time_io CARPET::evolution_steps_count CARPET::local_grid_points_per_second CARPET::total_grid_points_per_second CARPET::local_grid_point_updates_count CARPET::total_grid_point_updates_count CARPET::local_interior_points_per_second CARPET::total_interior_points_per_second CARPET::local_interior_point_updates_count CARPET::total_interior_point_updates_count CARPET::io_per_second CARPET::io_bytes_per_second CARPET::io_bytes_ascii_per_second CARPET::io_bytes_binary_per_second CARPET::io_count CARPET::io_bytes_count CARPET::io_bytes_ascii_count CARPET::io_bytes_binary_count CARPET::comm_per_second CARPET::comm_bytes_per_second CARPET::comm_count CARPET::comm_bytes_count CARPET::time_levels CARPET::current_walltime CARPET::syncs_count PUNCTURETRACKER::pt_loc_t[0] PUNCTURETRACKER::pt_loc_t[1] PUNCTURETRACKER::pt_loc_t[2] PUNCTURETRACKER::pt_loc_t[3] PUNCTURETRACKER::pt_loc_t[4] PUNCTURETRACKER::pt_loc_t[5] PUNCTURETRACKER::pt_loc_t[6] PUNCTURETRACKER::pt_loc_t[7] PUNCTURETRACKER::pt_loc_t[8] PUNCTURETRACKER::pt_loc_t[9] PUNCTURETRACKER::pt_loc_x[0] PUNCTURETRACKER::pt_loc_x[1] PUNCTURETRACKER::pt_loc_x[2] PUNCTURETRACKER::pt_loc_x[3] PUNCTURETRACKER::pt_loc_x[4] PUNCTURETRACKER::pt_loc_x[5] PUNCTURETRACKER::pt_loc_x[6] PUNCTURETRACKER::pt_loc_x[7] PUNCTURETRACKER::pt_loc_x[8] PUNCTURETRACKER::pt_loc_x[9] PUNCTURETRACKER::pt_loc_y[0] PUNCTURETRACKER::pt_loc_y[1] PUNCTURETRACKER::pt_loc_y[2] PUNCTURETRACKER::pt_loc_y[3] PUNCTURETRACKER::pt_loc_y[4] PUNCTURETRACKER::pt_loc_y[5] PUNCTURETRACKER::pt_loc_y[6] PUNCTURETRACKER::pt_loc_y[7] PUNCTURETRACKER::pt_loc_y[8] PUNCTURETRACKER::pt_loc_y[9] PUNCTURETRACKER::pt_loc_z[0] PUNCTURETRACKER::pt_loc_z[1] PUNCTURETRACKER::pt_loc_z[2] PUNCTURETRACKER::pt_loc_z[3] PUNCTURETRACKER::pt_loc_z[4] PUNCTURETRACKER::pt_loc_z[5] PUNCTURETRACKER::pt_loc_z[6] PUNCTURETRACKER::pt_loc_z[7] PUNCTURETRACKER::pt_loc_z[8] PUNCTURETRACKER::pt_loc_z[9] QUASILOCALMEASURES::qlm_time[0] QUASILOCALMEASURES::qlm_time[1] QUASILOCALMEASURES::qlm_time[2] QUASILOCALMEASURES::qlm_equatorial_circumference[0] QUASILOCALMEASURES::qlm_equatorial_circumference[1] QUASILOCALMEASURES::qlm_equatorial_circumference[2] QUASILOCALMEASURES::qlm_polar_circumference_0[0] QUASILOCALMEASURES::qlm_polar_circumference_0[1] QUASILOCALMEASURES::qlm_polar_circumference_0[2] QUASILOCALMEASURES::qlm_polar_circumference_pi_2[0] QUASILOCALMEASURES::qlm_polar_circumference_pi_2[1] QUASILOCALMEASURES::qlm_polar_circumference_pi_2[2] QUASILOCALMEASURES::qlm_area[0] QUASILOCALMEASURES::qlm_area[1] QUASILOCALMEASURES::qlm_area[2] QUASILOCALMEASURES::qlm_irreducible_mass[0] QUASILOCALMEASURES::qlm_irreducible_mass[1] QUASILOCALMEASURES::qlm_irreducible_mass[2] QUASILOCALMEASURES::qlm_radius[0] QUASILOCALMEASURES::qlm_radius[1] QUASILOCALMEASURES::qlm_radius[2] QUASILOCALMEASURES::qlm_spin_guess[0] QUASILOCALMEASURES::qlm_spin_guess[1] QUASILOCALMEASURES::qlm_spin_guess[2] QUASILOCALMEASURES::qlm_mass_guess[0] QUASILOCALMEASURES::qlm_mass_guess[1] QUASILOCALMEASURES::qlm_mass_guess[2] QUASILOCALMEASURES::qlm_killing_eigenvalue_re[0] QUASILOCALMEASURES::qlm_killing_eigenvalue_re[1] QUASILOCALMEASURES::qlm_killing_eigenvalue_re[2] QUASILOCALMEASURES::qlm_killing_eigenvalue_im[0] QUASILOCALMEASURES::qlm_killing_eigenvalue_im[1] QUASILOCALMEASURES::qlm_killing_eigenvalue_im[2] QUASILOCALMEASURES::qlm_spin[0] QUASILOCALMEASURES::qlm_spin[1] QUASILOCALMEASURES::qlm_spin[2] QUASILOCALMEASURES::qlm_npspin[0] QUASILOCALMEASURES::qlm_npspin[1] QUASILOCALMEASURES::qlm_npspin[2] QUASILOCALMEASURES::qlm_wsspin[0] QUASILOCALMEASURES::qlm_wsspin[1] QUASILOCALMEASURES::qlm_wsspin[2] QUASILOCALMEASURES::qlm_cvspin[0] QUASILOCALMEASURES::qlm_cvspin[1] QUASILOCALMEASURES::qlm_cvspin[2] QUASILOCALMEASURES::qlm_coordspinx[0] QUASILOCALMEASURES::qlm_coordspinx[1] QUASILOCALMEASURES::qlm_coordspinx[2] QUASILOCALMEASURES::qlm_coordspiny[0] QUASILOCALMEASURES::qlm_coordspiny[1] QUASILOCALMEASURES::qlm_coordspiny[2] QUASILOCALMEASURES::qlm_coordspinz[0] QUASILOCALMEASURES::qlm_coordspinz[1] QUASILOCALMEASURES::qlm_coordspinz[2] QUASILOCALMEASURES::qlm_mass[0] QUASILOCALMEASURES::qlm_mass[1] QUASILOCALMEASURES::qlm_mass[2] QUASILOCALMEASURES::qlm_adm_energy[0] QUASILOCALMEASURES::qlm_adm_energy[1] QUASILOCALMEASURES::qlm_adm_energy[2] QUASILOCALMEASURES::qlm_adm_momentum_x[0] QUASILOCALMEASURES::qlm_adm_momentum_x[1] QUASILOCALMEASURES::qlm_adm_momentum_x[2] QUASILOCALMEASURES::qlm_adm_momentum_y[0] QUASILOCALMEASURES::qlm_adm_momentum_y[1] QUASILOCALMEASURES::qlm_adm_momentum_y[2] QUASILOCALMEASURES::qlm_adm_momentum_z[0] QUASILOCALMEASURES::qlm_adm_momentum_z[1] QUASILOCALMEASURES::qlm_adm_momentum_z[2] QUASILOCALMEASURES::qlm_adm_angular_momentum_x[0] QUASILOCALMEASURES::qlm_adm_angular_momentum_x[1] QUASILOCALMEASURES::qlm_adm_angular_momentum_x[2] QUASILOCALMEASURES::qlm_adm_angular_momentum_y[0] QUASILOCALMEASURES::qlm_adm_angular_momentum_y[1] QUASILOCALMEASURES::qlm_adm_angular_momentum_y[2] QUASILOCALMEASURES::qlm_adm_angular_momentum_z[0] QUASILOCALMEASURES::qlm_adm_angular_momentum_z[1] QUASILOCALMEASURES::qlm_adm_angular_momentum_z[2] QUASILOCALMEASURES::qlm_w_energy[0] QUASILOCALMEASURES::qlm_w_energy[1] QUASILOCALMEASURES::qlm_w_energy[2] QUASILOCALMEASURES::qlm_w_momentum_x[0] QUASILOCALMEASURES::qlm_w_momentum_x[1] QUASILOCALMEASURES::qlm_w_momentum_x[2] QUASILOCALMEASURES::qlm_w_momentum_y[0] QUASILOCALMEASURES::qlm_w_momentum_y[1] QUASILOCALMEASURES::qlm_w_momentum_y[2] QUASILOCALMEASURES::qlm_w_momentum_z[0] QUASILOCALMEASURES::qlm_w_momentum_z[1] QUASILOCALMEASURES::qlm_w_momentum_z[2] QUASILOCALMEASURES::qlm_w_angular_momentum_x[0] QUASILOCALMEASURES::qlm_w_angular_momentum_x[1] QUASILOCALMEASURES::qlm_w_angular_momentum_x[2] QUASILOCALMEASURES::qlm_w_angular_momentum_y[0] QUASILOCALMEASURES::qlm_w_angular_momentum_y[1] QUASILOCALMEASURES::qlm_w_angular_momentum_y[2] QUASILOCALMEASURES::qlm_w_angular_momentum_z[0] QUASILOCALMEASURES::qlm_w_angular_momentum_z[1] QUASILOCALMEASURES::qlm_w_angular_momentum_z[2] INFO (CarpetIOASCII): I/O Method 'IOASCII_1D' registered: 1D AMR output of grid variables to ASCII files INFO (CarpetIOASCII): Periodic 1D AMR output requested for: ML_BSSN::phi ML_BSSN::trK WEYLSCAL4::Psi4r INFO (CarpetIOASCII): I/O Method 'IOASCII_2D' registered: 2D AMR output of grid variables to ASCII files INFO (CarpetIOASCII): I/O Method 'IOASCII_3D' registered: 3D AMR output of grid variables to ASCII files -------------------------------------------------------------------------------- INFO (Formaline): Writing tarballs with the Cactus sources into the directory "GW150914_28/cactus-source" INFO (MoL): Using Runge-Kutta 4 as the time integrator. INFO (SymBase): Symmetry on lower x-face: multipatch INFO (SymBase): Symmetry on upper x-face: multipatch INFO (SymBase): Symmetry on lower y-face: multipatch INFO (SymBase): Symmetry on upper y-face: multipatch INFO (SymBase): Symmetry on lower z-face: multipatch INFO (SymBase): Symmetry on upper z-face: multipatch INFO (MoL): The maximum number of evolved variables is 1404. 25 are registered. INFO (MoL): The maximum number of slow evolved variables is 1404. 0 are registered. INFO (MoL): The maximum number of constrained variables is 1404. 20 are registered. INFO (MoL): The maximum number of SandR variables is 1404. 0 are registered. INFO (MoL): The maximum number of evolved array variables is 1404. 0 are registered. INFO (MoL): The maximum number of constrained array variables is 1404. 0 are registered. INFO (MoL): The maximum number of SandR array variables is 1404. 0 are registered. INFO (MoL): The maximum size of any array variables is 0. INFO (Vectors): Testing vectorisation... [errors may result in segfaults] INFO (Vectors): 375/375 tests passed INFO (CarpetRegrid2): Enforcing grid structure properties, iteration 0 INFO (CarpetRegrid2): Enforcing grid structure properties, iteration 1 INFO (CarpetRegrid2): Enforcing grid structure properties, iteration 0 INFO (CarpetRegrid2): Enforcing grid structure properties, iteration 1 INFO (CarpetRegrid2): Enforcing grid structure properties, iteration 0 INFO (CarpetRegrid2): Enforcing grid structure properties, iteration 1 INFO (CarpetRegrid2): Enforcing grid structure properties, iteration 0 INFO (CarpetRegrid2): Enforcing grid structure properties, iteration 1 INFO (CarpetRegrid2): Enforcing grid structure properties, iteration 0 INFO (CarpetRegrid2): Enforcing grid structure properties, iteration 1 INFO (CarpetRegrid2): Enforcing grid structure properties, iteration 0 INFO (CarpetRegrid2): Enforcing grid structure properties, iteration 1 INFO (Carpet): Grid structure (superregions, grid points): [0][0][0] exterior: [0,0,0] : [22,22,16] ([23,23,17] + PADDING) 8993 [1][0][0] exterior: [8,8,5] : [36,36,24] ([29,29,20] + PADDING) 16820 [2][0][0] exterior: [24,24,15] : [64,64,40] ([41,41,26] + PADDING) 43706 [3][0][0] exterior: [56,56,35] : [120,120,72] ([65,65,38] + PADDING) 160550 [4][0][0] exterior: [120,136,75] : [232,216,120] ([113,81,46] + PADDING) 421038 [5][0][0] exterior: [309,320,155] : [393,384,192] ([85,65,38] + PADDING) 209950 [6][0][0] exterior: [666,687,315] : [738,721,337] ([73,35,23] + PADDING) 58765 INFO (Carpet): Grid structure (superregions, coordinates): [0][0][0] exterior: [-188.455384615385,-188.455384615385,-85.6615384615385] : [188.455384615385,188.455384615385,188.455384615385] : [17.1323076923077,17.1323076923077,17.1323076923077] [1][0][0] exterior: [-119.926153846154,-119.926153846154,-42.8307692307692] : [119.926153846154,119.926153846154,119.926153846154] : [8.56615384615385,8.56615384615385,8.56615384615385] [2][0][0] exterior: [-85.6615384615385,-85.6615384615385,-21.4153846153846] : [85.6615384615384,85.6615384615384,85.6615384615385] : [4.28307692307692,4.28307692307692,4.28307692307692] [3][0][0] exterior: [-68.5292307692308,-68.5292307692308,-10.7076923076923] : [68.5292307692308,68.5292307692308,68.5292307692308] : [2.14153846153846,2.14153846153846,2.14153846153846] [4][0][0] exterior: [-59.9630769230769,-42.8307692307692,-5.35384615384615] : [59.9630769230769,42.8307692307692,42.8307692307692] : [1.07076923076923,1.07076923076923,1.07076923076923] [5][0][0] exterior: [-23.0215384615385,-17.1323076923077,-2.67692307692307] : [21.9507692307692,17.1323076923077,17.1323076923077] : [0.535384615384615,0.535384615384615,0.535384615384615] [6][0][0] exterior: [-10.1723076923077,-4.55076923076925,-1.33846153846154] : [9.10153846153844,4.55076923076922,4.55076923076923] : [0.267692307692308,0.267692307692308,0.267692307692308] INFO (Carpet): Grid structure (superregions, grid points): [0][1][0] exterior: [0,0,0] : [15,20,138] ([16,21,139] + PADDING) 46704 INFO (Carpet): Grid structure (superregions, coordinates): [0][1][0] exterior: [-1.96349540849362,-3.92699081698724,-85.6615384615385] : [3.92699081698724,3.92699081698724,2278.59692307692] : [0.392699081698724,0.392699081698724,17.1323076923077] INFO (Carpet): Grid structure (superregions, grid points): [0][2][0] exterior: [0,0,0] : [15,20,138] ([16,21,139] + PADDING) 46704 INFO (Carpet): Grid structure (superregions, coordinates): [0][2][0] exterior: [-3.92699081698724,-3.92699081698724,-85.6615384615385] : [1.96349540849362,3.92699081698724,2278.59692307692] : [0.392699081698724,0.392699081698724,17.1323076923077] INFO (Carpet): Grid structure (superregions, grid points): [0][3][0] exterior: [0,0,0] : [15,20,138] ([16,21,139] + PADDING) 46704 INFO (Carpet): Grid structure (superregions, coordinates): [0][3][0] exterior: [-1.96349540849362,-3.92699081698724,-85.6615384615385] : [3.92699081698724,3.92699081698724,2278.59692307692] : [0.392699081698724,0.392699081698724,17.1323076923077] INFO (Carpet): Grid structure (superregions, grid points): [0][4][0] exterior: [0,0,0] : [15,20,138] ([16,21,139] + PADDING) 46704 INFO (Carpet): Grid structure (superregions, coordinates): [0][4][0] exterior: [-3.92699081698724,-3.92699081698724,-85.6615384615385] : [1.96349540849362,3.92699081698724,2278.59692307692] : [0.392699081698724,0.392699081698724,17.1323076923077] INFO (Carpet): Grid structure (superregions, grid points): [0][5][0] exterior: [0,0,0] : [20,20,138] ([21,21,139] + PADDING) 61299 INFO (Carpet): Grid structure (superregions, coordinates): [0][5][0] exterior: [-3.92699081698724,-3.92699081698724,-85.6615384615385] : [3.92699081698724,3.92699081698724,2278.59692307692] : [0.392699081698724,0.392699081698724,17.1323076923077] INFO (Carpet): Global grid structure statistics: INFO (Carpet): GF: rhs: 33k active, 118k owned (+261%), 210k total (+78%), 64 steps/time INFO (Carpet): GF: vars: 305, pts: 100M active, 221M owned (+122%), 431M total (+95%), 1.0 comp/proc INFO (Carpet): GA: vars: 1126, pts: 2M active, 2M total (+0%) INFO (Carpet): Total required memory: 3.456 GByte (for GAs and currently active GFs) INFO (Carpet): Load balance: min avg max sdv max/avg-1 INFO (Carpet): Level 0: 16M 16M 16M 0M owned 0% INFO (Carpet): Level 1: 1M 1M 1M 0M owned 0% INFO (Carpet): Level 2: 5M 5M 5M 0M owned 0% INFO (Carpet): Level 3: 26M 26M 26M 0M owned 0% INFO (Carpet): Level 4: 104M 104M 104M 0M owned 0% INFO (Carpet): Level 5: 56M 56M 56M 0M owned 0% INFO (Carpet): Level 6: 14M 14M 14M 0M owned 0% INFO (CartGrid3D): Grid Spacings: INFO (CartGrid3D): dx=>1.7132308e+01 dy=>1.7132308e+01 dz=>1.7132308e+01 INFO (CartGrid3D): Computational Coordinates: INFO (CartGrid3D): x=>[-188.455,188.455] y=>[-188.455,188.455] z=>[-85.662,188.455] INFO (CartGrid3D): Indices of Physical Coordinates: INFO (CartGrid3D): x=>[0,22] y=>[0,22] z=>[0,16] WARNING[L3,P0] (Cactus): CCTK_CoordRange: Range already registered for system 'cart3d' WARNING[L3,P0] (Cactus): CCTK_CoordRegisterRangePhysIndex: Range already registered for system 'cart3d' WARNING[L3,P0] (Cactus): CCTK_CoordRange: Range already registered for system 'cart3d' WARNING[L3,P0] (Cactus): CCTK_CoordRegisterRangePhysIndex: Range already registered for system 'cart3d' WARNING[L3,P0] (Cactus): CCTK_CoordRange: Range already registered for system 'cart3d' WARNING[L3,P0] (Cactus): CCTK_CoordRegisterRangePhysIndex: Range already registered for system 'cart3d' INFO (CartGrid3D): Grid Spacings: INFO (CartGrid3D): dx=>3.9269908e-01 dy=>3.9269908e-01 dz=>1.7132308e+01 INFO (CartGrid3D): Computational Coordinates: INFO (CartGrid3D): x=>[-1.963, 3.927] y=>[-3.927, 3.927] z=>[-85.662,2278.597] INFO (CartGrid3D): Indices of Physical Coordinates: INFO (CartGrid3D): x=>[0,15] y=>[0,20] z=>[0,138] WARNING[L3,P0] (Cactus): CCTK_CoordRange: Range already registered for system 'cart3d' WARNING[L3,P0] (Cactus): CCTK_CoordRegisterRangePhysIndex: Range already registered for system 'cart3d' WARNING[L3,P0] (Cactus): CCTK_CoordRange: Range already registered for system 'cart3d' WARNING[L3,P0] (Cactus): CCTK_CoordRegisterRangePhysIndex: Range already registered for system 'cart3d' WARNING[L3,P0] (Cactus): CCTK_CoordRange: Range already registered for system 'cart3d' WARNING[L3,P0] (Cactus): CCTK_CoordRegisterRangePhysIndex: Range already registered for system 'cart3d' INFO (CartGrid3D): Grid Spacings: INFO (CartGrid3D): dx=>3.9269908e-01 dy=>3.9269908e-01 dz=>1.7132308e+01 INFO (CartGrid3D): Computational Coordinates: INFO (CartGrid3D): x=>[-3.927, 1.963] y=>[-3.927, 3.927] z=>[-85.662,2278.597] INFO (CartGrid3D): Indices of Physical Coordinates: INFO (CartGrid3D): x=>[0,15] y=>[0,20] z=>[0,138] WARNING[L3,P0] (Cactus): CCTK_CoordRange: Range already registered for system 'cart3d' WARNING[L3,P0] (Cactus): CCTK_CoordRegisterRangePhysIndex: Range already registered for system 'cart3d' WARNING[L3,P0] (Cactus): CCTK_CoordRange: Range already registered for system 'cart3d' WARNING[L3,P0] (Cactus): CCTK_CoordRegisterRangePhysIndex: Range already registered for system 'cart3d' WARNING[L3,P0] (Cactus): CCTK_CoordRange: Range already registered for system 'cart3d' WARNING[L3,P0] (Cactus): CCTK_CoordRegisterRangePhysIndex: Range already registered for system 'cart3d' INFO (CartGrid3D): Grid Spacings: INFO (CartGrid3D): dx=>3.9269908e-01 dy=>3.9269908e-01 dz=>1.7132308e+01 INFO (CartGrid3D): Computational Coordinates: INFO (CartGrid3D): x=>[-1.963, 3.927] y=>[-3.927, 3.927] z=>[-85.662,2278.597] INFO (CartGrid3D): Indices of Physical Coordinates: INFO (CartGrid3D): x=>[0,15] y=>[0,20] z=>[0,138] WARNING[L3,P0] (Cactus): CCTK_CoordRange: Range already registered for system 'cart3d' WARNING[L3,P0] (Cactus): CCTK_CoordRegisterRangePhysIndex: Range already registered for system 'cart3d' WARNING[L3,P0] (Cactus): CCTK_CoordRange: Range already registered for system 'cart3d' WARNING[L3,P0] (Cactus): CCTK_CoordRegisterRangePhysIndex: Range already registered for system 'cart3d' WARNING[L3,P0] (Cactus): CCTK_CoordRange: Range already registered for system 'cart3d' WARNING[L3,P0] (Cactus): CCTK_CoordRegisterRangePhysIndex: Range already registered for system 'cart3d' INFO (CartGrid3D): Grid Spacings: INFO (CartGrid3D): dx=>3.9269908e-01 dy=>3.9269908e-01 dz=>1.7132308e+01 INFO (CartGrid3D): Computational Coordinates: INFO (CartGrid3D): x=>[-3.927, 1.963] y=>[-3.927, 3.927] z=>[-85.662,2278.597] INFO (CartGrid3D): Indices of Physical Coordinates: INFO (CartGrid3D): x=>[0,15] y=>[0,20] z=>[0,138] WARNING[L3,P0] (Cactus): CCTK_CoordRange: Range already registered for system 'cart3d' WARNING[L3,P0] (Cactus): CCTK_CoordRegisterRangePhysIndex: Range already registered for system 'cart3d' WARNING[L3,P0] (Cactus): CCTK_CoordRange: Range already registered for system 'cart3d' WARNING[L3,P0] (Cactus): CCTK_CoordRegisterRangePhysIndex: Range already registered for system 'cart3d' WARNING[L3,P0] (Cactus): CCTK_CoordRange: Range already registered for system 'cart3d' WARNING[L3,P0] (Cactus): CCTK_CoordRegisterRangePhysIndex: Range already registered for system 'cart3d' INFO (CartGrid3D): Grid Spacings: INFO (CartGrid3D): dx=>3.9269908e-01 dy=>3.9269908e-01 dz=>1.7132308e+01 INFO (CartGrid3D): Computational Coordinates: INFO (CartGrid3D): x=>[-3.927, 3.927] y=>[-3.927, 3.927] z=>[-85.662,2278.597] INFO (CartGrid3D): Indices of Physical Coordinates: INFO (CartGrid3D): x=>[0,20] y=>[0,20] z=>[0,138] INFO (AHFinderDirect): setting up AHFinderDirect data structures INFO (AHFinderDirect): to search for 3 horizons on 1 processor INFO (AHFinderDirect): constructing full sphere patch system INFO (AHFinderDirect): at origin (4.46154,0,0) INFO (AHFinderDirect): with 18 angular zones per right angle INFO (AHFinderDirect): ==> 2166 nominal, 3174 ghosted angular grid points INFO (AHFinderDirect): constructing full sphere patch system INFO (AHFinderDirect): at origin (-5.53846,0,0) INFO (AHFinderDirect): with 18 angular zones per right angle INFO (AHFinderDirect): ==> 2166 nominal, 3174 ghosted angular grid points INFO (AHFinderDirect): constructing full sphere patch system INFO (AHFinderDirect): at origin (0,0,0) INFO (AHFinderDirect): with 18 angular zones per right angle INFO (AHFinderDirect): ==> 2166 nominal, 3174 ghosted angular grid points WARNING level 0 from host panther process 0 in thorn CoordinatesSymmetry, file /system/user/crangano/einstein_toolkit/Cactus/configs/sim/build/CoordinatesSymmetry/apply.c:665: -> Cannot apply symmetry boundary zones in the lower x direction, since there seem to be more symmetry zones than interior zones Simfactory Done at date: Wed 10 Sep 2025 05:10:09 PM CEST From cranganore at ml.jku.at Wed Sep 10 13:14:03 2025 From: cranganore at ml.jku.at (Sandeep Suresh CrangaNORE) Date: Wed, 10 Sep 2025 20:14:03 +0200 Subject: [Users] Fwd: Issues Running BNS and GW150914 Example Simulations + storing BSSN variables In-Reply-To: <2dd52b48-05bb-4830-a119-ea83945babcb@ml.jku.at> References: <2dd52b48-05bb-4830-a119-ea83945babcb@ml.jku.at> Message-ID: <8dab991a-59d2-4607-bdc5-b1b8503c47df@ml.jku.at> Dear EinsteinToolkit team, I am forwarding the email, since it was sent to the Post by non-member to a members-only list -- incase, the email was not received by you. -------- Forwarded Message -------- Subject: Issues Running BNS and GW150914 Example Simulations + storing BSSN variables Date: Wed, 10 Sep 2025 19:25:49 +0200 From: Sandeep Suresh CrangaNORE To: users at einsteintoolkit.org CC: haas at illinois.edu, brandstetter at ml.jku.at, andreibodnar30 at gmail.com Dear EinsteinToolkit team, I am currently running the Binary Neutron Star (BNS) --?(https://einsteintoolkit.org/gallery/bns/index.html)?and GW150914?(https://einsteintoolkit.org/gallery/bbh/index.html)??binary black hole example simulations on our workstation environment: * *Processors:* AMD EPYC 7542, 32 cores per socket, 2 threads per core (64 logical cores) * *Memory:* 2?TiB RAM * *Kernel:* Linux 5.14.0-570.37.1.el9_6.x86_64 * *OPENMPI version:* mpirun (Open MPI) 4.1.1 Both simulations use the standard example parameter files and initial data provided in the toolkit. I shall describe the problems while running these simulations as two different sections: *BNS: *On running the command ./simfactory/bin/sim create-submit bns --parfile bns.par --procs= --num-threads= --walltime=xx:xx:xx I encounter the error: WARNING level 1 from host panther process 0 ? in thorn ML_BSSN_Helper, file /system/user/crangano/einstein_toolkit/BNS/configs/sim/build/ML_BSSN_Helper/Parameters.c:166: ? -> Parameter ML_BSSN::my_boundary_condition is outdated; please update the parameter file. Do not use this parameter, and set up RHS boundary conditions as usual. cactus_sim: grille3d.C:125: Grille3d::Grille3d(int, int, int, int, int, int, int): Assertion `nr > 0' failed. Rank 0 with PID 3273101 received signal 6 For this I have attached the?par file,?err file, the out file (the initial data file is the one that is already there online: G2_I12vs12_D4R33T21_45km.resu.xz ) that I am currently using. I tried to find the `nr` in the par file, but didn't find it. I am not sure if this is a MPI/num procs related problem, or is it something in the .par file that one needs to be change ? *GW150914: */Error 1 -- /Here, it is much more complicated to understand for me.?This persists even after disabling |CoordinatesSymmetry::reflection_z|.?Is it likely caused by coarse grid resolution relative to the number of symmetry/ghost zones. /Error 2/ -- Another error that I often encounter is File "/system/user/crangano/simulations/GW150914_28/output-0000/GW150914.rpar", line 126, in sphere_outer_radius = int((outermost_detector + final_time)/(i*hr))*i*hr ZeroDivisionError: float division by zero Error: Error while executing parameter file script /system/user/crangano/simulations/GW150914_28/output-0000/GW150914.rpar Aborting Simfactory. -- I guess this occurs in the following line of the *GW150914.rpar* sphere_outer_radius=int((outermost_detector+final_time)/(i*hr))*i*hr sphere_outer_radius=int(sphere_outer_radius/hr) *hr+hr# round up to a multiple of hr So, I changed line 78 of the *GW150914.rpar as follows: * # Number of cells across finest grid radius n=int("@N@") if"@N@"[0] !="@"else28 i=max(int(n/4), 1) Is this a valid fix, or does this affect the simulations ? For this too, I attach the err, out and the par file such that you can inspect it. *Storing BSSN variables*:We would like to store the BSSN evolved *3-metric* (|?_ij|), *lapse* (|?|), and *shift* (|?^i|) and also the corresponding coordinates (t, x^i) of these quantities? at regular intervals (e.g., every 128 steps) to manage disk usage efficiently. Moreover, since AMR is used, is there any way we can keep track of the changes in resolution of the coordinates, since we also aim to do spatial derivatives via our FD stencils -- Jacobians and Hessians of these quantities. If there are ways to directly dump the Jacobians and Hessians of these quantities (by adding lines on the par file), w/o us implementing (since we are not aware of the AMR being used at different regions to implement our FD stencils), that would be very useful too. IOHDF5::out_vars = " ML_BSSN::ML_metric ML_BSSN::ML_lapse ML_BSSN::ML_shift Grid::Coordinates{out_every=1000000000 refinement_levels={0}} ML_BSSN::ML_log_confac WeylScal4::Psi4r WeylScal4::Psi4i WeylScal4::curvIr{refinement_levels={3 5}} WeylScal4::curvIi{refinement_levels={3 5}} WeylScal4::curvJr{refinement_levels={3 5}} WeylScal4::curvJi{refinement_levels={3 5}} " I would appreciate guidance on the proper way to quickly resolve these errors. Ideally, we don't need something very specific -- Just want to get these simulations running with minimum tweaking, given that these are tested gallery example and more importantly store the time-evolved metric, lapse and the shift for both these usecases. Thanks a lot in advance for your help! If you need anything do let us know. Best regards, Sandeep -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- # Carpet parameter file for binary Neutron star system # physical ID is LORENE dataset G2_I12vs12_D4R33T21_45km.resu # #------------------------------------------------------------------------------ # Cactus parameters: #------------------------------------------------------------------------------ Cactus::cctk_run_title = "Meudon BNS" Cactus::cctk_full_warnings = "yes" Cactus::highlight_warning_messages = "no" Cactus::terminate = "time" Cactus::cctk_final_time = 2500.0 #------------------------------------------------------------------------------ # Activate all necessary thorns: #------------------------------------------------------------------------------ ActiveThorns = "Boundary CartGrid3D CoordBase Fortran InitBase IOUtil LocalReduce SymBase Time" ActiveThorns = "AEILocalInterp" ActiveThorns = "MoL Slab SpaceMask SphericalSurface" ActiveThorns = "Carpet CarpetInterp CarpetInterp2 CarpetIOASCII CarpetIOHDF5 CarpetIOScalar CarpetLib CarpetIOBasic CarpetReduce CarpetRegrid2 CarpetSlab CarpetTracker CarpetMask LoopControl" ActiveThorns = "Formaline" ActiveThorns = "HTTPD Socket" ActiveThorns = "NaNChecker TerminationTrigger TimerReport" ActiveThorns = "ADMbase ADMcoupling ADMmacros CoordGauge StaticConformal" ActiveThorns = "RotatingSymmetry180 ReflectionSymmetry" ActiveThorns = "Constants TmunuBase HydroBase " ActiveThorns = "QuasiLocalMeasures" ActiveThorns = "EOS_Omni" ActiveThorns = "GRHydro" ActiveThorns = "SummationByParts" ActiveThorns = "GenericFD NewRad" ActiveThorns = "ML_BSSN ML_BSSN_Helper ML_ADMConstraints" ActiveThorns = "Hydro_Analysis NSTracker" ActiveThorns = "Dissipation" ActiveThorns = "SystemStatistics SystemTopology" # Wave extraction (Psi4) ActiveThorns = "WeylScal4 Multipole" #------------------------------------------------------------------------------ # Diagnostic parameters: #------------------------------------------------------------------------------ Carpet::output_timers_every = 0 Carpet::storage_verbose = "no" Carpet::verbose = "no" Carpet::veryverbose = "no" Carpet::grid_structure_filename = "carpet-grid-structure" Carpet::grid_coordinates_filename = "carpet-grid-coordinates" CarpetLib::output_bboxes = "no" CarpetMask::verbose = "no" CarpetReduce::verbose = "no" CarpetRegrid2::verbose = "no" CarpetRegrid2::veryverbose = "no" CarpetTracker::verbose = "no" TimerReport::out_every = 4096 TimerReport::out_filename = "TimerReport" TimerReport::output_all_timers = "yes" TimerReport::output_all_timers_together = "yes" TimerReport::output_all_timers_readable = "yes" TimerReport::n_top_timers = 40 QuasiLocalMeasures::verbose = "no" SphericalSurface::verbose = "no" #------------------------------------------------------------------------------ # Utility parameters: #------------------------------------------------------------------------------ NaNChecker::check_every = 128 # twice for every_coarse NaNChecker::check_vars = " ADMBase::curv ADMBase::metric ADMBase::lapse ADMBase::shift HydroBase::rho HydroBase::eps HydroBase::press HydroBase::vel " NaNChecker::action_if_found = "terminate" #NaNChecker::action_if_found = "just warn" #"terminate", "just warn", "abort" #------------------------------------------------------------------------------ # Run parameters: #------------------------------------------------------------------------------ #------ # Grid: #------ MoL::ODE_Method = "rk4" MoL::MoL_Intermediate_Steps = 4 MoL::MoL_Num_Scratch_Levels = 1 # use dt = 0.4 dx (works for core collapse) Time::dtfac = 0.4 ActiveThorns = "CoordBase" CoordBase::domainsize = "minmax" CoordBase::xmin = 0.00 CoordBase::ymin = 0.00 CoordBase::zmin = 0.00 CoordBase::xmax = +400.00 CoordBase::ymax = +400.00 CoordBase::zmax = +400.00 CoordBase::dx = 16.00 CoordBase::dy = 16.00 CoordBase::dz = 16.00 CoordBase::boundary_size_x_lower = 3 CoordBase::boundary_size_y_lower = 3 CoordBase::boundary_size_z_lower = 3 CoordBase::boundary_size_x_upper = 3 CoordBase::boundary_size_y_upper = 3 CoordBase::boundary_size_z_upper = 3 CoordBase::boundary_shiftout_x_lower = 1 CoordBase::boundary_shiftout_y_lower = 0 CoordBase::boundary_shiftout_z_lower = 1 ReflectionSymmetry::reflection_z = "yes" ReflectionSymmetry::avoid_origin_z = "no" CartGrid3D::type = "coordbase" Carpet::domain_from_coordbase = "yes" Driver::ghost_size = 3 # General Carpet parameters: Carpet::enable_all_storage = "no" Carpet::use_buffer_zones = "yes" Carpet::schedule_barriers = "no" Carpet::poison_new_timelevels = "yes" Carpet::check_for_poison = "no" Carpet::init_3_timelevels = "no" Carpet::init_fill_timelevels = "yes" CarpetLib::poison_new_memory = "yes" CarpetLib::poison_value = 114 CarpetLib::check_bboxes = "no" CarpetLib::interleave_communications = "yes" CarpetLib::combine_sends = "yes" CarpetInterp::tree_search = "yes" CarpetInterp::check_tree_search = "no" CarpetRegrid2::freeze_unaligned_levels = "yes" CarpetRegrid2::freeze_unaligned_parent_levels = "yes" CarpetRegrid2::ensure_proper_nesting = "yes" CarpetRegrid2::snap_to_coarse = "yes" CarpetRegrid2::symmetry_rotating180 = "yes" # System specific Carpet parameters: Carpet::max_refinement_levels = 9 Carpet::prolongation_order_space = 5 Carpet::prolongation_order_time = 2 Carpet::refinement_centering = "vertex" CarpetRegrid2::regrid_every = 2 # as often as required CarpetRegrid2::num_centres = 3 # box sizes are given by: # * the stars seem to puff up to about 13M during the initial phase of the evolution # * I need 12 buffer points (RK4, 3 ghost zones) # * need three coarse points for interpolation onto last fine buffer point # these boxes are minimal in this sense. The coarser grid are completely # covered by the finer grids and their buffers. # add 4 coarse grid points in between to have some leeway against roundoff # grid step sizes are for coarsest anticipated simulation dx = 1.5M CarpetRegrid2::num_levels_1 = 7 CarpetRegrid2::position_x_1 = 15.1875 CarpetRegrid2::radius_1[1] =240.0 CarpetRegrid2::radius_1[2] =120.0 CarpetRegrid2::radius_1[3] = 60.0 CarpetRegrid2::radius_1[4] = 26.125 CarpetRegrid2::radius_1[5] = 17.875 CarpetRegrid2::radius_1[6] = 13 # star seems about 9.5 intially, then puffs up CarpetRegrid2::num_levels_2 = 7 CarpetRegrid2::position_x_2 = -15.1875 CarpetRegrid2::radius_2[1] =240.0 CarpetRegrid2::radius_2[2] =120.0 CarpetRegrid2::radius_2[3] = 60.0 CarpetRegrid2::radius_2[4] = 26.125 CarpetRegrid2::radius_2[5] = 17.875 CarpetRegrid2::radius_2[6] = 13 CarpetRegrid2::num_levels_3 = 1 CarpetRegrid2::radius_3[1] =240.0 CarpetRegrid2::radius_3[2] =120.0 CarpetRegrid2::radius_3[3] = 60.0 CarpetRegrid2::radius_3[4] = 30.0 CarpetRegrid2::radius_3[5] = 15.0 CarpetRegrid2::radius_3[6] = 7.5 CarpetRegrid2::radius_3[7] = 3.75 CarpetMask::excluded_surface [0] = 2 CarpetMask::excluded_surface_factor[0] = 1.0 CarpetTracker::surface_name[0] = "Righthand NS" CarpetTracker::surface_name[1] = "Lefthand NS" #------ # MODEL: #------ ActiveThorns = "Meudon_Bin_NS" HydroBase::initial_hydro = "Meudon_Bin_NS" ADMBase::initial_data = "Meudon_Bin_NS" ADMBase::initial_lapse = "Meudon_Bin_NS" ADMBase::initial_shift = "zero" ADMBase::initial_dtlapse = "Meudon_Bin_NS" ADMBase::initial_dtshift = "zero" # change this to be the full path to he initial data file Meudon_Bin_NS::filename ="/system/user/crangano/einstein_toolkit/BNS/G2_I12vs12_D4R33T21_45km.resu" # M_ADM 3.251 # separation 45km # K 123.6 # Gamma 2 EOS_Omni::poly_K = 123.613314525753 # For other (non-Polytropic) EOSs Meudon_Bin_NS::eos_table should be labelled # e.g. ="Hybrid" for hybrid EOS. #---------- # Numerics: #---------- InitBase::initial_data_setup_method = "init_some_levels" TmunuBase::stress_energy_storage = "yes" TmunuBase::stress_energy_at_RHS = "yes" TmunuBase::timelevels = 1 TmunuBase::prolongation_type = "none" TmunuBase::support_old_CalcTmunu_mechanism = "no" HydroBase::timelevels = 3 SpaceMask::use_mask = "yes" SphericalSurface::nsurfaces = 5 SphericalSurface::maxntheta = 39 SphericalSurface::maxnphi = 76 SphericalSurface::ntheta [0] = 39 SphericalSurface::nphi [0] = 76 SphericalSurface::nghoststheta[0] = 2 SphericalSurface::nghostsphi [0] = 2 SphericalSurface::name [0] = "Righthand NS" SphericalSurface::ntheta [1] = 39 SphericalSurface::nphi [1] = 76 SphericalSurface::nghoststheta[1] = 2 SphericalSurface::nghostsphi [1] = 2 SphericalSurface::name [1] = "Lefthand NS" SphericalSurface::ntheta [3] = 39 SphericalSurface::nphi [3] = 76 SphericalSurface::nghoststheta[3] = 2 SphericalSurface::nghostsphi [3] = 2 SphericalSurface::set_spherical[3] = yes SphericalSurface::radius [3] = 100 SphericalSurface::name [3] = "waveextract surface at 100" SphericalSurface::ntheta [4] = 39 SphericalSurface::nphi [4] = 76 SphericalSurface::nghoststheta[4] = 2 SphericalSurface::nghostsphi [4] = 2 SphericalSurface::set_spherical[4] = yes SphericalSurface::radius [4] = 250 SphericalSurface::name [4] = "waveextract surface at 250" #----------- # Evolution: #----------- HydroBase::evolution_method = "GRHydro" ADMMacros::spatial_order = 4 GRHydro::sources_spatial_order = 4 GRHydro::riemann_solver = "HLLE" # Marquina is currently not supported by MP GRHydro::recon_method = "ppm" GRHydro::GRHydro_stencil = 3 GRHydro::bound = "flat" GRHydro::rho_abs_min = 1.e-11 GRHydro::GRHydro_atmo_tolerance = 0.01 GRHydro::c2p_reset_pressure = "yes" GRHydro::GRHydro_eos_type = "General" GRHydro::GRHydro_eos_table = "Ideal_Fluid" # these can save some memory since they prevent MoL from allocating unnecessary # scratch space for saveandrestore variables GRHydro::GRHydro_MaxNumSandRVars = 0 GRHydro::use_enhanced_ppm = "yes" # Parameters are defaults, which in turn are from Colella & Sekora 2008 and # McCorquodale & Colella 2011 GRHydro::sync_conserved_only = "yes" GRHydro::reconstruct_Wv = "yes" GRHydro::c2p_resort_to_bisection = "yes" GRHydro::use_cxx_code = "yes" # MacLachlan evolution parameters ADMBase::metric_type = physical ADMBase::evolution_method = ML_BSSN ADMBase::lapse_evolution_method = ML_BSSN ADMBase::shift_evolution_method = ML_BSSN ADMBase::dtlapse_evolution_method = ML_BSSN ADMBase::dtshift_evolution_method = ML_BSSN ML_BSSN::timelevels = 3 ML_BSSN::my_initial_data = "ADMBase" ML_BSSN::my_initial_boundary_condition = "extrapolate-gammas" ML_BSSN::my_boundary_condition = "none" ML_BSSN::my_rhs_boundary_condition = "NewRad" Boundary::radpower = 2 # not really needed I think but who knows what NewRad uses ML_BSSN::harmonicN = 1 # 1+log ML_BSSN::harmonicF = 2.0 # 1+log ML_BSSN::ShiftGammaCoeff = 0.75 ML_BSSN::AlphaDriver = 0.0 ML_BSSN::BetaDriver = 1.0 ML_BSSN::advectLapse = 1.0 ML_BSSN::advectShift = 1.0 ML_BSSN::MinimumLapse = 1.0e-8 ML_BSSN::ML_log_confac_bound = "none" ML_BSSN::ML_metric_bound = "none" ML_BSSN::ML_Gamma_bound = "none" ML_BSSN::ML_trace_curv_bound = "none" ML_BSSN::ML_curv_bound = "none" ML_BSSN::ML_lapse_bound = "none" ML_BSSN::ML_dtlapse_bound = "none" ML_BSSN::ML_shift_bound = "none" ML_BSSN::ML_dtshift_bound = "none" ML_BSSN::UseSpatialBetaDriver = 1 ML_BSSN::SpatialBetaDriverRadius = 50 ML_BSSN::apply_dissipation = "never" Dissipation::epsdis = 0.1 Dissipation::order = 5 Dissipation::vars = " ML_BSSN::ML_log_confac ML_BSSN::ML_metric ML_BSSN::ML_trace_curv ML_BSSN::ML_curv ML_BSSN::ML_Gamma ML_BSSN::ML_lapse ML_BSSN::ML_shift ML_BSSN::ML_dtlapse ML_BSSN::ML_dtshift " #------------------------------------------------------------------------------ # Output: #------------------------------------------------------------------------------ IO::out_dir = $parfile IOBasic::outInfo_every = 1 IOBasic::outInfo_reductions = "maximum" IOBasic::outInfo_vars = " Carpet::physical_time_per_hour HydroBase::rho ML_ADMConstraints::ML_Ham SystemStatistics::maxrss_mb GRHydro::dens{reductions = 'sum maximum'} HydroBase::w_lorentz " IOScalar::outScalar_every = 256 # every_coarse IOScalar::all_reductions_in_one_file = "yes" IOScalar::one_file_per_group = "yes" IOScalar::outScalar_reductions = "minimum maximum average norm1 norm2" IOScalar::outScalar_vars = " ADMBase::lapse ADMBase::shift ADMBase::metric ADMBase::curv HydroBase::rho HydroBase::vel HydroBase::w_lorentz GRHydro::dens{reductions = 'minimum maximum average norm1 norm2 sum'} SystemStatistics::process_memory_mb SphericalSurface::sf_radius ML_ADMConstraints::ML_Ham " IOASCII::one_file_per_group = "yes" IOASCII::compact_format = "yes" IOASCII::out0D_every = 256 # every_coarse IOASCII::out0D_vars = " Carpet::timing QuasiLocalMeasures::qlm_scalars SphericalSurface::sf_active SphericalSurface::sf_valid SphericalSurface::sf_info SphericalSurface::sf_radius SphericalSurface::sf_origin SphericalSurface::sf_coordinate_descriptors Hydro_Analysis::Hydro_Analysis_rho_max_loc Hydro_Analysis::Hydro_Analysis_rho_max_origin_distance " #Set these IOASCII options for initial data only: IOASCII::out1D_every = 0 IOASCII::out1D_d = "no" IOASCII::out1D_vars = " HydroBase::rho HydroBase::vel ADMBase::lapse ADMBase::shift ADMBase::metric ADMBase::curv ML_ADMConstraints::ML_Ham " CarpetIOHDF5::one_file_per_group = "no" # this is required by multipatch CarpetIOHDF5::open_one_input_file_at_a_time = "yes" CarpetIOHDF5::out2D_every = 1536 # 6*every coarse CarpetIOHDF5::out2D_xy = "yes" CarpetIOHDF5::out2D_xz = "no" CarpetIOHDF5::out2D_yz = "no" CarpetIOHDF5::out2D_xyplane_z = 0.0 CarpetIOHDF5::out2D_vars = " CarpetReduce::weight Grid::coordinates HydroBase::rho HydroBase::vel HydroBase::eps ADMBase::lapse ADMBase::shift ADMBase::metric ML_ADMConstraints::ML_Ham # Hydro_Analysis::Hydro_Analysis_Temperature " IOHDF5::out3D_every = 8192 # = 32*every_coarse IOHDF5::out3D_vars = " CarpetReduce::weight HydroBase::rho HydroBase::vel HydroBase::eps ADMBase::lapse ADMBase::shift ML_ADMConstraints::ML_Ham grid::coordinates " #------------------------------------------------------------------------------ # Analysis: #------------------------------------------------------------------------------ Hydro_Analysis::Hydro_Analysis_comp_rho_max = "true" Hydro_Analysis::Hydro_Analysis_rho_max_loc_only_positive_x = "true" Hydro_Analysis::Hydro_Analysis_comp_rho_max_origin_distance = "yes" Hydro_Analysis::Hydro_Analysis_average_multiple_maxima_locations = "yes" Hydro_Analysis::Hydro_Analysis_interpolator_name = "Lagrange polynomial interpolation (tensor product)" NSTracker::NSTracker_SF_Name = "Righthand NS" NSTracker::NSTracker_SF_Name_Opposite = "Lefthand NS" NSTracker::NSTracker_max_distance = 3 NSTracker::NSTracker_verbose = "yes" NSTracker::NSTracker_tracked_location = "Hydro_Analysis::Hydro_Analysis_rho_max_loc" QuasiLocalMeasures::num_surfaces = 2 QuasiLocalMeasures::spatial_order = 4 QuasiLocalMeasures::interpolator = "Lagrange polynomial interpolation" QuasiLocalMeasures::interpolator_options = "order=4" QuasiLocalMeasures::surface_name[0] = "waveextract surface at 100" QuasiLocalMeasures::surface_name[1] = "waveextract surface at 250" ################################################################################ ################################################################################ # Wave extraction ################################################################################ ################################################################################ WeylScal4::offset = 1e-8 WeylScal4::fd_order = "4th" WeylScal4::verbose = 0 Multipole::nradii = 8 Multipole::out_every = 128 Multipole::radius[0] = 45 Multipole::radius[1] = 70 Multipole::radius[2] = 100 Multipole::radius[3] = 125 Multipole::radius[4] = 150 Multipole::radius[5] = 200 Multipole::radius[6] = 250 Multipole::radius[7] = 300 Multipole::variables = "WeylScal4::Psi4r{sw=-2 cmplx='WeylScal4::Psi4i' name='Psi4'}" Multipole::l_max = 6 #------------------------------------------------------------------------------ # Checkpoint/Recovery: #------------------------------------------------------------------------------ IOHDF5::checkpoint = "yes" IO::checkpoint_dir = $parfile IO::checkpoint_ID = "yes" IO::checkpoint_on_terminate = "yes" # disable extra con2prim in Post_Recover_Variables to ensure bit-identical # recovery from checkpoints MoL::run_MoL_PostStep_in_Post_Recover_Variables = "no" IO::recover = "autoprobe" IO::recover_dir = $parfile #------------------------------------------------------------------------------ # Control #------------------------------------------------------------------------------ HTTPD::user = "cactus" HTTPD::password = "einstein" ActiveThorns = "TerminationTrigger" TerminationTrigger::max_walltime = @WALLTIME_HOURS@ # Trigger termination 30 minutes before the walltime is reached TerminationTrigger::on_remaining_walltime = 30 TerminationTrigger::termination_from_file = yes TerminationTrigger::termination_file = "terminate.txt" TerminationTrigger::create_termination_file = yes ActiveThorns = "Trigger" Trigger::Trigger_Number = 4 Trigger::Trigger_Checked_Variable[0]="Hydro_Analysis::Hydro_Analysis_rho_max_origin_distance" Trigger::Trigger_Reduction [0]="" Trigger::Trigger_Relation [0]="<" Trigger::Trigger_Checked_Value [0]=10 Trigger::Trigger_Reaction [0]="steerscalar" Trigger::Trigger_Steered_Scalar [0] = "CarpetRegrid2::num_levels[2]" # == num_levels_3 Trigger::Trigger_Steered_Scalar_Value[0] = "7" Trigger::Trigger_Checked_Variable[1]="ADMBase::alp" Trigger::Trigger_Reduction [1]="minimum" Trigger::Trigger_Relation [1]="<" Trigger::Trigger_Checked_Value [1]=0.1 Trigger::Trigger_Reaction [1]="steerscalar" Trigger::Trigger_Steered_Scalar [1] = "CarpetRegrid2::num_levels[2]" # == num_levels_3 Trigger::Trigger_Steered_Scalar_Value[1] = "8" Trigger::Trigger_Checked_Variable[2]="SphericalSurface::sf_valid[2]" Trigger::Trigger_Reduction [2]="" Trigger::Trigger_Relation [2]=">" Trigger::Trigger_Checked_Value [2]=0. Trigger::Trigger_Reaction [2]="steerscalar" Trigger::Trigger_Steered_Scalar [2] = "CarpetRegrid2::num_levels[0]" # == num_levels_1 Trigger::Trigger_Steered_Scalar_Value[2] = "1" Trigger::Trigger_Checked_Variable[3]="SphericalSurface::sf_valid[2]" Trigger::Trigger_Reduction [3]="" Trigger::Trigger_Relation [3]=">" Trigger::Trigger_Checked_Value [3]=0. Trigger::Trigger_Reaction [3]="steerscalar" Trigger::Trigger_Steered_Scalar [3] = "CarpetRegrid2::num_levels[1]" # == num_levels_2 Trigger::Trigger_Steered_Scalar_Value[3] = "1" -------------- next part -------------- + set -e + cd /system/user/crangano/simulations/bns_sim/output-0000-active + echo Checking: + pwd + hostname + date + echo Environment: + export CACTUS_NUM_PROCS=1 + CACTUS_NUM_PROCS=1 + export CACTUS_NUM_THREADS=32 + CACTUS_NUM_THREADS=32 + export GMON_OUT_PREFIX=gmon.out + GMON_OUT_PREFIX=gmon.out + export OMP_NUM_THREADS=32 + OMP_NUM_THREADS=32 + env + sort + echo Starting: ++ date +%s + export CACTUS_STARTTIME=1757517541 + CACTUS_STARTTIME=1757517541 + '[' 1 = 1 ']' + '[' 0 -eq 0 ']' + /system/user/crangano/simulations/bns_sim/SIMFACTORY/exe/cactus_sim -L 3 /system/user/crangano/simulations/bns_sim/output-0000/bns.par WARNING level 1 from host panther process 0 in thorn ML_BSSN_Helper, file /system/user/crangano/einstein_toolkit/BNS/configs/sim/build/ML_BSSN_Helper/Parameters.c:10: -> Forcing ML_BSSN::initial_boundary_condition="extrapolate-gammas" because ML_BSSN::my_initial_boundary_condition="extrapolate-gammas" WARNING level 1 from host panther process 0 in thorn ML_BSSN_Helper, file /system/user/crangano/einstein_toolkit/BNS/configs/sim/build/ML_BSSN_Helper/Parameters.c:25: -> Forcing ML_BSSN::rhs_boundary_condition="NewRad" because ML_BSSN::my_rhs_boundary_condition="NewRad" WARNING level 1 from host panther process 0 in thorn ML_BSSN_Helper, file /system/user/crangano/einstein_toolkit/BNS/configs/sim/build/ML_BSSN_Helper/Parameters.c:45: -> Forcing ML_BSSN::epsDiss=0.0 because ML_BSSN::apply_dissipation="never" WARNING level 1 from host panther process 0 in thorn ML_BSSN_Helper, file /system/user/crangano/einstein_toolkit/BNS/configs/sim/build/ML_BSSN_Helper/Parameters.c:145: -> Parameter ML_BSSN::my_initial_data is outdated; please update the parameter file. Do not use this parameter, and set up initial conditions via ADMBase as usual. WARNING level 1 from host panther process 0 in thorn ML_BSSN_Helper, file /system/user/crangano/einstein_toolkit/BNS/configs/sim/build/ML_BSSN_Helper/Parameters.c:152: -> Parameter ML_BSSN::my_initial_boundary_condition is outdated; please update the parameter file. Do not use this parameter, and set up initial boundary conditions as usual. WARNING level 1 from host panther process 0 in thorn ML_BSSN_Helper, file /system/user/crangano/einstein_toolkit/BNS/configs/sim/build/ML_BSSN_Helper/Parameters.c:159: -> Parameter ML_BSSN::my_rhs_boundary_condition is outdated; please update the parameter file. Do not use this parameter, and set up RHS boundary conditions as usual. WARNING level 1 from host panther process 0 in thorn ML_BSSN_Helper, file /system/user/crangano/einstein_toolkit/BNS/configs/sim/build/ML_BSSN_Helper/Parameters.c:166: -> Parameter ML_BSSN::my_boundary_condition is outdated; please update the parameter file. Do not use this parameter, and set up RHS boundary conditions as usual. cactus_sim: grille3d.C:125: Grille3d::Grille3d(int, int, int, int, int, int, int): Assertion `nr > 0' failed. Rank 0 with PID 3273101 received signal 6 Writing backtrace to bns/backtrace.0.txt /system/user/crangano/simulations/bns_sim/output-0000/SIMFACTORY/RunScript: line 36: 3273101 Aborted (core dumped) /system/user/crangano/simulations/bns_sim/SIMFACTORY/exe/cactus_sim -L 3 /system/user/crangano/simulations/bns_sim/output-0000/bns.par -------------- next part -------------- Running simulation bns_sim Preparing: Checking: /system/user/crangano/simulations/bns_sim/output-0000-active panther Wed Sep 10 05:19:01 PM CEST 2025 Environment: Starting: INFO (Cactus): Increased logging level from 0 to 3 -------------------------------------------------------------------------------- 10 1 0101 ************************ 01 1010 10 The Cactus Code V4.18.0 1010 1101 011 www.cactuscode.org 1001 100101 ************************ 00010101 100011 (c) Copyright The Authors 0100 GNU Licensed. No Warranty 0101 -------------------------------------------------------------------------------- Cactus version: 4.18.0 Compile date: Sep 09 2025 (16:36:24) Run date: Sep 10 2025 (17:19:02+0200) Run host: panther (pid=3273101) Working directory: /system/user/crangano/simulations/bns_sim/output-0000 Executable: /system/user/crangano/simulations/bns_sim/SIMFACTORY/exe/cactus_sim Parameter file: /system/user/crangano/simulations/bns_sim/output-0000/bns.par -------------------------------------------------------------------------------- Activating thorn Cactus...Success -> active implementation Cactus Activation requested for --->Boundary CartGrid3D CoordBase Fortran InitBase IOUtil LocalReduce SymBase Time AEILocalInterp MoL Slab SpaceMask SphericalSurface Carpet CarpetInterp CarpetInterp2 CarpetIOASCII CarpetIOHDF5 CarpetIOScalar CarpetLib CarpetIOBasic CarpetReduce CarpetRegrid2 CarpetSlab CarpetTracker CarpetMask LoopControl Formaline HTTPD Socket NaNChecker TerminationTrigger TimerReport ADMbase ADMcoupling ADMmacros CoordGauge StaticConformal RotatingSymmetry180 ReflectionSymmetry Constants TmunuBase HydroBase QuasiLocalMeasures EOS_Omni GRHydro SummationByParts GenericFD NewRad ML_BSSN ML_BSSN_Helper ML_ADMConstraints Hydro_Analysis NSTracker Dissipation SystemStatistics SystemTopology WeylScal4 Multipole CoordBase Meudon_Bin_NS TerminationTrigger Trigger<--- Warning: thorn CoordBase already scheduled for activation Warning: thorn TerminationTrigger already scheduled for activation Thorn Carpet requests automatic activation of MPI Thorn Carpet requests automatic activation of Timers Thorn CarpetIOHDF5 requests automatic activation of HDF5 Thorn CarpetLib requests automatic activation of Vectors Thorn CarpetLib requests automatic activation of CycleClock Thorn Formaline requests automatic activation of pthreads Thorn GRHydro requests automatic activation of EOS_Polytrope Thorn LoopControl requests automatic activation of hwloc Thorn Meudon_Bin_NS requests automatic activation of LORENE Thorn QuasiLocalMeasures requests automatic activation of LAPACK Thorn QuasiLocalMeasures requests automatic activation of TGRtensor Thorn EOS_Polytrope requests automatic activation of EOS_Base Thorn HDF5 requests automatic activation of zlib Thorn LAPACK requests automatic activation of BLAS Thorn LORENE requests automatic activation of GSL Activating thorn ADMbase...Success -> active implementation ADMBase Activating thorn ADMcoupling...Success -> active implementation ADMCoupling Activating thorn ADMmacros...Success -> active implementation ADMMacros Activating thorn AEILocalInterp...Success -> active implementation AEILocalInterp Activating thorn BLAS...Success -> active implementation BLAS Activating thorn Boundary...Success -> active implementation boundary Activating thorn Carpet...Success -> active implementation Driver Activating thorn CarpetInterp...Success -> active implementation interp Activating thorn CarpetInterp2...Success -> active implementation interp2 Activating thorn CarpetIOASCII...Success -> active implementation IOASCII Activating thorn CarpetIOBasic...Success -> active implementation IOBasic Activating thorn CarpetIOHDF5...Success -> active implementation IOHDF5 Activating thorn CarpetIOScalar...Success -> active implementation IOScalar Activating thorn CarpetLib...Success -> active implementation CarpetLib Activating thorn CarpetMask...Success -> active implementation CarpetMask Activating thorn CarpetReduce...Success -> active implementation reduce Activating thorn CarpetRegrid2...Success -> active implementation CarpetRegrid2 Activating thorn CarpetSlab...Success -> active implementation Hyperslab Activating thorn CarpetTracker...Success -> active implementation CarpetTracker Activating thorn CartGrid3D...Success -> active implementation grid Activating thorn Constants...Success -> active implementation Constants Activating thorn CoordBase...Success -> active implementation CoordBase Activating thorn CoordGauge...Success -> active implementation CoordGauge Activating thorn CycleClock...Success -> active implementation CycleClock Activating thorn Dissipation...Success -> active implementation Dissipation Activating thorn EOS_Base...Success -> active implementation EOS_Base Activating thorn EOS_Omni...Success -> active implementation EOS_Omni Activating thorn EOS_Polytrope...Success -> active implementation EOS_2d_Polytrope Activating thorn Formaline...Success -> active implementation Formaline Activating thorn Fortran...Success -> active implementation Fortran Activating thorn GenericFD...Success -> active implementation GenericFD Activating thorn GRHydro...Success -> active implementation GRHydro Activating thorn GSL...Success -> active implementation GSL Activating thorn HDF5...Success -> active implementation HDF5 Activating thorn HTTPD...Success -> active implementation HTTPD Activating thorn hwloc...Success -> active implementation hwloc Activating thorn Hydro_Analysis...Success -> active implementation Hydro_Analysis Activating thorn HydroBase...Success -> active implementation HydroBase Activating thorn InitBase...Success -> active implementation InitBase Activating thorn IOUtil...Success -> active implementation IO Activating thorn LAPACK...Success -> active implementation LAPACK Activating thorn LocalReduce...Success -> active implementation LocalReduce Activating thorn LoopControl...Success -> active implementation LoopControl Activating thorn LORENE...Success -> active implementation LORENE Activating thorn Meudon_Bin_NS...Success -> active implementation Meudon_Bin_NS Activating thorn ML_ADMConstraints...Success -> active implementation ML_ADMConstraints Activating thorn ML_BSSN...Success -> active implementation ML_BSSN Activating thorn ML_BSSN_Helper...Success -> active implementation ML_BSSN_Helper Activating thorn MoL...Success -> active implementation MethodOfLines Activating thorn MPI...Success -> active implementation MPI Activating thorn Multipole...Success -> active implementation multipole Activating thorn NaNChecker...Success -> active implementation NaNChecker Activating thorn NewRad...Success -> active implementation NewRad Activating thorn NSTracker...Success -> active implementation NSTracker Activating thorn pthreads...Success -> active implementation PTHREADS Activating thorn QuasiLocalMeasures...Success -> active implementation QuasiLocalMeasures Activating thorn ReflectionSymmetry...Success -> active implementation ReflectionSymmetry Activating thorn RotatingSymmetry180...Success -> active implementation RotatingSymmetry180 Activating thorn Slab...Success -> active implementation Slab Activating thorn Socket...Success -> active implementation Socket Activating thorn SpaceMask...Success -> active implementation SpaceMask Activating thorn SphericalSurface...Success -> active implementation SphericalSurface Activating thorn StaticConformal...Success -> active implementation StaticConformal Activating thorn SummationByParts...Success -> active implementation SummationByParts Activating thorn SymBase...Success -> active implementation SymBase Activating thorn SystemStatistics...Success -> active implementation SystemStatistics Activating thorn SystemTopology...Success -> active implementation SystemTopology Activating thorn TerminationTrigger...Success -> active implementation TerminationTrigger Activating thorn TGRtensor...Success -> active implementation TGRtensor Activating thorn Time...Success -> active implementation time Activating thorn TimerReport...Success -> active implementation timerreport Activating thorn Timers...Success -> active implementation Timers Activating thorn TmunuBase...Success -> active implementation TmunuBase Activating thorn Trigger...Success -> active implementation trigger Activating thorn Vectors...Success -> active implementation Vectors Activating thorn WeylScal4...Success -> active implementation WeylScal4 Activating thorn zlib...Success -> active implementation zlib WARNING level 3 from host panther process 0 in thorn IOUtil, file /system/user/crangano/einstein_toolkit/BNS/configs/sim/build/IOUtil/CheckpointRecovery.cc:748: -> No HDF5 checkpoint files with basefilename 'checkpoint.chkpt' and file extension '.h5' found in recovery directory 'bns' -------------------------------------------------------------------------------- if (recover initial data) Recover parameters endif Startup routines [CCTK_STARTUP] Carpet::MultiModel_Startup: Multi-model Startup routine CycleClock::CycleClock_Setup: Set up CycleClock LoopControl::LC_setup: Set up LoopControl ML_BSSN_Helper::ML_BSSN_SetGroupTags: [meta] Set checkpointing and prolongation group tags GROUP hwloc_startup: hwloc startup group hwloc::hwloc_version: Output hwloc version Timers::Timer_Startup: Prepare hierarchical timers SystemTopology::ST_system_topology: Output and/or modify system topology and hardware locality CarpetInterp::CarpetInterpStartup: Startup routine CarpetReduce::CarpetReduceStartup: Startup routine CartGrid3D::SymmetryStartup: Register GH Extension for GridSymmetry CoordBase::CoordBase_Startup: Register a GH extension to store the coordinate system handles Carpet::Driver_Startup: Startup routine EOS_Omni::EOS_Omni_Startup: [global] Set up conversion factors and other fun stuff EOS_Polytrope::EOS_Polytrope_Startup: Setup the polytropic EOS Formaline::Formaline_PrintIDs: [meta] Print the build and simulation ids GRHydro::GRHydro_Startup: Startup banner GRHydro::GRHydro_RegisterMask: Register the hydro masks GROUP HTTP_Startup: HTTP daemon startup group HTTPD::HTTP_StartServer: [global] Start HTTP server GROUP HTTP_SetupPages: Group to setup stuff which needs to be done between starting the server and the first time it serves pages HTTPD::HTTP_FirstServ: [global] Serve first pages at startup HydroBase::HydroBase_StartUp: Startup banner IOUtil::IOUtil_Startup: Startup routine LocalReduce::LocalReduce_Startup: Startup routine CarpetIOBasic::CarpetIOBasicStartup: [global] Startup routine ML_ADMConstraints::ML_ADMConstraints_Startup: [meta] create banner ML_BSSN::ML_BSSN_Startup: [meta] create banner ML_BSSN_Helper::ML_BSSN_RegisterSlicing: [meta] Register slicing CarpetIOHDF5::CarpetIOHDF5_Startup: Startup routine MoL::MoL_Startup: Startup banner Slab::Slab_InitMPIDatatypes: Create MPI datatypes for complex variables in C SymBase::SymBase_Startup: Register GH Extension for SymBase CarpetIOScalar::CarpetIOScalarStartup: [global] Startup routine TerminationTrigger::TerminationTrigger_StartSignalHandler: Start signal handler CarpetIOASCII::CarpetIOASCIIStartup: [global] Startup routine Trigger::Trigger_Startup: Startup Routine Vectors::Vectors_Startup: Print startup message WeylScal4::WeylScal4_Startup: [meta] create banner AEILocalInterp::AEILocalInterp_U_Startup: register CCTK_InterpLocalUniform() interpolation operators Startup routines which need an existing grid hierarchy [CCTK_WRAGH] ADMBase::Einstein_InitSymBound: [global] Set up GF symmetries Boundary::Boundary_RegisterBCs: [global] Register boundary conditions that this thorn provides CarpetRegrid2::CarpetRegrid2_Initialise: [global] Initialise locations of refined regions CartGrid3D::RegisterCartGrid3DCoords: [meta] Register coordinates for the Cartesian grid CoordGauge::Einstein_ActivateSlicing: Initialize slicing, setup priorities for mixed slicings CoordGauge::Einstein_SetNextSlicing: Identify the slicing for the next iteration Formaline::Formaline_OutputSource: [meta] Output Cactus source tree Formaline::Formaline_RegisterWarnings: [meta] Register to receive warnings and info messages from the flesh Formaline::Formaline_AnnounceInitial: [global] Put some meta information about the current run into permanent storage GRHydro::GRHydro_ClearLastMoLPostStep: [global] Initialize InLastMoLPostStep to zero ML_BSSN_Helper::ML_BSSN_ParamCompat: [meta] Handle parameter backward compatibility MoL::MoL_SetupIndexArrays: Set up the MoL bookkeeping index arrays MoL::MoL_SetScheduleStatus: [global] Set the flag so it is ok to register with MoL TmunuBase::TmunuBase_SetStressEnergyState: [global] Set the stress_energy_state variable GROUP MoL_Register: The group where physics thorns register variables with MoL GRHydro::GRHydro_Register: Register variables for MoL ML_ADMConstraints::ML_ADMConstraints_RegisterVars: [meta] Register Variables for MoL ML_BSSN::ML_BSSN_RegisterVars: [meta] Register Variables for MoL ML_BSSN_Helper::ML_BSSN_RegisterConstrained: [meta] Register ADMBase variables as constrained WeylScal4::WeylScal4_RegisterVars: [meta] Register Variables for MoL Slab::Slab_InitTimers: Initialise timers SpaceMask::MaskSym: [global] Set grid symmetries for mask SpaceMask::MaskSym_emask: [global] Set grid symmetries for emask (compatibility mode) GROUP SymBase_Wrapper: Wrapper group for SymBase GROUP SymmetryRegister: Register your symmetries here CartGrid3D::CartGrid3D_RegisterSymmetryBoundaries: [meta] Register symmetry boundaries ML_ADMConstraints::ML_ADMConstraints_RegisterSymmetries: [meta] register symmetries ML_BSSN::ML_BSSN_RegisterSymmetries: [meta] register symmetries ReflectionSymmetry::ReflectionSymmetry_Register: Register reflection symmetry boundaries RotatingSymmetry180::Rot180_RegisterSymmetry: Register symmetry boundaries WeylScal4::WeylScal4_RegisterSymmetries: [meta] register symmetries SymBase::SymBase_Statistics: Print symmetry boundary face descriptions MoL::MoL_ReportNumberVariables: [meta] Report how many of each type of variable there are Parameter checking routines [CCTK_PARAMCHECK] ADMBase::ADMBase_ParamCheck: [global] Check consistency of parameters Boundary::Boundary_Check: Check dimension of grid variables Carpet::CarpetParamCheck: Parameter checking routine CarpetLib::CarpetLib_test_prolongate_3d_rf2: [global] Test prolongation operators CarpetMask::CarpetSurfaceParamCheck: [global] Check parameters CarpetRegrid2::CarpetRegrid2_ParamCheck: Check parameters CartGrid3D::ParamCheck_CartGrid3D: Check coordinates for CartGrid3D Dissipation::dissipation_paramcheck: Check dissipation parameters for consistency Fortran::CheckFortranParameters: Test whether Fortran parameters work correctly GRHydro::GRHydro_ParamCheck: Check parameters ML_BSSN_Helper::ML_BSSN_ParamCheck: [meta] Check parameters MoL::MoL_ParamCheck: Basic parameter checking Multipole::Multipole_ParamCheck: [global] Check Multipole parameters QuasiLocalMeasures::qlm_paramcheck: [global] Check quasi-local parameter settings RotatingSymmetry180::Rot180_CheckTensorTypes: Check tensor type definitions for consistency SphericalSurface::SphericalSurface_ParamCheck: [global] Check that all surface names are unique TerminationTrigger::TerminationTrigger_ParamCheck: Check consitency of parameters TmunuBase::TmunuBase_ParamCheck: [global] Check that no deprecated parameters are used. Trigger::Trigger_ParamCheck: Parameter checking Vectors::Vectors_Test: Run correctness tests. Initialisation if (NOT (recover initial data AND recovery_mode is 'strict')) [CCTK_PREREGRIDINITIAL] Set up grid hierarchy [CCTK_POSTREGRIDINITIAL] CartGrid3D::SpatialCoordinates: Set Coordinates after regridding GROUP MaskBase_SetupMask: Set up the weight function GROUP MaskBase_SetupMaskAll: Set up the weight function CarpetReduce::MaskBase_AllocateMask: [global] Allocate the weight function CarpetReduce::MaskBase_InitMask: [global] [loop-local] Initialise the weight function GROUP SetupIMaskInternal: Set up the integer weight function (schedule other routines in here) CarpetReduce::CoordBase_SetupMask: [global] [loop-local] Set up the outer boundaries of the weight function CarpetReduce::CarpetMaskSetup: [global] [loop-singlemap] Set up the weight function for the restriction regions GROUP SetupIMask: Set up the integer weight function (schedule other routines in here) CarpetMask::CarpetExcludedSetup: [global] [loop-local] Set up the weight function for the excluded regions CarpetMask::CarpetSurfaceSetup: [global] [loop-local] Set up the weight function for the excluded spherical surfaces CarpetReduce::MaskBase_SetMask: [global] [loop-local] Set the weight function GROUP SetupMask: Set up the real weight function (schedule other routines in here) CarpetReduce::MaskBase_TestMask: [global] Test the weight function Dissipation::setup_epsdis: Setup spatially varying dissipation SpaceMask::MaskZero: Initialise mask to zero GRHydro::GRHydro_RefinementLevel: Calculate current refinement level GROUP GRHydro_PrimitiveBoundaries: Apply boundary conditions to all primitives GRHydro::GRHydro_SelectPrimitiveBoundaries: [level] Select primitive variables for boundary conditions GROUP GRHydro_ApplyPrimitiveBCs: Apply boundary conditions to all primitive variables GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP HydroBase_ExcisionMaskSetup: Set up hydro excision mask HydroBase::HydroBase_InitExcisionMask: Initialize hydro excision mask to 'no excision everywhere' GROUP MoL_PseudoEvolutionBoundaries: Apply boundary conditions to pseudo-evolved quantities GROUP ML_ADMConstraints_evaluate_bc_group: ML_ADMConstraints_evaluate ML_ADMConstraints::ML_ADMConstraints_evaluate_SelectBCs: [level] ML_ADMConstraints_evaluate_SelectBCs GROUP ML_ADMConstraints_evaluate_ApplyBCs: Apply BCs for groups set in ML_ADMConstraints_evaluate GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP ML_BSSN_ConstraintsEverywhere_bc_group: ML_BSSN_ConstraintsEverywhere ML_BSSN::ML_BSSN_ConstraintsEverywhere_SelectBCs: [level] ML_BSSN_ConstraintsEverywhere_SelectBCs GROUP ML_BSSN_ConstraintsEverywhere_ApplyBCs: Apply BCs for groups set in ML_BSSN_ConstraintsEverywhere GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP ML_BSSN_ConstraintsInterior_bc_group: ML_BSSN_ConstraintsInterior ML_BSSN::ML_BSSN_ConstraintsInterior_SelectBCs: [level] ML_BSSN_ConstraintsInterior_SelectBCs GROUP ML_BSSN_ConstraintsInterior_ApplyBCs: Apply BCs for groups set in ML_BSSN_ConstraintsInterior GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP WeylScal4_psi4_calc_4th_bc_group: WeylScal4_psi4_calc_4th WeylScal4::WeylScal4_psi4_calc_4th_SelectBCs: [level] WeylScal4_psi4_calc_4th_SelectBCs GROUP WeylScal4_psi4_calc_4th_ApplyBCs: Apply BCs for groups set in WeylScal4_psi4_calc_4th GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions SpaceMask::MaskOne: Set mask to one GRHydro::GRHydro_SetupMask: Initialize the atmosphere mask [CCTK_BASEGRID] ADMBase::ADMBase_SetShiftStateOn: Set the shift_state variable to 1 ADMBase::ADMBase_SetDtLapseStateOn: Set the dtlapse_state variable to 1 ADMBase::ADMBase_SetDtShiftStateOn: Set the dtshift_state variable to 1 ADMMacros::ADMMacros_SetLocalSpatialOrder: Initialize the local_spatial_order CartGrid3D::SpatialSpacings: Set up ranges for spatial 3D Cartesian coordinates (on all grids) CartGrid3D::SpatialCoordinates: Set up spatial 3D Cartesian coordinates on the GH SphericalSurface::SphericalSurface_SetupRes: [global] [loop-local] Set surface resolution automatically Dissipation::dissipation_basegrid: Ensure that there are enough ghost zones GRHydro::GRHydro_Reset_Execution_Flags: [global] Initially set execution flags to 'YEAH, Execute'! GRHydro::GRHydro_InitSymBound: Schedule symmetries and check shift state GRHydro::reset_GRHydro_C2P_failed: Initialise the mask function that contains the points where C2P has failed (at BASEGRID) Hydro_Analysis::Hydro_Analysis_Init: [global] Initialize variables ML_ADMConstraints::ML_ADMConstraints_CheckBoundaries: [meta] check boundaries treatment ML_BSSN::ML_BSSN_CheckBoundaries: [meta] check boundaries treatment NaNChecker::NaNChecker_ResetCounter: [global] Reset the NaNChecker::NaNsFound counter SpaceMask::MaskZero: Initialise mask to zero SpaceMask::MaskOne: Set old style mask to one SphericalSurface::SphericalSurface_Setup: [global] Calculate surface coordinate descriptors GROUP MaskBase_SetupMask: Set up the weight function GROUP MaskBase_SetupMaskAll: Set up the weight function CarpetReduce::MaskBase_AllocateMask: [global] Allocate the weight function CarpetReduce::MaskBase_InitMask: [global] [loop-local] Initialise the weight function GROUP SetupIMaskInternal: Set up the integer weight function (schedule other routines in here) CarpetReduce::CoordBase_SetupMask: [global] [loop-local] Set up the outer boundaries of the weight function CarpetReduce::CarpetMaskSetup: [global] [loop-singlemap] Set up the weight function for the restriction regions GROUP SetupIMask: Set up the integer weight function (schedule other routines in here) CarpetMask::CarpetExcludedSetup: [global] [loop-local] Set up the weight function for the excluded regions CarpetMask::CarpetSurfaceSetup: [global] [loop-local] Set up the weight function for the excluded spherical surfaces CarpetReduce::MaskBase_SetMask: [global] [loop-local] Set the weight function GROUP SetupMask: Set up the real weight function (schedule other routines in here) CarpetReduce::MaskBase_TestMask: [global] Test the weight function SphericalSurface::SphericalSurface_Set: [global] Set surface radii to be used for initial setup in other thorns GROUP SphericalSurface_HasBeenSet: Set the spherical surfaces before this group, and use it afterwards SphericalSurface::SphericalSurface_CheckState: [global] Test the state of the spherical surfaces SummationByParts::SBP_SetNormMask: Setup the mask for the calculation of the norm SymBase::SymBase_Check: Check whether the driver set up the grid consistently TerminationTrigger::TerminationTrigger_ResetTrigger: Clear trigger state TerminationTrigger::TerminationTrigger_StartTimer: Start timer TerminationTrigger::TerminationTrigger_CreateFile: Create termination file Time::Time_Initialise: [global] Initialise Time variables Time::TemporalSpacings: [singlemap] Set timestep based on Courant condition (courant_static) WeylScal4::WeylScal4_CheckBoundaries: [meta] check boundaries treatment [CCTK_INITIAL] StaticConformal::StaticConformal_InitialiseState: Set the conformal_state variable to 0 GROUP ADMBase_InitialData: Schedule group for calculating ADM initial data GRHydro::GRHydro_EOSHandle: [global] Set the EOS number CarpetIOASCII::CarpetIOASCIIInit: [global] Initialisation routine CarpetIOBasic::CarpetIOBasicInit: [global] Initialisation routine CarpetIOHDF5::CarpetIOHDF5_Init: [global] Initialisation routine CarpetIOScalar::CarpetIOScalarInit: [global] Initialisation routine GRHydro::GRHydro_Rho_Minima_Setup: Set up minimum for the rest-mass density in the atmosphere (before intial data) GRHydro::GRHydro_SetupMask: Initialize the atmosphere mask GRHydro::GRHydro_RefinementLevel: Calculate current refinement level GROUP ADMBase_InitialGauge: Schedule group for the ADM initial gauge condition ADMBase::ADMBase_ShiftZero: Set the shift to 0 at all points ADMBase::ADMBase_DtShiftZero: Set the dtshift to 0 at all points GROUP HydroBase_Initial: HydroBase initial data group GROUP GRHydro_Initial: GRHydro initial data group GROUP HydroBase_ExcisionMaskSetup: Set up hydro excision mask HydroBase::HydroBase_InitExcisionMask: Initialize hydro excision mask to 'no excision everywhere' Meudon_Bin_NS::Meudon_Bin_NS_initialise: Set up binary neutron star initial data GROUP ADMBase_PostInitial: Schedule group for modifying the ADM initial data, such as e.g. adding noise GRHydro::GRHydro_SqrtSpatialDeterminant: Calculate sdetg GRHydro::GRHydro_InitialAtmosphereReset: Use mask to enforce atmosphere at initial time ML_BSSN::ML_BSSN_InitialADMBase1Everywhere: ML_BSSN_InitialADMBase1Everywhere ML_BSSN::ML_BSSN_InitialADMBase2Interior: ML_BSSN_InitialADMBase2Interior ML_BSSN::ML_BSSN_InitialADMBase2BoundaryScalar: ML_BSSN_InitialADMBase2BoundaryScalar ML_BSSN_Helper::ML_BSSN_ExtrapolateGammas: Extrapolate Gammas and time derivatives of lapse and shift MoL::MoL_StartLoop: [level] Initialise the step size control QuasiLocalMeasures::qlm_init: [global] Initialise quasi-local calculations GROUP HydroBase_Prim2ConInitial: Recover the conservative variables from the primitive variables GRHydro::Primitive2ConservativeCells: Convert initial data given in primive variables to conserved variables [CCTK_POSTINITIAL] CarpetIOHDF5::CarpetIOHDF5_CloseFiles: [global] Close all filereader input files GRHydro::GRHydro_Scalar_Setup: Set up and check scalars for efficiency GROUP MoL_PostStepModify: The group for physics thorns to schedule enforcing constraints ML_BSSN::ML_BSSN_EnforceEverywhere: ML_BSSN_EnforceEverywhere GROUP MoL_PostStep: Ensure that everything is correct after the initial data have been set up ML_BSSN::ML_BSSN_SelectBoundConds: [level] select boundary conditions GRHydro::GRHydro_RefinementLevel: Calculate current refinement level GRHydro::GRHydro_SetLastMoLPostStep: [level] Set grid scalar InLastMoLPostStep if this is the last MoL PostStep call GROUP ML_BSSN_ApplyBCs: Apply boundary conditions controlled by thorn Boundary GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions ML_ADMConstraints::ML_ADMConstraints_SelectBoundConds: [level] select boundary conditions GROUP ML_ADMConstraints_ApplyBCs: Apply boundary conditions controlled by thorn Boundary GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions ML_BSSN::ML_BSSN_ADMBaseInterior: ML_BSSN_ADMBaseInterior ML_BSSN::ML_BSSN_ADMBaseBoundaryScalar: ML_BSSN_ADMBaseBoundaryScalar ML_BSSN::ML_BSSN_ADMBaseEverywhere: ML_BSSN_ADMBaseEverywhere ML_BSSN_Helper::ML_BSSN_ADMBase_SelectBCs: [level] Select boundary conditions for ADMBase variables GROUP ML_BSSN_ADMBase_ApplyBCs: Apply boundary conditions to ADMBase variables GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP ADMBase_SetADMVars: Set the ADM variables before this group, and use them afterwards GROUP HydroBase_PostStep: Post step tasks for hydro thorns GROUP GRHydro_PostStep: Post step tasks for GRHydro GROUP GRHydro_AtmosphereMaskBoundaries: Apply boundary conditions to primitives GRHydro::GRHydro_SelectAtmosphereMaskBoundaries: [level] Select atmosphere mask for boundary conditions GROUP GRHydro_ApplyAtmosphereMaskBCs: Apply boundary conditions to real-valued atmosphere mask GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GRHydro::GRHydroPostSyncAtmosphereMask: Set integer atmosphere mask from synchronized real atmosphere mask if (GRHydro::InLastMoLPostStep) GRHydro::GRHydro_AtmosphereReset: Reset the atmosphere if (GRHydro::InLastMoLPostStep) GROUP GRHydro_PrimitiveInitialGuessesBoundaries: Apply boundary conditions to those primitives used as initial guesses GRHydro::GRHydro_SelectPrimitiveInitialGuessesBoundaries: [level] Select initial guess primitive variables for boudary conditions GROUP GRHydro_ApplyPrimitiveInitialGuessBCs: Apply boundary conditions to initial guess primitive variables GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP HydroBase_Boundaries: HydroBase-internal Boundary conditions group GROUP Do_GRHydro_Boundaries: GRHydro Boundary conditions group GROUP HydroBase_Select_Boundaries: Group to schedule the boundary condition functions if (GRHydro::execute_MoL_PostStep) GRHydro::GRHydro_Bound: [level] Select GRHydro boundary conditions GROUP HydroBase_ApplyBCs: Apply the boundary conditions of HydroBase GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP HydroBase_Con2Prim: Convert from conservative to primitive variables if (GRHydro::execute_MoL_Step) GRHydro::GRHydro_SqrtSpatialDeterminant: Calculate sdetg if (GRHydro::execute_MoL_PostStep) GRHydro::Con2Prim: Convert back to primitive variables (general) GROUP SetTmunu: Group for calculating the stress-energy tensor TmunuBase::TmunuBase_ZeroTmunu: Initialise the stress-energy tensor to zero GROUP AddToTmunu: Add to the stress-energy tensor here GRHydro::GRHydro_Tmunu: Compute the energy-momentum tensor WeylScal4::WeylScal4_SelectBoundConds: [level] select boundary conditions GROUP WeylScal4_ApplyBCs: Apply boundary conditions controlled by thorn Boundary GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP MoL_PseudoEvolution: Calculate pseudo-evolved quantities GROUP ADMBase_SetADMVars: Set the ADM variables before this group, and use them afterwards GROUP ML_ADMConstraints_evaluate_group: ML_ADMConstraints_evaluate ML_ADMConstraints::ML_ADMConstraints_evaluate: ML_ADMConstraints_evaluate GROUP ML_ADMConstraints_evaluate_bc_group: ML_ADMConstraints_evaluate ML_ADMConstraints::ML_ADMConstraints_evaluate_SelectBCs: [level] ML_ADMConstraints_evaluate_SelectBCs GROUP ML_ADMConstraints_evaluate_ApplyBCs: Apply BCs for groups set in ML_ADMConstraints_evaluate GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP ML_BSSN_ConstraintsEverywhere_group: ML_BSSN_ConstraintsEverywhere ML_BSSN::ML_BSSN_ConstraintsEverywhere: ML_BSSN_ConstraintsEverywhere GROUP ML_BSSN_ConstraintsEverywhere_bc_group: ML_BSSN_ConstraintsEverywhere ML_BSSN::ML_BSSN_ConstraintsEverywhere_SelectBCs: [level] ML_BSSN_ConstraintsEverywhere_SelectBCs GROUP ML_BSSN_ConstraintsEverywhere_ApplyBCs: Apply BCs for groups set in ML_BSSN_ConstraintsEverywhere GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP ML_BSSN_ConstraintsInterior_group: ML_BSSN_ConstraintsInterior ML_BSSN::ML_BSSN_ConstraintsInterior: ML_BSSN_ConstraintsInterior GROUP ML_BSSN_ConstraintsInterior_bc_group: ML_BSSN_ConstraintsInterior ML_BSSN::ML_BSSN_ConstraintsInterior_SelectBCs: [level] ML_BSSN_ConstraintsInterior_SelectBCs GROUP ML_BSSN_ConstraintsInterior_ApplyBCs: Apply BCs for groups set in ML_BSSN_ConstraintsInterior GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP WeylScal4_psi4_calc_4th_group: WeylScal4_psi4_calc_4th WeylScal4::WeylScal4_psi4_calc_4th: WeylScal4_psi4_calc_4th GROUP WeylScal4_psi4_calc_4th_bc_group: WeylScal4_psi4_calc_4th WeylScal4::WeylScal4_psi4_calc_4th_SelectBCs: [level] WeylScal4_psi4_calc_4th_SelectBCs GROUP WeylScal4_psi4_calc_4th_ApplyBCs: Apply BCs for groups set in WeylScal4_psi4_calc_4th GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions SummationByParts::SBP_CheckGridSizes: Check grid sizes and ghost zones Initialise finer grids recursively Restrict from finer grids [CCTK_POSTRESTRICTINITIAL] GROUP MoL_PostStep: Ensure that everything is correct after restriction ML_BSSN::ML_BSSN_SelectBoundConds: [level] select boundary conditions GRHydro::GRHydro_RefinementLevel: Calculate current refinement level GRHydro::GRHydro_SetLastMoLPostStep: [level] Set grid scalar InLastMoLPostStep if this is the last MoL PostStep call GROUP ML_BSSN_ApplyBCs: Apply boundary conditions controlled by thorn Boundary GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions ML_ADMConstraints::ML_ADMConstraints_SelectBoundConds: [level] select boundary conditions GROUP ML_ADMConstraints_ApplyBCs: Apply boundary conditions controlled by thorn Boundary GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions ML_BSSN::ML_BSSN_ADMBaseInterior: ML_BSSN_ADMBaseInterior ML_BSSN::ML_BSSN_ADMBaseBoundaryScalar: ML_BSSN_ADMBaseBoundaryScalar ML_BSSN::ML_BSSN_ADMBaseEverywhere: ML_BSSN_ADMBaseEverywhere ML_BSSN_Helper::ML_BSSN_ADMBase_SelectBCs: [level] Select boundary conditions for ADMBase variables GROUP ML_BSSN_ADMBase_ApplyBCs: Apply boundary conditions to ADMBase variables GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP ADMBase_SetADMVars: Set the ADM variables before this group, and use them afterwards GROUP HydroBase_PostStep: Post step tasks for hydro thorns GROUP GRHydro_PostStep: Post step tasks for GRHydro GROUP GRHydro_AtmosphereMaskBoundaries: Apply boundary conditions to primitives GRHydro::GRHydro_SelectAtmosphereMaskBoundaries: [level] Select atmosphere mask for boundary conditions GROUP GRHydro_ApplyAtmosphereMaskBCs: Apply boundary conditions to real-valued atmosphere mask GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GRHydro::GRHydroPostSyncAtmosphereMask: Set integer atmosphere mask from synchronized real atmosphere mask if (GRHydro::InLastMoLPostStep) GRHydro::GRHydro_AtmosphereReset: Reset the atmosphere if (GRHydro::InLastMoLPostStep) GROUP GRHydro_PrimitiveInitialGuessesBoundaries: Apply boundary conditions to those primitives used as initial guesses GRHydro::GRHydro_SelectPrimitiveInitialGuessesBoundaries: [level] Select initial guess primitive variables for boudary conditions GROUP GRHydro_ApplyPrimitiveInitialGuessBCs: Apply boundary conditions to initial guess primitive variables GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP HydroBase_Boundaries: HydroBase-internal Boundary conditions group GROUP Do_GRHydro_Boundaries: GRHydro Boundary conditions group GROUP HydroBase_Select_Boundaries: Group to schedule the boundary condition functions if (GRHydro::execute_MoL_PostStep) GRHydro::GRHydro_Bound: [level] Select GRHydro boundary conditions GROUP HydroBase_ApplyBCs: Apply the boundary conditions of HydroBase GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP HydroBase_Con2Prim: Convert from conservative to primitive variables if (GRHydro::execute_MoL_Step) GRHydro::GRHydro_SqrtSpatialDeterminant: Calculate sdetg if (GRHydro::execute_MoL_PostStep) GRHydro::Con2Prim: Convert back to primitive variables (general) GROUP SetTmunu: Group for calculating the stress-energy tensor TmunuBase::TmunuBase_ZeroTmunu: Initialise the stress-energy tensor to zero GROUP AddToTmunu: Add to the stress-energy tensor here GRHydro::GRHydro_Tmunu: Compute the energy-momentum tensor WeylScal4::WeylScal4_SelectBoundConds: [level] select boundary conditions GROUP WeylScal4_ApplyBCs: Apply boundary conditions controlled by thorn Boundary GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP MoL_PseudoEvolutionBoundaries: Apply boundary conditions to pseudo-evolved quantities GROUP ML_ADMConstraints_evaluate_bc_group: ML_ADMConstraints_evaluate ML_ADMConstraints::ML_ADMConstraints_evaluate_SelectBCs: [level] ML_ADMConstraints_evaluate_SelectBCs GROUP ML_ADMConstraints_evaluate_ApplyBCs: Apply BCs for groups set in ML_ADMConstraints_evaluate GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP ML_BSSN_ConstraintsEverywhere_bc_group: ML_BSSN_ConstraintsEverywhere ML_BSSN::ML_BSSN_ConstraintsEverywhere_SelectBCs: [level] ML_BSSN_ConstraintsEverywhere_SelectBCs GROUP ML_BSSN_ConstraintsEverywhere_ApplyBCs: Apply BCs for groups set in ML_BSSN_ConstraintsEverywhere GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP ML_BSSN_ConstraintsInterior_bc_group: ML_BSSN_ConstraintsInterior ML_BSSN::ML_BSSN_ConstraintsInterior_SelectBCs: [level] ML_BSSN_ConstraintsInterior_SelectBCs GROUP ML_BSSN_ConstraintsInterior_ApplyBCs: Apply BCs for groups set in ML_BSSN_ConstraintsInterior GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP WeylScal4_psi4_calc_4th_bc_group: WeylScal4_psi4_calc_4th WeylScal4::WeylScal4_psi4_calc_4th_SelectBCs: [level] WeylScal4_psi4_calc_4th_SelectBCs GROUP WeylScal4_psi4_calc_4th_ApplyBCs: Apply BCs for groups set in WeylScal4_psi4_calc_4th GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions [CCTK_POSTPOSTINITIAL] GRHydro::GRHydro_Rho_Minima_Setup_Final: Set the value of the rest-mass density of the atmosphere which will be used during the evolution GRHydro::GRHydro_InitialAtmosphereReset: Use mask to enforce atmosphere at initial time GROUP Con2Prim: Convert from conservative to primitive variables (might be redundant) if (GRHydro::execute_MoL_Step) GRHydro::GRHydro_SqrtSpatialDeterminant: Calculate sdetg if (GRHydro::execute_MoL_PostStep) GRHydro::Con2Prim: Convert back to primitive variables (general) GROUP SetTmunu: Calculate the stress-energy tensor TmunuBase::TmunuBase_ZeroTmunu: Initialise the stress-energy tensor to zero GROUP AddToTmunu: Add to the stress-energy tensor here GRHydro::GRHydro_Tmunu: Compute the energy-momentum tensor GROUP ADMConstraintsGroup: Evaluate ADM constraints, and perform symmetry boundary conditions [CCTK_POSTSTEP] SphericalSurface::SphericalSurface_Set: [global] Set surface radii GROUP SphericalSurface_HasBeenSet: Set the spherical surfaces before this group, and use it afterwards SphericalSurface::SphericalSurface_CheckState: [global] Test the state of the spherical surfaces GRHydro::GRHydro_RefinementLevel: Calculate current refinement level (for the check of the C2P mask) GRHydro::check_GRHydro_C2P_failed: Check the mask function that contains the points where C2P has failed and report an error in case a failure is found HTTPD::HTTP_Work: [global] Working routine HTTPD::HTTP_ContentWork: [global] Content Working routine GROUP HydroBase_ExcisionHasBeenSet: Group to schedule thorns changing the mask before and thorns using the mask after GROUP Hydro_Analysis: Group for Hydro_Analysis routines Hydro_Analysis::Hydro_Analysis_PrepareReduction: [global] [loop-local] Compute the local reduction results Hydro_Analysis::Hydro_Analysis_Reduction: [global] Compute the global reduction results GROUP Hydro_Analysis_LocationSearch: Look for the location of the maximum density Hydro_Analysis::Hydro_Analysis_LocationSearch_Setup: [global] Prepare data structures for search Hydro_Analysis::Hydro_Analysis_LocationSearch_Search: [global] [loop-local] Search for the location of the maximum density Hydro_Analysis::Hydro_Analysis_LocationSearch_Combine: [global] Communicate and verify the location of the maximum density Hydro_Analysis::Hydro_Analysis_FindSeparation: [global] Compute the proper distance between the maximum of the density and the origin (along a straight coordinate line) NSTracker::NSTracker: [global] Look for the star and set grid accordingly GROUP zzz_NaNChecker_NaNCheck: Check for NaNs and count them in NaNChecker::NaNsFound NaNChecker::NaNChecker_NaNCheck_Prepare: [level] Prepare data structures to check for NaNs NaNChecker::NaNChecker_NaNCheck_Check: [local] Check for NaNs NaNChecker::NaNChecker_NaNCheck_Finish: [level] Count NaNs in NaNChecker::NaNsFound NaNChecker::NaNChecker_TakeAction: [global] [loop-level] Output NaNChecker::NaNmask and take action according to NaNChecker::action_if_found SpaceMask::CheckMask: Ensure that all mask values are legal Dissipation::setup_epsdis: Setup spatially varying dissipation GROUP MaskBase_SetupMask: Set up the weight function GROUP MaskBase_SetupMaskAll: Set up the weight function CarpetReduce::MaskBase_AllocateMask: [global] Allocate the weight function CarpetReduce::MaskBase_InitMask: [global] [loop-local] Initialise the weight function GROUP SetupIMaskInternal: Set up the integer weight function (schedule other routines in here) CarpetReduce::CoordBase_SetupMask: [global] [loop-local] Set up the outer boundaries of the weight function CarpetReduce::CarpetMaskSetup: [global] [loop-singlemap] Set up the weight function for the restriction regions GROUP SetupIMask: Set up the integer weight function (schedule other routines in here) CarpetMask::CarpetExcludedSetup: [global] [loop-local] Set up the weight function for the excluded regions CarpetMask::CarpetSurfaceSetup: [global] [loop-local] Set up the weight function for the excluded spherical surfaces CarpetReduce::MaskBase_SetMask: [global] [loop-local] Set the weight function GROUP SetupMask: Set up the real weight function (schedule other routines in here) CarpetReduce::MaskBase_TestMask: [global] Test the weight function endif if (recover initial data) [CCTK_BASEGRID] ADMBase::ADMBase_SetShiftStateOn: Set the shift_state variable to 1 ADMBase::ADMBase_SetDtLapseStateOn: Set the dtlapse_state variable to 1 ADMBase::ADMBase_SetDtShiftStateOn: Set the dtshift_state variable to 1 ADMMacros::ADMMacros_SetLocalSpatialOrder: Initialize the local_spatial_order CartGrid3D::SpatialSpacings: Set up ranges for spatial 3D Cartesian coordinates (on all grids) CartGrid3D::SpatialCoordinates: Set up spatial 3D Cartesian coordinates on the GH SphericalSurface::SphericalSurface_SetupRes: [global] [loop-local] Set surface resolution automatically Dissipation::dissipation_basegrid: Ensure that there are enough ghost zones GRHydro::GRHydro_Reset_Execution_Flags: [global] Initially set execution flags to 'YEAH, Execute'! GRHydro::GRHydro_InitSymBound: Schedule symmetries and check shift state GRHydro::reset_GRHydro_C2P_failed: Initialise the mask function that contains the points where C2P has failed (at BASEGRID) Hydro_Analysis::Hydro_Analysis_Init: [global] Initialize variables ML_ADMConstraints::ML_ADMConstraints_CheckBoundaries: [meta] check boundaries treatment ML_BSSN::ML_BSSN_CheckBoundaries: [meta] check boundaries treatment NaNChecker::NaNChecker_ResetCounter: [global] Reset the NaNChecker::NaNsFound counter SpaceMask::MaskZero: Initialise mask to zero SpaceMask::MaskOne: Set old style mask to one SphericalSurface::SphericalSurface_Setup: [global] Calculate surface coordinate descriptors GROUP MaskBase_SetupMask: Set up the weight function GROUP MaskBase_SetupMaskAll: Set up the weight function CarpetReduce::MaskBase_AllocateMask: [global] Allocate the weight function CarpetReduce::MaskBase_InitMask: [global] [loop-local] Initialise the weight function GROUP SetupIMaskInternal: Set up the integer weight function (schedule other routines in here) CarpetReduce::CoordBase_SetupMask: [global] [loop-local] Set up the outer boundaries of the weight function CarpetReduce::CarpetMaskSetup: [global] [loop-singlemap] Set up the weight function for the restriction regions GROUP SetupIMask: Set up the integer weight function (schedule other routines in here) CarpetMask::CarpetExcludedSetup: [global] [loop-local] Set up the weight function for the excluded regions CarpetMask::CarpetSurfaceSetup: [global] [loop-local] Set up the weight function for the excluded spherical surfaces CarpetReduce::MaskBase_SetMask: [global] [loop-local] Set the weight function GROUP SetupMask: Set up the real weight function (schedule other routines in here) CarpetReduce::MaskBase_TestMask: [global] Test the weight function SphericalSurface::SphericalSurface_Set: [global] Set surface radii to be used for initial setup in other thorns GROUP SphericalSurface_HasBeenSet: Set the spherical surfaces before this group, and use it afterwards SphericalSurface::SphericalSurface_CheckState: [global] Test the state of the spherical surfaces SummationByParts::SBP_SetNormMask: Setup the mask for the calculation of the norm SymBase::SymBase_Check: Check whether the driver set up the grid consistently TerminationTrigger::TerminationTrigger_ResetTrigger: Clear trigger state TerminationTrigger::TerminationTrigger_StartTimer: Start timer TerminationTrigger::TerminationTrigger_CreateFile: Create termination file Time::Time_Initialise: [global] Initialise Time variables Time::TemporalSpacings: [singlemap] Set timestep based on Courant condition (courant_static) WeylScal4::WeylScal4_CheckBoundaries: [meta] check boundaries treatment [CCTK_RECOVER_VARIABLES] IOUtil::IOUtil_RecoverGH: [level] Checkpoint recovery routine [CCTK_POST_RECOVER_VARIABLES] CarpetIOHDF5::CarpetIOHDF5_InitCheckpointingIntervals: [global] Initialisation of checkpointing intervals after recovery CarpetIOHDF5::CarpetIOHDF5_CloseFiles: [meta] Close all initial data checkpoint files after recovery GROUP MaskBase_SetupMask: Set up the weight function GROUP MaskBase_SetupMaskAll: Set up the weight function CarpetReduce::MaskBase_AllocateMask: [global] Allocate the weight function CarpetReduce::MaskBase_InitMask: [global] [loop-local] Initialise the weight function GROUP SetupIMaskInternal: Set up the integer weight function (schedule other routines in here) CarpetReduce::CoordBase_SetupMask: [global] [loop-local] Set up the outer boundaries of the weight function CarpetReduce::CarpetMaskSetup: [global] [loop-singlemap] Set up the weight function for the restriction regions GROUP SetupIMask: Set up the integer weight function (schedule other routines in here) CarpetMask::CarpetExcludedSetup: [global] [loop-local] Set up the weight function for the excluded regions CarpetMask::CarpetSurfaceSetup: [global] [loop-local] Set up the weight function for the excluded spherical surfaces CarpetReduce::MaskBase_SetMask: [global] [loop-local] Set the weight function GROUP SetupMask: Set up the real weight function (schedule other routines in here) CarpetReduce::MaskBase_TestMask: [global] Test the weight function GRHydro::GRHydro_EOSHandle: [global] Set the EOS number GRHydro::GRHydroCopyIntegerMask: Initialize the real valued atmosphere mask after checkpoint recovery GRHydro::GRHydro_SqrtSpatialDeterminant: Calculate sdetg GROUP HydroBase_ExcisionMaskSetup: Set up hydro excision mask HydroBase::HydroBase_InitExcisionMask: Initialize hydro excision mask to 'no excision everywhere' GROUP zzz_NaNChecker_NaNCheck: Check for NaNs and count them in NaNChecker::NaNsFound NaNChecker::NaNChecker_NaNCheck_Prepare: [level] Prepare data structures to check for NaNs NaNChecker::NaNChecker_NaNCheck_Check: [local] Check for NaNs NaNChecker::NaNChecker_NaNCheck_Finish: [level] Count NaNs in NaNChecker::NaNsFound NaNChecker::NaNChecker_TakeAction: [global] [loop-level] Output NaNChecker::NaNmask and take action according to NaNChecker::action_if_found TerminationTrigger::TerminationTrigger_ResetMinutes: [global] Reset Watchtime endif if (checkpoint initial data) [CCTK_CPINITIAL] CarpetIOHDF5::CarpetIOHDF5_InitialDataCheckpoint: [meta] Initial data checkpoint routine endif if (analysis) [CCTK_ANALYSIS] CarpetLib::CarpetLib_printtimestats: [global] Print timing statistics if desired CarpetLib::CarpetLib_printmemstats: [global] Print memory statistics if desired Formaline::Formaline_AnnounceUpdate: [global] Put some meta information about the current run into permanent storage LoopControl::LC_statistics_analysis: [meta] Output LoopControl statistics GROUP ML_BSSN_EvolutionAnalysis: Calculate RHS at analysis ML_BSSN::ML_BSSN_EvolutionAnalysisInit: ML_BSSN_EvolutionAnalysisInit ML_BSSN::ML_BSSN_EvolutionAnalysisInterior: ML_BSSN_EvolutionAnalysisInterior ML_BSSN_Helper::ML_BSSN_NewRad: Apply NewRad boundary conditions to RHS Multipole::Multipole_Calc: [global] Calculate Multipoles QuasiLocalMeasures::qlm_calculate: [global] Calculate quasi-local quantities SystemStatistics::SystemStatistics_Collect: [global] Collect system statistics TerminationTrigger::TerminationTrigger_CheckWalltime: Check elapsed job walltime TerminationTrigger::TerminationTrigger_CheckSignal: Check if we received a termination signal TerminationTrigger::TerminationTrigger_CheckFile: Check termination file TimerReport::zzz_TimerReport_Output: [global] Print the timer report Trigger::Trigger_Check: [global] test endif Output grid variables do loop over timesteps [CCTK_PREREGRID] CarpetTracker::CarpetTracker_SetPositions: [global] Set positions of refined regions Change grid hierarchy [CCTK_POSTREGRID] CartGrid3D::SpatialCoordinates: Set Coordinates after regridding GROUP MaskBase_SetupMask: Set up the weight function GROUP MaskBase_SetupMaskAll: Set up the weight function CarpetReduce::MaskBase_AllocateMask: [global] Allocate the weight function CarpetReduce::MaskBase_InitMask: [global] [loop-local] Initialise the weight function GROUP SetupIMaskInternal: Set up the integer weight function (schedule other routines in here) CarpetReduce::CoordBase_SetupMask: [global] [loop-local] Set up the outer boundaries of the weight function CarpetReduce::CarpetMaskSetup: [global] [loop-singlemap] Set up the weight function for the restriction regions GROUP SetupIMask: Set up the integer weight function (schedule other routines in here) CarpetMask::CarpetExcludedSetup: [global] [loop-local] Set up the weight function for the excluded regions CarpetMask::CarpetSurfaceSetup: [global] [loop-local] Set up the weight function for the excluded spherical surfaces CarpetReduce::MaskBase_SetMask: [global] [loop-local] Set the weight function GROUP SetupMask: Set up the real weight function (schedule other routines in here) CarpetReduce::MaskBase_TestMask: [global] Test the weight function Dissipation::setup_epsdis: Setup spatially varying dissipation SpaceMask::MaskZero: Initialise mask to zero GROUP GRHydro_PrimitiveBoundaries: Apply boundary conditions to all primitives GRHydro::GRHydro_SelectPrimitiveBoundaries: [level] Select primitive variables for boundary conditions GROUP GRHydro_ApplyPrimitiveBCs: Apply boundary conditions to all primitive variables GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP HydroBase_ExcisionMaskSetup: Set up hydro excision mask HydroBase::HydroBase_InitExcisionMask: Initialize hydro excision mask to 'no excision everywhere' SpaceMask::MaskOne: Set mask to one GRHydro::GRHydro_SetupMask: Initialize the atmosphere mask GROUP MoL_PostStep: Ensure that everything is correct after regridding ML_BSSN::ML_BSSN_SelectBoundConds: [level] select boundary conditions GRHydro::GRHydro_RefinementLevel: Calculate current refinement level GRHydro::GRHydro_SetLastMoLPostStep: [level] Set grid scalar InLastMoLPostStep if this is the last MoL PostStep call GROUP ML_BSSN_ApplyBCs: Apply boundary conditions controlled by thorn Boundary GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions ML_ADMConstraints::ML_ADMConstraints_SelectBoundConds: [level] select boundary conditions GROUP ML_ADMConstraints_ApplyBCs: Apply boundary conditions controlled by thorn Boundary GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions ML_BSSN::ML_BSSN_ADMBaseInterior: ML_BSSN_ADMBaseInterior ML_BSSN::ML_BSSN_ADMBaseBoundaryScalar: ML_BSSN_ADMBaseBoundaryScalar ML_BSSN::ML_BSSN_ADMBaseEverywhere: ML_BSSN_ADMBaseEverywhere ML_BSSN_Helper::ML_BSSN_ADMBase_SelectBCs: [level] Select boundary conditions for ADMBase variables GROUP ML_BSSN_ADMBase_ApplyBCs: Apply boundary conditions to ADMBase variables GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP ADMBase_SetADMVars: Set the ADM variables before this group, and use them afterwards GROUP HydroBase_PostStep: Post step tasks for hydro thorns GROUP GRHydro_PostStep: Post step tasks for GRHydro GROUP GRHydro_AtmosphereMaskBoundaries: Apply boundary conditions to primitives GRHydro::GRHydro_SelectAtmosphereMaskBoundaries: [level] Select atmosphere mask for boundary conditions GROUP GRHydro_ApplyAtmosphereMaskBCs: Apply boundary conditions to real-valued atmosphere mask GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GRHydro::GRHydroPostSyncAtmosphereMask: Set integer atmosphere mask from synchronized real atmosphere mask if (GRHydro::InLastMoLPostStep) GRHydro::GRHydro_AtmosphereReset: Reset the atmosphere if (GRHydro::InLastMoLPostStep) GROUP GRHydro_PrimitiveInitialGuessesBoundaries: Apply boundary conditions to those primitives used as initial guesses GRHydro::GRHydro_SelectPrimitiveInitialGuessesBoundaries: [level] Select initial guess primitive variables for boudary conditions GROUP GRHydro_ApplyPrimitiveInitialGuessBCs: Apply boundary conditions to initial guess primitive variables GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP HydroBase_Boundaries: HydroBase-internal Boundary conditions group GROUP Do_GRHydro_Boundaries: GRHydro Boundary conditions group GROUP HydroBase_Select_Boundaries: Group to schedule the boundary condition functions if (GRHydro::execute_MoL_PostStep) GRHydro::GRHydro_Bound: [level] Select GRHydro boundary conditions GROUP HydroBase_ApplyBCs: Apply the boundary conditions of HydroBase GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP HydroBase_Con2Prim: Convert from conservative to primitive variables if (GRHydro::execute_MoL_Step) GRHydro::GRHydro_SqrtSpatialDeterminant: Calculate sdetg if (GRHydro::execute_MoL_PostStep) GRHydro::Con2Prim: Convert back to primitive variables (general) GROUP SetTmunu: Group for calculating the stress-energy tensor TmunuBase::TmunuBase_ZeroTmunu: Initialise the stress-energy tensor to zero GROUP AddToTmunu: Add to the stress-energy tensor here GRHydro::GRHydro_Tmunu: Compute the energy-momentum tensor WeylScal4::WeylScal4_SelectBoundConds: [level] select boundary conditions GROUP WeylScal4_ApplyBCs: Apply boundary conditions controlled by thorn Boundary GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP MoL_PseudoEvolutionBoundaries: Apply boundary conditions to pseudo-evolved quantities GROUP ML_ADMConstraints_evaluate_bc_group: ML_ADMConstraints_evaluate ML_ADMConstraints::ML_ADMConstraints_evaluate_SelectBCs: [level] ML_ADMConstraints_evaluate_SelectBCs GROUP ML_ADMConstraints_evaluate_ApplyBCs: Apply BCs for groups set in ML_ADMConstraints_evaluate GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP ML_BSSN_ConstraintsEverywhere_bc_group: ML_BSSN_ConstraintsEverywhere ML_BSSN::ML_BSSN_ConstraintsEverywhere_SelectBCs: [level] ML_BSSN_ConstraintsEverywhere_SelectBCs GROUP ML_BSSN_ConstraintsEverywhere_ApplyBCs: Apply BCs for groups set in ML_BSSN_ConstraintsEverywhere GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP ML_BSSN_ConstraintsInterior_bc_group: ML_BSSN_ConstraintsInterior ML_BSSN::ML_BSSN_ConstraintsInterior_SelectBCs: [level] ML_BSSN_ConstraintsInterior_SelectBCs GROUP ML_BSSN_ConstraintsInterior_ApplyBCs: Apply BCs for groups set in ML_BSSN_ConstraintsInterior GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP WeylScal4_psi4_calc_4th_bc_group: WeylScal4_psi4_calc_4th WeylScal4::WeylScal4_psi4_calc_4th_SelectBCs: [level] WeylScal4_psi4_calc_4th_SelectBCs GROUP WeylScal4_psi4_calc_4th_ApplyBCs: Apply BCs for groups set in WeylScal4_psi4_calc_4th GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions Rotate timelevels iteration = iteration+1 t = t+dt [CCTK_PRESTEP] CoordGauge::Einstein_SetNextSlicing: Identify the slicing for the next iteration GRHydro::reset_GRHydro_C2P_failed: Reset the mask function that contains the points where C2P has failed (at PRESTEP) LoopControl::LC_steer: [meta] Update LoopControl algorithm preferences NaNChecker::NaNChecker_ResetCounter: [global] Reset the NaNChecker::NaNsFound counter [CCTK_EVOL] MoL::MoL_StartLoop: [level] Initialise the step size control while (MoL::MoL_Stepsize_Bad) GROUP MoL_Evolution: A single Cactus evolution step using MoL GROUP MoL_StartStep: MoL internal setup for the evolution step MoL::MoL_SetCounter: [level] Set the counter for the ODE method to loop over MoL::MoL_SetTime: [level] Ensure the correct time and timestep are used MoL::MoL_AllocateScratchSpace: [level] Allocate storage for scratch levels GROUP MoL_PreStep: Physics thorns can schedule preloop setup routines in here GRHydro::GRHydro_Scalar_Setup: Set up and check scalars for efficiency MoL::MoL_AllocateScratch: Allocate sufficient space for array scratch variables MoL::MoL_InitialCopy: Ensure the data is in the correct timelevel while (MoL::MoL_Intermediate_Step) GROUP MoL_Step: The loop over the intermediate steps for the ODE integrator MoL::MoL_InitRHS: Initialise the RHS functions GROUP MoL_CalcRHS: Physics thorns schedule the calculation of the discrete spatial operator in here GROUP HydroBase_RHS: Groups for scheduling tasks for calculating RHS of hydro variables if (GRHydro::execute_MoL_Step) GROUP GRHydroRHS: Calculate the update terms GRHydro::SourceTerms: Source term calculation GRHydro::GRHydroStartLoop: [level] Set the flux_direction variable while (GRHydro::flux_direction) GROUP FluxTerms: Calculation of intercell fluxes GRHydro::GRHydro_RefinementLevel: Calculate current refinement level GRHydro::Reconstruct: Reconstruct the functions at the cell boundaries GRHydro::Riemann: Solve the local Riemann problems GRHydro::UpdateCalcul: Calculate the update term from the fluxes GRHydro::GRHydroAdvanceLoop: [level] Decrement the flux_direction variable end while GRHydro::GRHydroUpdateAtmosphereMask: Alter the update terms if inside the atmosphere region ML_BSSN::ML_BSSN_EvolutionBoundaryScalar: ML_BSSN_EvolutionBoundaryScalar GROUP ML_BSSN_EvolutionInteriorSplitBy: ML_BSSN::ML_BSSN_EvolutionInteriorSplitBy1: ML_BSSN_EvolutionInteriorSplitBy1 ML_BSSN::ML_BSSN_EvolutionInteriorSplitBy2: ML_BSSN_EvolutionInteriorSplitBy2 ML_BSSN::ML_BSSN_EvolutionInteriorSplitBy3: ML_BSSN_EvolutionInteriorSplitBy3 ML_BSSN_Helper::ML_BSSN_NewRad: Apply NewRad boundary conditions to RHS GROUP MoL_PostRHS: Modify RHS functions Dissipation::dissipation_add: Add Kreiss-Oliger dissipation to the right hand sides GROUP MoL_RHSBoundaries: Any 'final' modifications to the RHS functions (boundaries etc.) MoL::MoL_Add: Updates calculated with the efficient Runge-Kutta 4 method MoL::MoL_DecrementCounter: [level] Alter the counter number MoL::MoL_ResetTime: [level] If necessary, change the time GROUP MoL_PostStepModify: The group for physics thorns to schedule enforcing constraints ML_BSSN::ML_BSSN_EnforceEverywhere: ML_BSSN_EnforceEverywhere GROUP MoL_PostStep: The group for physics thorns to schedule boundary calls etc. ML_BSSN::ML_BSSN_SelectBoundConds: [level] select boundary conditions GRHydro::GRHydro_RefinementLevel: Calculate current refinement level GRHydro::GRHydro_SetLastMoLPostStep: [level] Set grid scalar InLastMoLPostStep if this is the last MoL PostStep call GROUP ML_BSSN_ApplyBCs: Apply boundary conditions controlled by thorn Boundary GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions ML_ADMConstraints::ML_ADMConstraints_SelectBoundConds: [level] select boundary conditions GROUP ML_ADMConstraints_ApplyBCs: Apply boundary conditions controlled by thorn Boundary GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions ML_BSSN::ML_BSSN_ADMBaseInterior: ML_BSSN_ADMBaseInterior ML_BSSN::ML_BSSN_ADMBaseBoundaryScalar: ML_BSSN_ADMBaseBoundaryScalar ML_BSSN::ML_BSSN_ADMBaseEverywhere: ML_BSSN_ADMBaseEverywhere ML_BSSN_Helper::ML_BSSN_ADMBase_SelectBCs: [level] Select boundary conditions for ADMBase variables GROUP ML_BSSN_ADMBase_ApplyBCs: Apply boundary conditions to ADMBase variables GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP ADMBase_SetADMVars: Set the ADM variables before this group, and use them afterwards GROUP HydroBase_PostStep: Post step tasks for hydro thorns GROUP GRHydro_PostStep: Post step tasks for GRHydro GROUP GRHydro_AtmosphereMaskBoundaries: Apply boundary conditions to primitives GRHydro::GRHydro_SelectAtmosphereMaskBoundaries: [level] Select atmosphere mask for boundary conditions GROUP GRHydro_ApplyAtmosphereMaskBCs: Apply boundary conditions to real-valued atmosphere mask GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GRHydro::GRHydroPostSyncAtmosphereMask: Set integer atmosphere mask from synchronized real atmosphere mask if (GRHydro::InLastMoLPostStep) GRHydro::GRHydro_AtmosphereReset: Reset the atmosphere if (GRHydro::InLastMoLPostStep) GROUP GRHydro_PrimitiveInitialGuessesBoundaries: Apply boundary conditions to those primitives used as initial guesses GRHydro::GRHydro_SelectPrimitiveInitialGuessesBoundaries: [level] Select initial guess primitive variables for boudary conditions GROUP GRHydro_ApplyPrimitiveInitialGuessBCs: Apply boundary conditions to initial guess primitive variables GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP HydroBase_Boundaries: HydroBase-internal Boundary conditions group GROUP Do_GRHydro_Boundaries: GRHydro Boundary conditions group GROUP HydroBase_Select_Boundaries: Group to schedule the boundary condition functions if (GRHydro::execute_MoL_PostStep) GRHydro::GRHydro_Bound: [level] Select GRHydro boundary conditions GROUP HydroBase_ApplyBCs: Apply the boundary conditions of HydroBase GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP HydroBase_Con2Prim: Convert from conservative to primitive variables if (GRHydro::execute_MoL_Step) GRHydro::GRHydro_SqrtSpatialDeterminant: Calculate sdetg if (GRHydro::execute_MoL_PostStep) GRHydro::Con2Prim: Convert back to primitive variables (general) GROUP SetTmunu: Group for calculating the stress-energy tensor TmunuBase::TmunuBase_ZeroTmunu: Initialise the stress-energy tensor to zero GROUP AddToTmunu: Add to the stress-energy tensor here GRHydro::GRHydro_Tmunu: Compute the energy-momentum tensor WeylScal4::WeylScal4_SelectBoundConds: [level] select boundary conditions GROUP WeylScal4_ApplyBCs: Apply boundary conditions controlled by thorn Boundary GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GRHydro::GRHydro_ClearLastMoLPostStep: [level] Reset InLastMoLPostStep to zero MoL::MoL_ResetDeltaTime: [level] If necessary, change the timestep end while MoL::MoL_FinishLoop: [level] Control the step size MoL::MoL_RestoreSandR: Restoring the Save and Restore variables to the original state MoL::MoL_FreeScratchSpace: [level] Free storage for scratch levels end while GRHydro::sync_GRHydro_C2P_failed: Syncronise the mask function that contains the points where C2P has failed GROUP MoL_PseudoEvolution: Calculate pseudo-evolved quantities GROUP ADMBase_SetADMVars: Set the ADM variables before this group, and use them afterwards GROUP ML_ADMConstraints_evaluate_group: ML_ADMConstraints_evaluate ML_ADMConstraints::ML_ADMConstraints_evaluate: ML_ADMConstraints_evaluate GROUP ML_ADMConstraints_evaluate_bc_group: ML_ADMConstraints_evaluate ML_ADMConstraints::ML_ADMConstraints_evaluate_SelectBCs: [level] ML_ADMConstraints_evaluate_SelectBCs GROUP ML_ADMConstraints_evaluate_ApplyBCs: Apply BCs for groups set in ML_ADMConstraints_evaluate GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP ML_BSSN_ConstraintsEverywhere_group: ML_BSSN_ConstraintsEverywhere ML_BSSN::ML_BSSN_ConstraintsEverywhere: ML_BSSN_ConstraintsEverywhere GROUP ML_BSSN_ConstraintsEverywhere_bc_group: ML_BSSN_ConstraintsEverywhere ML_BSSN::ML_BSSN_ConstraintsEverywhere_SelectBCs: [level] ML_BSSN_ConstraintsEverywhere_SelectBCs GROUP ML_BSSN_ConstraintsEverywhere_ApplyBCs: Apply BCs for groups set in ML_BSSN_ConstraintsEverywhere GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP ML_BSSN_ConstraintsInterior_group: ML_BSSN_ConstraintsInterior ML_BSSN::ML_BSSN_ConstraintsInterior: ML_BSSN_ConstraintsInterior GROUP ML_BSSN_ConstraintsInterior_bc_group: ML_BSSN_ConstraintsInterior ML_BSSN::ML_BSSN_ConstraintsInterior_SelectBCs: [level] ML_BSSN_ConstraintsInterior_SelectBCs GROUP ML_BSSN_ConstraintsInterior_ApplyBCs: Apply BCs for groups set in ML_BSSN_ConstraintsInterior GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP WeylScal4_psi4_calc_4th_group: WeylScal4_psi4_calc_4th WeylScal4::WeylScal4_psi4_calc_4th: WeylScal4_psi4_calc_4th GROUP WeylScal4_psi4_calc_4th_bc_group: WeylScal4_psi4_calc_4th WeylScal4::WeylScal4_psi4_calc_4th_SelectBCs: [level] WeylScal4_psi4_calc_4th_SelectBCs GROUP WeylScal4_psi4_calc_4th_ApplyBCs: Apply BCs for groups set in WeylScal4_psi4_calc_4th GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions Evolve finer grids recursively Restrict from finer grids [CCTK_POSTRESTRICT] GROUP MoL_PostStep: Ensure that everything is correct after restriction ML_BSSN::ML_BSSN_SelectBoundConds: [level] select boundary conditions GRHydro::GRHydro_RefinementLevel: Calculate current refinement level GRHydro::GRHydro_SetLastMoLPostStep: [level] Set grid scalar InLastMoLPostStep if this is the last MoL PostStep call GROUP ML_BSSN_ApplyBCs: Apply boundary conditions controlled by thorn Boundary GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions ML_ADMConstraints::ML_ADMConstraints_SelectBoundConds: [level] select boundary conditions GROUP ML_ADMConstraints_ApplyBCs: Apply boundary conditions controlled by thorn Boundary GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions ML_BSSN::ML_BSSN_ADMBaseInterior: ML_BSSN_ADMBaseInterior ML_BSSN::ML_BSSN_ADMBaseBoundaryScalar: ML_BSSN_ADMBaseBoundaryScalar ML_BSSN::ML_BSSN_ADMBaseEverywhere: ML_BSSN_ADMBaseEverywhere ML_BSSN_Helper::ML_BSSN_ADMBase_SelectBCs: [level] Select boundary conditions for ADMBase variables GROUP ML_BSSN_ADMBase_ApplyBCs: Apply boundary conditions to ADMBase variables GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP ADMBase_SetADMVars: Set the ADM variables before this group, and use them afterwards GROUP HydroBase_PostStep: Post step tasks for hydro thorns GROUP GRHydro_PostStep: Post step tasks for GRHydro GROUP GRHydro_AtmosphereMaskBoundaries: Apply boundary conditions to primitives GRHydro::GRHydro_SelectAtmosphereMaskBoundaries: [level] Select atmosphere mask for boundary conditions GROUP GRHydro_ApplyAtmosphereMaskBCs: Apply boundary conditions to real-valued atmosphere mask GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GRHydro::GRHydroPostSyncAtmosphereMask: Set integer atmosphere mask from synchronized real atmosphere mask if (GRHydro::InLastMoLPostStep) GRHydro::GRHydro_AtmosphereReset: Reset the atmosphere if (GRHydro::InLastMoLPostStep) GROUP GRHydro_PrimitiveInitialGuessesBoundaries: Apply boundary conditions to those primitives used as initial guesses GRHydro::GRHydro_SelectPrimitiveInitialGuessesBoundaries: [level] Select initial guess primitive variables for boudary conditions GROUP GRHydro_ApplyPrimitiveInitialGuessBCs: Apply boundary conditions to initial guess primitive variables GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP HydroBase_Boundaries: HydroBase-internal Boundary conditions group GROUP Do_GRHydro_Boundaries: GRHydro Boundary conditions group GROUP HydroBase_Select_Boundaries: Group to schedule the boundary condition functions if (GRHydro::execute_MoL_PostStep) GRHydro::GRHydro_Bound: [level] Select GRHydro boundary conditions GROUP HydroBase_ApplyBCs: Apply the boundary conditions of HydroBase GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP HydroBase_Con2Prim: Convert from conservative to primitive variables if (GRHydro::execute_MoL_Step) GRHydro::GRHydro_SqrtSpatialDeterminant: Calculate sdetg if (GRHydro::execute_MoL_PostStep) GRHydro::Con2Prim: Convert back to primitive variables (general) GROUP SetTmunu: Group for calculating the stress-energy tensor TmunuBase::TmunuBase_ZeroTmunu: Initialise the stress-energy tensor to zero GROUP AddToTmunu: Add to the stress-energy tensor here GRHydro::GRHydro_Tmunu: Compute the energy-momentum tensor WeylScal4::WeylScal4_SelectBoundConds: [level] select boundary conditions GROUP WeylScal4_ApplyBCs: Apply boundary conditions controlled by thorn Boundary GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP MoL_PseudoEvolutionBoundaries: Apply boundary conditions to pseudo-evolved quantities GROUP ML_ADMConstraints_evaluate_bc_group: ML_ADMConstraints_evaluate ML_ADMConstraints::ML_ADMConstraints_evaluate_SelectBCs: [level] ML_ADMConstraints_evaluate_SelectBCs GROUP ML_ADMConstraints_evaluate_ApplyBCs: Apply BCs for groups set in ML_ADMConstraints_evaluate GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP ML_BSSN_ConstraintsEverywhere_bc_group: ML_BSSN_ConstraintsEverywhere ML_BSSN::ML_BSSN_ConstraintsEverywhere_SelectBCs: [level] ML_BSSN_ConstraintsEverywhere_SelectBCs GROUP ML_BSSN_ConstraintsEverywhere_ApplyBCs: Apply BCs for groups set in ML_BSSN_ConstraintsEverywhere GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP ML_BSSN_ConstraintsInterior_bc_group: ML_BSSN_ConstraintsInterior ML_BSSN::ML_BSSN_ConstraintsInterior_SelectBCs: [level] ML_BSSN_ConstraintsInterior_SelectBCs GROUP ML_BSSN_ConstraintsInterior_ApplyBCs: Apply BCs for groups set in ML_BSSN_ConstraintsInterior GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP WeylScal4_psi4_calc_4th_bc_group: WeylScal4_psi4_calc_4th WeylScal4::WeylScal4_psi4_calc_4th_SelectBCs: [level] WeylScal4_psi4_calc_4th_SelectBCs GROUP WeylScal4_psi4_calc_4th_ApplyBCs: Apply BCs for groups set in WeylScal4_psi4_calc_4th GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions [CCTK_POSTSTEP] SphericalSurface::SphericalSurface_Set: [global] Set surface radii GROUP SphericalSurface_HasBeenSet: Set the spherical surfaces before this group, and use it afterwards SphericalSurface::SphericalSurface_CheckState: [global] Test the state of the spherical surfaces GRHydro::GRHydro_RefinementLevel: Calculate current refinement level (for the check of the C2P mask) GRHydro::check_GRHydro_C2P_failed: Check the mask function that contains the points where C2P has failed and report an error in case a failure is found HTTPD::HTTP_Work: [global] Working routine HTTPD::HTTP_ContentWork: [global] Content Working routine GROUP HydroBase_ExcisionHasBeenSet: Group to schedule thorns changing the mask before and thorns using the mask after GROUP Hydro_Analysis: Group for Hydro_Analysis routines Hydro_Analysis::Hydro_Analysis_PrepareReduction: [global] [loop-local] Compute the local reduction results Hydro_Analysis::Hydro_Analysis_Reduction: [global] Compute the global reduction results GROUP Hydro_Analysis_LocationSearch: Look for the location of the maximum density Hydro_Analysis::Hydro_Analysis_LocationSearch_Setup: [global] Prepare data structures for search Hydro_Analysis::Hydro_Analysis_LocationSearch_Search: [global] [loop-local] Search for the location of the maximum density Hydro_Analysis::Hydro_Analysis_LocationSearch_Combine: [global] Communicate and verify the location of the maximum density Hydro_Analysis::Hydro_Analysis_FindSeparation: [global] Compute the proper distance between the maximum of the density and the origin (along a straight coordinate line) NSTracker::NSTracker: [global] Look for the star and set grid accordingly GROUP zzz_NaNChecker_NaNCheck: Check for NaNs and count them in NaNChecker::NaNsFound NaNChecker::NaNChecker_NaNCheck_Prepare: [level] Prepare data structures to check for NaNs NaNChecker::NaNChecker_NaNCheck_Check: [local] Check for NaNs NaNChecker::NaNChecker_NaNCheck_Finish: [level] Count NaNs in NaNChecker::NaNsFound NaNChecker::NaNChecker_TakeAction: [global] [loop-level] Output NaNChecker::NaNmask and take action according to NaNChecker::action_if_found SpaceMask::CheckMask: Ensure that all mask values are legal Dissipation::setup_epsdis: Setup spatially varying dissipation GROUP MaskBase_SetupMask: Set up the weight function GROUP MaskBase_SetupMaskAll: Set up the weight function CarpetReduce::MaskBase_AllocateMask: [global] Allocate the weight function CarpetReduce::MaskBase_InitMask: [global] [loop-local] Initialise the weight function GROUP SetupIMaskInternal: Set up the integer weight function (schedule other routines in here) CarpetReduce::CoordBase_SetupMask: [global] [loop-local] Set up the outer boundaries of the weight function CarpetReduce::CarpetMaskSetup: [global] [loop-singlemap] Set up the weight function for the restriction regions GROUP SetupIMask: Set up the integer weight function (schedule other routines in here) CarpetMask::CarpetExcludedSetup: [global] [loop-local] Set up the weight function for the excluded regions CarpetMask::CarpetSurfaceSetup: [global] [loop-local] Set up the weight function for the excluded spherical surfaces CarpetReduce::MaskBase_SetMask: [global] [loop-local] Set the weight function GROUP SetupMask: Set up the real weight function (schedule other routines in here) CarpetReduce::MaskBase_TestMask: [global] Test the weight function if (checkpoint) [CCTK_CHECKPOINT] CarpetIOHDF5::CarpetIOHDF5_EvolutionCheckpoint: [meta] Evolution checkpoint routine TimerReport::zzz_TimerReport_Checkpoint: [global] Print the timer report endif if (analysis) [CCTK_ANALYSIS] CarpetLib::CarpetLib_printtimestats: [global] Print timing statistics if desired CarpetLib::CarpetLib_printmemstats: [global] Print memory statistics if desired Formaline::Formaline_AnnounceUpdate: [global] Put some meta information about the current run into permanent storage LoopControl::LC_statistics_analysis: [meta] Output LoopControl statistics GROUP ML_BSSN_EvolutionAnalysis: Calculate RHS at analysis ML_BSSN::ML_BSSN_EvolutionAnalysisInit: ML_BSSN_EvolutionAnalysisInit ML_BSSN::ML_BSSN_EvolutionAnalysisInterior: ML_BSSN_EvolutionAnalysisInterior ML_BSSN_Helper::ML_BSSN_NewRad: Apply NewRad boundary conditions to RHS Multipole::Multipole_Calc: [global] Calculate Multipoles QuasiLocalMeasures::qlm_calculate: [global] Calculate quasi-local quantities SystemStatistics::SystemStatistics_Collect: [global] Collect system statistics TerminationTrigger::TerminationTrigger_CheckWalltime: Check elapsed job walltime TerminationTrigger::TerminationTrigger_CheckSignal: Check if we received a termination signal TerminationTrigger::TerminationTrigger_CheckFile: Check termination file TimerReport::zzz_TimerReport_Output: [global] Print the timer report Trigger::Trigger_Check: [global] test endif Output grid variables enddo Termination routines [CCTK_TERMINATE] CarpetIOHDF5::CarpetIOHDF5_TerminationCheckpoint: [meta] Termination checkpoint routine Formaline::Formaline_AnnounceFinal: [global] Put some meta information about the current run into permanent storage LoopControl::LC_statistics_terminate: [meta] Output LoopControl statistics MoL::MoL_FreeIndexArrays: Free the MoL bookkeeping index arrays TimerReport::zzz_TimerReport_Output: [global] Print the timer report Shutdown routines [CCTK_SHUTDOWN] HTTPD::HTTP_Shutdown: [global] HTTP daemon shutdown Timers::Timer_Shutdown: Prepare hierarchical timers Routines run after changing the grid hierarchy: [CCTK_POSTREGRID] CartGrid3D::SpatialCoordinates: Set Coordinates after regridding GROUP MaskBase_SetupMask: Set up the weight function GROUP MaskBase_SetupMaskAll: Set up the weight function CarpetReduce::MaskBase_AllocateMask: [global] Allocate the weight function CarpetReduce::MaskBase_InitMask: [global] [loop-local] Initialise the weight function GROUP SetupIMaskInternal: Set up the integer weight function (schedule other routines in here) CarpetReduce::CoordBase_SetupMask: [global] [loop-local] Set up the outer boundaries of the weight function CarpetReduce::CarpetMaskSetup: [global] [loop-singlemap] Set up the weight function for the restriction regions GROUP SetupIMask: Set up the integer weight function (schedule other routines in here) CarpetMask::CarpetExcludedSetup: [global] [loop-local] Set up the weight function for the excluded regions CarpetMask::CarpetSurfaceSetup: [global] [loop-local] Set up the weight function for the excluded spherical surfaces CarpetReduce::MaskBase_SetMask: [global] [loop-local] Set the weight function GROUP SetupMask: Set up the real weight function (schedule other routines in here) CarpetReduce::MaskBase_TestMask: [global] Test the weight function Dissipation::setup_epsdis: Setup spatially varying dissipation SpaceMask::MaskZero: Initialise mask to zero GROUP GRHydro_PrimitiveBoundaries: Apply boundary conditions to all primitives GRHydro::GRHydro_SelectPrimitiveBoundaries: [level] Select primitive variables for boundary conditions GROUP GRHydro_ApplyPrimitiveBCs: Apply boundary conditions to all primitive variables GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP HydroBase_ExcisionMaskSetup: Set up hydro excision mask HydroBase::HydroBase_InitExcisionMask: Initialize hydro excision mask to 'no excision everywhere' SpaceMask::MaskOne: Set mask to one GRHydro::GRHydro_SetupMask: Initialize the atmosphere mask GROUP MoL_PostStep: Ensure that everything is correct after regridding ML_BSSN::ML_BSSN_SelectBoundConds: [level] select boundary conditions GRHydro::GRHydro_RefinementLevel: Calculate current refinement level GRHydro::GRHydro_SetLastMoLPostStep: [level] Set grid scalar InLastMoLPostStep if this is the last MoL PostStep call GROUP ML_BSSN_ApplyBCs: Apply boundary conditions controlled by thorn Boundary GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions ML_ADMConstraints::ML_ADMConstraints_SelectBoundConds: [level] select boundary conditions GROUP ML_ADMConstraints_ApplyBCs: Apply boundary conditions controlled by thorn Boundary GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions ML_BSSN::ML_BSSN_ADMBaseInterior: ML_BSSN_ADMBaseInterior ML_BSSN::ML_BSSN_ADMBaseBoundaryScalar: ML_BSSN_ADMBaseBoundaryScalar ML_BSSN::ML_BSSN_ADMBaseEverywhere: ML_BSSN_ADMBaseEverywhere ML_BSSN_Helper::ML_BSSN_ADMBase_SelectBCs: [level] Select boundary conditions for ADMBase variables GROUP ML_BSSN_ADMBase_ApplyBCs: Apply boundary conditions to ADMBase variables GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP ADMBase_SetADMVars: Set the ADM variables before this group, and use them afterwards GROUP HydroBase_PostStep: Post step tasks for hydro thorns GROUP GRHydro_PostStep: Post step tasks for GRHydro GROUP GRHydro_AtmosphereMaskBoundaries: Apply boundary conditions to primitives GRHydro::GRHydro_SelectAtmosphereMaskBoundaries: [level] Select atmosphere mask for boundary conditions GROUP GRHydro_ApplyAtmosphereMaskBCs: Apply boundary conditions to real-valued atmosphere mask GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GRHydro::GRHydroPostSyncAtmosphereMask: Set integer atmosphere mask from synchronized real atmosphere mask if (GRHydro::InLastMoLPostStep) GRHydro::GRHydro_AtmosphereReset: Reset the atmosphere if (GRHydro::InLastMoLPostStep) GROUP GRHydro_PrimitiveInitialGuessesBoundaries: Apply boundary conditions to those primitives used as initial guesses GRHydro::GRHydro_SelectPrimitiveInitialGuessesBoundaries: [level] Select initial guess primitive variables for boudary conditions GROUP GRHydro_ApplyPrimitiveInitialGuessBCs: Apply boundary conditions to initial guess primitive variables GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP HydroBase_Boundaries: HydroBase-internal Boundary conditions group GROUP Do_GRHydro_Boundaries: GRHydro Boundary conditions group GROUP HydroBase_Select_Boundaries: Group to schedule the boundary condition functions if (GRHydro::execute_MoL_PostStep) GRHydro::GRHydro_Bound: [level] Select GRHydro boundary conditions GROUP HydroBase_ApplyBCs: Apply the boundary conditions of HydroBase GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP HydroBase_Con2Prim: Convert from conservative to primitive variables if (GRHydro::execute_MoL_Step) GRHydro::GRHydro_SqrtSpatialDeterminant: Calculate sdetg if (GRHydro::execute_MoL_PostStep) GRHydro::Con2Prim: Convert back to primitive variables (general) GROUP SetTmunu: Group for calculating the stress-energy tensor TmunuBase::TmunuBase_ZeroTmunu: Initialise the stress-energy tensor to zero GROUP AddToTmunu: Add to the stress-energy tensor here GRHydro::GRHydro_Tmunu: Compute the energy-momentum tensor WeylScal4::WeylScal4_SelectBoundConds: [level] select boundary conditions GROUP WeylScal4_ApplyBCs: Apply boundary conditions controlled by thorn Boundary GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP MoL_PseudoEvolutionBoundaries: Apply boundary conditions to pseudo-evolved quantities GROUP ML_ADMConstraints_evaluate_bc_group: ML_ADMConstraints_evaluate ML_ADMConstraints::ML_ADMConstraints_evaluate_SelectBCs: [level] ML_ADMConstraints_evaluate_SelectBCs GROUP ML_ADMConstraints_evaluate_ApplyBCs: Apply BCs for groups set in ML_ADMConstraints_evaluate GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP ML_BSSN_ConstraintsEverywhere_bc_group: ML_BSSN_ConstraintsEverywhere ML_BSSN::ML_BSSN_ConstraintsEverywhere_SelectBCs: [level] ML_BSSN_ConstraintsEverywhere_SelectBCs GROUP ML_BSSN_ConstraintsEverywhere_ApplyBCs: Apply BCs for groups set in ML_BSSN_ConstraintsEverywhere GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP ML_BSSN_ConstraintsInterior_bc_group: ML_BSSN_ConstraintsInterior ML_BSSN::ML_BSSN_ConstraintsInterior_SelectBCs: [level] ML_BSSN_ConstraintsInterior_SelectBCs GROUP ML_BSSN_ConstraintsInterior_ApplyBCs: Apply BCs for groups set in ML_BSSN_ConstraintsInterior GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP WeylScal4_psi4_calc_4th_bc_group: WeylScal4_psi4_calc_4th WeylScal4::WeylScal4_psi4_calc_4th_SelectBCs: [level] WeylScal4_psi4_calc_4th_SelectBCs GROUP WeylScal4_psi4_calc_4th_ApplyBCs: Apply BCs for groups set in WeylScal4_psi4_calc_4th GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions -------------------------------------------------------------------------------- INFO (Carpet): Multi-Model listing: model 0: "world" INFO (Carpet): Multi-Model process distribution: process 0: model 0 "world" INFO (Carpet): Multi-Model: This is process 0, model 0 "world" INFO (CarpetLib): Process startup time was 1.76 seconds Current core file size limit: hard=[unlimited], soft=[unlimited] Current addres space size limit: hard=[unlimited], soft=[unlimited] Current data segment size limit: hard=[unlimited], soft=[unlimited] Current resident set size limit: hard=[unlimited], soft=[unlimited] INFO (CycleClock): Measuring CycleClock tick via OpenMP... INFO (CycleClock): Calibrated CycleClock: 0.345488 ns per clock tick (2.89446 GHz) INFO (hwloc): library version 2.0.4, API version 0x20000 INFO (SystemTopology): MPI process-to-host mapping: This is MPI process 0 of 1 MPI hosts: 0: panther This MPI process runs on host 0 of 1 On this host, this is MPI process 0 of 1 INFO (SystemTopology): Topology support: Discovery support: discovery->pu : yes CPU binding support: cpubind->set_thisproc_cpubind : yes cpubind->get_thisproc_cpubind : yes cpubind->set_proc_cpubind : yes cpubind->get_proc_cpubind : yes cpubind->set_thisthread_cpubind : yes cpubind->get_thisthread_cpubind : yes cpubind->set_thread_cpubind : yes cpubind->get_thread_cpubind : yes cpubind->get_thisproc_last_cpu_location : yes cpubind->get_proc_last_cpu_location : yes cpubind->get_thisthread_last_cpu_location: yes Memory binding support: membind->set_thisproc_membind : no membind->get_thisproc_membind : no membind->set_proc_membind : no membind->get_proc_membind : no membind->set_thisthread_membind : yes membind->get_thisthread_membind : yes membind->set_area_membind : yes membind->get_area_membind : yes membind->alloc_membind : yes membind->firsttouch_membind : yes membind->bind_membind : yes membind->interleave_membind : yes membind->nexttouch_membind : no membind->migrate_membind : yes INFO (SystemTopology): Hardware objects in this node: Machine L#0: (P#0, total=1043114796KB, DMIProductName="PowerEdge R7515", DMIProductVersion=, DMIBoardVendor="Dell Inc.", DMIBoardName=04F3CJ, DMIBoardVersion=A02, DMIChassisVendor="Dell Inc.", DMIChassisType=23, DMIChassisVersion=, DMIChassisAssetTag=, DMIBIOSVendor="Dell Inc.", DMIBIOSVersion=2.18.1, DMIBIOSDate=02/03/2025, DMISysVendor="Dell Inc.", Backend=Linux, OSName=Linux, OSRelease=5.14.0-570.37.1.el9_6.x86_64, OSVersion="#1 SMP PREEMPT_DYNAMIC Thu Aug 28 10:41:06 UTC 2025", HostName=panther, Architecture=x86_64, hwlocVersion=2.0.4, ProcessName=cactus_sim) Package L#0: (P#0, total=1043114796KB, CPUVendor=AuthenticAMD, CPUFamilyNumber=23, CPUModelNumber=49, CPUModel="AMD EPYC 7542 32-Core Processor ", CPUStepping=0) L3Cache L#0: (P#-1, size=16384KB, linesize=64, ways=16, Inclusive=0) L2Cache L#0: (P#-1, size=512KB, linesize=64, ways=8, Inclusive=1) L1dCache L#0: (P#-1, size=32KB, linesize=64, ways=8, Inclusive=0) Core L#0: (P#0) PU L#0: (P#0) PU L#1: (P#32) L2Cache L#1: (P#-1, size=512KB, linesize=64, ways=8, Inclusive=1) L1dCache L#1: (P#-1, size=32KB, linesize=64, ways=8, Inclusive=0) Core L#1: (P#1) PU L#2: (P#1) PU L#3: (P#33) L2Cache L#2: (P#-1, size=512KB, linesize=64, ways=8, Inclusive=1) L1dCache L#2: (P#-1, size=32KB, linesize=64, ways=8, Inclusive=0) Core L#2: (P#2) PU L#4: (P#2) PU L#5: (P#34) L2Cache L#3: (P#-1, size=512KB, linesize=64, ways=8, Inclusive=1) L1dCache L#3: (P#-1, size=32KB, linesize=64, ways=8, Inclusive=0) Core L#3: (P#3) PU L#6: (P#3) PU L#7: (P#35) L3Cache L#1: (P#-1, size=16384KB, linesize=64, ways=16, Inclusive=0) L2Cache L#4: (P#-1, size=512KB, linesize=64, ways=8, Inclusive=1) L1dCache L#4: (P#-1, size=32KB, linesize=64, ways=8, Inclusive=0) Core L#4: (P#4) PU L#8: (P#4) PU L#9: (P#36) L2Cache L#5: (P#-1, size=512KB, linesize=64, ways=8, Inclusive=1) L1dCache L#5: (P#-1, size=32KB, linesize=64, ways=8, Inclusive=0) Core L#5: (P#5) PU L#10: (P#5) PU L#11: (P#37) L2Cache L#6: (P#-1, size=512KB, linesize=64, ways=8, Inclusive=1) L1dCache L#6: (P#-1, size=32KB, linesize=64, ways=8, Inclusive=0) Core L#6: (P#6) PU L#12: (P#6) PU L#13: (P#38) L2Cache L#7: (P#-1, size=512KB, linesize=64, ways=8, Inclusive=1) L1dCache L#7: (P#-1, size=32KB, linesize=64, ways=8, Inclusive=0) Core L#7: (P#7) PU L#14: (P#7) PU L#15: (P#39) L3Cache L#2: (P#-1, size=16384KB, linesize=64, ways=16, Inclusive=0) L2Cache L#8: (P#-1, size=512KB, linesize=64, ways=8, Inclusive=1) L1dCache L#8: (P#-1, size=32KB, linesize=64, ways=8, Inclusive=0) Core L#8: (P#8) PU L#16: (P#8) PU L#17: (P#40) L2Cache L#9: (P#-1, size=512KB, linesize=64, ways=8, Inclusive=1) L1dCache L#9: (P#-1, size=32KB, linesize=64, ways=8, Inclusive=0) Core L#9: (P#9) PU L#18: (P#9) PU L#19: (P#41) L2Cache L#10: (P#-1, size=512KB, linesize=64, ways=8, Inclusive=1) L1dCache L#10: (P#-1, size=32KB, linesize=64, ways=8, Inclusive=0) Core L#10: (P#10) PU L#20: (P#10) PU L#21: (P#42) L2Cache L#11: (P#-1, size=512KB, linesize=64, ways=8, Inclusive=1) L1dCache L#11: (P#-1, size=32KB, linesize=64, ways=8, Inclusive=0) Core L#11: (P#11) PU L#22: (P#11) PU L#23: (P#43) L3Cache L#3: (P#-1, size=16384KB, linesize=64, ways=16, Inclusive=0) L2Cache L#12: (P#-1, size=512KB, linesize=64, ways=8, Inclusive=1) L1dCache L#12: (P#-1, size=32KB, linesize=64, ways=8, Inclusive=0) Core L#12: (P#12) PU L#24: (P#12) PU L#25: (P#44) L2Cache L#13: (P#-1, size=512KB, linesize=64, ways=8, Inclusive=1) L1dCache L#13: (P#-1, size=32KB, linesize=64, ways=8, Inclusive=0) Core L#13: (P#13) PU L#26: (P#13) PU L#27: (P#45) L2Cache L#14: (P#-1, size=512KB, linesize=64, ways=8, Inclusive=1) L1dCache L#14: (P#-1, size=32KB, linesize=64, ways=8, Inclusive=0) Core L#14: (P#14) PU L#28: (P#14) PU L#29: (P#46) L2Cache L#15: (P#-1, size=512KB, linesize=64, ways=8, Inclusive=1) L1dCache L#15: (P#-1, size=32KB, linesize=64, ways=8, Inclusive=0) Core L#15: (P#15) PU L#30: (P#15) PU L#31: (P#47) L3Cache L#4: (P#-1, size=16384KB, linesize=64, ways=16, Inclusive=0) L2Cache L#16: (P#-1, size=512KB, linesize=64, ways=8, Inclusive=1) L1dCache L#16: (P#-1, size=32KB, linesize=64, ways=8, Inclusive=0) Core L#16: (P#16) PU L#32: (P#16) PU L#33: (P#48) L2Cache L#17: (P#-1, size=512KB, linesize=64, ways=8, Inclusive=1) L1dCache L#17: (P#-1, size=32KB, linesize=64, ways=8, Inclusive=0) Core L#17: (P#17) PU L#34: (P#17) PU L#35: (P#49) L2Cache L#18: (P#-1, size=512KB, linesize=64, ways=8, Inclusive=1) L1dCache L#18: (P#-1, size=32KB, linesize=64, ways=8, Inclusive=0) Core L#18: (P#18) PU L#36: (P#18) PU L#37: (P#50) L2Cache L#19: (P#-1, size=512KB, linesize=64, ways=8, Inclusive=1) L1dCache L#19: (P#-1, size=32KB, linesize=64, ways=8, Inclusive=0) Core L#19: (P#19) PU L#38: (P#19) PU L#39: (P#51) L3Cache L#5: (P#-1, size=16384KB, linesize=64, ways=16, Inclusive=0) L2Cache L#20: (P#-1, size=512KB, linesize=64, ways=8, Inclusive=1) L1dCache L#20: (P#-1, size=32KB, linesize=64, ways=8, Inclusive=0) Core L#20: (P#20) PU L#40: (P#20) PU L#41: (P#52) L2Cache L#21: (P#-1, size=512KB, linesize=64, ways=8, Inclusive=1) L1dCache L#21: (P#-1, size=32KB, linesize=64, ways=8, Inclusive=0) Core L#21: (P#21) PU L#42: (P#21) PU L#43: (P#53) L2Cache L#22: (P#-1, size=512KB, linesize=64, ways=8, Inclusive=1) L1dCache L#22: (P#-1, size=32KB, linesize=64, ways=8, Inclusive=0) Core L#22: (P#22) PU L#44: (P#22) PU L#45: (P#54) L2Cache L#23: (P#-1, size=512KB, linesize=64, ways=8, Inclusive=1) L1dCache L#23: (P#-1, size=32KB, linesize=64, ways=8, Inclusive=0) Core L#23: (P#23) PU L#46: (P#23) PU L#47: (P#55) L3Cache L#6: (P#-1, size=16384KB, linesize=64, ways=16, Inclusive=0) L2Cache L#24: (P#-1, size=512KB, linesize=64, ways=8, Inclusive=1) L1dCache L#24: (P#-1, size=32KB, linesize=64, ways=8, Inclusive=0) Core L#24: (P#24) PU L#48: (P#24) PU L#49: (P#56) L2Cache L#25: (P#-1, size=512KB, linesize=64, ways=8, Inclusive=1) L1dCache L#25: (P#-1, size=32KB, linesize=64, ways=8, Inclusive=0) Core L#25: (P#25) PU L#50: (P#25) PU L#51: (P#57) L2Cache L#26: (P#-1, size=512KB, linesize=64, ways=8, Inclusive=1) L1dCache L#26: (P#-1, size=32KB, linesize=64, ways=8, Inclusive=0) Core L#26: (P#26) PU L#52: (P#26) PU L#53: (P#58) L2Cache L#27: (P#-1, size=512KB, linesize=64, ways=8, Inclusive=1) L1dCache L#27: (P#-1, size=32KB, linesize=64, ways=8, Inclusive=0) Core L#27: (P#27) PU L#54: (P#27) PU L#55: (P#59) L3Cache L#7: (P#-1, size=16384KB, linesize=64, ways=16, Inclusive=0) L2Cache L#28: (P#-1, size=512KB, linesize=64, ways=8, Inclusive=1) L1dCache L#28: (P#-1, size=32KB, linesize=64, ways=8, Inclusive=0) Core L#28: (P#28) PU L#56: (P#28) PU L#57: (P#60) L2Cache L#29: (P#-1, size=512KB, linesize=64, ways=8, Inclusive=1) L1dCache L#29: (P#-1, size=32KB, linesize=64, ways=8, Inclusive=0) Core L#29: (P#29) PU L#58: (P#29) PU L#59: (P#61) L2Cache L#30: (P#-1, size=512KB, linesize=64, ways=8, Inclusive=1) L1dCache L#30: (P#-1, size=32KB, linesize=64, ways=8, Inclusive=0) Core L#30: (P#30) PU L#60: (P#30) PU L#61: (P#62) L2Cache L#31: (P#-1, size=512KB, linesize=64, ways=8, Inclusive=1) L1dCache L#31: (P#-1, size=32KB, linesize=64, ways=8, Inclusive=0) Core L#31: (P#31) PU L#62: (P#31) PU L#63: (P#63) INFO (SystemTopology): Thread CPU bindings: MPI process 0 on host 0 (process 0 of 1 on this host) OpenMP thread 0: PU set L#{0-63} P#{0-63} OpenMP thread 1: PU set L#{0-63} P#{0-63} OpenMP thread 2: PU set L#{0-63} P#{0-63} OpenMP thread 3: PU set L#{0-63} P#{0-63} OpenMP thread 4: PU set L#{0-63} P#{0-63} OpenMP thread 5: PU set L#{0-63} P#{0-63} OpenMP thread 6: PU set L#{0-63} P#{0-63} OpenMP thread 7: PU set L#{0-63} P#{0-63} OpenMP thread 8: PU set L#{0-63} P#{0-63} OpenMP thread 9: PU set L#{0-63} P#{0-63} OpenMP thread 10: PU set L#{0-63} P#{0-63} OpenMP thread 11: PU set L#{0-63} P#{0-63} OpenMP thread 12: PU set L#{0-63} P#{0-63} OpenMP thread 13: PU set L#{0-63} P#{0-63} OpenMP thread 14: PU set L#{0-63} P#{0-63} OpenMP thread 15: PU set L#{0-63} P#{0-63} OpenMP thread 16: PU set L#{0-63} P#{0-63} OpenMP thread 17: PU set L#{0-63} P#{0-63} OpenMP thread 18: PU set L#{0-63} P#{0-63} OpenMP thread 19: PU set L#{0-63} P#{0-63} OpenMP thread 20: PU set L#{0-63} P#{0-63} OpenMP thread 21: PU set L#{0-63} P#{0-63} OpenMP thread 22: PU set L#{0-63} P#{0-63} OpenMP thread 23: PU set L#{0-63} P#{0-63} OpenMP thread 24: PU set L#{0-63} P#{0-63} OpenMP thread 25: PU set L#{0-63} P#{0-63} OpenMP thread 26: PU set L#{0-63} P#{0-63} OpenMP thread 27: PU set L#{0-63} P#{0-63} OpenMP thread 28: PU set L#{0-63} P#{0-63} OpenMP thread 29: PU set L#{0-63} P#{0-63} OpenMP thread 30: PU set L#{0-63} P#{0-63} OpenMP thread 31: PU set L#{0-63} P#{0-63} INFO (SystemTopology): Setting thread CPU bindings: INFO (SystemTopology): Thread CPU bindings: MPI process 0 on host 0 (process 0 of 1 on this host) OpenMP thread 0: PU set L#{0} P#{0} OpenMP thread 1: PU set L#{2} P#{1} OpenMP thread 2: PU set L#{4} P#{2} OpenMP thread 3: PU set L#{6} P#{3} OpenMP thread 4: PU set L#{8} P#{4} OpenMP thread 5: PU set L#{10} P#{5} OpenMP thread 6: PU set L#{12} P#{6} OpenMP thread 7: PU set L#{14} P#{7} OpenMP thread 8: PU set L#{16} P#{8} OpenMP thread 9: PU set L#{18} P#{9} OpenMP thread 10: PU set L#{20} P#{10} OpenMP thread 11: PU set L#{22} P#{11} OpenMP thread 12: PU set L#{24} P#{12} OpenMP thread 13: PU set L#{26} P#{13} OpenMP thread 14: PU set L#{28} P#{14} OpenMP thread 15: PU set L#{30} P#{15} OpenMP thread 16: PU set L#{32} P#{16} OpenMP thread 17: PU set L#{34} P#{17} OpenMP thread 18: PU set L#{36} P#{18} OpenMP thread 19: PU set L#{38} P#{19} OpenMP thread 20: PU set L#{40} P#{20} OpenMP thread 21: PU set L#{42} P#{21} OpenMP thread 22: PU set L#{44} P#{22} OpenMP thread 23: PU set L#{46} P#{23} OpenMP thread 24: PU set L#{48} P#{24} OpenMP thread 25: PU set L#{50} P#{25} OpenMP thread 26: PU set L#{52} P#{26} OpenMP thread 27: PU set L#{54} P#{27} OpenMP thread 28: PU set L#{56} P#{28} OpenMP thread 29: PU set L#{58} P#{29} OpenMP thread 30: PU set L#{60} P#{30} OpenMP thread 31: PU set L#{62} P#{31} INFO (SystemTopology): Extracting CPU/cache/memory properties: There are 2 PUs per core (aka hardware SMT threads) There are 1 threads per core (aka SMT threads used) Cache (unknown name) has type "data" depth 1 size 32768 linesize 64 associativity 8 stride 4096, for 2 PUs Cache (unknown name) has type "unified" depth 2 size 524288 linesize 64 associativity 8 stride 65536, for 2 PUs Cache (unknown name) has type "unified" depth 3 size 16777216 linesize 64 associativity 16 stride 1048576, for 8 PUs INFO (Formaline): Configuration id: config-sim-panther-system-user-crangano-einstein_toolkit-BNS INFO (Formaline): Build id: build-sim-panther-crangano-2025.09.09-14.35.11-2994181 INFO (Formaline): Simulation id: run-bns-panther-crangano-2025.09.10-15.19.02-3273101 INFO (Formaline): Run id: run-bns-panther-crangano-2025.09.10-15.19.02-3273101 Server started on http://panther:5555/ Not announcing location via Twitter. INFO (Vectors): Using vector size 2 for architecture SSE2+SSE4.1 (64-bit precision) -------------------------------------------------------------------------------- AMR driver provided by Carpet -------------------------------------------------------------------------------- GRHydro: relativistic hydrodynamics, no ice. -------------------------------------------------------------------------------- HydroBase: Let it flow. -------------------------------------------------------------------------------- AMR info I/O provided by CarpetIOBasic -------------------------------------------------------------------------------- ML_ADMConstraints -------------------------------------------------------------------------------- ML_BSSN -------------------------------------------------------------------------------- AMR HDF5 I/O provided by CarpetIOHDF5 -------------------------------------------------------------------------------- AMR 0D HDF5 I/O provided by CarpetIOHDF5 -------------------------------------------------------------------------------- AMR 1D HDF5 I/O provided by CarpetIOHDF5 -------------------------------------------------------------------------------- AMR 2D HDF5 I/O provided by CarpetIOHDF5 -------------------------------------------------------------------------------- AMR 3D HDF5 I/O provided by CarpetIOHDF5 -------------------------------------------------------------------------------- MoL: Generalized time integration. -------------------------------------------------------------------------------- AMR scalar I/O provided by CarpetIOScalar -------------------------------------------------------------------------------- AMR 0D ASCII I/O provided by CarpetIOASCII -------------------------------------------------------------------------------- AMR 1D ASCII I/O provided by CarpetIOASCII -------------------------------------------------------------------------------- AMR 2D ASCII I/O provided by CarpetIOASCII -------------------------------------------------------------------------------- AMR 3D ASCII I/O provided by CarpetIOASCII -------------------------------------------------------------------------------- WeylScal4 -------------------------------------------------------------------------------- INFO (Carpet): MPI is enabled INFO (Carpet): Carpet is running on 1 processes INFO (Carpet): This is process 0 INFO (Carpet): OpenMP is enabled INFO (Carpet): This process contains 32 threads, this is thread 0 INFO (Carpet): There are 32 threads in total INFO (Carpet): There are 32 threads per process INFO (Carpet): This process runs on host panther, pid=3273101 INFO (Carpet): This process runs on 32 cores: 0-31 INFO (Carpet): Thread 0 runs on 1 core: 0 INFO (Carpet): Thread 1 runs on 1 core: 1 INFO (Carpet): Thread 2 runs on 1 core: 2 INFO (Carpet): Thread 3 runs on 1 core: 3 INFO (Carpet): Thread 4 runs on 1 core: 4 INFO (Carpet): Thread 5 runs on 1 core: 5 INFO (Carpet): Thread 6 runs on 1 core: 6 INFO (Carpet): Thread 7 runs on 1 core: 7 INFO (Carpet): Thread 8 runs on 1 core: 8 INFO (Carpet): Thread 9 runs on 1 core: 9 INFO (Carpet): Thread 10 runs on 1 core: 10 INFO (Carpet): Thread 11 runs on 1 core: 11 INFO (Carpet): Thread 12 runs on 1 core: 12 INFO (Carpet): Thread 13 runs on 1 core: 13 INFO (Carpet): Thread 14 runs on 1 core: 14 INFO (Carpet): Thread 15 runs on 1 core: 15 INFO (Carpet): Thread 16 runs on 1 core: 16 INFO (Carpet): Thread 17 runs on 1 core: 17 INFO (Carpet): Thread 18 runs on 1 core: 18 INFO (Carpet): Thread 19 runs on 1 core: 19 INFO (Carpet): Thread 20 runs on 1 core: 20 INFO (Carpet): Thread 21 runs on 1 core: 21 INFO (Carpet): Thread 22 runs on 1 core: 22 INFO (Carpet): Thread 23 runs on 1 core: 23 INFO (Carpet): Thread 24 runs on 1 core: 24 INFO (Carpet): Thread 25 runs on 1 core: 25 INFO (Carpet): Thread 26 runs on 1 core: 26 INFO (Carpet): Thread 27 runs on 1 core: 27 INFO (Carpet): Thread 28 runs on 1 core: 28 INFO (Carpet): Thread 29 runs on 1 core: 29 INFO (Carpet): Thread 30 runs on 1 core: 30 INFO (Carpet): Thread 31 runs on 1 core: 31 INFO (Carpet): This simulation is running in 3 dimensions INFO (Carpet): Boundary specification for map 0: nboundaryzones: [[3,3,3],[3,3,3]] is_internal : [[0,0,0],[0,0,0]] is_staggered : [[0,0,0],[0,0,0]] shiftout : [[1,0,1],[0,0,0]] INFO (Carpet): CoordBase domain specification for map 0: physical extent: [0,0,0] : [400,400,400] ([400,400,400]) interior extent: [0,16,0] : [384,384,384] ([384,368,384]) exterior extent: [-48,-32,-48] : [432,432,432] ([480,464,480]) base_spacing : [16,16,16] INFO (Carpet): Adapted domain specification for map 0: convergence factor: 2 convergence level : 0 physical extent : [0,0,0] : [400,400,400] ([400,400,400]) interior extent : [0,16,0] : [384,384,384] ([384,368,384]) exterior extent : [-48,-32,-48] : [432,432,432] ([480,464,480]) spacing : [16,16,16] INFO (Carpet): Base grid specification for map 0: number of grid points : [31,30,31] number of coarse grid ghost points: [[3,3,3],[3,3,3]] INFO (Carpet): Buffer zone counts (excluding ghosts): [0]: [[0,0,0],[0,0,0]] [1]: [[9,9,9],[9,9,9]] [2]: [[9,9,9],[9,9,9]] [3]: [[9,9,9],[9,9,9]] [4]: [[9,9,9],[9,9,9]] [5]: [[9,9,9],[9,9,9]] [6]: [[9,9,9],[9,9,9]] [7]: [[9,9,9],[9,9,9]] [8]: [[9,9,9],[9,9,9]] INFO (Carpet): Overlap zone counts: [0]: [[0,0,0],[0,0,0]] [1]: [[0,0,0],[0,0,0]] [2]: [[0,0,0],[0,0,0]] [3]: [[0,0,0],[0,0,0]] [4]: [[0,0,0],[0,0,0]] [5]: [[0,0,0],[0,0,0]] [6]: [[0,0,0],[0,0,0]] [7]: [[0,0,0],[0,0,0]] [8]: [[0,0,0],[0,0,0]] INFO (Carpet): Group and variable statistics: INFO (Carpet): There are 1162 grid functions in 159 groups INFO (Carpet): There are 604 grid scalars in 96 groups INFO (Carpet): There are 130 1-dimensional grid arrays in 13 groups INFO (Carpet): There are 143 2-dimensional grid arrays in 18 groups INFO (Carpet): There are 0 3-dimensional grid arrays in 0 groups INFO (Carpet): (The number of variables counts all time levels) INFO (CarpetIOHDF5): I/O Method 'IOHDF5' registered: AMR output of grid variables to HDF5 files INFO (CarpetIOHDF5): I/O Method 'IOHDF5_0D' registered: 0D AMR output of grid variables to HDF5 files INFO (CarpetIOHDF5): I/O Method 'IOHDF5_1D' registered: 1D AMR output of grid variables to HDF5 files INFO (CarpetIOHDF5): I/O Method 'IOHDF5_2D' registered: 2D AMR output of grid variables to HDF5 files INFO (CarpetIOHDF5): Periodic 2D AMR output requested for: ADMBASE::gxx ADMBASE::gxy ADMBASE::gxz ADMBASE::gyy ADMBASE::gyz ADMBASE::gzz ADMBASE::alp ADMBASE::betax ADMBASE::betay ADMBASE::betaz CARPETREDUCE::weight GRID::x GRID::y GRID::z GRID::r HYDROBASE::rho HYDROBASE::eps HYDROBASE::vel[0] HYDROBASE::vel[1] HYDROBASE::vel[2] ML_ADMCONSTRAINTS::H INFO (CarpetIOHDF5): I/O Method 'IOHDF5_3D' registered: 3D AMR output of grid variables to HDF5 files INFO (CarpetIOHDF5): Periodic 3D AMR output requested for: ADMBASE::alp ADMBASE::betax ADMBASE::betay ADMBASE::betaz CARPETREDUCE::weight GRID::x GRID::y GRID::z GRID::r HYDROBASE::rho HYDROBASE::eps HYDROBASE::vel[0] HYDROBASE::vel[1] HYDROBASE::vel[2] ML_ADMCONSTRAINTS::H INFO (CarpetIOScalar): Periodic scalar output requested for: ADMBASE::gxx ADMBASE::gxy ADMBASE::gxz ADMBASE::gyy ADMBASE::gyz ADMBASE::gzz ADMBASE::kxx ADMBASE::kxy ADMBASE::kxz ADMBASE::kyy ADMBASE::kyz ADMBASE::kzz ADMBASE::alp ADMBASE::betax ADMBASE::betay ADMBASE::betaz GRHYDRO::dens HYDROBASE::rho HYDROBASE::vel[0] HYDROBASE::vel[1] HYDROBASE::vel[2] HYDROBASE::w_lorentz ML_ADMCONSTRAINTS::H SPHERICALSURFACE::sf_radius[0] SPHERICALSURFACE::sf_radius[1] SPHERICALSURFACE::sf_radius[2] SPHERICALSURFACE::sf_radius[3] SPHERICALSURFACE::sf_radius[4] SYSTEMSTATISTICS::maxrss_mb SYSTEMSTATISTICS::majflt_mb SYSTEMSTATISTICS::arena_mb SYSTEMSTATISTICS::ordblks_mb SYSTEMSTATISTICS::hblks_mb SYSTEMSTATISTICS::hblkhd_mb SYSTEMSTATISTICS::uordblks_mb SYSTEMSTATISTICS::fordblks_mb SYSTEMSTATISTICS::keepcost_mb SYSTEMSTATISTICS::swap_used_mb INFO (CarpetIOASCII): I/O Method 'IOASCII_0D' registered: 0D AMR output of grid variables to ASCII files INFO (CarpetIOASCII): Periodic 0D AMR output requested for: CARPET::physical_time_per_hour CARPET::current_physical_time_per_hour CARPET::time_total CARPET::time_evolution CARPET::time_computing CARPET::time_communicating CARPET::time_io CARPET::evolution_steps_count CARPET::local_grid_points_per_second CARPET::total_grid_points_per_second CARPET::local_grid_point_updates_count CARPET::total_grid_point_updates_count CARPET::local_interior_points_per_second CARPET::total_interior_points_per_second CARPET::local_interior_point_updates_count CARPET::total_interior_point_updates_count CARPET::io_per_second CARPET::io_bytes_per_second CARPET::io_bytes_ascii_per_second CARPET::io_bytes_binary_per_second CARPET::io_count CARPET::io_bytes_count CARPET::io_bytes_ascii_count CARPET::io_bytes_binary_count CARPET::comm_per_second CARPET::comm_bytes_per_second CARPET::comm_count CARPET::comm_bytes_count CARPET::time_levels CARPET::current_walltime CARPET::syncs_count HYDRO_ANALYSIS::Hydro_Analysis_rho_max_loc[0] HYDRO_ANALYSIS::Hydro_Analysis_rho_max_loc[1] HYDRO_ANALYSIS::Hydro_Analysis_rho_max_loc[2] HYDRO_ANALYSIS::Hydro_Analysis_rho_max_origin_distance QUASILOCALMEASURES::qlm_time[0] QUASILOCALMEASURES::qlm_time[1] QUASILOCALMEASURES::qlm_equatorial_circumference[0] QUASILOCALMEASURES::qlm_equatorial_circumference[1] QUASILOCALMEASURES::qlm_polar_circumference_0[0] QUASILOCALMEASURES::qlm_polar_circumference_0[1] QUASILOCALMEASURES::qlm_polar_circumference_pi_2[0] QUASILOCALMEASURES::qlm_polar_circumference_pi_2[1] QUASILOCALMEASURES::qlm_area[0] QUASILOCALMEASURES::qlm_area[1] QUASILOCALMEASURES::qlm_irreducible_mass[0] QUASILOCALMEASURES::qlm_irreducible_mass[1] QUASILOCALMEASURES::qlm_radius[0] QUASILOCALMEASURES::qlm_radius[1] QUASILOCALMEASURES::qlm_spin_guess[0] QUASILOCALMEASURES::qlm_spin_guess[1] QUASILOCALMEASURES::qlm_mass_guess[0] QUASILOCALMEASURES::qlm_mass_guess[1] QUASILOCALMEASURES::qlm_killing_eigenvalue_re[0] QUASILOCALMEASURES::qlm_killing_eigenvalue_re[1] QUASILOCALMEASURES::qlm_killing_eigenvalue_im[0] QUASILOCALMEASURES::qlm_killing_eigenvalue_im[1] QUASILOCALMEASURES::qlm_spin[0] QUASILOCALMEASURES::qlm_spin[1] QUASILOCALMEASURES::qlm_npspin[0] QUASILOCALMEASURES::qlm_npspin[1] QUASILOCALMEASURES::qlm_wsspin[0] QUASILOCALMEASURES::qlm_wsspin[1] QUASILOCALMEASURES::qlm_cvspin[0] QUASILOCALMEASURES::qlm_cvspin[1] QUASILOCALMEASURES::qlm_coordspinx[0] QUASILOCALMEASURES::qlm_coordspinx[1] QUASILOCALMEASURES::qlm_coordspiny[0] QUASILOCALMEASURES::qlm_coordspiny[1] QUASILOCALMEASURES::qlm_coordspinz[0] QUASILOCALMEASURES::qlm_coordspinz[1] QUASILOCALMEASURES::qlm_mass[0] QUASILOCALMEASURES::qlm_mass[1] QUASILOCALMEASURES::qlm_adm_energy[0] QUASILOCALMEASURES::qlm_adm_energy[1] QUASILOCALMEASURES::qlm_adm_momentum_x[0] QUASILOCALMEASURES::qlm_adm_momentum_x[1] QUASILOCALMEASURES::qlm_adm_momentum_y[0] QUASILOCALMEASURES::qlm_adm_momentum_y[1] QUASILOCALMEASURES::qlm_adm_momentum_z[0] QUASILOCALMEASURES::qlm_adm_momentum_z[1] QUASILOCALMEASURES::qlm_adm_angular_momentum_x[0] QUASILOCALMEASURES::qlm_adm_angular_momentum_x[1] QUASILOCALMEASURES::qlm_adm_angular_momentum_y[0] QUASILOCALMEASURES::qlm_adm_angular_momentum_y[1] QUASILOCALMEASURES::qlm_adm_angular_momentum_z[0] QUASILOCALMEASURES::qlm_adm_angular_momentum_z[1] QUASILOCALMEASURES::qlm_w_energy[0] QUASILOCALMEASURES::qlm_w_energy[1] QUASILOCALMEASURES::qlm_w_momentum_x[0] QUASILOCALMEASURES::qlm_w_momentum_x[1] QUASILOCALMEASURES::qlm_w_momentum_y[0] QUASILOCALMEASURES::qlm_w_momentum_y[1] QUASILOCALMEASURES::qlm_w_momentum_z[0] QUASILOCALMEASURES::qlm_w_momentum_z[1] QUASILOCALMEASURES::qlm_w_angular_momentum_x[0] QUASILOCALMEASURES::qlm_w_angular_momentum_x[1] QUASILOCALMEASURES::qlm_w_angular_momentum_y[0] QUASILOCALMEASURES::qlm_w_angular_momentum_y[1] QUASILOCALMEASURES::qlm_w_angular_momentum_z[0] QUASILOCALMEASURES::qlm_w_angular_momentum_z[1] SPHERICALSURFACE::sf_active[0] SPHERICALSURFACE::sf_active[1] SPHERICALSURFACE::sf_active[2] SPHERICALSURFACE::sf_active[3] SPHERICALSURFACE::sf_active[4] SPHERICALSURFACE::sf_valid[0] SPHERICALSURFACE::sf_valid[1] SPHERICALSURFACE::sf_valid[2] SPHERICALSURFACE::sf_valid[3] SPHERICALSURFACE::sf_valid[4] SPHERICALSURFACE::sf_area[0] SPHERICALSURFACE::sf_area[1] SPHERICALSURFACE::sf_area[2] SPHERICALSURFACE::sf_area[3] SPHERICALSURFACE::sf_area[4] SPHERICALSURFACE::sf_mean_radius[0] SPHERICALSURFACE::sf_mean_radius[1] SPHERICALSURFACE::sf_mean_radius[2] SPHERICALSURFACE::sf_mean_radius[3] SPHERICALSURFACE::sf_mean_radius[4] SPHERICALSURFACE::sf_centroid_x[0] SPHERICALSURFACE::sf_centroid_x[1] SPHERICALSURFACE::sf_centroid_x[2] SPHERICALSURFACE::sf_centroid_x[3] SPHERICALSURFACE::sf_centroid_x[4] SPHERICALSURFACE::sf_centroid_y[0] SPHERICALSURFACE::sf_centroid_y[1] SPHERICALSURFACE::sf_centroid_y[2] SPHERICALSURFACE::sf_centroid_y[3] SPHERICALSURFACE::sf_centroid_y[4] SPHERICALSURFACE::sf_centroid_z[0] SPHERICALSURFACE::sf_centroid_z[1] SPHERICALSURFACE::sf_centroid_z[2] SPHERICALSURFACE::sf_centroid_z[3] SPHERICALSURFACE::sf_centroid_z[4] SPHERICALSURFACE::sf_quadrupole_xx[0] SPHERICALSURFACE::sf_quadrupole_xx[1] SPHERICALSURFACE::sf_quadrupole_xx[2] SPHERICALSURFACE::sf_quadrupole_xx[3] SPHERICALSURFACE::sf_quadrupole_xx[4] SPHERICALSURFACE::sf_quadrupole_xy[0] SPHERICALSURFACE::sf_quadrupole_xy[1] SPHERICALSURFACE::sf_quadrupole_xy[2] SPHERICALSURFACE::sf_quadrupole_xy[3] SPHERICALSURFACE::sf_quadrupole_xy[4] SPHERICALSURFACE::sf_quadrupole_xz[0] SPHERICALSURFACE::sf_quadrupole_xz[1] SPHERICALSURFACE::sf_quadrupole_xz[2] SPHERICALSURFACE::sf_quadrupole_xz[3] SPHERICALSURFACE::sf_quadrupole_xz[4] SPHERICALSURFACE::sf_quadrupole_yy[0] SPHERICALSURFACE::sf_quadrupole_yy[1] SPHERICALSURFACE::sf_quadrupole_yy[2] SPHERICALSURFACE::sf_quadrupole_yy[3] SPHERICALSURFACE::sf_quadrupole_yy[4] SPHERICALSURFACE::sf_quadrupole_yz[0] SPHERICALSURFACE::sf_quadrupole_yz[1] SPHERICALSURFACE::sf_quadrupole_yz[2] SPHERICALSURFACE::sf_quadrupole_yz[3] SPHERICALSURFACE::sf_quadrupole_yz[4] SPHERICALSURFACE::sf_quadrupole_zz[0] SPHERICALSURFACE::sf_quadrupole_zz[1] SPHERICALSURFACE::sf_quadrupole_zz[2] SPHERICALSURFACE::sf_quadrupole_zz[3] SPHERICALSURFACE::sf_quadrupole_zz[4] SPHERICALSURFACE::sf_min_radius[0] SPHERICALSURFACE::sf_min_radius[1] SPHERICALSURFACE::sf_min_radius[2] SPHERICALSURFACE::sf_min_radius[3] SPHERICALSURFACE::sf_min_radius[4] SPHERICALSURFACE::sf_max_radius[0] SPHERICALSURFACE::sf_max_radius[1] SPHERICALSURFACE::sf_max_radius[2] SPHERICALSURFACE::sf_max_radius[3] SPHERICALSURFACE::sf_max_radius[4] SPHERICALSURFACE::sf_min_x[0] SPHERICALSURFACE::sf_min_x[1] SPHERICALSURFACE::sf_min_x[2] SPHERICALSURFACE::sf_min_x[3] SPHERICALSURFACE::sf_min_x[4] SPHERICALSURFACE::sf_min_y[0] SPHERICALSURFACE::sf_min_y[1] SPHERICALSURFACE::sf_min_y[2] SPHERICALSURFACE::sf_min_y[3] SPHERICALSURFACE::sf_min_y[4] SPHERICALSURFACE::sf_min_z[0] SPHERICALSURFACE::sf_min_z[1] SPHERICALSURFACE::sf_min_z[2] SPHERICALSURFACE::sf_min_z[3] SPHERICALSURFACE::sf_min_z[4] SPHERICALSURFACE::sf_max_x[0] SPHERICALSURFACE::sf_max_x[1] SPHERICALSURFACE::sf_max_x[2] SPHERICALSURFACE::sf_max_x[3] SPHERICALSURFACE::sf_max_x[4] SPHERICALSURFACE::sf_max_y[0] SPHERICALSURFACE::sf_max_y[1] SPHERICALSURFACE::sf_max_y[2] SPHERICALSURFACE::sf_max_y[3] SPHERICALSURFACE::sf_max_y[4] SPHERICALSURFACE::sf_max_z[0] SPHERICALSURFACE::sf_max_z[1] SPHERICALSURFACE::sf_max_z[2] SPHERICALSURFACE::sf_max_z[3] SPHERICALSURFACE::sf_max_z[4] SPHERICALSURFACE::sf_radius[0] SPHERICALSURFACE::sf_radius[1] SPHERICALSURFACE::sf_radius[2] SPHERICALSURFACE::sf_radius[3] SPHERICALSURFACE::sf_radius[4] SPHERICALSURFACE::sf_origin_x[0] SPHERICALSURFACE::sf_origin_x[1] SPHERICALSURFACE::sf_origin_x[2] SPHERICALSURFACE::sf_origin_x[3] SPHERICALSURFACE::sf_origin_x[4] SPHERICALSURFACE::sf_origin_y[0] SPHERICALSURFACE::sf_origin_y[1] SPHERICALSURFACE::sf_origin_y[2] SPHERICALSURFACE::sf_origin_y[3] SPHERICALSURFACE::sf_origin_y[4] SPHERICALSURFACE::sf_origin_z[0] SPHERICALSURFACE::sf_origin_z[1] SPHERICALSURFACE::sf_origin_z[2] SPHERICALSURFACE::sf_origin_z[3] SPHERICALSURFACE::sf_origin_z[4] SPHERICALSURFACE::sf_origin_theta[0] SPHERICALSURFACE::sf_origin_theta[1] SPHERICALSURFACE::sf_origin_theta[2] SPHERICALSURFACE::sf_origin_theta[3] SPHERICALSURFACE::sf_origin_theta[4] SPHERICALSURFACE::sf_origin_phi[0] SPHERICALSURFACE::sf_origin_phi[1] SPHERICALSURFACE::sf_origin_phi[2] SPHERICALSURFACE::sf_origin_phi[3] SPHERICALSURFACE::sf_origin_phi[4] SPHERICALSURFACE::sf_delta_theta[0] SPHERICALSURFACE::sf_delta_theta[1] SPHERICALSURFACE::sf_delta_theta[2] SPHERICALSURFACE::sf_delta_theta[3] SPHERICALSURFACE::sf_delta_theta[4] SPHERICALSURFACE::sf_delta_phi[0] SPHERICALSURFACE::sf_delta_phi[1] SPHERICALSURFACE::sf_delta_phi[2] SPHERICALSURFACE::sf_delta_phi[3] SPHERICALSURFACE::sf_delta_phi[4] INFO (CarpetIOASCII): I/O Method 'IOASCII_1D' registered: 1D AMR output of grid variables to ASCII files INFO (CarpetIOASCII): Periodic 1D AMR output requested for: ADMBASE::gxx ADMBASE::gxy ADMBASE::gxz ADMBASE::gyy ADMBASE::gyz ADMBASE::gzz ADMBASE::kxx ADMBASE::kxy ADMBASE::kxz ADMBASE::kyy ADMBASE::kyz ADMBASE::kzz ADMBASE::alp ADMBASE::betax ADMBASE::betay ADMBASE::betaz HYDROBASE::rho HYDROBASE::vel[0] HYDROBASE::vel[1] HYDROBASE::vel[2] ML_ADMCONSTRAINTS::H INFO (CarpetIOASCII): I/O Method 'IOASCII_2D' registered: 2D AMR output of grid variables to ASCII files INFO (CarpetIOASCII): I/O Method 'IOASCII_3D' registered: 3D AMR output of grid variables to ASCII files -------------------------------------------------------------------------------- INFO (Formaline): Writing tarballs with the Cactus sources into the directory "bns/cactus-source" WARNING level 1 from host panther process 0 in thorn ML_BSSN_Helper, file /system/user/crangano/einstein_toolkit/BNS/configs/sim/build/ML_BSSN_Helper/Parameters.c:10: -> Forcing ML_BSSN::initial_boundary_condition="extrapolate-gammas" because ML_BSSN::my_initial_boundary_condition="extrapolate-gammas" WARNING level 1 from host panther process 0 in thorn ML_BSSN_Helper, file /system/user/crangano/einstein_toolkit/BNS/configs/sim/build/ML_BSSN_Helper/Parameters.c:25: -> Forcing ML_BSSN::rhs_boundary_condition="NewRad" because ML_BSSN::my_rhs_boundary_condition="NewRad" WARNING level 1 from host panther process 0 in thorn ML_BSSN_Helper, file /system/user/crangano/einstein_toolkit/BNS/configs/sim/build/ML_BSSN_Helper/Parameters.c:45: -> Forcing ML_BSSN::epsDiss=0.0 because ML_BSSN::apply_dissipation="never" INFO (MoL): Using Runge-Kutta 4 as the time integrator. INFO (SymBase): Symmetry on lower x-face: rotating_symmetry_180 INFO (SymBase): Symmetry on lower z-face: reflection_symmetry INFO (MoL): The maximum number of evolved variables is 1246. 30 are registered. INFO (MoL): The maximum number of slow evolved variables is 1246. 0 are registered. INFO (MoL): The maximum number of constrained variables is 1246. 37 are registered. INFO (MoL): The maximum number of SandR variables is 1246. 0 are registered. INFO (MoL): The maximum number of evolved array variables is 1246. 0 are registered. INFO (MoL): The maximum number of constrained array variables is 1246. 0 are registered. INFO (MoL): The maximum number of SandR array variables is 1246. 0 are registered. INFO (MoL): The maximum size of any array variables is 0. WARNING level 1 from host panther process 0 in thorn ML_BSSN_Helper, file /system/user/crangano/einstein_toolkit/BNS/configs/sim/build/ML_BSSN_Helper/Parameters.c:145: -> Parameter ML_BSSN::my_initial_data is outdated; please update the parameter file. Do not use this parameter, and set up initial conditions via ADMBase as usual. WARNING level 1 from host panther process 0 in thorn ML_BSSN_Helper, file /system/user/crangano/einstein_toolkit/BNS/configs/sim/build/ML_BSSN_Helper/Parameters.c:152: -> Parameter ML_BSSN::my_initial_boundary_condition is outdated; please update the parameter file. Do not use this parameter, and set up initial boundary conditions as usual. WARNING level 1 from host panther process 0 in thorn ML_BSSN_Helper, file /system/user/crangano/einstein_toolkit/BNS/configs/sim/build/ML_BSSN_Helper/Parameters.c:159: -> Parameter ML_BSSN::my_rhs_boundary_condition is outdated; please update the parameter file. Do not use this parameter, and set up RHS boundary conditions as usual. WARNING level 1 from host panther process 0 in thorn ML_BSSN_Helper, file /system/user/crangano/einstein_toolkit/BNS/configs/sim/build/ML_BSSN_Helper/Parameters.c:166: -> Parameter ML_BSSN::my_boundary_condition is outdated; please update the parameter file. Do not use this parameter, and set up RHS boundary conditions as usual. INFO (Vectors): Testing vectorisation... [errors may result in segfaults] INFO (Vectors): 375/375 tests passed INFO (CarpetRegrid2): Enforcing grid structure properties, iteration 0 INFO (CarpetRegrid2): Enforcing grid structure properties, iteration 1 INFO (CarpetRegrid2): Enforcing grid structure properties, iteration 0 INFO (CarpetRegrid2): Enforcing grid structure properties, iteration 1 INFO (CarpetRegrid2): Enforcing grid structure properties, iteration 0 INFO (CarpetRegrid2): Enforcing grid structure properties, iteration 1 INFO (CarpetRegrid2): Enforcing grid structure properties, iteration 0 INFO (CarpetRegrid2): Enforcing grid structure properties, iteration 1 INFO (CarpetRegrid2): Enforcing grid structure properties, iteration 0 INFO (CarpetRegrid2): Enforcing grid structure properties, iteration 1 INFO (CarpetRegrid2): Enforcing grid structure properties, iteration 0 INFO (CarpetRegrid2): Enforcing grid structure properties, iteration 1 INFO (Carpet): Grid structure (superregions, grid points): [0][0][0] exterior: [0,0,0] : [30,29,30] ([31,30,31] + PADDING) 28830 [1][0][0] exterior: [3,2,3] : [47,56,45] ([45,55,43] + PADDING) 106425 [2][0][0] exterior: [9,6,9] : [55,47,51] ([47,42,43] + PADDING) 84882 [2][0][1] exterior: [9,69,9] : [55,110,51] ([47,42,43] + PADDING) 84882 [3][0][0] exterior: [21,14,21] : [71,55,63] ([51,42,43] + PADDING) 92106 [3][0][1] exterior: [21,177,21] : [71,218,63] ([51,42,43] + PADDING) 92106 [4][0][0] exterior: [45,30,45] : [105,73,89] ([61,44,45] + PADDING) 120780 [4][0][1] exterior: [45,391,45] : [105,434,89] ([61,44,45] + PADDING) 120780 [5][0][0] exterior: [93,62,93] : [175,111,143] ([83,50,51] + PADDING) 211650 [5][0][1] exterior: [93,817,93] : [175,866,143] ([83,50,51] + PADDING) 211650 [6][0][0] exterior: [189,126,189] : [315,189,253] ([127,64,65] + PADDING) 528320 [6][0][1] exterior: [189,1667,189] : [315,1730,253] ([127,64,65] + PADDING) 528320 INFO (Carpet): Grid structure (superregions, coordinates): [0][0][0] exterior: [-48,-32,-48] : [432,432,432] : [16,16,16] [1][0][0] exterior: [-24,-16,-24] : [328,416,312] : [8,8,8] [2][0][0] exterior: [-12,-8,-12] : [172,156,156] : [4,4,4] [2][0][1] exterior: [-12,244,-12] : [172,408,156] : [4,4,4] [3][0][0] exterior: [-6,-4,-6] : [94,78,78] : [2,2,2] [3][0][1] exterior: [-6,322,-6] : [94,404,78] : [2,2,2] [4][0][0] exterior: [-3,-2,-3] : [57,41,41] : [1,1,1] [4][0][1] exterior: [-3,359,-3] : [57,402,41] : [1,1,1] [5][0][0] exterior: [-1.5,-1,-1.5] : [39.5,23.5,23.5] : [0.5,0.5,0.5] [5][0][1] exterior: [-1.5,376.5,-1.5] : [39.5,401,23.5] : [0.5,0.5,0.5] [6][0][0] exterior: [-0.75,-0.5,-0.75] : [30.75,15.25,15.25] : [0.25,0.25,0.25] [6][0][1] exterior: [-0.75,384.75,-0.75] : [30.75,400.5,15.25] : [0.25,0.25,0.25] INFO (Carpet): Global grid structure statistics: INFO (Carpet): GF: rhs: 393k active, 604k owned (+54%), 780k total (+29%), 127 steps/time INFO (Carpet): GF: vars: 319, pts: 367M active, 601M owned (+64%), 812M total (+35%), 1.0 comp/proc INFO (Carpet): GA: vars: 808, pts: 0M active, 0M total (+0%) INFO (Carpet): Total required memory: 6.462 GByte (for GAs and currently active GFs) INFO (Carpet): Load balance: min avg max sdv max/avg-1 INFO (Carpet): Level 0: 5M 5M 5M 0M owned 0% INFO (Carpet): Level 1: 26M 26M 26M 0M owned 0% INFO (Carpet): Level 2: 44M 44M 44M 0M owned 0% INFO (Carpet): Level 3: 48M 48M 48M 0M owned 0% INFO (Carpet): Level 4: 64M 64M 64M 0M owned 0% INFO (Carpet): Level 5: 115M 115M 115M 0M owned 0% INFO (Carpet): Level 6: 299M 299M 299M 0M owned 0% INFO (CartGrid3D): Grid Spacings: INFO (CartGrid3D): dx=>1.6000000e+01 dy=>1.6000000e+01 dz=>1.6000000e+01 INFO (CartGrid3D): Computational Coordinates: INFO (CartGrid3D): x=>[-48.000,432.000] y=>[-32.000,432.000] z=>[-48.000,432.000] INFO (CartGrid3D): Indices of Physical Coordinates: INFO (CartGrid3D): x=>[0,30] y=>[0,29] z=>[0,30] INFO (TerminationTrigger): Reminding you every 60 minutes about remaining walltime INFO (Time): Timestep set to 6.4 (courant_static) INFO (GRHydro): Trying to get EOS handles INFO (GRHydro): Trying to get EOS handles INFO (GRHydro): GRHydro will use the Ideal_Fluid equation of state. INFO (GRHydro): Setting up the atmosphere mask: all points are not_atmosphere INFO (Meudon_Bin_NS): Setting up LORENE Bin_NS initial data INFO (Meudon_Bin_NS): Setting up coordinates INFO (Meudon_Bin_NS): Meudon_Bin_NS will use the 2D_Polytrope equation of state. INFO (Meudon_Bin_NS): Meudon_Bin_NS will use the 1 eos handle INFO (Meudon_Bin_NS): Reading from file "/system/user/crangano/einstein_toolkit/BNS/G2_I12vs12_D4R33T21_45km.resu" Simfactory Done at date: Wed 10 Sep 2025 05:19:25 PM CEST -------------- next part -------------- ################################################################################ # Script variables ################################################################################ # D = 10.0 # M = 1.0 # Pmx = 0.00084541526517121 # Pmy = -0.09530152296974252 # Pmz = 0 # Pphi = 0.09530152296974252 # Ppx = -0.00084541526517121 # Ppy = 0.09530152296974252 # Ppz = 0 # Pr = -0.00084541526517121 # Smx = 0.0 # Smy = 0.0 # Smz = -0.09156449704142013 # Spx = 0.0 # Spy = 0.0 # Spz = 0.09509112426035504 # __file__ = /system/user/crangano/einstein_toolkit/Cactus/repos/einsteinexamples/par/GW150914/GW150914.rpar # __name__ = __main__ # ahrm = 0.4461538461538462 # ahrp = 0.5538461538461539 # center_offset = -0.5384615384615383 # chim = -0.46 # chip = 0.31 # col_width = 19 # dt0 = 0.2753406593406593 # dt_it = 0.002151098901098901 # dtdx = 0.45 # dtdx0 = 0.225 # e = 2.718281828459045 # expected_merger = 1000.0 # final_time = 1700.0 # find_cah = 371902 # h0 = 1.2237362637362637 # h0_min = 1.4276923076923076 # half_D = 5.0 # hfm_min = 0.022307692307692306 # horizon_every = 256 # hr = 1.2237362637362637 # hr_min = 1.92 # i = 7 # inf = inf # key = xp # levelsm = [0,17.132308,8.566154,4.283077,2.141538,1.070769,0.535385] # levelsp = [0,21.267692,10.633846,5.316923,2.658462,1.329231,0.664615] # maxrls = 9 # mm = 0.4461538461538462 # mp = 0.5538461538461539 # n = 28 # n_angular = 28 # n_min = 24 # nan = nan # out2d_every = 1024 # out3d_every = 0 # out_every = 128 # outermost_detector = 500.0 # pi = 3.141592653589793 # q = 1.2413793103448276 # rl0_every = 128 # rl1_every = 128 # rl_max = 8 # rlsm = 7 # rlsp = 7 # rm = 0.5353846153846153 # rp = 0.6646153846153846 # sphere_inner_radius = 51.39692307692307 # sphere_outer_radius = 2194.1591208791206 # tau = 6.283185307179586 # time_after_merger = 200.0 # val = 4.461538461538462 # wave_extract_every = 256 # waveform_length = 1200.0 # xm = -5.538461538461538 # xp = 4.461538461538462 ################################################################################ # Active thorns ################################################################################ ActiveThorns = " ADMBase ML_ADMConstraints ADMCoupling ADMMacros AEILocalInterp AHFinderDirect Boundary Carpet CarpetIOASCII CarpetIOBasic CarpetIOHDF5 CarpetIOScalar CarpetInterp CarpetInterp2 CarpetLib CarpetReduce CarpetRegrid2 CarpetTracker CartGrid3D CoordBase CoordGauge Coordinates CoordinatesSymmetry Formaline GlobalDerivative hwloc IOUtil InitBase Interpolate2 QuasiLocalMeasures LocalInterp LoopControl MoL NaNChecker PunctureTracker Slab SpaceMask SphericalSurface StaticConformal SummationByParts SymBase SystemStatistics SystemTopology TerminationTrigger TensorTypes Time TmunuBase TwoPunctures Vectors ML_BSSN ML_BSSN_Helper NewRad GenericFD WeylScal4 Multipole # WaveExtractCPM ADMDerivatives " ################################################################################ # Grid structure ################################################################################ Carpet::domain_from_multipatch = yes CartGrid3D::type = "multipatch" CartGrid3D::set_coordinate_ranges_on = "all maps" Coordinates::coordinate_system = "Thornburg04" Coordinates::h_cartesian = 1.2237362637362637 Coordinates::h_radial = 1.2237362637362637 Coordinates::sphere_inner_radius = 51.39692307692307 Coordinates::sphere_outer_radius = 2194.1591208791206 Coordinates::n_angular = 28 Driver::ghost_size = 5 Coordinates::patch_boundary_size = 5 Coordinates::additional_overlap_size = 3 Coordinates::outer_boundary_size = 5 # change these (to their defaults) to disable z-zymmetry CoordinatesSymmetry::reflection_z = no CoordinatesSymmetry::stagger = no Coordinates::symmetry = "+z bitant" Coordinates::additional_symmetry_size = 1 Coordinates::verbose = no Time::timestep_method = "given" Time::timestep = 0.2753406593406593 Carpet::time_refinement_factors = "[1,1,2,4,8,16,32,64,128,256]" ################################################################################ # Mesh refinement ################################################################################ Carpet::max_refinement_levels = 9 CarpetRegrid2::num_centres = 2 CarpetRegrid2::num_levels_1 = 7 CarpetRegrid2::position_x_1 = 4.461538461538462 CarpetRegrid2::radius_1 = [0,21.267692,10.633846,5.316923,2.658462,1.329231,0.664615] CarpetRegrid2::num_levels_2 = 7 CarpetRegrid2::position_x_2 = -5.538461538461538 CarpetRegrid2::radius_2 = [0,17.132308,8.566154,4.283077,2.141538,1.070769,0.535385] Carpet::use_buffer_zones = yes Carpet::prolongation_order_space = 5 Carpet::prolongation_order_time = 2 CarpetRegrid2::regrid_every = 128 CarpetRegrid2::verbose = no Carpet::grid_coordinates_filename = "carpet-grid.asc" ################################################################################ # Initial data ################################################################################ ADMBase::initial_data = "twopunctures" ADMBase::initial_lapse = "twopunctures-averaged" ADMBase::initial_shift = "zero" ADMBase::initial_dtlapse = "zero" ADMBase::initial_dtshift = "zero" TwoPunctures::target_M_plus = 0.5538461538461539 TwoPunctures::target_M_minus = 0.4461538461538462 # Initial guesses for iterative solve TwoPunctures::par_m_plus = 0.5538461538461539 TwoPunctures::par_m_minus = 0.4461538461538462 TwoPunctures::par_b = 5.0 TwoPunctures::center_offset[0] = -0.5384615384615383 TwoPunctures::par_P_plus[0] = -0.00084541526517121 TwoPunctures::par_P_plus[1] = 0.09530152296974252 TwoPunctures::par_P_plus[2] = 0 TwoPunctures::par_P_minus[0] = 0.00084541526517121 TwoPunctures::par_P_minus[1] = -0.09530152296974252 TwoPunctures::par_P_minus[2] = 0 TwoPunctures::par_S_plus[0] = 0.0 TwoPunctures::par_S_plus[1] = 0.0 TwoPunctures::par_S_plus[2] = 0.09509112426035504 TwoPunctures::par_S_minus[0] = 0.0 TwoPunctures::par_S_minus[1] = 0.0 TwoPunctures::par_S_minus[2] = -0.09156449704142013 TwoPunctures::grid_setup_method = "evaluation" TwoPunctures::give_bare_mass = no TwoPunctures::TP_epsilon = 1e-6 Carpet::init_fill_timelevels = yes InitBase::initial_data_setup_method = "init_single_level" ################################################################################ # Evolution and boundary ################################################################################ GenericFD::jacobian_group = "Coordinates::jacobian" GenericFD::jacobian_derivative_group = "Coordinates::jacobian2" GenericFD::jacobian_identity_map = 0 ADMBase::evolution_method = "ML_BSSN" ADMBase::lapse_evolution_method = "ML_BSSN" ADMBase::shift_evolution_method = "ML_BSSN" ADMBase::dtlapse_evolution_method = "ML_BSSN" ADMBase::dtshift_evolution_method = "ML_BSSN" ML_BSSN::fdOrder = 8 # 1+log slicing requires harmonicN = 1 and harmonicF = 2.0 ML_BSSN::harmonicN = 1 ML_BSSN::harmonicF = 2.0 ML_BSSN::ShiftGammaCoeff = 0.75 ML_BSSN::BetaDriver = 1.0 ML_BSSN::advectLapse = 1 ML_BSSN::advectShift = 1 ML_BSSN::MinimumLapse = 1.0e-8 # conformalaMethod = 1 for W, 0 for phi ML_BSSN::conformalMethod = 1 # We apply dissipation using GlobalDerivaitive so disable it here ML_BSSN::epsDiss = 0.0 ML_BSSN::dt_lapse_shift_method = "noLapseShiftAdvection" ML_BSSN::initial_boundary_condition = "extrapolate-gammas" ML_BSSN::rhs_boundary_condition = "scalar" Boundary::radpower = 2 ################################################################################ # BH tracking ################################################################################ CarpetTracker::surface [0] = 0 CarpetTracker::surface [1] = 1 PunctureTracker::track [0] = yes PunctureTracker::initial_x [0] = 4.461538461538462 PunctureTracker::which_surface_to_store_info[0] = 0 PunctureTracker::track [1] = yes PunctureTracker::initial_x [1] = -5.538461538461538 PunctureTracker::which_surface_to_store_info[1] = 1 ################################################################################ # Spatial finite differencing ################################################################################ SummationByParts::order = 8 # Drop order instead of using upwinded stencils, only for advection derivatives SummationByParts::sbp_upwind_deriv = no SummationByParts::sbp_1st_deriv = yes SummationByParts::sbp_2nd_deriv = no SummationByParts::onesided_interpatch_boundaries = no SummationByParts::onesided_outer_boundaries = yes SummationByParts::use_dissipation = no GlobalDerivative::use_dissipation = yes SummationByParts::scale_with_h = yes SummationByParts::dissipation_type = "Kreiss-Oliger" SummationByParts::epsdis = 0.15 # Because dt/dx is half that of the other levels we also need half the dissipation GlobalDerivative::epsdis_for_level [0] = 0.075 # Variables for dissipation SummationByParts::vars = " ML_BSSN::ML_log_confac ML_BSSN::ML_metric ML_BSSN::ML_trace_curv ML_BSSN::ML_curv ML_BSSN::ML_Gamma ML_BSSN::ML_lapse ML_BSSN::ML_shift ML_BSSN::ML_dtlapse ML_BSSN::ML_dtshift " ################################################################################ # Time integration ################################################################################ MoL::ODE_Method = "rk4" MoL::MoL_Intermediate_Steps = 4 MoL::MoL_Num_Scratch_Levels = 1 ################################################################################ # Interpolation ################################################################################ CarpetInterp::check_tree_search = no CarpetInterp::tree_search = yes # Use 5-th order interpatch interpolation on the Llama grid Interpolate::interpolator_order = 5 ################################################################################ # Psi4 computation ################################################################################ WeylScal4::fdOrder = 8 WeylScal4::calc_scalars = "psis" WeylScal4::calc_invariants = "always" ################################################################################ # Psi4 mode decomposition ################################################################################ # Radii are chosen to be evenly spaced in 1/r as that is the variable # extrapolation is performed in Multipole::nradii = 7 Multipole::radius[0] = 100 Multipole::radius[1] = 115 Multipole::radius[2] = 136 Multipole::radius[3] = 167 Multipole::radius[4] = 214 Multipole::radius[5] = 300 Multipole::radius[6] = 500 Multipole::ntheta = 120 Multipole::nphi = 240 Multipole::variables = "WeylScal4::Psi4r{sw=-2 cmplx='WeylScal4::Psi4i' name='psi4'}" Multipole::out_every = 256 Multipole::l_max = 8 Multipole::output_hdf5 = yes # Disable ASCII output to avoid creating a large number of files Multipole::output_ascii = no ################################################################################ # Gauge invariant perturbations of Schwarzschild (ZM-CPM variables) ################################################################################ # WaveExtractCPM::out_every = 256 # WaveExtractCPM::use_carpetinterp2 = no # WaveExtractCPM::calc_when_necessary = no # WaveExtractCPM::verbose = 0 # WaveExtractCPM::maximum_detector_number = 7 # WaveExtractCPM::switch_output_format = 100 # WaveExtractCPM::rsch2_computation = "average Schwarzschild metric" # WaveExtractCPM::l_mode = 8 # WaveExtractCPM::m_mode = 8 # WaveExtractCPM::detector_radius [0] = 100 # WaveExtractCPM::detector_radius [1] = 115 # WaveExtractCPM::detector_radius [2] = 136 # WaveExtractCPM::detector_radius [3] = 167 # WaveExtractCPM::detector_radius [4] = 214 # WaveExtractCPM::detector_radius [5] = 300 # WaveExtractCPM::detector_radius [6] = 500 # WaveExtractCPM::maxntheta = 120 # WaveExtractCPM::maxnphi = 240 # WaveExtractCPM::output_hdf5 = yes # WaveExtractCPM::output_ascii = no # WaveExtractCPM::output_h = yes # WaveExtractCPM::output_Psi = yes ################################################################################ # Apparent Horizons ################################################################################ AHFinderDirect::N_horizons = 3 AHFinderDirect::find_every = 256 AHFinderDirect::output_h_every = 0 AHFinderDirect::max_Newton_iterations__initial = 50 AHFinderDirect::max_Newton_iterations__subsequent = 50 AHFinderDirect::max_allowable_Theta_growth_iterations = 10 AHFinderDirect::max_allowable_Theta_nonshrink_iterations = 10 AHFinderDirect::geometry_interpolator_name = "Lagrange polynomial interpolation" AHFinderDirect::geometry_interpolator_pars = "order=4" AHFinderDirect::surface_interpolator_name = "Lagrange polynomial interpolation" AHFinderDirect::surface_interpolator_pars = "order=4" AHFinderDirect::verbose_level = "physics details" AHFinderDirect::move_origins = yes AHFinderDirect::origin_x [1] = 4.461538461538462 AHFinderDirect::initial_guess__coord_sphere__x_center[1] = 4.461538461538462 AHFinderDirect::initial_guess__coord_sphere__radius [1] = 0.6646153846153846 AHFinderDirect::which_surface_to_store_info [1] = 2 AHFinderDirect::set_mask_for_individual_horizon [1] = no AHFinderDirect::reset_horizon_after_not_finding [1] = no AHFinderDirect::track_origin_from_grid_scalar [1] = yes AHFinderDirect::track_origin_source_x [1] = "PunctureTracker::pt_loc_x[0]" AHFinderDirect::track_origin_source_y [1] = "PunctureTracker::pt_loc_y[0]" AHFinderDirect::track_origin_source_z [1] = "PunctureTracker::pt_loc_z[0]" AHFinderDirect::max_allowable_horizon_radius [1] = 3 AHFinderDirect::origin_x [2] = -5.538461538461538 AHFinderDirect::initial_guess__coord_sphere__x_center[2] = -5.538461538461538 AHFinderDirect::initial_guess__coord_sphere__radius [2] = 0.5353846153846153 AHFinderDirect::which_surface_to_store_info [2] = 3 AHFinderDirect::set_mask_for_individual_horizon [2] = no AHFinderDirect::reset_horizon_after_not_finding [2] = no AHFinderDirect::track_origin_from_grid_scalar [2] = yes AHFinderDirect::track_origin_source_x [2] = "PunctureTracker::pt_loc_x[1]" AHFinderDirect::track_origin_source_y [2] = "PunctureTracker::pt_loc_y[1]" AHFinderDirect::track_origin_source_z [2] = "PunctureTracker::pt_loc_z[1]" AHFinderDirect::max_allowable_horizon_radius [2] = 3 AHFinderDirect::origin_x [3] = 0 AHFinderDirect::find_after_individual [3] = 371902 AHFinderDirect::initial_guess__coord_sphere__x_center[3] = 0 AHFinderDirect::initial_guess__coord_sphere__radius [3] = 1.0 AHFinderDirect::which_surface_to_store_info [3] = 4 AHFinderDirect::set_mask_for_individual_horizon [3] = no AHFinderDirect::max_allowable_horizon_radius [3] = 6 ################################################################################ # Spherical surfaces ################################################################################ SphericalSurface::nsurfaces = 5 SphericalSurface::maxntheta = 66 SphericalSurface::maxnphi = 124 SphericalSurface::verbose = no # Surfaces 0 and 1 are used by PunctureTracker # Horizon 1 SphericalSurface::ntheta [2] = 41 SphericalSurface::nphi [2] = 80 SphericalSurface::nghoststheta [2] = 2 SphericalSurface::nghostsphi [2] = 2 # Horizon 2 SphericalSurface::ntheta [3] = 41 SphericalSurface::nphi [3] = 80 SphericalSurface::nghoststheta [3] = 2 SphericalSurface::nghostsphi [3] = 2 # Common horizon SphericalSurface::ntheta [4] = 41 SphericalSurface::nphi [4] = 80 SphericalSurface::nghoststheta [4] = 2 SphericalSurface::nghostsphi [4] = 2 ################################################################################ # Isolated Horizons ################################################################################ QuasiLocalMeasures::verbose = no QuasiLocalMeasures::veryverbose = no QuasiLocalMeasures::interpolator = "Lagrange polynomial interpolation" QuasiLocalMeasures::interpolator_options = "order=4" QuasiLocalMeasures::spatial_order = 4 QuasiLocalMeasures::num_surfaces = 3 QuasiLocalMeasures::surface_index [0] = 2 QuasiLocalMeasures::surface_index [1] = 3 QuasiLocalMeasures::surface_index [2] = 4 QuasiLocalMeasures::output_vtk_every = 0 ################################################################################ # Correctness checking ################################################################################ Carpet::poison_new_timelevels = no Carpet::check_for_poison = no NaNChecker::check_every = 256 NanChecker::check_after = 0 NaNChecker::report_max = 10 NaNChecker::verbose = "all" NaNChecker::action_if_found = terminate NaNChecker::out_NaNmask = yes NaNChecker::check_vars = " ML_BSSN::ML_log_confac " ################################################################################ # Timers ################################################################################ Carpet::output_timer_tree_every = 1024 Carpet::output_initialise_timer_tree = yes ################################################################################ # Output ################################################################################ IO::out_dir = "@SIMULATION_NAME@" IOScalar::one_file_per_group = yes IOASCII::one_file_per_group = yes IOBasic::outInfo_every = 1 IOBasic::outInfo_reductions = "minimum maximum" IOBasic::outInfo_vars = " ML_BSSN::ML_log_confac Carpet::physical_time_per_hour SystemStatistics::maxrss_mb SystemStatistics::swap_used_mb " IOScalar::outScalar_every = 256 IOScalar::outScalar_reductions = "minimum maximum average" IOScalar::outScalar_vars = "SystemStatistics::process_memory_mb" IOASCII::out0D_every = 256 IOASCII::out0D_vars = " Carpet::timing PunctureTracker::pt_loc QuasiLocalMeasures::qlm_scalars{out_every = 256} " IOASCII::out1D_every = 0 IOASCII::out1D_d = no IOASCII::out1D_x = yes IOASCII::out1D_y = no IOASCII::out1D_z = yes IOASCII::out1D_vars = " ML_BSSN::ML_log_confac ML_BSSN::ML_trace_curv WeylScal4::Psi4r " IOASCII::out2D_every = 0 IOASCII::out2D_vars = " " IOHDF5::out_every = 0 IOHDF5::out_vars = " Grid::Coordinates{out_every=1000000000 refinement_levels={0}} ML_BSSN::ML_log_confac WeylScal4::Psi4r WeylScal4::Psi4i WeylScal4::curvIr{refinement_levels={3 5}} WeylScal4::curvIi{refinement_levels={3 5}} WeylScal4::curvJr{refinement_levels={3 5}} WeylScal4::curvJi{refinement_levels={3 5}} " IOHDF5::out2D_every = 1024 IOHDF5::out2D_vars = " Grid::Coordinates{out_every=1000000000 refinement_levels={0}} ML_BSSN::alpha ML_BSSN::phi ML_BSSN::trK WeylScal4::Psi4r WeylScal4::Psi4i " ################################################################################ # Checkpointing and recovery ################################################################################ CarpetIOHDF5::checkpoint = yes IO::checkpoint_ID = no IO::recover = "autoprobe" IO::out_proc_every = 2 IO::checkpoint_on_terminate = yes IO::checkpoint_dir = "../checkpoints" IO::recover_dir = "../checkpoints" IO::abort_on_io_errors = yes CarpetIOHDF5::open_one_input_file_at_a_time = yes CarpetIOHDF5::compression_level = 0 ################################################################################ # Run termination ################################################################################ TerminationTrigger::max_walltime = @WALLTIME_HOURS@ # Trigger termination 30 minutes before the walltime is reached TerminationTrigger::on_remaining_walltime = 30 TerminationTrigger::output_remtime_every_minutes = 30 TerminationTrigger::termination_from_file = yes TerminationTrigger::termination_file = "terminate.txt" TerminationTrigger::create_termination_file = yes Cactus::terminate = time Cactus::cctk_final_time = 1700.0 -------------- next part -------------- + set -e + cd /system/user/crangano/simulations/GW150914_28/output-0000-active + echo Checking: + pwd + hostname + date + echo Environment: + export CACTUS_NUM_PROCS=1 + CACTUS_NUM_PROCS=1 + export CACTUS_NUM_THREADS=32 + CACTUS_NUM_THREADS=32 + export GMON_OUT_PREFIX=gmon.out + GMON_OUT_PREFIX=gmon.out + export OMP_NUM_THREADS=32 + OMP_NUM_THREADS=32 + sort + env + echo Starting: ++ date +%s + export CACTUS_STARTTIME=1757517004 + CACTUS_STARTTIME=1757517004 + '[' 1 = 1 ']' + '[' 0 -eq 0 ']' + /system/user/crangano/simulations/GW150914_28/SIMFACTORY/exe/cactus_sim -L 3 /system/user/crangano/simulations/GW150914_28/output-0000/GW150914.par -------------------------------------------------------------------------- No OpenFabrics connection schemes reported that they were able to be used on a specific port. As such, the openib BTL (OpenFabrics support) will be disabled for this port. Local host: panther Local device: bnxt_re0 Local port: 1 CPCs attempted: rdmacm, udcm -------------------------------------------------------------------------- -------------------------------------------------------------------------- Open MPI failed an OFI Libfabric library call (fi_endpoint). This is highly unusual; your job may behave unpredictably (and/or abort) after this. Local host: panther Location: mtl_ofi_component.c:513 Error: Invalid argument (22) -------------------------------------------------------------------------- WARNING level 0 from host panther process 0 in thorn CoordinatesSymmetry, file /system/user/crangano/einstein_toolkit/Cactus/configs/sim/build/CoordinatesSymmetry/apply.c:665: -> Cannot apply symmetry boundary zones in the lower x direction, since there seem to be more symmetry zones than interior zones cactus_sim: /system/user/crangano/einstein_toolkit/Cactus/configs/sim/build/Carpet/helpers.cc:275: int Carpet::Abort(const cGH*, int): Assertion `0' failed. Rank 0 with PID 3271897 received signal 6 Writing backtrace to GW150914_28/backtrace.0.txt [panther:3271901] 1 more process has sent help message help-mpi-btl-openib-cpc-base.txt / no cpcs for port [panther:3271901] Set MCA parameter "orte_base_help_aggregate" to 0 to see all help / error messages /system/user/crangano/simulations/GW150914_28/output-0000/SIMFACTORY/RunScript: line 36: 3271897 Aborted (core dumped) /system/user/crangano/simulations/GW150914_28/SIMFACTORY/exe/cactus_sim -L 3 /system/user/crangano/simulations/GW150914_28/output-0000/GW150914.par -------------- next part -------------- Simulation name: GW150914_28 Running simulation GW150914_28 Preparing: Checking: /system/user/crangano/simulations/GW150914_28/output-0000-active panther Wed Sep 10 05:10:04 PM CEST 2025 Environment: Starting: [1757517005.372585993] panther:pid3271897.cactus_sim: Unable to alloc recv buffer MR on bnxt_re0: Cannot allocate memory [1757517005.372630518] panther:pid3271897.cactus_sim: Unable to allocate UD recv buffer pool [1757517005.376325443] panther:pid3271897.cactus_sim: Unable to alloc recv buffer MR on bnxt_re0: Cannot allocate memory [1757517005.376328448] panther:pid3271897.cactus_sim: Unable to allocate UD recv buffer pool [1757517005.378921438] panther:pid3271897.cactus_sim: Unable to alloc recv buffer MR on bnxt_re0: Cannot allocate memory [1757517005.378924854] panther:pid3271897.cactus_sim: Unable to allocate UD recv buffer pool [1757517005.439461572] panther:pid3271897.cactus_sim: Unable to alloc recv buffer MR on bnxt_re0: Cannot allocate memory [1757517005.439464628] panther:pid3271897.cactus_sim: Unable to allocate UD recv buffer pool INFO (Cactus): Increased logging level from 0 to 3 -------------------------------------------------------------------------------- 10 1 0101 ************************ 01 1010 10 The Cactus Code V4.18.0 1010 1101 011 www.cactuscode.org 1001 100101 ************************ 00010101 100011 (c) Copyright The Authors 0100 GNU Licensed. No Warranty 0101 -------------------------------------------------------------------------------- Cactus version: 4.18.0 Compile date: Sep 10 2025 (16:44:10) Run date: Sep 10 2025 (17:10:05+0200) Run host: panther (pid=3271897) Working directory: /system/user/crangano/simulations/GW150914_28/output-0000 Executable: /system/user/crangano/simulations/GW150914_28/SIMFACTORY/exe/cactus_sim Parameter file: /system/user/crangano/simulations/GW150914_28/output-0000/GW150914.par -------------------------------------------------------------------------------- Activating thorn Cactus...Success -> active implementation Cactus Activation requested for --->ADMBase ML_ADMConstraints ADMCoupling ADMMacros AEILocalInterp AHFinderDirect Boundary Carpet CarpetIOASCII CarpetIOBasic CarpetIOHDF5 CarpetIOScalar CarpetInterp CarpetInterp2 CarpetLib CarpetReduce CarpetRegrid2 CarpetTracker CartGrid3D CoordBase CoordGauge Coordinates CoordinatesSymmetry Formaline GlobalDerivative hwloc IOUtil InitBase Interpolate2 QuasiLocalMeasures LocalInterp LoopControl MoL NaNChecker PunctureTracker Slab SpaceMask SphericalSurface StaticConformal SummationByParts SymBase SystemStatistics SystemTopology TerminationTrigger TensorTypes Time TmunuBase TwoPunctures Vectors ML_BSSN ML_BSSN_Helper NewRad GenericFD WeylScal4 Multipole ADMDerivatives<--- Thorn AHFinderDirect requests automatic activation of HDF5 Thorn Carpet requests automatic activation of MPI Thorn Carpet requests automatic activation of Timers Thorn CarpetLib requests automatic activation of CycleClock Thorn Coordinates requests automatic activation of TGRtensor Thorn Formaline requests automatic activation of pthreads Thorn hwloc requests automatic activation of zlib Thorn Interpolate2 requests automatic activation of LAPACK Thorn QuasiLocalMeasures requests automatic activation of Fortran Thorn TwoPunctures requests automatic activation of GSL Thorn LAPACK requests automatic activation of BLAS Activating thorn ADMBase...Success -> active implementation ADMBase Activating thorn ADMCoupling...Success -> active implementation ADMCoupling Activating thorn ADMDerivatives...Success -> active implementation ADMDerivatives Activating thorn ADMMacros...Success -> active implementation ADMMacros Activating thorn AEILocalInterp...Success -> active implementation AEILocalInterp Activating thorn AHFinderDirect...Success -> active implementation AHFinderDirect Activating thorn BLAS...Success -> active implementation BLAS Activating thorn Boundary...Success -> active implementation boundary Activating thorn Carpet...Success -> active implementation Driver Activating thorn CarpetInterp...Success -> active implementation interp Activating thorn CarpetInterp2...Success -> active implementation interp2 Activating thorn CarpetIOASCII...Success -> active implementation IOASCII Activating thorn CarpetIOBasic...Success -> active implementation IOBasic Activating thorn CarpetIOHDF5...Success -> active implementation IOHDF5 Activating thorn CarpetIOScalar...Success -> active implementation IOScalar Activating thorn CarpetLib...Success -> active implementation CarpetLib Activating thorn CarpetReduce...Success -> active implementation reduce Activating thorn CarpetRegrid2...Success -> active implementation CarpetRegrid2 Activating thorn CarpetTracker...Success -> active implementation CarpetTracker Activating thorn CartGrid3D...Success -> active implementation grid Activating thorn CoordBase...Success -> active implementation CoordBase Activating thorn CoordGauge...Success -> active implementation CoordGauge Activating thorn Coordinates...Success -> active implementation Coordinates Activating thorn CoordinatesSymmetry...Success -> active implementation CoordinatesSymmetry Activating thorn CycleClock...Success -> active implementation CycleClock Activating thorn Formaline...Success -> active implementation Formaline Activating thorn Fortran...Success -> active implementation Fortran Activating thorn GenericFD...Success -> active implementation GenericFD Activating thorn GlobalDerivative...Success -> active implementation GlobalDerivative Activating thorn GSL...Success -> active implementation GSL Activating thorn HDF5...Success -> active implementation HDF5 Activating thorn hwloc...Success -> active implementation hwloc Activating thorn InitBase...Success -> active implementation InitBase Activating thorn Interpolate2...Success -> active implementation Interpolate Activating thorn IOUtil...Success -> active implementation IO Activating thorn LAPACK...Success -> active implementation LAPACK Activating thorn LocalInterp...Success -> active implementation LocalInterp Activating thorn LoopControl...Success -> active implementation LoopControl Activating thorn ML_ADMConstraints...Success -> active implementation ML_ADMConstraints Activating thorn ML_BSSN...Success -> active implementation ML_BSSN Activating thorn ML_BSSN_Helper...Success -> active implementation ML_BSSN_Helper Activating thorn MoL...Success -> active implementation MethodOfLines Activating thorn MPI...Success -> active implementation MPI Activating thorn Multipole...Success -> active implementation multipole Activating thorn NaNChecker...Success -> active implementation NaNChecker Activating thorn NewRad...Success -> active implementation NewRad Activating thorn pthreads...Success -> active implementation PTHREADS Activating thorn PunctureTracker...Success -> active implementation PunctureTracker Activating thorn QuasiLocalMeasures...Success -> active implementation QuasiLocalMeasures Activating thorn Slab...Success -> active implementation Slab Activating thorn SpaceMask...Success -> active implementation SpaceMask Activating thorn SphericalSurface...Success -> active implementation SphericalSurface Activating thorn StaticConformal...Success -> active implementation StaticConformal Activating thorn SummationByParts...Success -> active implementation SummationByParts Activating thorn SymBase...Success -> active implementation SymBase Activating thorn SystemStatistics...Success -> active implementation SystemStatistics Activating thorn SystemTopology...Success -> active implementation SystemTopology Activating thorn TensorTypes...Success -> active implementation TensorTypes Activating thorn TerminationTrigger...Success -> active implementation TerminationTrigger Activating thorn TGRtensor...Success -> active implementation TGRtensor Activating thorn Time...Success -> active implementation time Activating thorn Timers...Success -> active implementation Timers Activating thorn TmunuBase...Success -> active implementation TmunuBase Activating thorn TwoPunctures...Success -> active implementation TwoPunctures Activating thorn Vectors...Success -> active implementation Vectors Activating thorn WeylScal4...Success -> active implementation WeylScal4 Activating thorn zlib...Success -> active implementation zlib INFO (IOUtil): Recovery directory '../checkpoints' doesn't exist -------------------------------------------------------------------------------- if (recover initial data) Recover parameters endif Startup routines [CCTK_STARTUP] Carpet::MultiModel_Startup: Multi-model Startup routine Coordinates::Coordinates_ChoosePatchSystem: [meta] Choose patch system CycleClock::CycleClock_Setup: Set up CycleClock LoopControl::LC_setup: Set up LoopControl ML_BSSN_Helper::ML_BSSN_SetGroupTags: [meta] Set checkpointing and prolongation group tags GROUP hwloc_startup: hwloc startup group hwloc::hwloc_version: Output hwloc version Timers::Timer_Startup: Prepare hierarchical timers CarpetInterp::CarpetInterpStartup: Startup routine CarpetReduce::CarpetReduceStartup: Startup routine CartGrid3D::SymmetryStartup: Register GH Extension for GridSymmetry CoordBase::CoordBase_Startup: Register a GH extension to store the coordinate system handles SystemTopology::ST_system_topology: Output and/or modify system topology and hardware locality Carpet::Driver_Startup: Startup routine Formaline::Formaline_PrintIDs: [meta] Print the build and simulation ids IOUtil::IOUtil_Startup: Startup routine LocalInterp::LocalInterp_Startup: register LocalInterp's interpolation operators CarpetIOHDF5::CarpetIOHDF5_Startup: Startup routine ML_ADMConstraints::ML_ADMConstraints_Startup: [meta] create banner ML_BSSN::ML_BSSN_Startup: [meta] create banner ML_BSSN_Helper::ML_BSSN_RegisterSlicing: [meta] Register slicing CarpetIOScalar::CarpetIOScalarStartup: [global] Startup routine MoL::MoL_Startup: Startup banner Slab::Slab_InitMPIDatatypes: Create MPI datatypes for complex variables in C SymBase::SymBase_Startup: Register GH Extension for SymBase CarpetIOBasic::CarpetIOBasicStartup: [global] Startup routine TerminationTrigger::TerminationTrigger_StartSignalHandler: Start signal handler AEILocalInterp::AEILocalInterp_U_Startup: register CCTK_InterpLocalUniform() interpolation operators Vectors::Vectors_Startup: Print startup message WeylScal4::WeylScal4_Startup: [meta] create banner CarpetIOASCII::CarpetIOASCIIStartup: [global] Startup routine Startup routines which need an existing grid hierarchy [CCTK_WRAGH] ADMBase::Einstein_InitSymBound: [global] Set up GF symmetries Boundary::Boundary_RegisterBCs: [global] Register boundary conditions that this thorn provides CarpetRegrid2::CarpetRegrid2_Initialise: [global] Initialise locations of refined regions CartGrid3D::RegisterCartGrid3DCoords: [meta] Register coordinates for the Cartesian grid CoordGauge::Einstein_ActivateSlicing: Initialize slicing, setup priorities for mixed slicings CoordGauge::Einstein_SetNextSlicing: Identify the slicing for the next iteration Formaline::Formaline_OutputSource: [meta] Output Cactus source tree Formaline::Formaline_RegisterWarnings: [meta] Register to receive warnings and info messages from the flesh Formaline::Formaline_AnnounceInitial: [global] Put some meta information about the current run into permanent storage ML_BSSN_Helper::ML_BSSN_ParamCompat: [meta] Handle parameter backward compatibility MoL::MoL_SetupIndexArrays: Set up the MoL bookkeeping index arrays MoL::MoL_SetScheduleStatus: [global] Set the flag so it is ok to register with MoL TmunuBase::TmunuBase_SetStressEnergyState: [global] Set the stress_energy_state variable GROUP MoL_Register: The group where physics thorns register variables with MoL ML_ADMConstraints::ML_ADMConstraints_RegisterVars: [meta] Register Variables for MoL ML_BSSN::ML_BSSN_RegisterVars: [meta] Register Variables for MoL ML_BSSN_Helper::ML_BSSN_RegisterConstrained: [meta] Register ADMBase variables as constrained WeylScal4::WeylScal4_RegisterVars: [meta] Register Variables for MoL Slab::Slab_InitTimers: Initialise timers GROUP SymBase_Wrapper: Wrapper group for SymBase GROUP SymmetryRegister: Register your symmetries here CartGrid3D::CartGrid3D_RegisterSymmetryBoundaries: [meta] Register symmetry boundaries Coordinates::Coordinates_RegisterSymmetry: [meta] Register inter-patch boundaries as symmetries ML_ADMConstraints::ML_ADMConstraints_RegisterSymmetries: [meta] register symmetries ML_BSSN::ML_BSSN_RegisterSymmetries: [meta] register symmetries WeylScal4::WeylScal4_RegisterSymmetries: [meta] register symmetries SymBase::SymBase_Statistics: Print symmetry boundary face descriptions MoL::MoL_ReportNumberVariables: [meta] Report how many of each type of variable there are Parameter checking routines [CCTK_PARAMCHECK] ADMBase::ADMBase_ParamCheck: [global] Check consistency of parameters Boundary::Boundary_Check: Check dimension of grid variables Carpet::CarpetParamCheck: Parameter checking routine CarpetLib::CarpetLib_test_prolongate_3d_rf2: [global] Test prolongation operators CarpetRegrid2::CarpetRegrid2_ParamCheck: Check parameters CartGrid3D::ParamCheck_CartGrid3D: Check coordinates for CartGrid3D Coordinates::Coordinates_ParamCheck: Check thorn parameters for consistency. Fortran::CheckFortranParameters: Test whether Fortran parameters work correctly GlobalDerivative::GlobalDeriv_ParamCheck: [global] Check parameters ML_BSSN_Helper::ML_BSSN_ParamCheck: [meta] Check parameters MoL::MoL_ParamCheck: Basic parameter checking Multipole::Multipole_ParamCheck: [global] Check Multipole parameters QuasiLocalMeasures::qlm_paramcheck: [global] Check quasi-local parameter settings SphericalSurface::SphericalSurface_ParamCheck: [global] Check that all surface names are unique TerminationTrigger::TerminationTrigger_ParamCheck: Check consitency of parameters TmunuBase::TmunuBase_ParamCheck: [global] Check that no deprecated parameters are used. TwoPunctures::TwoPunctures_ParamCheck: Check parameters and thorn needs Vectors::Vectors_Test: Run correctness tests. Initialisation if (NOT (recover initial data AND recovery_mode is 'strict')) [CCTK_PREREGRIDINITIAL] Set up grid hierarchy [CCTK_POSTREGRIDINITIAL] AHFinderDirect::AHFinderDirect_maybe_do_masks: set mask(s) based on apparent horizon position(s) CartGrid3D::SpatialCoordinates: Set Coordinates after regridding GROUP MaskBase_SetupMask: Set up the weight function GROUP MaskBase_SetupMaskAll: Set up the weight function CarpetReduce::MaskBase_AllocateMask: [global] Allocate the weight function CarpetReduce::MaskBase_InitMask: [global] [loop-local] Initialise the weight function GROUP SetupIMaskInternal: Set up the integer weight function (schedule other routines in here) CarpetReduce::CoordBase_SetupMask: [global] [loop-local] Set up the outer boundaries of the weight function CarpetReduce::CarpetMaskSetup: [global] [loop-singlemap] Set up the weight function for the restriction regions GROUP SetupIMask: Set up the integer weight function (schedule other routines in here) CarpetReduce::MaskBase_SetMask: [global] [loop-local] Set the weight function GROUP SetupMask: Set up the real weight function (schedule other routines in here) CarpetReduce::MaskBase_TestMask: [global] Test the weight function GROUP MultiPatch_SpatialCoordinates: Determine the global coordinates of each gridpoint. Coordinates::Coordinates_SetGlobalCoords: [local] Determine the global coordinates of each gridpoint. Coordinates::Coordinates_SetJacobian: [local] Fill in the Jacobian matrices for Thornburg04 coordinates. Coordinates::Coordinates_SetVolumeForm: [local] Set weight mask (for Carpet reduction operators) GROUP MoL_PseudoEvolutionBoundaries: Apply boundary conditions to pseudo-evolved quantities GROUP ML_ADMConstraints_evaluate_bc_group: ML_ADMConstraints_evaluate ML_ADMConstraints::ML_ADMConstraints_evaluate_SelectBCs: [level] ML_ADMConstraints_evaluate_SelectBCs GROUP ML_ADMConstraints_evaluate_ApplyBCs: Apply BCs for groups set in ML_ADMConstraints_evaluate GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions CoordinatesSymmetry::CoordinatesSymmetry_Apply_FirstPass: Apply multipatch symmetries Interpolate2::Interpolate2ApplyBC: [level] Apply interpolating inter-patch boundaries CoordinatesSymmetry::CoordinatesSymmetry_Apply_SecondPass: Apply multipatch symmetries for remaining points not filled by interpatch interpolation Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP ML_BSSN_ConstraintsEverywhere_bc_group: ML_BSSN_ConstraintsEverywhere ML_BSSN::ML_BSSN_ConstraintsEverywhere_SelectBCs: [level] ML_BSSN_ConstraintsEverywhere_SelectBCs GROUP ML_BSSN_ConstraintsEverywhere_ApplyBCs: Apply BCs for groups set in ML_BSSN_ConstraintsEverywhere GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions CoordinatesSymmetry::CoordinatesSymmetry_Apply_FirstPass: Apply multipatch symmetries Interpolate2::Interpolate2ApplyBC: [level] Apply interpolating inter-patch boundaries CoordinatesSymmetry::CoordinatesSymmetry_Apply_SecondPass: Apply multipatch symmetries for remaining points not filled by interpatch interpolation Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP ML_BSSN_ConstraintsInterior_bc_group: ML_BSSN_ConstraintsInterior ML_BSSN::ML_BSSN_ConstraintsInterior_SelectBCs: [level] ML_BSSN_ConstraintsInterior_SelectBCs GROUP ML_BSSN_ConstraintsInterior_ApplyBCs: Apply BCs for groups set in ML_BSSN_ConstraintsInterior GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions CoordinatesSymmetry::CoordinatesSymmetry_Apply_FirstPass: Apply multipatch symmetries Interpolate2::Interpolate2ApplyBC: [level] Apply interpolating inter-patch boundaries CoordinatesSymmetry::CoordinatesSymmetry_Apply_SecondPass: Apply multipatch symmetries for remaining points not filled by interpatch interpolation Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP WeylScal4_psis_calc_Nth_bc_group: WeylScal4_psis_calc_Nth WeylScal4::WeylScal4_psis_calc_Nth_SelectBCs: [level] WeylScal4_psis_calc_Nth_SelectBCs GROUP WeylScal4_psis_calc_Nth_ApplyBCs: Apply BCs for groups set in WeylScal4_psis_calc_Nth GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions CoordinatesSymmetry::CoordinatesSymmetry_Apply_FirstPass: Apply multipatch symmetries Interpolate2::Interpolate2ApplyBC: [level] Apply interpolating inter-patch boundaries CoordinatesSymmetry::CoordinatesSymmetry_Apply_SecondPass: Apply multipatch symmetries for remaining points not filled by interpatch interpolation Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions [CCTK_BASEGRID] ADMBase::ADMBase_SetShiftStateOn: Set the shift_state variable to 1 ADMBase::ADMBase_SetDtLapseStateOn: Set the dtlapse_state variable to 1 ADMBase::ADMBase_SetDtShiftStateOn: Set the dtshift_state variable to 1 ADMMacros::ADMMacros_SetLocalSpatialOrder: Initialize the local_spatial_order CartGrid3D::SpatialSpacings: [singlemap] Set up ranges for spatial 3D Cartesian coordinates (on all maps) CartGrid3D::SpatialCoordinates: Set up spatial 3D Cartesian coordinates on the GH SphericalSurface::SphericalSurface_SetupRes: [global] [loop-local] Set surface resolution automatically AHFinderDirect::AHFinderDirect_setup: [global] setup data structures GROUP MultiPatch_SpatialCoordinates: Determine the global coordinates of each gridpoint. Coordinates::Coordinates_SetGlobalCoords: [local] Determine the global coordinates of each gridpoint. Coordinates::Coordinates_SetJacobian: [local] Fill in the Jacobian matrices for Thornburg04 coordinates. Coordinates::Coordinates_SetVolumeForm: [local] Set weight mask (for Carpet reduction operators) Interpolate2::Interpolate2Init: Initialise interpolating inter-patch boundaries GROUP Interpolate2Test: Test interpolating inter-patch boundaries Interpolate2::Interpolate2TestInit: Test: Initialise test grid function Interpolate2::Interpolate2TestSelectBCs: [level] Test: Interpolate test grid function GROUP Interpolate2TestApplyBCs: Test: Interpolate test grid function GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions CoordinatesSymmetry::CoordinatesSymmetry_Apply_FirstPass: Apply multipatch symmetries Interpolate2::Interpolate2ApplyBC: [level] Apply interpolating inter-patch boundaries CoordinatesSymmetry::CoordinatesSymmetry_Apply_SecondPass: Apply multipatch symmetries for remaining points not filled by interpatch interpolation Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions Interpolate2::Interpolate2TestCheck: Test: Check test grid function ML_ADMConstraints::ML_ADMConstraints_CheckBoundaries: [meta] check boundaries treatment ML_BSSN::ML_BSSN_CheckBoundaries: [meta] check boundaries treatment NaNChecker::NaNChecker_ResetCounter: [global] Reset the NaNChecker::NaNsFound counter SphericalSurface::SphericalSurface_Setup: [global] Calculate surface coordinate descriptors GROUP MaskBase_SetupMask: Set up the weight function GROUP MaskBase_SetupMaskAll: Set up the weight function CarpetReduce::MaskBase_AllocateMask: [global] Allocate the weight function CarpetReduce::MaskBase_InitMask: [global] [loop-local] Initialise the weight function GROUP SetupIMaskInternal: Set up the integer weight function (schedule other routines in here) CarpetReduce::CoordBase_SetupMask: [global] [loop-local] Set up the outer boundaries of the weight function CarpetReduce::CarpetMaskSetup: [global] [loop-singlemap] Set up the weight function for the restriction regions GROUP SetupIMask: Set up the integer weight function (schedule other routines in here) CarpetReduce::MaskBase_SetMask: [global] [loop-local] Set the weight function GROUP SetupMask: Set up the real weight function (schedule other routines in here) CarpetReduce::MaskBase_TestMask: [global] Test the weight function SphericalSurface::SphericalSurface_Set: [global] Set surface radii to be used for initial setup in other thorns GROUP SphericalSurface_HasBeenSet: Set the spherical surfaces before this group, and use it afterwards SphericalSurface::SphericalSurface_CheckState: [global] Test the state of the spherical surfaces SummationByParts::SBP_SetNormMask: Setup the mask for the calculation of the norm SymBase::SymBase_Check: Check whether the driver set up the grid consistently TerminationTrigger::TerminationTrigger_ResetTrigger: Clear trigger state TerminationTrigger::TerminationTrigger_StartTimer: Start timer TerminationTrigger::TerminationTrigger_CreateFile: Create termination file Time::Time_Initialise: [global] Initialise Time variables Time::TemporalSpacings: [singlemap] Set fixed timestep WeylScal4::WeylScal4_CheckBoundaries: [meta] check boundaries treatment [CCTK_INITIAL] StaticConformal::StaticConformal_InitialiseState: Set the conformal_state variable to 0 GROUP ADMBase_InitialData: Schedule group for calculating ADM initial data GROUP TwoPunctures_Group: TwoPunctures initial data group TwoPunctures::TwoPunctures: Create puncture black hole initial data TwoPunctures::TwoPunctures_Metadata: [global] Output TwoPunctures metadata GROUP ADMBase_InitialGauge: Schedule group for the ADM initial gauge condition ADMBase::ADMBase_ShiftZero: Set the shift to 0 at all points ADMBase::ADMBase_DtLapseZero: Set the dtlapse to 0 at all points ADMBase::ADMBase_DtShiftZero: Set the dtshift to 0 at all points CarpetIOASCII::CarpetIOASCIIInit: [global] Initialisation routine CarpetIOBasic::CarpetIOBasicInit: [global] Initialisation routine CarpetIOHDF5::CarpetIOHDF5_Init: [global] Initialisation routine CarpetIOScalar::CarpetIOScalarInit: [global] Initialisation routine GROUP ADMBase_PostInitial: Schedule group for modifying the ADM initial data, such as e.g. adding noise ML_BSSN::ML_BSSN_InitialADMBase1Everywhere: ML_BSSN_InitialADMBase1Everywhere ML_BSSN::ML_BSSN_InitialADMBase2Interior: ML_BSSN_InitialADMBase2Interior MoL::MoL_StartLoop: [level] Initialise the step size control PunctureTracker::PunctureTracker_Init: [global] Calculate initial location of punctures QuasiLocalMeasures::qlm_init: [global] Initialise quasi-local calculations ML_BSSN_Helper::ML_BSSN_ExtrapolateGammas: Extrapolate Gammas and time derivatives of lapse and shift [CCTK_POSTINITIAL] CarpetIOHDF5::CarpetIOHDF5_CloseFiles: [global] Close all filereader input files GROUP MoL_PostStepModify: The group for physics thorns to schedule enforcing constraints ML_BSSN::ML_BSSN_EnforceEverywhere: ML_BSSN_EnforceEverywhere GROUP MoL_PostStep: Ensure that everything is correct after the initial data have been set up ML_BSSN::ML_BSSN_SelectBoundConds: [level] select boundary conditions ML_ADMConstraints::ML_ADMConstraints_SelectBoundConds: [level] select boundary conditions GROUP ML_ADMConstraints_ApplyBCs: Apply boundary conditions controlled by thorn Boundary GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions CoordinatesSymmetry::CoordinatesSymmetry_Apply_FirstPass: Apply multipatch symmetries Interpolate2::Interpolate2ApplyBC: [level] Apply interpolating inter-patch boundaries CoordinatesSymmetry::CoordinatesSymmetry_Apply_SecondPass: Apply multipatch symmetries for remaining points not filled by interpatch interpolation Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP ML_BSSN_ApplyBCs: Apply boundary conditions controlled by thorn Boundary GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions CoordinatesSymmetry::CoordinatesSymmetry_Apply_FirstPass: Apply multipatch symmetries Interpolate2::Interpolate2ApplyBC: [level] Apply interpolating inter-patch boundaries CoordinatesSymmetry::CoordinatesSymmetry_Apply_SecondPass: Apply multipatch symmetries for remaining points not filled by interpatch interpolation Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions ML_BSSN::ML_BSSN_ADMBaseInterior: ML_BSSN_ADMBaseInterior ML_BSSN::ML_BSSN_ADMBaseBoundaryScalar: ML_BSSN_ADMBaseBoundaryScalar ML_BSSN::ML_BSSN_ADMBaseEverywhere: ML_BSSN_ADMBaseEverywhere ML_BSSN_Helper::ML_BSSN_ADMBase_SelectBCs: [level] Select boundary conditions for ADMBase variables GROUP ML_BSSN_ADMBase_ApplyBCs: Apply boundary conditions to ADMBase variables GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions CoordinatesSymmetry::CoordinatesSymmetry_Apply_FirstPass: Apply multipatch symmetries Interpolate2::Interpolate2ApplyBC: [level] Apply interpolating inter-patch boundaries CoordinatesSymmetry::CoordinatesSymmetry_Apply_SecondPass: Apply multipatch symmetries for remaining points not filled by interpatch interpolation Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP ADMBase_SetADMVars: Set the ADM variables before this group, and use them afterwards WeylScal4::WeylScal4_SelectBoundConds: [level] select boundary conditions GROUP WeylScal4_ApplyBCs: Apply boundary conditions controlled by thorn Boundary GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions CoordinatesSymmetry::CoordinatesSymmetry_Apply_FirstPass: Apply multipatch symmetries Interpolate2::Interpolate2ApplyBC: [level] Apply interpolating inter-patch boundaries CoordinatesSymmetry::CoordinatesSymmetry_Apply_SecondPass: Apply multipatch symmetries for remaining points not filled by interpatch interpolation Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP MoL_PseudoEvolution: Calculate pseudo-evolved quantities GROUP ADMBase_SetADMVars: Set the ADM variables before this group, and use them afterwards GROUP ML_ADMConstraints_evaluate_group: ML_ADMConstraints_evaluate ML_ADMConstraints::ML_ADMConstraints_evaluate: ML_ADMConstraints_evaluate GROUP ML_ADMConstraints_evaluate_bc_group: ML_ADMConstraints_evaluate ML_ADMConstraints::ML_ADMConstraints_evaluate_SelectBCs: [level] ML_ADMConstraints_evaluate_SelectBCs GROUP ML_ADMConstraints_evaluate_ApplyBCs: Apply BCs for groups set in ML_ADMConstraints_evaluate GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions CoordinatesSymmetry::CoordinatesSymmetry_Apply_FirstPass: Apply multipatch symmetries Interpolate2::Interpolate2ApplyBC: [level] Apply interpolating inter-patch boundaries CoordinatesSymmetry::CoordinatesSymmetry_Apply_SecondPass: Apply multipatch symmetries for remaining points not filled by interpatch interpolation Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP ML_BSSN_ConstraintsEverywhere_group: ML_BSSN_ConstraintsEverywhere ML_BSSN::ML_BSSN_ConstraintsEverywhere: ML_BSSN_ConstraintsEverywhere GROUP ML_BSSN_ConstraintsEverywhere_bc_group: ML_BSSN_ConstraintsEverywhere ML_BSSN::ML_BSSN_ConstraintsEverywhere_SelectBCs: [level] ML_BSSN_ConstraintsEverywhere_SelectBCs GROUP ML_BSSN_ConstraintsEverywhere_ApplyBCs: Apply BCs for groups set in ML_BSSN_ConstraintsEverywhere GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions CoordinatesSymmetry::CoordinatesSymmetry_Apply_FirstPass: Apply multipatch symmetries Interpolate2::Interpolate2ApplyBC: [level] Apply interpolating inter-patch boundaries CoordinatesSymmetry::CoordinatesSymmetry_Apply_SecondPass: Apply multipatch symmetries for remaining points not filled by interpatch interpolation Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP ML_BSSN_ConstraintsInterior_group: ML_BSSN_ConstraintsInterior ML_BSSN::ML_BSSN_ConstraintsInterior: ML_BSSN_ConstraintsInterior GROUP ML_BSSN_ConstraintsInterior_bc_group: ML_BSSN_ConstraintsInterior ML_BSSN::ML_BSSN_ConstraintsInterior_SelectBCs: [level] ML_BSSN_ConstraintsInterior_SelectBCs GROUP ML_BSSN_ConstraintsInterior_ApplyBCs: Apply BCs for groups set in ML_BSSN_ConstraintsInterior GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions CoordinatesSymmetry::CoordinatesSymmetry_Apply_FirstPass: Apply multipatch symmetries Interpolate2::Interpolate2ApplyBC: [level] Apply interpolating inter-patch boundaries CoordinatesSymmetry::CoordinatesSymmetry_Apply_SecondPass: Apply multipatch symmetries for remaining points not filled by interpatch interpolation Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP WeylScal4_psis_calc_Nth_group: WeylScal4_psis_calc_Nth WeylScal4::WeylScal4_psis_calc_Nth: WeylScal4_psis_calc_Nth GROUP WeylScal4_psis_calc_Nth_bc_group: WeylScal4_psis_calc_Nth WeylScal4::WeylScal4_psis_calc_Nth_SelectBCs: [level] WeylScal4_psis_calc_Nth_SelectBCs GROUP WeylScal4_psis_calc_Nth_ApplyBCs: Apply BCs for groups set in WeylScal4_psis_calc_Nth GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions CoordinatesSymmetry::CoordinatesSymmetry_Apply_FirstPass: Apply multipatch symmetries Interpolate2::Interpolate2ApplyBC: [level] Apply interpolating inter-patch boundaries CoordinatesSymmetry::CoordinatesSymmetry_Apply_SecondPass: Apply multipatch symmetries for remaining points not filled by interpatch interpolation Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP WeylScal4_invars_calc_Nth_group: WeylScal4_invars_calc_Nth WeylScal4::WeylScal4_invars_calc_Nth: WeylScal4_invars_calc_Nth SummationByParts::SBP_CheckGridSizes: Check grid sizes and ghost zones Initialise finer grids recursively Restrict from finer grids [CCTK_POSTRESTRICTINITIAL] GROUP MoL_PostStep: Ensure that everything is correct after restriction ML_BSSN::ML_BSSN_SelectBoundConds: [level] select boundary conditions ML_ADMConstraints::ML_ADMConstraints_SelectBoundConds: [level] select boundary conditions GROUP ML_ADMConstraints_ApplyBCs: Apply boundary conditions controlled by thorn Boundary GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions CoordinatesSymmetry::CoordinatesSymmetry_Apply_FirstPass: Apply multipatch symmetries Interpolate2::Interpolate2ApplyBC: [level] Apply interpolating inter-patch boundaries CoordinatesSymmetry::CoordinatesSymmetry_Apply_SecondPass: Apply multipatch symmetries for remaining points not filled by interpatch interpolation Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP ML_BSSN_ApplyBCs: Apply boundary conditions controlled by thorn Boundary GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions CoordinatesSymmetry::CoordinatesSymmetry_Apply_FirstPass: Apply multipatch symmetries Interpolate2::Interpolate2ApplyBC: [level] Apply interpolating inter-patch boundaries CoordinatesSymmetry::CoordinatesSymmetry_Apply_SecondPass: Apply multipatch symmetries for remaining points not filled by interpatch interpolation Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions ML_BSSN::ML_BSSN_ADMBaseInterior: ML_BSSN_ADMBaseInterior ML_BSSN::ML_BSSN_ADMBaseBoundaryScalar: ML_BSSN_ADMBaseBoundaryScalar ML_BSSN::ML_BSSN_ADMBaseEverywhere: ML_BSSN_ADMBaseEverywhere ML_BSSN_Helper::ML_BSSN_ADMBase_SelectBCs: [level] Select boundary conditions for ADMBase variables GROUP ML_BSSN_ADMBase_ApplyBCs: Apply boundary conditions to ADMBase variables GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions CoordinatesSymmetry::CoordinatesSymmetry_Apply_FirstPass: Apply multipatch symmetries Interpolate2::Interpolate2ApplyBC: [level] Apply interpolating inter-patch boundaries CoordinatesSymmetry::CoordinatesSymmetry_Apply_SecondPass: Apply multipatch symmetries for remaining points not filled by interpatch interpolation Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP ADMBase_SetADMVars: Set the ADM variables before this group, and use them afterwards WeylScal4::WeylScal4_SelectBoundConds: [level] select boundary conditions GROUP WeylScal4_ApplyBCs: Apply boundary conditions controlled by thorn Boundary GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions CoordinatesSymmetry::CoordinatesSymmetry_Apply_FirstPass: Apply multipatch symmetries Interpolate2::Interpolate2ApplyBC: [level] Apply interpolating inter-patch boundaries CoordinatesSymmetry::CoordinatesSymmetry_Apply_SecondPass: Apply multipatch symmetries for remaining points not filled by interpatch interpolation Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP MoL_PseudoEvolutionBoundaries: Apply boundary conditions to pseudo-evolved quantities GROUP ML_ADMConstraints_evaluate_bc_group: ML_ADMConstraints_evaluate ML_ADMConstraints::ML_ADMConstraints_evaluate_SelectBCs: [level] ML_ADMConstraints_evaluate_SelectBCs GROUP ML_ADMConstraints_evaluate_ApplyBCs: Apply BCs for groups set in ML_ADMConstraints_evaluate GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions CoordinatesSymmetry::CoordinatesSymmetry_Apply_FirstPass: Apply multipatch symmetries Interpolate2::Interpolate2ApplyBC: [level] Apply interpolating inter-patch boundaries CoordinatesSymmetry::CoordinatesSymmetry_Apply_SecondPass: Apply multipatch symmetries for remaining points not filled by interpatch interpolation Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP ML_BSSN_ConstraintsEverywhere_bc_group: ML_BSSN_ConstraintsEverywhere ML_BSSN::ML_BSSN_ConstraintsEverywhere_SelectBCs: [level] ML_BSSN_ConstraintsEverywhere_SelectBCs GROUP ML_BSSN_ConstraintsEverywhere_ApplyBCs: Apply BCs for groups set in ML_BSSN_ConstraintsEverywhere GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions CoordinatesSymmetry::CoordinatesSymmetry_Apply_FirstPass: Apply multipatch symmetries Interpolate2::Interpolate2ApplyBC: [level] Apply interpolating inter-patch boundaries CoordinatesSymmetry::CoordinatesSymmetry_Apply_SecondPass: Apply multipatch symmetries for remaining points not filled by interpatch interpolation Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP ML_BSSN_ConstraintsInterior_bc_group: ML_BSSN_ConstraintsInterior ML_BSSN::ML_BSSN_ConstraintsInterior_SelectBCs: [level] ML_BSSN_ConstraintsInterior_SelectBCs GROUP ML_BSSN_ConstraintsInterior_ApplyBCs: Apply BCs for groups set in ML_BSSN_ConstraintsInterior GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions CoordinatesSymmetry::CoordinatesSymmetry_Apply_FirstPass: Apply multipatch symmetries Interpolate2::Interpolate2ApplyBC: [level] Apply interpolating inter-patch boundaries CoordinatesSymmetry::CoordinatesSymmetry_Apply_SecondPass: Apply multipatch symmetries for remaining points not filled by interpatch interpolation Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP WeylScal4_psis_calc_Nth_bc_group: WeylScal4_psis_calc_Nth WeylScal4::WeylScal4_psis_calc_Nth_SelectBCs: [level] WeylScal4_psis_calc_Nth_SelectBCs GROUP WeylScal4_psis_calc_Nth_ApplyBCs: Apply BCs for groups set in WeylScal4_psis_calc_Nth GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions CoordinatesSymmetry::CoordinatesSymmetry_Apply_FirstPass: Apply multipatch symmetries Interpolate2::Interpolate2ApplyBC: [level] Apply interpolating inter-patch boundaries CoordinatesSymmetry::CoordinatesSymmetry_Apply_SecondPass: Apply multipatch symmetries for remaining points not filled by interpatch interpolation Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions [CCTK_POSTPOSTINITIAL] [CCTK_POSTSTEP] GROUP ADMDerivatives_Group: Group for ADMDerivatives computations. ADMDerivatives::ADMDerivatives_CalcDerivatives: [local] Calculate requested derivatives of ADM lapse, shift, 3-metric ADMDerivatives::ADMDerivatives_radial_SelectBC: [level] Selecting BC for metric radial derivatives ADMDerivatives::ADMDerivatives_time_SelectBC: [level] Selecting BC for metric time derivatives GROUP ADMDerivatives_ApplyBCs: Apply boundary conditions controlled by thorn Boundary GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions CoordinatesSymmetry::CoordinatesSymmetry_Apply_FirstPass: Apply multipatch symmetries Interpolate2::Interpolate2ApplyBC: [level] Apply interpolating inter-patch boundaries CoordinatesSymmetry::CoordinatesSymmetry_Apply_SecondPass: Apply multipatch symmetries for remaining points not filled by interpatch interpolation Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions AHFinderDirect::AHFinderDirect_import_mask: [global] [loop-local] import the excision mask AHFinderDirect::AHFinderDirect_find_horizons: [global] find apparent horizon(s) after this time step AHFinderDirect::AHFinderDirect_store: [global] store apparent horizon(s) into spherical surface(s) AHFinderDirect::AHFinderDirect_save: [global] save apparent horizon(s) into Cactus variables AHFinderDirect::AHFinderDirect_maybe_do_masks: set mask(s) based on apparent horizon position(s) GROUP zzz_NaNChecker_NaNCheck: Check for NaNs and count them in NaNChecker::NaNsFound NaNChecker::NaNChecker_NaNCheck_Prepare: [level] Prepare data structures to check for NaNs NaNChecker::NaNChecker_NaNCheck_Check: [local] Check for NaNs NaNChecker::NaNChecker_NaNCheck_Finish: [level] Count NaNs in NaNChecker::NaNsFound NaNChecker::NaNChecker_TakeAction: [global] [loop-level] Output NaNChecker::NaNmask and take action according to NaNChecker::action_if_found PunctureTracker::PunctureTracker_SetPositions: [global] Copy puncture positions to spherical surfaces SphericalSurface::SphericalSurface_Set: [global] Set surface radii GROUP SphericalSurface_HasBeenSet: Set the spherical surfaces before this group, and use it afterwards SphericalSurface::SphericalSurface_CheckState: [global] Test the state of the spherical surfaces endif if (recover initial data) [CCTK_BASEGRID] ADMBase::ADMBase_SetShiftStateOn: Set the shift_state variable to 1 ADMBase::ADMBase_SetDtLapseStateOn: Set the dtlapse_state variable to 1 ADMBase::ADMBase_SetDtShiftStateOn: Set the dtshift_state variable to 1 ADMMacros::ADMMacros_SetLocalSpatialOrder: Initialize the local_spatial_order CartGrid3D::SpatialSpacings: [singlemap] Set up ranges for spatial 3D Cartesian coordinates (on all maps) CartGrid3D::SpatialCoordinates: Set up spatial 3D Cartesian coordinates on the GH SphericalSurface::SphericalSurface_SetupRes: [global] [loop-local] Set surface resolution automatically AHFinderDirect::AHFinderDirect_setup: [global] setup data structures GROUP MultiPatch_SpatialCoordinates: Determine the global coordinates of each gridpoint. Coordinates::Coordinates_SetGlobalCoords: [local] Determine the global coordinates of each gridpoint. Coordinates::Coordinates_SetJacobian: [local] Fill in the Jacobian matrices for Thornburg04 coordinates. Coordinates::Coordinates_SetVolumeForm: [local] Set weight mask (for Carpet reduction operators) Interpolate2::Interpolate2Init: Initialise interpolating inter-patch boundaries GROUP Interpolate2Test: Test interpolating inter-patch boundaries Interpolate2::Interpolate2TestInit: Test: Initialise test grid function Interpolate2::Interpolate2TestSelectBCs: [level] Test: Interpolate test grid function GROUP Interpolate2TestApplyBCs: Test: Interpolate test grid function GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions CoordinatesSymmetry::CoordinatesSymmetry_Apply_FirstPass: Apply multipatch symmetries Interpolate2::Interpolate2ApplyBC: [level] Apply interpolating inter-patch boundaries CoordinatesSymmetry::CoordinatesSymmetry_Apply_SecondPass: Apply multipatch symmetries for remaining points not filled by interpatch interpolation Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions Interpolate2::Interpolate2TestCheck: Test: Check test grid function ML_ADMConstraints::ML_ADMConstraints_CheckBoundaries: [meta] check boundaries treatment ML_BSSN::ML_BSSN_CheckBoundaries: [meta] check boundaries treatment NaNChecker::NaNChecker_ResetCounter: [global] Reset the NaNChecker::NaNsFound counter SphericalSurface::SphericalSurface_Setup: [global] Calculate surface coordinate descriptors GROUP MaskBase_SetupMask: Set up the weight function GROUP MaskBase_SetupMaskAll: Set up the weight function CarpetReduce::MaskBase_AllocateMask: [global] Allocate the weight function CarpetReduce::MaskBase_InitMask: [global] [loop-local] Initialise the weight function GROUP SetupIMaskInternal: Set up the integer weight function (schedule other routines in here) CarpetReduce::CoordBase_SetupMask: [global] [loop-local] Set up the outer boundaries of the weight function CarpetReduce::CarpetMaskSetup: [global] [loop-singlemap] Set up the weight function for the restriction regions GROUP SetupIMask: Set up the integer weight function (schedule other routines in here) CarpetReduce::MaskBase_SetMask: [global] [loop-local] Set the weight function GROUP SetupMask: Set up the real weight function (schedule other routines in here) CarpetReduce::MaskBase_TestMask: [global] Test the weight function SphericalSurface::SphericalSurface_Set: [global] Set surface radii to be used for initial setup in other thorns GROUP SphericalSurface_HasBeenSet: Set the spherical surfaces before this group, and use it afterwards SphericalSurface::SphericalSurface_CheckState: [global] Test the state of the spherical surfaces SummationByParts::SBP_SetNormMask: Setup the mask for the calculation of the norm SymBase::SymBase_Check: Check whether the driver set up the grid consistently TerminationTrigger::TerminationTrigger_ResetTrigger: Clear trigger state TerminationTrigger::TerminationTrigger_StartTimer: Start timer TerminationTrigger::TerminationTrigger_CreateFile: Create termination file Time::Time_Initialise: [global] Initialise Time variables Time::TemporalSpacings: [singlemap] Set fixed timestep WeylScal4::WeylScal4_CheckBoundaries: [meta] check boundaries treatment [CCTK_RECOVER_VARIABLES] IOUtil::IOUtil_RecoverGH: [level] Checkpoint recovery routine [CCTK_POST_RECOVER_VARIABLES] GROUP ADMDerivatives_Group: Group for ADMDerivatives computations. ADMDerivatives::ADMDerivatives_CalcDerivatives: [local] Calculate requested derivatives of ADM lapse, shift, 3-metric ADMDerivatives::ADMDerivatives_radial_SelectBC: [level] Selecting BC for metric radial derivatives ADMDerivatives::ADMDerivatives_time_SelectBC: [level] Selecting BC for metric time derivatives GROUP ADMDerivatives_ApplyBCs: Apply boundary conditions controlled by thorn Boundary GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions CoordinatesSymmetry::CoordinatesSymmetry_Apply_FirstPass: Apply multipatch symmetries Interpolate2::Interpolate2ApplyBC: [level] Apply interpolating inter-patch boundaries CoordinatesSymmetry::CoordinatesSymmetry_Apply_SecondPass: Apply multipatch symmetries for remaining points not filled by interpatch interpolation Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions AHFinderDirect::AHFinderDirect_recover: [global] import horizon data from Cactus variables CarpetIOHDF5::CarpetIOHDF5_InitCheckpointingIntervals: [global] Initialisation of checkpointing intervals after recovery CarpetIOHDF5::CarpetIOHDF5_CloseFiles: [meta] Close all initial data checkpoint files after recovery GROUP MaskBase_SetupMask: Set up the weight function GROUP MaskBase_SetupMaskAll: Set up the weight function CarpetReduce::MaskBase_AllocateMask: [global] Allocate the weight function CarpetReduce::MaskBase_InitMask: [global] [loop-local] Initialise the weight function GROUP SetupIMaskInternal: Set up the integer weight function (schedule other routines in here) CarpetReduce::CoordBase_SetupMask: [global] [loop-local] Set up the outer boundaries of the weight function CarpetReduce::CarpetMaskSetup: [global] [loop-singlemap] Set up the weight function for the restriction regions GROUP SetupIMask: Set up the integer weight function (schedule other routines in here) CarpetReduce::MaskBase_SetMask: [global] [loop-local] Set the weight function GROUP SetupMask: Set up the real weight function (schedule other routines in here) CarpetReduce::MaskBase_TestMask: [global] Test the weight function GROUP MoL_PostStep: Ensure that everything is correct after recovery ML_BSSN::ML_BSSN_SelectBoundConds: [level] select boundary conditions ML_ADMConstraints::ML_ADMConstraints_SelectBoundConds: [level] select boundary conditions GROUP ML_ADMConstraints_ApplyBCs: Apply boundary conditions controlled by thorn Boundary GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions CoordinatesSymmetry::CoordinatesSymmetry_Apply_FirstPass: Apply multipatch symmetries Interpolate2::Interpolate2ApplyBC: [level] Apply interpolating inter-patch boundaries CoordinatesSymmetry::CoordinatesSymmetry_Apply_SecondPass: Apply multipatch symmetries for remaining points not filled by interpatch interpolation Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP ML_BSSN_ApplyBCs: Apply boundary conditions controlled by thorn Boundary GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions CoordinatesSymmetry::CoordinatesSymmetry_Apply_FirstPass: Apply multipatch symmetries Interpolate2::Interpolate2ApplyBC: [level] Apply interpolating inter-patch boundaries CoordinatesSymmetry::CoordinatesSymmetry_Apply_SecondPass: Apply multipatch symmetries for remaining points not filled by interpatch interpolation Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions ML_BSSN::ML_BSSN_ADMBaseInterior: ML_BSSN_ADMBaseInterior ML_BSSN::ML_BSSN_ADMBaseBoundaryScalar: ML_BSSN_ADMBaseBoundaryScalar ML_BSSN::ML_BSSN_ADMBaseEverywhere: ML_BSSN_ADMBaseEverywhere ML_BSSN_Helper::ML_BSSN_ADMBase_SelectBCs: [level] Select boundary conditions for ADMBase variables GROUP ML_BSSN_ADMBase_ApplyBCs: Apply boundary conditions to ADMBase variables GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions CoordinatesSymmetry::CoordinatesSymmetry_Apply_FirstPass: Apply multipatch symmetries Interpolate2::Interpolate2ApplyBC: [level] Apply interpolating inter-patch boundaries CoordinatesSymmetry::CoordinatesSymmetry_Apply_SecondPass: Apply multipatch symmetries for remaining points not filled by interpatch interpolation Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP ADMBase_SetADMVars: Set the ADM variables before this group, and use them afterwards WeylScal4::WeylScal4_SelectBoundConds: [level] select boundary conditions GROUP WeylScal4_ApplyBCs: Apply boundary conditions controlled by thorn Boundary GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions CoordinatesSymmetry::CoordinatesSymmetry_Apply_FirstPass: Apply multipatch symmetries Interpolate2::Interpolate2ApplyBC: [level] Apply interpolating inter-patch boundaries CoordinatesSymmetry::CoordinatesSymmetry_Apply_SecondPass: Apply multipatch symmetries for remaining points not filled by interpatch interpolation Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP zzz_NaNChecker_NaNCheck: Check for NaNs and count them in NaNChecker::NaNsFound NaNChecker::NaNChecker_NaNCheck_Prepare: [level] Prepare data structures to check for NaNs NaNChecker::NaNChecker_NaNCheck_Check: [local] Check for NaNs NaNChecker::NaNChecker_NaNCheck_Finish: [level] Count NaNs in NaNChecker::NaNsFound NaNChecker::NaNChecker_TakeAction: [global] [loop-level] Output NaNChecker::NaNmask and take action according to NaNChecker::action_if_found TerminationTrigger::TerminationTrigger_ResetMinutes: [global] Reset Watchtime endif if (checkpoint initial data) [CCTK_CPINITIAL] CarpetIOHDF5::CarpetIOHDF5_InitialDataCheckpoint: [meta] Initial data checkpoint routine endif if (analysis) [CCTK_ANALYSIS] CarpetLib::CarpetLib_printtimestats: [global] Print timing statistics if desired CarpetLib::CarpetLib_printmemstats: [global] Print memory statistics if desired Formaline::Formaline_AnnounceUpdate: [global] Put some meta information about the current run into permanent storage LoopControl::LC_statistics_analysis: [meta] Output LoopControl statistics GROUP ML_BSSN_EvolutionAnalysis: Calculate RHS at analysis ML_BSSN::ML_BSSN_EvolutionAnalysisInit: ML_BSSN_EvolutionAnalysisInit ML_BSSN::ML_BSSN_EvolutionAnalysisInterior: ML_BSSN_EvolutionAnalysisInterior Multipole::Multipole_Calc: [global] Calculate Multipoles QuasiLocalMeasures::qlm_calculate: [global] Calculate quasi-local quantities SystemStatistics::SystemStatistics_Collect: [global] Collect system statistics TerminationTrigger::TerminationTrigger_CheckWalltime: Check elapsed job walltime TerminationTrigger::TerminationTrigger_CheckSignal: Check if we received a termination signal TerminationTrigger::TerminationTrigger_CheckFile: Check termination file endif Output grid variables do loop over timesteps [CCTK_PREREGRID] CarpetTracker::CarpetTracker_SetPositions: [global] Set positions of refined regions Change grid hierarchy [CCTK_POSTREGRID] AHFinderDirect::AHFinderDirect_maybe_do_masks: set mask(s) based on apparent horizon position(s) CartGrid3D::SpatialCoordinates: Set Coordinates after regridding GROUP MaskBase_SetupMask: Set up the weight function GROUP MaskBase_SetupMaskAll: Set up the weight function CarpetReduce::MaskBase_AllocateMask: [global] Allocate the weight function CarpetReduce::MaskBase_InitMask: [global] [loop-local] Initialise the weight function GROUP SetupIMaskInternal: Set up the integer weight function (schedule other routines in here) CarpetReduce::CoordBase_SetupMask: [global] [loop-local] Set up the outer boundaries of the weight function CarpetReduce::CarpetMaskSetup: [global] [loop-singlemap] Set up the weight function for the restriction regions GROUP SetupIMask: Set up the integer weight function (schedule other routines in here) CarpetReduce::MaskBase_SetMask: [global] [loop-local] Set the weight function GROUP SetupMask: Set up the real weight function (schedule other routines in here) CarpetReduce::MaskBase_TestMask: [global] Test the weight function GROUP MultiPatch_SpatialCoordinates: Determine the global coordinates of each gridpoint. Coordinates::Coordinates_SetGlobalCoords: [local] Determine the global coordinates of each gridpoint. Coordinates::Coordinates_SetJacobian: [local] Fill in the Jacobian matrices for Thornburg04 coordinates. Coordinates::Coordinates_SetVolumeForm: [local] Set weight mask (for Carpet reduction operators) GROUP MoL_PostStep: Ensure that everything is correct after regridding ML_BSSN::ML_BSSN_SelectBoundConds: [level] select boundary conditions ML_ADMConstraints::ML_ADMConstraints_SelectBoundConds: [level] select boundary conditions GROUP ML_ADMConstraints_ApplyBCs: Apply boundary conditions controlled by thorn Boundary GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions CoordinatesSymmetry::CoordinatesSymmetry_Apply_FirstPass: Apply multipatch symmetries Interpolate2::Interpolate2ApplyBC: [level] Apply interpolating inter-patch boundaries CoordinatesSymmetry::CoordinatesSymmetry_Apply_SecondPass: Apply multipatch symmetries for remaining points not filled by interpatch interpolation Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP ML_BSSN_ApplyBCs: Apply boundary conditions controlled by thorn Boundary GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions CoordinatesSymmetry::CoordinatesSymmetry_Apply_FirstPass: Apply multipatch symmetries Interpolate2::Interpolate2ApplyBC: [level] Apply interpolating inter-patch boundaries CoordinatesSymmetry::CoordinatesSymmetry_Apply_SecondPass: Apply multipatch symmetries for remaining points not filled by interpatch interpolation Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions ML_BSSN::ML_BSSN_ADMBaseInterior: ML_BSSN_ADMBaseInterior ML_BSSN::ML_BSSN_ADMBaseBoundaryScalar: ML_BSSN_ADMBaseBoundaryScalar ML_BSSN::ML_BSSN_ADMBaseEverywhere: ML_BSSN_ADMBaseEverywhere ML_BSSN_Helper::ML_BSSN_ADMBase_SelectBCs: [level] Select boundary conditions for ADMBase variables GROUP ML_BSSN_ADMBase_ApplyBCs: Apply boundary conditions to ADMBase variables GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions CoordinatesSymmetry::CoordinatesSymmetry_Apply_FirstPass: Apply multipatch symmetries Interpolate2::Interpolate2ApplyBC: [level] Apply interpolating inter-patch boundaries CoordinatesSymmetry::CoordinatesSymmetry_Apply_SecondPass: Apply multipatch symmetries for remaining points not filled by interpatch interpolation Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP ADMBase_SetADMVars: Set the ADM variables before this group, and use them afterwards WeylScal4::WeylScal4_SelectBoundConds: [level] select boundary conditions GROUP WeylScal4_ApplyBCs: Apply boundary conditions controlled by thorn Boundary GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions CoordinatesSymmetry::CoordinatesSymmetry_Apply_FirstPass: Apply multipatch symmetries Interpolate2::Interpolate2ApplyBC: [level] Apply interpolating inter-patch boundaries CoordinatesSymmetry::CoordinatesSymmetry_Apply_SecondPass: Apply multipatch symmetries for remaining points not filled by interpatch interpolation Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP MoL_PseudoEvolutionBoundaries: Apply boundary conditions to pseudo-evolved quantities GROUP ML_ADMConstraints_evaluate_bc_group: ML_ADMConstraints_evaluate ML_ADMConstraints::ML_ADMConstraints_evaluate_SelectBCs: [level] ML_ADMConstraints_evaluate_SelectBCs GROUP ML_ADMConstraints_evaluate_ApplyBCs: Apply BCs for groups set in ML_ADMConstraints_evaluate GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions CoordinatesSymmetry::CoordinatesSymmetry_Apply_FirstPass: Apply multipatch symmetries Interpolate2::Interpolate2ApplyBC: [level] Apply interpolating inter-patch boundaries CoordinatesSymmetry::CoordinatesSymmetry_Apply_SecondPass: Apply multipatch symmetries for remaining points not filled by interpatch interpolation Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP ML_BSSN_ConstraintsEverywhere_bc_group: ML_BSSN_ConstraintsEverywhere ML_BSSN::ML_BSSN_ConstraintsEverywhere_SelectBCs: [level] ML_BSSN_ConstraintsEverywhere_SelectBCs GROUP ML_BSSN_ConstraintsEverywhere_ApplyBCs: Apply BCs for groups set in ML_BSSN_ConstraintsEverywhere GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions CoordinatesSymmetry::CoordinatesSymmetry_Apply_FirstPass: Apply multipatch symmetries Interpolate2::Interpolate2ApplyBC: [level] Apply interpolating inter-patch boundaries CoordinatesSymmetry::CoordinatesSymmetry_Apply_SecondPass: Apply multipatch symmetries for remaining points not filled by interpatch interpolation Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP ML_BSSN_ConstraintsInterior_bc_group: ML_BSSN_ConstraintsInterior ML_BSSN::ML_BSSN_ConstraintsInterior_SelectBCs: [level] ML_BSSN_ConstraintsInterior_SelectBCs GROUP ML_BSSN_ConstraintsInterior_ApplyBCs: Apply BCs for groups set in ML_BSSN_ConstraintsInterior GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions CoordinatesSymmetry::CoordinatesSymmetry_Apply_FirstPass: Apply multipatch symmetries Interpolate2::Interpolate2ApplyBC: [level] Apply interpolating inter-patch boundaries CoordinatesSymmetry::CoordinatesSymmetry_Apply_SecondPass: Apply multipatch symmetries for remaining points not filled by interpatch interpolation Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP WeylScal4_psis_calc_Nth_bc_group: WeylScal4_psis_calc_Nth WeylScal4::WeylScal4_psis_calc_Nth_SelectBCs: [level] WeylScal4_psis_calc_Nth_SelectBCs GROUP WeylScal4_psis_calc_Nth_ApplyBCs: Apply BCs for groups set in WeylScal4_psis_calc_Nth GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions CoordinatesSymmetry::CoordinatesSymmetry_Apply_FirstPass: Apply multipatch symmetries Interpolate2::Interpolate2ApplyBC: [level] Apply interpolating inter-patch boundaries CoordinatesSymmetry::CoordinatesSymmetry_Apply_SecondPass: Apply multipatch symmetries for remaining points not filled by interpatch interpolation Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions Rotate timelevels iteration = iteration+1 t = t+dt [CCTK_PRESTEP] CoordGauge::Einstein_SetNextSlicing: Identify the slicing for the next iteration LoopControl::LC_steer: [meta] Update LoopControl algorithm preferences NaNChecker::NaNChecker_ResetCounter: [global] Reset the NaNChecker::NaNsFound counter [CCTK_EVOL] MoL::MoL_StartLoop: [level] Initialise the step size control while (MoL::MoL_Stepsize_Bad) GROUP MoL_Evolution: A single Cactus evolution step using MoL GROUP MoL_StartStep: MoL internal setup for the evolution step MoL::MoL_SetCounter: [level] Set the counter for the ODE method to loop over MoL::MoL_SetTime: [level] Ensure the correct time and timestep are used MoL::MoL_AllocateScratchSpace: [level] Allocate storage for scratch levels GROUP MoL_PreStep: Physics thorns can schedule preloop setup routines in here MoL::MoL_AllocateScratch: Allocate sufficient space for array scratch variables MoL::MoL_InitialCopy: Ensure the data is in the correct timelevel while (MoL::MoL_Intermediate_Step) GROUP MoL_Step: The loop over the intermediate steps for the ODE integrator MoL::MoL_InitRHS: Initialise the RHS functions GROUP MoL_CalcRHS: Physics thorns schedule the calculation of the discrete spatial operator in here ML_BSSN::ML_BSSN_EvolutionBoundaryScalar: ML_BSSN_EvolutionBoundaryScalar GROUP ML_BSSN_EvolutionInteriorSplitBy: ML_BSSN::ML_BSSN_EvolutionInteriorSplitBy1: ML_BSSN_EvolutionInteriorSplitBy1 ML_BSSN::ML_BSSN_EvolutionInteriorSplitBy2: ML_BSSN_EvolutionInteriorSplitBy2 ML_BSSN::ML_BSSN_EvolutionInteriorSplitBy3: ML_BSSN_EvolutionInteriorSplitBy3 GROUP MoL_PostRHS: Modify RHS functions GlobalDerivative::GlobalDeriv_Dissipation: [local] Apply global dissipation to registered variables GROUP MoL_RHSBoundaries: Any 'final' modifications to the RHS functions (boundaries etc.) MoL::MoL_Add: Updates calculated with the efficient Runge-Kutta 4 method MoL::MoL_DecrementCounter: [level] Alter the counter number MoL::MoL_ResetTime: [level] If necessary, change the time GROUP MoL_PostStepModify: The group for physics thorns to schedule enforcing constraints ML_BSSN::ML_BSSN_EnforceEverywhere: ML_BSSN_EnforceEverywhere GROUP MoL_PostStep: The group for physics thorns to schedule boundary calls etc. ML_BSSN::ML_BSSN_SelectBoundConds: [level] select boundary conditions ML_ADMConstraints::ML_ADMConstraints_SelectBoundConds: [level] select boundary conditions GROUP ML_ADMConstraints_ApplyBCs: Apply boundary conditions controlled by thorn Boundary GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions CoordinatesSymmetry::CoordinatesSymmetry_Apply_FirstPass: Apply multipatch symmetries Interpolate2::Interpolate2ApplyBC: [level] Apply interpolating inter-patch boundaries CoordinatesSymmetry::CoordinatesSymmetry_Apply_SecondPass: Apply multipatch symmetries for remaining points not filled by interpatch interpolation Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP ML_BSSN_ApplyBCs: Apply boundary conditions controlled by thorn Boundary GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions CoordinatesSymmetry::CoordinatesSymmetry_Apply_FirstPass: Apply multipatch symmetries Interpolate2::Interpolate2ApplyBC: [level] Apply interpolating inter-patch boundaries CoordinatesSymmetry::CoordinatesSymmetry_Apply_SecondPass: Apply multipatch symmetries for remaining points not filled by interpatch interpolation Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions ML_BSSN::ML_BSSN_ADMBaseInterior: ML_BSSN_ADMBaseInterior ML_BSSN::ML_BSSN_ADMBaseBoundaryScalar: ML_BSSN_ADMBaseBoundaryScalar ML_BSSN::ML_BSSN_ADMBaseEverywhere: ML_BSSN_ADMBaseEverywhere ML_BSSN_Helper::ML_BSSN_ADMBase_SelectBCs: [level] Select boundary conditions for ADMBase variables GROUP ML_BSSN_ADMBase_ApplyBCs: Apply boundary conditions to ADMBase variables GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions CoordinatesSymmetry::CoordinatesSymmetry_Apply_FirstPass: Apply multipatch symmetries Interpolate2::Interpolate2ApplyBC: [level] Apply interpolating inter-patch boundaries CoordinatesSymmetry::CoordinatesSymmetry_Apply_SecondPass: Apply multipatch symmetries for remaining points not filled by interpatch interpolation Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP ADMBase_SetADMVars: Set the ADM variables before this group, and use them afterwards WeylScal4::WeylScal4_SelectBoundConds: [level] select boundary conditions GROUP WeylScal4_ApplyBCs: Apply boundary conditions controlled by thorn Boundary GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions CoordinatesSymmetry::CoordinatesSymmetry_Apply_FirstPass: Apply multipatch symmetries Interpolate2::Interpolate2ApplyBC: [level] Apply interpolating inter-patch boundaries CoordinatesSymmetry::CoordinatesSymmetry_Apply_SecondPass: Apply multipatch symmetries for remaining points not filled by interpatch interpolation Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions MoL::MoL_ResetDeltaTime: [level] If necessary, change the timestep end while MoL::MoL_FinishLoop: [level] Control the step size MoL::MoL_RestoreSandR: Restoring the Save and Restore variables to the original state MoL::MoL_FreeScratchSpace: [level] Free storage for scratch levels end while GROUP MoL_PseudoEvolution: Calculate pseudo-evolved quantities GROUP ADMBase_SetADMVars: Set the ADM variables before this group, and use them afterwards GROUP ML_ADMConstraints_evaluate_group: ML_ADMConstraints_evaluate ML_ADMConstraints::ML_ADMConstraints_evaluate: ML_ADMConstraints_evaluate GROUP ML_ADMConstraints_evaluate_bc_group: ML_ADMConstraints_evaluate ML_ADMConstraints::ML_ADMConstraints_evaluate_SelectBCs: [level] ML_ADMConstraints_evaluate_SelectBCs GROUP ML_ADMConstraints_evaluate_ApplyBCs: Apply BCs for groups set in ML_ADMConstraints_evaluate GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions CoordinatesSymmetry::CoordinatesSymmetry_Apply_FirstPass: Apply multipatch symmetries Interpolate2::Interpolate2ApplyBC: [level] Apply interpolating inter-patch boundaries CoordinatesSymmetry::CoordinatesSymmetry_Apply_SecondPass: Apply multipatch symmetries for remaining points not filled by interpatch interpolation Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP ML_BSSN_ConstraintsEverywhere_group: ML_BSSN_ConstraintsEverywhere ML_BSSN::ML_BSSN_ConstraintsEverywhere: ML_BSSN_ConstraintsEverywhere GROUP ML_BSSN_ConstraintsEverywhere_bc_group: ML_BSSN_ConstraintsEverywhere ML_BSSN::ML_BSSN_ConstraintsEverywhere_SelectBCs: [level] ML_BSSN_ConstraintsEverywhere_SelectBCs GROUP ML_BSSN_ConstraintsEverywhere_ApplyBCs: Apply BCs for groups set in ML_BSSN_ConstraintsEverywhere GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions CoordinatesSymmetry::CoordinatesSymmetry_Apply_FirstPass: Apply multipatch symmetries Interpolate2::Interpolate2ApplyBC: [level] Apply interpolating inter-patch boundaries CoordinatesSymmetry::CoordinatesSymmetry_Apply_SecondPass: Apply multipatch symmetries for remaining points not filled by interpatch interpolation Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP ML_BSSN_ConstraintsInterior_group: ML_BSSN_ConstraintsInterior ML_BSSN::ML_BSSN_ConstraintsInterior: ML_BSSN_ConstraintsInterior GROUP ML_BSSN_ConstraintsInterior_bc_group: ML_BSSN_ConstraintsInterior ML_BSSN::ML_BSSN_ConstraintsInterior_SelectBCs: [level] ML_BSSN_ConstraintsInterior_SelectBCs GROUP ML_BSSN_ConstraintsInterior_ApplyBCs: Apply BCs for groups set in ML_BSSN_ConstraintsInterior GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions CoordinatesSymmetry::CoordinatesSymmetry_Apply_FirstPass: Apply multipatch symmetries Interpolate2::Interpolate2ApplyBC: [level] Apply interpolating inter-patch boundaries CoordinatesSymmetry::CoordinatesSymmetry_Apply_SecondPass: Apply multipatch symmetries for remaining points not filled by interpatch interpolation Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP WeylScal4_psis_calc_Nth_group: WeylScal4_psis_calc_Nth WeylScal4::WeylScal4_psis_calc_Nth: WeylScal4_psis_calc_Nth GROUP WeylScal4_psis_calc_Nth_bc_group: WeylScal4_psis_calc_Nth WeylScal4::WeylScal4_psis_calc_Nth_SelectBCs: [level] WeylScal4_psis_calc_Nth_SelectBCs GROUP WeylScal4_psis_calc_Nth_ApplyBCs: Apply BCs for groups set in WeylScal4_psis_calc_Nth GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions CoordinatesSymmetry::CoordinatesSymmetry_Apply_FirstPass: Apply multipatch symmetries Interpolate2::Interpolate2ApplyBC: [level] Apply interpolating inter-patch boundaries CoordinatesSymmetry::CoordinatesSymmetry_Apply_SecondPass: Apply multipatch symmetries for remaining points not filled by interpatch interpolation Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP WeylScal4_invars_calc_Nth_group: WeylScal4_invars_calc_Nth WeylScal4::WeylScal4_invars_calc_Nth: WeylScal4_invars_calc_Nth PunctureTracker::PunctureTracker_Track: [global] Calculate new location of punctures Evolve finer grids recursively Restrict from finer grids [CCTK_POSTRESTRICT] GROUP MoL_PostStep: Ensure that everything is correct after restriction ML_BSSN::ML_BSSN_SelectBoundConds: [level] select boundary conditions ML_ADMConstraints::ML_ADMConstraints_SelectBoundConds: [level] select boundary conditions GROUP ML_ADMConstraints_ApplyBCs: Apply boundary conditions controlled by thorn Boundary GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions CoordinatesSymmetry::CoordinatesSymmetry_Apply_FirstPass: Apply multipatch symmetries Interpolate2::Interpolate2ApplyBC: [level] Apply interpolating inter-patch boundaries CoordinatesSymmetry::CoordinatesSymmetry_Apply_SecondPass: Apply multipatch symmetries for remaining points not filled by interpatch interpolation Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP ML_BSSN_ApplyBCs: Apply boundary conditions controlled by thorn Boundary GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions CoordinatesSymmetry::CoordinatesSymmetry_Apply_FirstPass: Apply multipatch symmetries Interpolate2::Interpolate2ApplyBC: [level] Apply interpolating inter-patch boundaries CoordinatesSymmetry::CoordinatesSymmetry_Apply_SecondPass: Apply multipatch symmetries for remaining points not filled by interpatch interpolation Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions ML_BSSN::ML_BSSN_ADMBaseInterior: ML_BSSN_ADMBaseInterior ML_BSSN::ML_BSSN_ADMBaseBoundaryScalar: ML_BSSN_ADMBaseBoundaryScalar ML_BSSN::ML_BSSN_ADMBaseEverywhere: ML_BSSN_ADMBaseEverywhere ML_BSSN_Helper::ML_BSSN_ADMBase_SelectBCs: [level] Select boundary conditions for ADMBase variables GROUP ML_BSSN_ADMBase_ApplyBCs: Apply boundary conditions to ADMBase variables GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions CoordinatesSymmetry::CoordinatesSymmetry_Apply_FirstPass: Apply multipatch symmetries Interpolate2::Interpolate2ApplyBC: [level] Apply interpolating inter-patch boundaries CoordinatesSymmetry::CoordinatesSymmetry_Apply_SecondPass: Apply multipatch symmetries for remaining points not filled by interpatch interpolation Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP ADMBase_SetADMVars: Set the ADM variables before this group, and use them afterwards WeylScal4::WeylScal4_SelectBoundConds: [level] select boundary conditions GROUP WeylScal4_ApplyBCs: Apply boundary conditions controlled by thorn Boundary GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions CoordinatesSymmetry::CoordinatesSymmetry_Apply_FirstPass: Apply multipatch symmetries Interpolate2::Interpolate2ApplyBC: [level] Apply interpolating inter-patch boundaries CoordinatesSymmetry::CoordinatesSymmetry_Apply_SecondPass: Apply multipatch symmetries for remaining points not filled by interpatch interpolation Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP MoL_PseudoEvolutionBoundaries: Apply boundary conditions to pseudo-evolved quantities GROUP ML_ADMConstraints_evaluate_bc_group: ML_ADMConstraints_evaluate ML_ADMConstraints::ML_ADMConstraints_evaluate_SelectBCs: [level] ML_ADMConstraints_evaluate_SelectBCs GROUP ML_ADMConstraints_evaluate_ApplyBCs: Apply BCs for groups set in ML_ADMConstraints_evaluate GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions CoordinatesSymmetry::CoordinatesSymmetry_Apply_FirstPass: Apply multipatch symmetries Interpolate2::Interpolate2ApplyBC: [level] Apply interpolating inter-patch boundaries CoordinatesSymmetry::CoordinatesSymmetry_Apply_SecondPass: Apply multipatch symmetries for remaining points not filled by interpatch interpolation Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP ML_BSSN_ConstraintsEverywhere_bc_group: ML_BSSN_ConstraintsEverywhere ML_BSSN::ML_BSSN_ConstraintsEverywhere_SelectBCs: [level] ML_BSSN_ConstraintsEverywhere_SelectBCs GROUP ML_BSSN_ConstraintsEverywhere_ApplyBCs: Apply BCs for groups set in ML_BSSN_ConstraintsEverywhere GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions CoordinatesSymmetry::CoordinatesSymmetry_Apply_FirstPass: Apply multipatch symmetries Interpolate2::Interpolate2ApplyBC: [level] Apply interpolating inter-patch boundaries CoordinatesSymmetry::CoordinatesSymmetry_Apply_SecondPass: Apply multipatch symmetries for remaining points not filled by interpatch interpolation Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP ML_BSSN_ConstraintsInterior_bc_group: ML_BSSN_ConstraintsInterior ML_BSSN::ML_BSSN_ConstraintsInterior_SelectBCs: [level] ML_BSSN_ConstraintsInterior_SelectBCs GROUP ML_BSSN_ConstraintsInterior_ApplyBCs: Apply BCs for groups set in ML_BSSN_ConstraintsInterior GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions CoordinatesSymmetry::CoordinatesSymmetry_Apply_FirstPass: Apply multipatch symmetries Interpolate2::Interpolate2ApplyBC: [level] Apply interpolating inter-patch boundaries CoordinatesSymmetry::CoordinatesSymmetry_Apply_SecondPass: Apply multipatch symmetries for remaining points not filled by interpatch interpolation Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP WeylScal4_psis_calc_Nth_bc_group: WeylScal4_psis_calc_Nth WeylScal4::WeylScal4_psis_calc_Nth_SelectBCs: [level] WeylScal4_psis_calc_Nth_SelectBCs GROUP WeylScal4_psis_calc_Nth_ApplyBCs: Apply BCs for groups set in WeylScal4_psis_calc_Nth GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions CoordinatesSymmetry::CoordinatesSymmetry_Apply_FirstPass: Apply multipatch symmetries Interpolate2::Interpolate2ApplyBC: [level] Apply interpolating inter-patch boundaries CoordinatesSymmetry::CoordinatesSymmetry_Apply_SecondPass: Apply multipatch symmetries for remaining points not filled by interpatch interpolation Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions [CCTK_POSTSTEP] GROUP ADMDerivatives_Group: Group for ADMDerivatives computations. ADMDerivatives::ADMDerivatives_CalcDerivatives: [local] Calculate requested derivatives of ADM lapse, shift, 3-metric ADMDerivatives::ADMDerivatives_radial_SelectBC: [level] Selecting BC for metric radial derivatives ADMDerivatives::ADMDerivatives_time_SelectBC: [level] Selecting BC for metric time derivatives GROUP ADMDerivatives_ApplyBCs: Apply boundary conditions controlled by thorn Boundary GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions CoordinatesSymmetry::CoordinatesSymmetry_Apply_FirstPass: Apply multipatch symmetries Interpolate2::Interpolate2ApplyBC: [level] Apply interpolating inter-patch boundaries CoordinatesSymmetry::CoordinatesSymmetry_Apply_SecondPass: Apply multipatch symmetries for remaining points not filled by interpatch interpolation Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions AHFinderDirect::AHFinderDirect_import_mask: [global] [loop-local] import the excision mask AHFinderDirect::AHFinderDirect_find_horizons: [global] find apparent horizon(s) after this time step AHFinderDirect::AHFinderDirect_store: [global] store apparent horizon(s) into spherical surface(s) AHFinderDirect::AHFinderDirect_save: [global] save apparent horizon(s) into Cactus variables AHFinderDirect::AHFinderDirect_maybe_do_masks: set mask(s) based on apparent horizon position(s) GROUP zzz_NaNChecker_NaNCheck: Check for NaNs and count them in NaNChecker::NaNsFound NaNChecker::NaNChecker_NaNCheck_Prepare: [level] Prepare data structures to check for NaNs NaNChecker::NaNChecker_NaNCheck_Check: [local] Check for NaNs NaNChecker::NaNChecker_NaNCheck_Finish: [level] Count NaNs in NaNChecker::NaNsFound NaNChecker::NaNChecker_TakeAction: [global] [loop-level] Output NaNChecker::NaNmask and take action according to NaNChecker::action_if_found PunctureTracker::PunctureTracker_SetPositions: [global] Copy puncture positions to spherical surfaces SphericalSurface::SphericalSurface_Set: [global] Set surface radii GROUP SphericalSurface_HasBeenSet: Set the spherical surfaces before this group, and use it afterwards SphericalSurface::SphericalSurface_CheckState: [global] Test the state of the spherical surfaces if (checkpoint) [CCTK_CHECKPOINT] CarpetIOHDF5::CarpetIOHDF5_EvolutionCheckpoint: [meta] Evolution checkpoint routine endif if (analysis) [CCTK_ANALYSIS] CarpetLib::CarpetLib_printtimestats: [global] Print timing statistics if desired CarpetLib::CarpetLib_printmemstats: [global] Print memory statistics if desired Formaline::Formaline_AnnounceUpdate: [global] Put some meta information about the current run into permanent storage LoopControl::LC_statistics_analysis: [meta] Output LoopControl statistics GROUP ML_BSSN_EvolutionAnalysis: Calculate RHS at analysis ML_BSSN::ML_BSSN_EvolutionAnalysisInit: ML_BSSN_EvolutionAnalysisInit ML_BSSN::ML_BSSN_EvolutionAnalysisInterior: ML_BSSN_EvolutionAnalysisInterior Multipole::Multipole_Calc: [global] Calculate Multipoles QuasiLocalMeasures::qlm_calculate: [global] Calculate quasi-local quantities SystemStatistics::SystemStatistics_Collect: [global] Collect system statistics TerminationTrigger::TerminationTrigger_CheckWalltime: Check elapsed job walltime TerminationTrigger::TerminationTrigger_CheckSignal: Check if we received a termination signal TerminationTrigger::TerminationTrigger_CheckFile: Check termination file endif Output grid variables enddo Termination routines [CCTK_TERMINATE] CarpetIOHDF5::CarpetIOHDF5_TerminationCheckpoint: [meta] Termination checkpoint routine Formaline::Formaline_AnnounceFinal: [global] Put some meta information about the current run into permanent storage LoopControl::LC_statistics_terminate: [meta] Output LoopControl statistics MoL::MoL_FreeIndexArrays: Free the MoL bookkeeping index arrays Shutdown routines [CCTK_SHUTDOWN] Timers::Timer_Shutdown: Prepare hierarchical timers Routines run after changing the grid hierarchy: [CCTK_POSTREGRID] AHFinderDirect::AHFinderDirect_maybe_do_masks: set mask(s) based on apparent horizon position(s) CartGrid3D::SpatialCoordinates: Set Coordinates after regridding GROUP MaskBase_SetupMask: Set up the weight function GROUP MaskBase_SetupMaskAll: Set up the weight function CarpetReduce::MaskBase_AllocateMask: [global] Allocate the weight function CarpetReduce::MaskBase_InitMask: [global] [loop-local] Initialise the weight function GROUP SetupIMaskInternal: Set up the integer weight function (schedule other routines in here) CarpetReduce::CoordBase_SetupMask: [global] [loop-local] Set up the outer boundaries of the weight function CarpetReduce::CarpetMaskSetup: [global] [loop-singlemap] Set up the weight function for the restriction regions GROUP SetupIMask: Set up the integer weight function (schedule other routines in here) CarpetReduce::MaskBase_SetMask: [global] [loop-local] Set the weight function GROUP SetupMask: Set up the real weight function (schedule other routines in here) CarpetReduce::MaskBase_TestMask: [global] Test the weight function GROUP MultiPatch_SpatialCoordinates: Determine the global coordinates of each gridpoint. Coordinates::Coordinates_SetGlobalCoords: [local] Determine the global coordinates of each gridpoint. Coordinates::Coordinates_SetJacobian: [local] Fill in the Jacobian matrices for Thornburg04 coordinates. Coordinates::Coordinates_SetVolumeForm: [local] Set weight mask (for Carpet reduction operators) GROUP MoL_PostStep: Ensure that everything is correct after regridding ML_BSSN::ML_BSSN_SelectBoundConds: [level] select boundary conditions ML_ADMConstraints::ML_ADMConstraints_SelectBoundConds: [level] select boundary conditions GROUP ML_ADMConstraints_ApplyBCs: Apply boundary conditions controlled by thorn Boundary GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions CoordinatesSymmetry::CoordinatesSymmetry_Apply_FirstPass: Apply multipatch symmetries Interpolate2::Interpolate2ApplyBC: [level] Apply interpolating inter-patch boundaries CoordinatesSymmetry::CoordinatesSymmetry_Apply_SecondPass: Apply multipatch symmetries for remaining points not filled by interpatch interpolation Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP ML_BSSN_ApplyBCs: Apply boundary conditions controlled by thorn Boundary GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions CoordinatesSymmetry::CoordinatesSymmetry_Apply_FirstPass: Apply multipatch symmetries Interpolate2::Interpolate2ApplyBC: [level] Apply interpolating inter-patch boundaries CoordinatesSymmetry::CoordinatesSymmetry_Apply_SecondPass: Apply multipatch symmetries for remaining points not filled by interpatch interpolation Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions ML_BSSN::ML_BSSN_ADMBaseInterior: ML_BSSN_ADMBaseInterior ML_BSSN::ML_BSSN_ADMBaseBoundaryScalar: ML_BSSN_ADMBaseBoundaryScalar ML_BSSN::ML_BSSN_ADMBaseEverywhere: ML_BSSN_ADMBaseEverywhere ML_BSSN_Helper::ML_BSSN_ADMBase_SelectBCs: [level] Select boundary conditions for ADMBase variables GROUP ML_BSSN_ADMBase_ApplyBCs: Apply boundary conditions to ADMBase variables GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions CoordinatesSymmetry::CoordinatesSymmetry_Apply_FirstPass: Apply multipatch symmetries Interpolate2::Interpolate2ApplyBC: [level] Apply interpolating inter-patch boundaries CoordinatesSymmetry::CoordinatesSymmetry_Apply_SecondPass: Apply multipatch symmetries for remaining points not filled by interpatch interpolation Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP ADMBase_SetADMVars: Set the ADM variables before this group, and use them afterwards WeylScal4::WeylScal4_SelectBoundConds: [level] select boundary conditions GROUP WeylScal4_ApplyBCs: Apply boundary conditions controlled by thorn Boundary GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions CoordinatesSymmetry::CoordinatesSymmetry_Apply_FirstPass: Apply multipatch symmetries Interpolate2::Interpolate2ApplyBC: [level] Apply interpolating inter-patch boundaries CoordinatesSymmetry::CoordinatesSymmetry_Apply_SecondPass: Apply multipatch symmetries for remaining points not filled by interpatch interpolation Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP MoL_PseudoEvolutionBoundaries: Apply boundary conditions to pseudo-evolved quantities GROUP ML_ADMConstraints_evaluate_bc_group: ML_ADMConstraints_evaluate ML_ADMConstraints::ML_ADMConstraints_evaluate_SelectBCs: [level] ML_ADMConstraints_evaluate_SelectBCs GROUP ML_ADMConstraints_evaluate_ApplyBCs: Apply BCs for groups set in ML_ADMConstraints_evaluate GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions CoordinatesSymmetry::CoordinatesSymmetry_Apply_FirstPass: Apply multipatch symmetries Interpolate2::Interpolate2ApplyBC: [level] Apply interpolating inter-patch boundaries CoordinatesSymmetry::CoordinatesSymmetry_Apply_SecondPass: Apply multipatch symmetries for remaining points not filled by interpatch interpolation Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP ML_BSSN_ConstraintsEverywhere_bc_group: ML_BSSN_ConstraintsEverywhere ML_BSSN::ML_BSSN_ConstraintsEverywhere_SelectBCs: [level] ML_BSSN_ConstraintsEverywhere_SelectBCs GROUP ML_BSSN_ConstraintsEverywhere_ApplyBCs: Apply BCs for groups set in ML_BSSN_ConstraintsEverywhere GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions CoordinatesSymmetry::CoordinatesSymmetry_Apply_FirstPass: Apply multipatch symmetries Interpolate2::Interpolate2ApplyBC: [level] Apply interpolating inter-patch boundaries CoordinatesSymmetry::CoordinatesSymmetry_Apply_SecondPass: Apply multipatch symmetries for remaining points not filled by interpatch interpolation Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP ML_BSSN_ConstraintsInterior_bc_group: ML_BSSN_ConstraintsInterior ML_BSSN::ML_BSSN_ConstraintsInterior_SelectBCs: [level] ML_BSSN_ConstraintsInterior_SelectBCs GROUP ML_BSSN_ConstraintsInterior_ApplyBCs: Apply BCs for groups set in ML_BSSN_ConstraintsInterior GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions CoordinatesSymmetry::CoordinatesSymmetry_Apply_FirstPass: Apply multipatch symmetries Interpolate2::Interpolate2ApplyBC: [level] Apply interpolating inter-patch boundaries CoordinatesSymmetry::CoordinatesSymmetry_Apply_SecondPass: Apply multipatch symmetries for remaining points not filled by interpatch interpolation Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP WeylScal4_psis_calc_Nth_bc_group: WeylScal4_psis_calc_Nth WeylScal4::WeylScal4_psis_calc_Nth_SelectBCs: [level] WeylScal4_psis_calc_Nth_SelectBCs GROUP WeylScal4_psis_calc_Nth_ApplyBCs: Apply BCs for groups set in WeylScal4_psis_calc_Nth GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions CoordinatesSymmetry::CoordinatesSymmetry_Apply_FirstPass: Apply multipatch symmetries Interpolate2::Interpolate2ApplyBC: [level] Apply interpolating inter-patch boundaries CoordinatesSymmetry::CoordinatesSymmetry_Apply_SecondPass: Apply multipatch symmetries for remaining points not filled by interpatch interpolation Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions -------------------------------------------------------------------------------- INFO (Carpet): Multi-Model listing: model 0: "world" INFO (Carpet): Multi-Model process distribution: process 0: model 0 "world" INFO (Carpet): Multi-Model: This is process 0, model 0 "world" INFO (CarpetLib): Process startup time was 1.49 seconds Current core file size limit: hard=[unlimited], soft=[unlimited] Current addres space size limit: hard=[unlimited], soft=[unlimited] Current data segment size limit: hard=[unlimited], soft=[unlimited] Current resident set size limit: hard=[unlimited], soft=[unlimited] INFO (CycleClock): Measuring CycleClock tick via OpenMP... INFO (CycleClock): Calibrated CycleClock: 0.345487 ns per clock tick (2.89446 GHz) INFO (hwloc): library version 2.0.4, API version 0x20000 INFO (SystemTopology): MPI process-to-host mapping: This is MPI process 0 of 1 MPI hosts: 0: panther This MPI process runs on host 0 of 1 On this host, this is MPI process 0 of 1 INFO (SystemTopology): Topology support: Discovery support: discovery->pu : yes CPU binding support: cpubind->set_thisproc_cpubind : yes cpubind->get_thisproc_cpubind : yes cpubind->set_proc_cpubind : yes cpubind->get_proc_cpubind : yes cpubind->set_thisthread_cpubind : yes cpubind->get_thisthread_cpubind : yes cpubind->set_thread_cpubind : yes cpubind->get_thread_cpubind : yes cpubind->get_thisproc_last_cpu_location : yes cpubind->get_proc_last_cpu_location : yes cpubind->get_thisthread_last_cpu_location: yes Memory binding support: membind->set_thisproc_membind : no membind->get_thisproc_membind : no membind->set_proc_membind : no membind->get_proc_membind : no membind->set_thisthread_membind : yes membind->get_thisthread_membind : yes membind->set_area_membind : yes membind->get_area_membind : yes membind->alloc_membind : yes membind->firsttouch_membind : yes membind->bind_membind : yes membind->interleave_membind : yes membind->nexttouch_membind : no membind->migrate_membind : yes INFO (SystemTopology): Hardware objects in this node: Machine L#0: (P#0, total=1043114796KB, DMIProductName="PowerEdge R7515", DMIProductVersion=, DMIBoardVendor="Dell Inc.", DMIBoardName=04F3CJ, DMIBoardVersion=A02, DMIChassisVendor="Dell Inc.", DMIChassisType=23, DMIChassisVersion=, DMIChassisAssetTag=, DMIBIOSVendor="Dell Inc.", DMIBIOSVersion=2.18.1, DMIBIOSDate=02/03/2025, DMISysVendor="Dell Inc.", Backend=Linux, OSName=Linux, OSRelease=5.14.0-570.37.1.el9_6.x86_64, OSVersion="#1 SMP PREEMPT_DYNAMIC Thu Aug 28 10:41:06 UTC 2025", HostName=panther, Architecture=x86_64, hwlocVersion=2.0.4, ProcessName=cactus_sim) Package L#0: (P#0, total=1043114796KB, CPUVendor=AuthenticAMD, CPUFamilyNumber=23, CPUModelNumber=49, CPUModel="AMD EPYC 7542 32-Core Processor ", CPUStepping=0) L3Cache L#0: (P#-1, size=16384KB, linesize=64, ways=16, Inclusive=0) L2Cache L#0: (P#-1, size=512KB, linesize=64, ways=8, Inclusive=1) L1dCache L#0: (P#-1, size=32KB, linesize=64, ways=8, Inclusive=0) Core L#0: (P#0) PU L#0: (P#0) PU L#1: (P#32) L2Cache L#1: (P#-1, size=512KB, linesize=64, ways=8, Inclusive=1) L1dCache L#1: (P#-1, size=32KB, linesize=64, ways=8, Inclusive=0) Core L#1: (P#1) PU L#2: (P#1) PU L#3: (P#33) L2Cache L#2: (P#-1, size=512KB, linesize=64, ways=8, Inclusive=1) L1dCache L#2: (P#-1, size=32KB, linesize=64, ways=8, Inclusive=0) Core L#2: (P#2) PU L#4: (P#2) PU L#5: (P#34) L2Cache L#3: (P#-1, size=512KB, linesize=64, ways=8, Inclusive=1) L1dCache L#3: (P#-1, size=32KB, linesize=64, ways=8, Inclusive=0) Core L#3: (P#3) PU L#6: (P#3) PU L#7: (P#35) L3Cache L#1: (P#-1, size=16384KB, linesize=64, ways=16, Inclusive=0) L2Cache L#4: (P#-1, size=512KB, linesize=64, ways=8, Inclusive=1) L1dCache L#4: (P#-1, size=32KB, linesize=64, ways=8, Inclusive=0) Core L#4: (P#4) PU L#8: (P#4) PU L#9: (P#36) L2Cache L#5: (P#-1, size=512KB, linesize=64, ways=8, Inclusive=1) L1dCache L#5: (P#-1, size=32KB, linesize=64, ways=8, Inclusive=0) Core L#5: (P#5) PU L#10: (P#5) PU L#11: (P#37) L2Cache L#6: (P#-1, size=512KB, linesize=64, ways=8, Inclusive=1) L1dCache L#6: (P#-1, size=32KB, linesize=64, ways=8, Inclusive=0) Core L#6: (P#6) PU L#12: (P#6) PU L#13: (P#38) L2Cache L#7: (P#-1, size=512KB, linesize=64, ways=8, Inclusive=1) L1dCache L#7: (P#-1, size=32KB, linesize=64, ways=8, Inclusive=0) Core L#7: (P#7) PU L#14: (P#7) PU L#15: (P#39) L3Cache L#2: (P#-1, size=16384KB, linesize=64, ways=16, Inclusive=0) L2Cache L#8: (P#-1, size=512KB, linesize=64, ways=8, Inclusive=1) L1dCache L#8: (P#-1, size=32KB, linesize=64, ways=8, Inclusive=0) Core L#8: (P#8) PU L#16: (P#8) PU L#17: (P#40) L2Cache L#9: (P#-1, size=512KB, linesize=64, ways=8, Inclusive=1) L1dCache L#9: (P#-1, size=32KB, linesize=64, ways=8, Inclusive=0) Core L#9: (P#9) PU L#18: (P#9) PU L#19: (P#41) L2Cache L#10: (P#-1, size=512KB, linesize=64, ways=8, Inclusive=1) L1dCache L#10: (P#-1, size=32KB, linesize=64, ways=8, Inclusive=0) Core L#10: (P#10) PU L#20: (P#10) PU L#21: (P#42) L2Cache L#11: (P#-1, size=512KB, linesize=64, ways=8, Inclusive=1) L1dCache L#11: (P#-1, size=32KB, linesize=64, ways=8, Inclusive=0) Core L#11: (P#11) PU L#22: (P#11) PU L#23: (P#43) L3Cache L#3: (P#-1, size=16384KB, linesize=64, ways=16, Inclusive=0) L2Cache L#12: (P#-1, size=512KB, linesize=64, ways=8, Inclusive=1) L1dCache L#12: (P#-1, size=32KB, linesize=64, ways=8, Inclusive=0) Core L#12: (P#12) PU L#24: (P#12) PU L#25: (P#44) L2Cache L#13: (P#-1, size=512KB, linesize=64, ways=8, Inclusive=1) L1dCache L#13: (P#-1, size=32KB, linesize=64, ways=8, Inclusive=0) Core L#13: (P#13) PU L#26: (P#13) PU L#27: (P#45) L2Cache L#14: (P#-1, size=512KB, linesize=64, ways=8, Inclusive=1) L1dCache L#14: (P#-1, size=32KB, linesize=64, ways=8, Inclusive=0) Core L#14: (P#14) PU L#28: (P#14) PU L#29: (P#46) L2Cache L#15: (P#-1, size=512KB, linesize=64, ways=8, Inclusive=1) L1dCache L#15: (P#-1, size=32KB, linesize=64, ways=8, Inclusive=0) Core L#15: (P#15) PU L#30: (P#15) PU L#31: (P#47) L3Cache L#4: (P#-1, size=16384KB, linesize=64, ways=16, Inclusive=0) L2Cache L#16: (P#-1, size=512KB, linesize=64, ways=8, Inclusive=1) L1dCache L#16: (P#-1, size=32KB, linesize=64, ways=8, Inclusive=0) Core L#16: (P#16) PU L#32: (P#16) PU L#33: (P#48) L2Cache L#17: (P#-1, size=512KB, linesize=64, ways=8, Inclusive=1) L1dCache L#17: (P#-1, size=32KB, linesize=64, ways=8, Inclusive=0) Core L#17: (P#17) PU L#34: (P#17) PU L#35: (P#49) L2Cache L#18: (P#-1, size=512KB, linesize=64, ways=8, Inclusive=1) L1dCache L#18: (P#-1, size=32KB, linesize=64, ways=8, Inclusive=0) Core L#18: (P#18) PU L#36: (P#18) PU L#37: (P#50) L2Cache L#19: (P#-1, size=512KB, linesize=64, ways=8, Inclusive=1) L1dCache L#19: (P#-1, size=32KB, linesize=64, ways=8, Inclusive=0) Core L#19: (P#19) PU L#38: (P#19) PU L#39: (P#51) L3Cache L#5: (P#-1, size=16384KB, linesize=64, ways=16, Inclusive=0) L2Cache L#20: (P#-1, size=512KB, linesize=64, ways=8, Inclusive=1) L1dCache L#20: (P#-1, size=32KB, linesize=64, ways=8, Inclusive=0) Core L#20: (P#20) PU L#40: (P#20) PU L#41: (P#52) L2Cache L#21: (P#-1, size=512KB, linesize=64, ways=8, Inclusive=1) L1dCache L#21: (P#-1, size=32KB, linesize=64, ways=8, Inclusive=0) Core L#21: (P#21) PU L#42: (P#21) PU L#43: (P#53) L2Cache L#22: (P#-1, size=512KB, linesize=64, ways=8, Inclusive=1) L1dCache L#22: (P#-1, size=32KB, linesize=64, ways=8, Inclusive=0) Core L#22: (P#22) PU L#44: (P#22) PU L#45: (P#54) L2Cache L#23: (P#-1, size=512KB, linesize=64, ways=8, Inclusive=1) L1dCache L#23: (P#-1, size=32KB, linesize=64, ways=8, Inclusive=0) Core L#23: (P#23) PU L#46: (P#23) PU L#47: (P#55) L3Cache L#6: (P#-1, size=16384KB, linesize=64, ways=16, Inclusive=0) L2Cache L#24: (P#-1, size=512KB, linesize=64, ways=8, Inclusive=1) L1dCache L#24: (P#-1, size=32KB, linesize=64, ways=8, Inclusive=0) Core L#24: (P#24) PU L#48: (P#24) PU L#49: (P#56) L2Cache L#25: (P#-1, size=512KB, linesize=64, ways=8, Inclusive=1) L1dCache L#25: (P#-1, size=32KB, linesize=64, ways=8, Inclusive=0) Core L#25: (P#25) PU L#50: (P#25) PU L#51: (P#57) L2Cache L#26: (P#-1, size=512KB, linesize=64, ways=8, Inclusive=1) L1dCache L#26: (P#-1, size=32KB, linesize=64, ways=8, Inclusive=0) Core L#26: (P#26) PU L#52: (P#26) PU L#53: (P#58) L2Cache L#27: (P#-1, size=512KB, linesize=64, ways=8, Inclusive=1) L1dCache L#27: (P#-1, size=32KB, linesize=64, ways=8, Inclusive=0) Core L#27: (P#27) PU L#54: (P#27) PU L#55: (P#59) L3Cache L#7: (P#-1, size=16384KB, linesize=64, ways=16, Inclusive=0) L2Cache L#28: (P#-1, size=512KB, linesize=64, ways=8, Inclusive=1) L1dCache L#28: (P#-1, size=32KB, linesize=64, ways=8, Inclusive=0) Core L#28: (P#28) PU L#56: (P#28) PU L#57: (P#60) L2Cache L#29: (P#-1, size=512KB, linesize=64, ways=8, Inclusive=1) L1dCache L#29: (P#-1, size=32KB, linesize=64, ways=8, Inclusive=0) Core L#29: (P#29) PU L#58: (P#29) PU L#59: (P#61) L2Cache L#30: (P#-1, size=512KB, linesize=64, ways=8, Inclusive=1) L1dCache L#30: (P#-1, size=32KB, linesize=64, ways=8, Inclusive=0) Core L#30: (P#30) PU L#60: (P#30) PU L#61: (P#62) L2Cache L#31: (P#-1, size=512KB, linesize=64, ways=8, Inclusive=1) L1dCache L#31: (P#-1, size=32KB, linesize=64, ways=8, Inclusive=0) Core L#31: (P#31) PU L#62: (P#31) PU L#63: (P#63) INFO (SystemTopology): Thread CPU bindings: MPI process 0 on host 0 (process 0 of 1 on this host) OpenMP thread 0: PU set L#{0-63} P#{0-63} OpenMP thread 1: PU set L#{0-63} P#{0-63} OpenMP thread 2: PU set L#{0-63} P#{0-63} OpenMP thread 3: PU set L#{0-63} P#{0-63} OpenMP thread 4: PU set L#{0-63} P#{0-63} OpenMP thread 5: PU set L#{0-63} P#{0-63} OpenMP thread 6: PU set L#{0-63} P#{0-63} OpenMP thread 7: PU set L#{0-63} P#{0-63} OpenMP thread 8: PU set L#{0-63} P#{0-63} OpenMP thread 9: PU set L#{0-63} P#{0-63} OpenMP thread 10: PU set L#{0-63} P#{0-63} OpenMP thread 11: PU set L#{0-63} P#{0-63} OpenMP thread 12: PU set L#{0-63} P#{0-63} OpenMP thread 13: PU set L#{0-63} P#{0-63} OpenMP thread 14: PU set L#{0-63} P#{0-63} OpenMP thread 15: PU set L#{0-63} P#{0-63} OpenMP thread 16: PU set L#{0-63} P#{0-63} OpenMP thread 17: PU set L#{0-63} P#{0-63} OpenMP thread 18: PU set L#{0-63} P#{0-63} OpenMP thread 19: PU set L#{0-63} P#{0-63} OpenMP thread 20: PU set L#{0-63} P#{0-63} OpenMP thread 21: PU set L#{0-63} P#{0-63} OpenMP thread 22: PU set L#{0-63} P#{0-63} OpenMP thread 23: PU set L#{0-63} P#{0-63} OpenMP thread 24: PU set L#{0-63} P#{0-63} OpenMP thread 25: PU set L#{0-63} P#{0-63} OpenMP thread 26: PU set L#{0-63} P#{0-63} OpenMP thread 27: PU set L#{0-63} P#{0-63} OpenMP thread 28: PU set L#{0-63} P#{0-63} OpenMP thread 29: PU set L#{0-63} P#{0-63} OpenMP thread 30: PU set L#{0-63} P#{0-63} OpenMP thread 31: PU set L#{0-63} P#{0-63} INFO (SystemTopology): Setting thread CPU bindings: INFO (SystemTopology): Thread CPU bindings: MPI process 0 on host 0 (process 0 of 1 on this host) OpenMP thread 0: PU set L#{0} P#{0} OpenMP thread 1: PU set L#{2} P#{1} OpenMP thread 2: PU set L#{4} P#{2} OpenMP thread 3: PU set L#{6} P#{3} OpenMP thread 4: PU set L#{8} P#{4} OpenMP thread 5: PU set L#{10} P#{5} OpenMP thread 6: PU set L#{12} P#{6} OpenMP thread 7: PU set L#{14} P#{7} OpenMP thread 8: PU set L#{16} P#{8} OpenMP thread 9: PU set L#{18} P#{9} OpenMP thread 10: PU set L#{20} P#{10} OpenMP thread 11: PU set L#{22} P#{11} OpenMP thread 12: PU set L#{24} P#{12} OpenMP thread 13: PU set L#{26} P#{13} OpenMP thread 14: PU set L#{28} P#{14} OpenMP thread 15: PU set L#{30} P#{15} OpenMP thread 16: PU set L#{32} P#{16} OpenMP thread 17: PU set L#{34} P#{17} OpenMP thread 18: PU set L#{36} P#{18} OpenMP thread 19: PU set L#{38} P#{19} OpenMP thread 20: PU set L#{40} P#{20} OpenMP thread 21: PU set L#{42} P#{21} OpenMP thread 22: PU set L#{44} P#{22} OpenMP thread 23: PU set L#{46} P#{23} OpenMP thread 24: PU set L#{48} P#{24} OpenMP thread 25: PU set L#{50} P#{25} OpenMP thread 26: PU set L#{52} P#{26} OpenMP thread 27: PU set L#{54} P#{27} OpenMP thread 28: PU set L#{56} P#{28} OpenMP thread 29: PU set L#{58} P#{29} OpenMP thread 30: PU set L#{60} P#{30} OpenMP thread 31: PU set L#{62} P#{31} INFO (SystemTopology): Extracting CPU/cache/memory properties: There are 2 PUs per core (aka hardware SMT threads) There are 1 threads per core (aka SMT threads used) Cache (unknown name) has type "data" depth 1 size 32768 linesize 64 associativity 8 stride 4096, for 2 PUs Cache (unknown name) has type "unified" depth 2 size 524288 linesize 64 associativity 8 stride 65536, for 2 PUs Cache (unknown name) has type "unified" depth 3 size 16777216 linesize 64 associativity 16 stride 1048576, for 8 PUs INFO (Formaline): Configuration id: config-sim-panther-system-user-crangano-einstein_toolkit-Cactus INFO (Formaline): Build id: build-sim-panther-crangano-2025.09.10-14.43.00-3265191 INFO (Formaline): Simulation id: run-GW150914-panther-crangano-2025.09.10-15.10.05-3271897 INFO (Formaline): Run id: run-GW150914-panther-crangano-2025.09.10-15.10.05-3271897 INFO (Vectors): Using vector size 2 for architecture SSE2+SSE4.1 (64-bit precision) -------------------------------------------------------------------------------- AMR driver provided by Carpet -------------------------------------------------------------------------------- AMR HDF5 I/O provided by CarpetIOHDF5 -------------------------------------------------------------------------------- AMR 0D HDF5 I/O provided by CarpetIOHDF5 -------------------------------------------------------------------------------- AMR 1D HDF5 I/O provided by CarpetIOHDF5 -------------------------------------------------------------------------------- AMR 2D HDF5 I/O provided by CarpetIOHDF5 -------------------------------------------------------------------------------- AMR 3D HDF5 I/O provided by CarpetIOHDF5 -------------------------------------------------------------------------------- ML_ADMConstraints -------------------------------------------------------------------------------- ML_BSSN -------------------------------------------------------------------------------- AMR scalar I/O provided by CarpetIOScalar -------------------------------------------------------------------------------- MoL: Generalized time integration. -------------------------------------------------------------------------------- AMR info I/O provided by CarpetIOBasic -------------------------------------------------------------------------------- WeylScal4 -------------------------------------------------------------------------------- AMR 0D ASCII I/O provided by CarpetIOASCII -------------------------------------------------------------------------------- AMR 1D ASCII I/O provided by CarpetIOASCII -------------------------------------------------------------------------------- AMR 2D ASCII I/O provided by CarpetIOASCII -------------------------------------------------------------------------------- AMR 3D ASCII I/O provided by CarpetIOASCII -------------------------------------------------------------------------------- INFO (Carpet): MPI is enabled INFO (Carpet): Carpet is running on 1 processes INFO (Carpet): This is process 0 INFO (Carpet): OpenMP is enabled INFO (Carpet): This process contains 32 threads, this is thread 0 INFO (Carpet): There are 32 threads in total INFO (Carpet): There are 32 threads per process INFO (Carpet): This process runs on host panther, pid=3271897 INFO (Carpet): This process runs on 32 cores: 0-31 INFO (Carpet): Thread 0 runs on 1 core: 0 INFO (Carpet): Thread 1 runs on 1 core: 1 INFO (Carpet): Thread 2 runs on 1 core: 2 INFO (Carpet): Thread 3 runs on 1 core: 3 INFO (Carpet): Thread 4 runs on 1 core: 4 INFO (Carpet): Thread 5 runs on 1 core: 5 INFO (Carpet): Thread 6 runs on 1 core: 6 INFO (Carpet): Thread 7 runs on 1 core: 7 INFO (Carpet): Thread 8 runs on 1 core: 8 INFO (Carpet): Thread 9 runs on 1 core: 9 INFO (Carpet): Thread 10 runs on 1 core: 10 INFO (Carpet): Thread 11 runs on 1 core: 11 INFO (Carpet): Thread 12 runs on 1 core: 12 INFO (Carpet): Thread 13 runs on 1 core: 13 INFO (Carpet): Thread 14 runs on 1 core: 14 INFO (Carpet): Thread 15 runs on 1 core: 15 INFO (Carpet): Thread 16 runs on 1 core: 16 INFO (Carpet): Thread 17 runs on 1 core: 17 INFO (Carpet): Thread 18 runs on 1 core: 18 INFO (Carpet): Thread 19 runs on 1 core: 19 INFO (Carpet): Thread 20 runs on 1 core: 20 INFO (Carpet): Thread 21 runs on 1 core: 21 INFO (Carpet): Thread 22 runs on 1 core: 22 INFO (Carpet): Thread 23 runs on 1 core: 23 INFO (Carpet): Thread 24 runs on 1 core: 24 INFO (Carpet): Thread 25 runs on 1 core: 25 INFO (Carpet): Thread 26 runs on 1 core: 26 INFO (Carpet): Thread 27 runs on 1 core: 27 INFO (Carpet): Thread 28 runs on 1 core: 28 INFO (Carpet): Thread 29 runs on 1 core: 29 INFO (Carpet): Thread 30 runs on 1 core: 30 INFO (Carpet): Thread 31 runs on 1 core: 31 INFO (Carpet): This simulation is running in 3 dimensions INFO (Carpet): Boundary specification for map 0: nboundaryzones: [[5,5,5],[5,5,5]] is_internal : [[0,0,0],[0,0,0]] is_staggered : [[0,0,0],[0,0,0]] shiftout : [[4,4,1],[4,4,4]] INFO (Carpet): CoordBase domain specification for map 0: physical extent: [-51.3969,-51.3969,0] : [51.3969,51.3969,51.3969] ([102.794,102.794,51.3969]) interior extent: [-102.794,-102.794,0] : [102.794,102.794,102.794] ([205.588,205.588,102.794]) exterior extent: [-188.455,-188.455,-85.6615] : [188.455,188.455,188.455] ([376.911,376.911,274.117]) base_spacing : [17.1323,17.1323,17.1323] INFO (Carpet): Adapted domain specification for map 0: convergence factor: 2 convergence level : 0 physical extent : [-51.3969,-51.3969,0] : [51.3969,51.3969,51.3969] ([102.794,102.794,51.3969]) interior extent : [-102.794,-102.794,0] : [102.794,102.794,102.794] ([205.588,205.588,102.794]) exterior extent : [-188.455,-188.455,-85.6615] : [188.455,188.455,188.455] ([376.911,376.911,274.117]) spacing : [17.1323,17.1323,17.1323] INFO (Carpet): Base grid specification for map 0: number of grid points : [23,23,17] number of coarse grid ghost points: [[5,5,5],[5,5,5]] INFO (Carpet): Buffer zone counts (excluding ghosts): [0]: [[0,0,0],[0,0,0]] [1]: [[15,15,15],[15,15,15]] [2]: [[15,15,15],[15,15,15]] [3]: [[15,15,15],[15,15,15]] [4]: [[15,15,15],[15,15,15]] [5]: [[15,15,15],[15,15,15]] [6]: [[15,15,15],[15,15,15]] [7]: [[15,15,15],[15,15,15]] [8]: [[15,15,15],[15,15,15]] INFO (Carpet): Overlap zone counts: [0]: [[0,0,0],[0,0,0]] [1]: [[0,0,0],[0,0,0]] [2]: [[0,0,0],[0,0,0]] [3]: [[0,0,0],[0,0,0]] [4]: [[0,0,0],[0,0,0]] [5]: [[0,0,0],[0,0,0]] [6]: [[0,0,0],[0,0,0]] [7]: [[0,0,0],[0,0,0]] [8]: [[0,0,0],[0,0,0]] INFO (Carpet): Boundary specification for map 1: nboundaryzones: [[5,5,5],[5,5,5]] is_internal : [[0,0,0],[0,0,0]] is_staggered : [[0,0,0],[0,0,0]] shiftout : [[1,4,4],[4,4,0]] INFO (Carpet): CoordBase domain specification for map 1: physical extent: [0,-0.785398,51.3969] : [0.785398,0.785398,2210.07] ([0.785398,1.5708,2158.67]) interior extent: [0,-1.9635,0] : [1.9635,1.9635,2192.94] ([1.9635,3.92699,2192.94]) exterior extent: [-1.9635,-3.92699,-85.6615] : [3.92699,3.92699,2278.6] ([5.89049,7.85398,2364.26]) base_spacing : [0.392699,0.392699,17.1323] INFO (Carpet): Adapted domain specification for map 1: convergence factor: 2 convergence level : 0 physical extent : [0,-0.785398,51.3969] : [0.785398,0.785398,2210.07] ([0.785398,1.5708,2158.67]) interior extent : [0,-1.9635,0] : [1.9635,1.9635,2192.94] ([1.9635,3.92699,2192.94]) exterior extent : [-1.9635,-3.92699,-85.6615] : [3.92699,3.92699,2278.6] ([5.89049,7.85398,2364.26]) spacing : [0.392699,0.392699,17.1323] INFO (Carpet): Base grid specification for map 1: number of grid points : [16,21,139] number of coarse grid ghost points: [[5,5,5],[5,5,5]] INFO (Carpet): Buffer zone counts (excluding ghosts): [0]: [[0,0,0],[0,0,0]] [1]: [[15,15,15],[15,15,15]] [2]: [[15,15,15],[15,15,15]] [3]: [[15,15,15],[15,15,15]] [4]: [[15,15,15],[15,15,15]] [5]: [[15,15,15],[15,15,15]] [6]: [[15,15,15],[15,15,15]] [7]: [[15,15,15],[15,15,15]] [8]: [[15,15,15],[15,15,15]] INFO (Carpet): Overlap zone counts: [0]: [[0,0,0],[0,0,0]] [1]: [[0,0,0],[0,0,0]] [2]: [[0,0,0],[0,0,0]] [3]: [[0,0,0],[0,0,0]] [4]: [[0,0,0],[0,0,0]] [5]: [[0,0,0],[0,0,0]] [6]: [[0,0,0],[0,0,0]] [7]: [[0,0,0],[0,0,0]] [8]: [[0,0,0],[0,0,0]] INFO (Carpet): Boundary specification for map 2: nboundaryzones: [[5,5,5],[5,5,5]] is_internal : [[0,0,0],[0,0,0]] is_staggered : [[0,0,0],[0,0,0]] shiftout : [[4,4,4],[1,4,0]] INFO (Carpet): CoordBase domain specification for map 2: physical extent: [-0.785398,-0.785398,51.3969] : [0,0.785398,2210.07] ([0.785398,1.5708,2158.67]) interior extent: [-1.9635,-1.9635,0] : [0,1.9635,2192.94] ([1.9635,3.92699,2192.94]) exterior extent: [-3.92699,-3.92699,-85.6615] : [1.9635,3.92699,2278.6] ([5.89049,7.85398,2364.26]) base_spacing : [0.392699,0.392699,17.1323] INFO (Carpet): Adapted domain specification for map 2: convergence factor: 2 convergence level : 0 physical extent : [-0.785398,-0.785398,51.3969] : [0,0.785398,2210.07] ([0.785398,1.5708,2158.67]) interior extent : [-1.9635,-1.9635,0] : [0,1.9635,2192.94] ([1.9635,3.92699,2192.94]) exterior extent : [-3.92699,-3.92699,-85.6615] : [1.9635,3.92699,2278.6] ([5.89049,7.85398,2364.26]) spacing : [0.392699,0.392699,17.1323] INFO (Carpet): Base grid specification for map 2: number of grid points : [16,21,139] number of coarse grid ghost points: [[5,5,5],[5,5,5]] INFO (Carpet): Buffer zone counts (excluding ghosts): [0]: [[0,0,0],[0,0,0]] [1]: [[15,15,15],[15,15,15]] [2]: [[15,15,15],[15,15,15]] [3]: [[15,15,15],[15,15,15]] [4]: [[15,15,15],[15,15,15]] [5]: [[15,15,15],[15,15,15]] [6]: [[15,15,15],[15,15,15]] [7]: [[15,15,15],[15,15,15]] [8]: [[15,15,15],[15,15,15]] INFO (Carpet): Overlap zone counts: [0]: [[0,0,0],[0,0,0]] [1]: [[0,0,0],[0,0,0]] [2]: [[0,0,0],[0,0,0]] [3]: [[0,0,0],[0,0,0]] [4]: [[0,0,0],[0,0,0]] [5]: [[0,0,0],[0,0,0]] [6]: [[0,0,0],[0,0,0]] [7]: [[0,0,0],[0,0,0]] [8]: [[0,0,0],[0,0,0]] INFO (Carpet): Boundary specification for map 3: nboundaryzones: [[5,5,5],[5,5,5]] is_internal : [[0,0,0],[0,0,0]] is_staggered : [[0,0,0],[0,0,0]] shiftout : [[1,4,4],[4,4,0]] INFO (Carpet): CoordBase domain specification for map 3: physical extent: [0,-0.785398,51.3969] : [0.785398,0.785398,2210.07] ([0.785398,1.5708,2158.67]) interior extent: [0,-1.9635,0] : [1.9635,1.9635,2192.94] ([1.9635,3.92699,2192.94]) exterior extent: [-1.9635,-3.92699,-85.6615] : [3.92699,3.92699,2278.6] ([5.89049,7.85398,2364.26]) base_spacing : [0.392699,0.392699,17.1323] INFO (Carpet): Adapted domain specification for map 3: convergence factor: 2 convergence level : 0 physical extent : [0,-0.785398,51.3969] : [0.785398,0.785398,2210.07] ([0.785398,1.5708,2158.67]) interior extent : [0,-1.9635,0] : [1.9635,1.9635,2192.94] ([1.9635,3.92699,2192.94]) exterior extent : [-1.9635,-3.92699,-85.6615] : [3.92699,3.92699,2278.6] ([5.89049,7.85398,2364.26]) spacing : [0.392699,0.392699,17.1323] INFO (Carpet): Base grid specification for map 3: number of grid points : [16,21,139] number of coarse grid ghost points: [[5,5,5],[5,5,5]] INFO (Carpet): Buffer zone counts (excluding ghosts): [0]: [[0,0,0],[0,0,0]] [1]: [[15,15,15],[15,15,15]] [2]: [[15,15,15],[15,15,15]] [3]: [[15,15,15],[15,15,15]] [4]: [[15,15,15],[15,15,15]] [5]: [[15,15,15],[15,15,15]] [6]: [[15,15,15],[15,15,15]] [7]: [[15,15,15],[15,15,15]] [8]: [[15,15,15],[15,15,15]] INFO (Carpet): Overlap zone counts: [0]: [[0,0,0],[0,0,0]] [1]: [[0,0,0],[0,0,0]] [2]: [[0,0,0],[0,0,0]] [3]: [[0,0,0],[0,0,0]] [4]: [[0,0,0],[0,0,0]] [5]: [[0,0,0],[0,0,0]] [6]: [[0,0,0],[0,0,0]] [7]: [[0,0,0],[0,0,0]] [8]: [[0,0,0],[0,0,0]] INFO (Carpet): Boundary specification for map 4: nboundaryzones: [[5,5,5],[5,5,5]] is_internal : [[0,0,0],[0,0,0]] is_staggered : [[0,0,0],[0,0,0]] shiftout : [[4,4,4],[1,4,0]] INFO (Carpet): CoordBase domain specification for map 4: physical extent: [-0.785398,-0.785398,51.3969] : [0,0.785398,2210.07] ([0.785398,1.5708,2158.67]) interior extent: [-1.9635,-1.9635,0] : [0,1.9635,2192.94] ([1.9635,3.92699,2192.94]) exterior extent: [-3.92699,-3.92699,-85.6615] : [1.9635,3.92699,2278.6] ([5.89049,7.85398,2364.26]) base_spacing : [0.392699,0.392699,17.1323] INFO (Carpet): Adapted domain specification for map 4: convergence factor: 2 convergence level : 0 physical extent : [-0.785398,-0.785398,51.3969] : [0,0.785398,2210.07] ([0.785398,1.5708,2158.67]) interior extent : [-1.9635,-1.9635,0] : [0,1.9635,2192.94] ([1.9635,3.92699,2192.94]) exterior extent : [-3.92699,-3.92699,-85.6615] : [1.9635,3.92699,2278.6] ([5.89049,7.85398,2364.26]) spacing : [0.392699,0.392699,17.1323] INFO (Carpet): Base grid specification for map 4: number of grid points : [16,21,139] number of coarse grid ghost points: [[5,5,5],[5,5,5]] INFO (Carpet): Buffer zone counts (excluding ghosts): [0]: [[0,0,0],[0,0,0]] [1]: [[15,15,15],[15,15,15]] [2]: [[15,15,15],[15,15,15]] [3]: [[15,15,15],[15,15,15]] [4]: [[15,15,15],[15,15,15]] [5]: [[15,15,15],[15,15,15]] [6]: [[15,15,15],[15,15,15]] [7]: [[15,15,15],[15,15,15]] [8]: [[15,15,15],[15,15,15]] INFO (Carpet): Overlap zone counts: [0]: [[0,0,0],[0,0,0]] [1]: [[0,0,0],[0,0,0]] [2]: [[0,0,0],[0,0,0]] [3]: [[0,0,0],[0,0,0]] [4]: [[0,0,0],[0,0,0]] [5]: [[0,0,0],[0,0,0]] [6]: [[0,0,0],[0,0,0]] [7]: [[0,0,0],[0,0,0]] [8]: [[0,0,0],[0,0,0]] INFO (Carpet): Boundary specification for map 5: nboundaryzones: [[5,5,5],[5,5,5]] is_internal : [[0,0,0],[0,0,0]] is_staggered : [[0,0,0],[0,0,0]] shiftout : [[4,4,4],[4,4,0]] INFO (Carpet): CoordBase domain specification for map 5: physical extent: [-0.785398,-0.785398,51.3969] : [0.785398,0.785398,2210.07] ([1.5708,1.5708,2158.67]) interior extent: [-1.9635,-1.9635,0] : [1.9635,1.9635,2192.94] ([3.92699,3.92699,2192.94]) exterior extent: [-3.92699,-3.92699,-85.6615] : [3.92699,3.92699,2278.6] ([7.85398,7.85398,2364.26]) base_spacing : [0.392699,0.392699,17.1323] INFO (Carpet): Adapted domain specification for map 5: convergence factor: 2 convergence level : 0 physical extent : [-0.785398,-0.785398,51.3969] : [0.785398,0.785398,2210.07] ([1.5708,1.5708,2158.67]) interior extent : [-1.9635,-1.9635,0] : [1.9635,1.9635,2192.94] ([3.92699,3.92699,2192.94]) exterior extent : [-3.92699,-3.92699,-85.6615] : [3.92699,3.92699,2278.6] ([7.85398,7.85398,2364.26]) spacing : [0.392699,0.392699,17.1323] INFO (Carpet): Base grid specification for map 5: number of grid points : [21,21,139] number of coarse grid ghost points: [[5,5,5],[5,5,5]] INFO (Carpet): Buffer zone counts (excluding ghosts): [0]: [[0,0,0],[0,0,0]] [1]: [[15,15,15],[15,15,15]] [2]: [[15,15,15],[15,15,15]] [3]: [[15,15,15],[15,15,15]] [4]: [[15,15,15],[15,15,15]] [5]: [[15,15,15],[15,15,15]] [6]: [[15,15,15],[15,15,15]] [7]: [[15,15,15],[15,15,15]] [8]: [[15,15,15],[15,15,15]] INFO (Carpet): Overlap zone counts: [0]: [[0,0,0],[0,0,0]] [1]: [[0,0,0],[0,0,0]] [2]: [[0,0,0],[0,0,0]] [3]: [[0,0,0],[0,0,0]] [4]: [[0,0,0],[0,0,0]] [5]: [[0,0,0],[0,0,0]] [6]: [[0,0,0],[0,0,0]] [7]: [[0,0,0],[0,0,0]] [8]: [[0,0,0],[0,0,0]] INFO (Carpet): Group and variable statistics: INFO (Carpet): There are 966 grid functions in 88 groups INFO (Carpet): There are 820 grid scalars in 78 groups INFO (Carpet): There are 111 1-dimensional grid arrays in 10 groups INFO (Carpet): There are 211 2-dimensional grid arrays in 18 groups INFO (Carpet): There are 3 3-dimensional grid arrays in 1 groups INFO (Carpet): (The number of variables counts all time levels) INFO (CarpetIOHDF5): I/O Method 'IOHDF5' registered: AMR output of grid variables to HDF5 files INFO (CarpetIOHDF5): Periodic AMR output requested for: GRID::x GRID::y GRID::z GRID::r ML_BSSN::phi WEYLSCAL4::Psi4r WEYLSCAL4::Psi4i WEYLSCAL4::curvIr WEYLSCAL4::curvIi WEYLSCAL4::curvJr WEYLSCAL4::curvJi INFO (CarpetIOHDF5): I/O Method 'IOHDF5_0D' registered: 0D AMR output of grid variables to HDF5 files INFO (CarpetIOHDF5): I/O Method 'IOHDF5_1D' registered: 1D AMR output of grid variables to HDF5 files INFO (CarpetIOHDF5): I/O Method 'IOHDF5_2D' registered: 2D AMR output of grid variables to HDF5 files INFO (CarpetIOHDF5): Periodic 2D AMR output requested for: GRID::x GRID::y GRID::z GRID::r ML_BSSN::phi ML_BSSN::trK ML_BSSN::alpha WEYLSCAL4::Psi4r WEYLSCAL4::Psi4i INFO (CarpetIOHDF5): I/O Method 'IOHDF5_3D' registered: 3D AMR output of grid variables to HDF5 files INFO (CarpetIOScalar): Periodic scalar output requested for: SYSTEMSTATISTICS::maxrss_mb SYSTEMSTATISTICS::majflt_mb SYSTEMSTATISTICS::arena_mb SYSTEMSTATISTICS::ordblks_mb SYSTEMSTATISTICS::hblks_mb SYSTEMSTATISTICS::hblkhd_mb SYSTEMSTATISTICS::uordblks_mb SYSTEMSTATISTICS::fordblks_mb SYSTEMSTATISTICS::keepcost_mb SYSTEMSTATISTICS::swap_used_mb INFO (CarpetIOASCII): I/O Method 'IOASCII_0D' registered: 0D AMR output of grid variables to ASCII files INFO (CarpetIOASCII): Periodic 0D AMR output requested for: CARPET::physical_time_per_hour CARPET::current_physical_time_per_hour CARPET::time_total CARPET::time_evolution CARPET::time_computing CARPET::time_communicating CARPET::time_io CARPET::evolution_steps_count CARPET::local_grid_points_per_second CARPET::total_grid_points_per_second CARPET::local_grid_point_updates_count CARPET::total_grid_point_updates_count CARPET::local_interior_points_per_second CARPET::total_interior_points_per_second CARPET::local_interior_point_updates_count CARPET::total_interior_point_updates_count CARPET::io_per_second CARPET::io_bytes_per_second CARPET::io_bytes_ascii_per_second CARPET::io_bytes_binary_per_second CARPET::io_count CARPET::io_bytes_count CARPET::io_bytes_ascii_count CARPET::io_bytes_binary_count CARPET::comm_per_second CARPET::comm_bytes_per_second CARPET::comm_count CARPET::comm_bytes_count CARPET::time_levels CARPET::current_walltime CARPET::syncs_count PUNCTURETRACKER::pt_loc_t[0] PUNCTURETRACKER::pt_loc_t[1] PUNCTURETRACKER::pt_loc_t[2] PUNCTURETRACKER::pt_loc_t[3] PUNCTURETRACKER::pt_loc_t[4] PUNCTURETRACKER::pt_loc_t[5] PUNCTURETRACKER::pt_loc_t[6] PUNCTURETRACKER::pt_loc_t[7] PUNCTURETRACKER::pt_loc_t[8] PUNCTURETRACKER::pt_loc_t[9] PUNCTURETRACKER::pt_loc_x[0] PUNCTURETRACKER::pt_loc_x[1] PUNCTURETRACKER::pt_loc_x[2] PUNCTURETRACKER::pt_loc_x[3] PUNCTURETRACKER::pt_loc_x[4] PUNCTURETRACKER::pt_loc_x[5] PUNCTURETRACKER::pt_loc_x[6] PUNCTURETRACKER::pt_loc_x[7] PUNCTURETRACKER::pt_loc_x[8] PUNCTURETRACKER::pt_loc_x[9] PUNCTURETRACKER::pt_loc_y[0] PUNCTURETRACKER::pt_loc_y[1] PUNCTURETRACKER::pt_loc_y[2] PUNCTURETRACKER::pt_loc_y[3] PUNCTURETRACKER::pt_loc_y[4] PUNCTURETRACKER::pt_loc_y[5] PUNCTURETRACKER::pt_loc_y[6] PUNCTURETRACKER::pt_loc_y[7] PUNCTURETRACKER::pt_loc_y[8] PUNCTURETRACKER::pt_loc_y[9] PUNCTURETRACKER::pt_loc_z[0] PUNCTURETRACKER::pt_loc_z[1] PUNCTURETRACKER::pt_loc_z[2] PUNCTURETRACKER::pt_loc_z[3] PUNCTURETRACKER::pt_loc_z[4] PUNCTURETRACKER::pt_loc_z[5] PUNCTURETRACKER::pt_loc_z[6] PUNCTURETRACKER::pt_loc_z[7] PUNCTURETRACKER::pt_loc_z[8] PUNCTURETRACKER::pt_loc_z[9] QUASILOCALMEASURES::qlm_time[0] QUASILOCALMEASURES::qlm_time[1] QUASILOCALMEASURES::qlm_time[2] QUASILOCALMEASURES::qlm_equatorial_circumference[0] QUASILOCALMEASURES::qlm_equatorial_circumference[1] QUASILOCALMEASURES::qlm_equatorial_circumference[2] QUASILOCALMEASURES::qlm_polar_circumference_0[0] QUASILOCALMEASURES::qlm_polar_circumference_0[1] QUASILOCALMEASURES::qlm_polar_circumference_0[2] QUASILOCALMEASURES::qlm_polar_circumference_pi_2[0] QUASILOCALMEASURES::qlm_polar_circumference_pi_2[1] QUASILOCALMEASURES::qlm_polar_circumference_pi_2[2] QUASILOCALMEASURES::qlm_area[0] QUASILOCALMEASURES::qlm_area[1] QUASILOCALMEASURES::qlm_area[2] QUASILOCALMEASURES::qlm_irreducible_mass[0] QUASILOCALMEASURES::qlm_irreducible_mass[1] QUASILOCALMEASURES::qlm_irreducible_mass[2] QUASILOCALMEASURES::qlm_radius[0] QUASILOCALMEASURES::qlm_radius[1] QUASILOCALMEASURES::qlm_radius[2] QUASILOCALMEASURES::qlm_spin_guess[0] QUASILOCALMEASURES::qlm_spin_guess[1] QUASILOCALMEASURES::qlm_spin_guess[2] QUASILOCALMEASURES::qlm_mass_guess[0] QUASILOCALMEASURES::qlm_mass_guess[1] QUASILOCALMEASURES::qlm_mass_guess[2] QUASILOCALMEASURES::qlm_killing_eigenvalue_re[0] QUASILOCALMEASURES::qlm_killing_eigenvalue_re[1] QUASILOCALMEASURES::qlm_killing_eigenvalue_re[2] QUASILOCALMEASURES::qlm_killing_eigenvalue_im[0] QUASILOCALMEASURES::qlm_killing_eigenvalue_im[1] QUASILOCALMEASURES::qlm_killing_eigenvalue_im[2] QUASILOCALMEASURES::qlm_spin[0] QUASILOCALMEASURES::qlm_spin[1] QUASILOCALMEASURES::qlm_spin[2] QUASILOCALMEASURES::qlm_npspin[0] QUASILOCALMEASURES::qlm_npspin[1] QUASILOCALMEASURES::qlm_npspin[2] QUASILOCALMEASURES::qlm_wsspin[0] QUASILOCALMEASURES::qlm_wsspin[1] QUASILOCALMEASURES::qlm_wsspin[2] QUASILOCALMEASURES::qlm_cvspin[0] QUASILOCALMEASURES::qlm_cvspin[1] QUASILOCALMEASURES::qlm_cvspin[2] QUASILOCALMEASURES::qlm_coordspinx[0] QUASILOCALMEASURES::qlm_coordspinx[1] QUASILOCALMEASURES::qlm_coordspinx[2] QUASILOCALMEASURES::qlm_coordspiny[0] QUASILOCALMEASURES::qlm_coordspiny[1] QUASILOCALMEASURES::qlm_coordspiny[2] QUASILOCALMEASURES::qlm_coordspinz[0] QUASILOCALMEASURES::qlm_coordspinz[1] QUASILOCALMEASURES::qlm_coordspinz[2] QUASILOCALMEASURES::qlm_mass[0] QUASILOCALMEASURES::qlm_mass[1] QUASILOCALMEASURES::qlm_mass[2] QUASILOCALMEASURES::qlm_adm_energy[0] QUASILOCALMEASURES::qlm_adm_energy[1] QUASILOCALMEASURES::qlm_adm_energy[2] QUASILOCALMEASURES::qlm_adm_momentum_x[0] QUASILOCALMEASURES::qlm_adm_momentum_x[1] QUASILOCALMEASURES::qlm_adm_momentum_x[2] QUASILOCALMEASURES::qlm_adm_momentum_y[0] QUASILOCALMEASURES::qlm_adm_momentum_y[1] QUASILOCALMEASURES::qlm_adm_momentum_y[2] QUASILOCALMEASURES::qlm_adm_momentum_z[0] QUASILOCALMEASURES::qlm_adm_momentum_z[1] QUASILOCALMEASURES::qlm_adm_momentum_z[2] QUASILOCALMEASURES::qlm_adm_angular_momentum_x[0] QUASILOCALMEASURES::qlm_adm_angular_momentum_x[1] QUASILOCALMEASURES::qlm_adm_angular_momentum_x[2] QUASILOCALMEASURES::qlm_adm_angular_momentum_y[0] QUASILOCALMEASURES::qlm_adm_angular_momentum_y[1] QUASILOCALMEASURES::qlm_adm_angular_momentum_y[2] QUASILOCALMEASURES::qlm_adm_angular_momentum_z[0] QUASILOCALMEASURES::qlm_adm_angular_momentum_z[1] QUASILOCALMEASURES::qlm_adm_angular_momentum_z[2] QUASILOCALMEASURES::qlm_w_energy[0] QUASILOCALMEASURES::qlm_w_energy[1] QUASILOCALMEASURES::qlm_w_energy[2] QUASILOCALMEASURES::qlm_w_momentum_x[0] QUASILOCALMEASURES::qlm_w_momentum_x[1] QUASILOCALMEASURES::qlm_w_momentum_x[2] QUASILOCALMEASURES::qlm_w_momentum_y[0] QUASILOCALMEASURES::qlm_w_momentum_y[1] QUASILOCALMEASURES::qlm_w_momentum_y[2] QUASILOCALMEASURES::qlm_w_momentum_z[0] QUASILOCALMEASURES::qlm_w_momentum_z[1] QUASILOCALMEASURES::qlm_w_momentum_z[2] QUASILOCALMEASURES::qlm_w_angular_momentum_x[0] QUASILOCALMEASURES::qlm_w_angular_momentum_x[1] QUASILOCALMEASURES::qlm_w_angular_momentum_x[2] QUASILOCALMEASURES::qlm_w_angular_momentum_y[0] QUASILOCALMEASURES::qlm_w_angular_momentum_y[1] QUASILOCALMEASURES::qlm_w_angular_momentum_y[2] QUASILOCALMEASURES::qlm_w_angular_momentum_z[0] QUASILOCALMEASURES::qlm_w_angular_momentum_z[1] QUASILOCALMEASURES::qlm_w_angular_momentum_z[2] INFO (CarpetIOASCII): I/O Method 'IOASCII_1D' registered: 1D AMR output of grid variables to ASCII files INFO (CarpetIOASCII): Periodic 1D AMR output requested for: ML_BSSN::phi ML_BSSN::trK WEYLSCAL4::Psi4r INFO (CarpetIOASCII): I/O Method 'IOASCII_2D' registered: 2D AMR output of grid variables to ASCII files INFO (CarpetIOASCII): I/O Method 'IOASCII_3D' registered: 3D AMR output of grid variables to ASCII files -------------------------------------------------------------------------------- INFO (Formaline): Writing tarballs with the Cactus sources into the directory "GW150914_28/cactus-source" INFO (MoL): Using Runge-Kutta 4 as the time integrator. INFO (SymBase): Symmetry on lower x-face: multipatch INFO (SymBase): Symmetry on upper x-face: multipatch INFO (SymBase): Symmetry on lower y-face: multipatch INFO (SymBase): Symmetry on upper y-face: multipatch INFO (SymBase): Symmetry on lower z-face: multipatch INFO (SymBase): Symmetry on upper z-face: multipatch INFO (MoL): The maximum number of evolved variables is 1404. 25 are registered. INFO (MoL): The maximum number of slow evolved variables is 1404. 0 are registered. INFO (MoL): The maximum number of constrained variables is 1404. 20 are registered. INFO (MoL): The maximum number of SandR variables is 1404. 0 are registered. INFO (MoL): The maximum number of evolved array variables is 1404. 0 are registered. INFO (MoL): The maximum number of constrained array variables is 1404. 0 are registered. INFO (MoL): The maximum number of SandR array variables is 1404. 0 are registered. INFO (MoL): The maximum size of any array variables is 0. INFO (Vectors): Testing vectorisation... [errors may result in segfaults] INFO (Vectors): 375/375 tests passed INFO (CarpetRegrid2): Enforcing grid structure properties, iteration 0 INFO (CarpetRegrid2): Enforcing grid structure properties, iteration 1 INFO (CarpetRegrid2): Enforcing grid structure properties, iteration 0 INFO (CarpetRegrid2): Enforcing grid structure properties, iteration 1 INFO (CarpetRegrid2): Enforcing grid structure properties, iteration 0 INFO (CarpetRegrid2): Enforcing grid structure properties, iteration 1 INFO (CarpetRegrid2): Enforcing grid structure properties, iteration 0 INFO (CarpetRegrid2): Enforcing grid structure properties, iteration 1 INFO (CarpetRegrid2): Enforcing grid structure properties, iteration 0 INFO (CarpetRegrid2): Enforcing grid structure properties, iteration 1 INFO (CarpetRegrid2): Enforcing grid structure properties, iteration 0 INFO (CarpetRegrid2): Enforcing grid structure properties, iteration 1 INFO (Carpet): Grid structure (superregions, grid points): [0][0][0] exterior: [0,0,0] : [22,22,16] ([23,23,17] + PADDING) 8993 [1][0][0] exterior: [8,8,5] : [36,36,24] ([29,29,20] + PADDING) 16820 [2][0][0] exterior: [24,24,15] : [64,64,40] ([41,41,26] + PADDING) 43706 [3][0][0] exterior: [56,56,35] : [120,120,72] ([65,65,38] + PADDING) 160550 [4][0][0] exterior: [120,136,75] : [232,216,120] ([113,81,46] + PADDING) 421038 [5][0][0] exterior: [309,320,155] : [393,384,192] ([85,65,38] + PADDING) 209950 [6][0][0] exterior: [666,687,315] : [738,721,337] ([73,35,23] + PADDING) 58765 INFO (Carpet): Grid structure (superregions, coordinates): [0][0][0] exterior: [-188.455384615385,-188.455384615385,-85.6615384615385] : [188.455384615385,188.455384615385,188.455384615385] : [17.1323076923077,17.1323076923077,17.1323076923077] [1][0][0] exterior: [-119.926153846154,-119.926153846154,-42.8307692307692] : [119.926153846154,119.926153846154,119.926153846154] : [8.56615384615385,8.56615384615385,8.56615384615385] [2][0][0] exterior: [-85.6615384615385,-85.6615384615385,-21.4153846153846] : [85.6615384615384,85.6615384615384,85.6615384615385] : [4.28307692307692,4.28307692307692,4.28307692307692] [3][0][0] exterior: [-68.5292307692308,-68.5292307692308,-10.7076923076923] : [68.5292307692308,68.5292307692308,68.5292307692308] : [2.14153846153846,2.14153846153846,2.14153846153846] [4][0][0] exterior: [-59.9630769230769,-42.8307692307692,-5.35384615384615] : [59.9630769230769,42.8307692307692,42.8307692307692] : [1.07076923076923,1.07076923076923,1.07076923076923] [5][0][0] exterior: [-23.0215384615385,-17.1323076923077,-2.67692307692307] : [21.9507692307692,17.1323076923077,17.1323076923077] : [0.535384615384615,0.535384615384615,0.535384615384615] [6][0][0] exterior: [-10.1723076923077,-4.55076923076925,-1.33846153846154] : [9.10153846153844,4.55076923076922,4.55076923076923] : [0.267692307692308,0.267692307692308,0.267692307692308] INFO (Carpet): Grid structure (superregions, grid points): [0][1][0] exterior: [0,0,0] : [15,20,138] ([16,21,139] + PADDING) 46704 INFO (Carpet): Grid structure (superregions, coordinates): [0][1][0] exterior: [-1.96349540849362,-3.92699081698724,-85.6615384615385] : [3.92699081698724,3.92699081698724,2278.59692307692] : [0.392699081698724,0.392699081698724,17.1323076923077] INFO (Carpet): Grid structure (superregions, grid points): [0][2][0] exterior: [0,0,0] : [15,20,138] ([16,21,139] + PADDING) 46704 INFO (Carpet): Grid structure (superregions, coordinates): [0][2][0] exterior: [-3.92699081698724,-3.92699081698724,-85.6615384615385] : [1.96349540849362,3.92699081698724,2278.59692307692] : [0.392699081698724,0.392699081698724,17.1323076923077] INFO (Carpet): Grid structure (superregions, grid points): [0][3][0] exterior: [0,0,0] : [15,20,138] ([16,21,139] + PADDING) 46704 INFO (Carpet): Grid structure (superregions, coordinates): [0][3][0] exterior: [-1.96349540849362,-3.92699081698724,-85.6615384615385] : [3.92699081698724,3.92699081698724,2278.59692307692] : [0.392699081698724,0.392699081698724,17.1323076923077] INFO (Carpet): Grid structure (superregions, grid points): [0][4][0] exterior: [0,0,0] : [15,20,138] ([16,21,139] + PADDING) 46704 INFO (Carpet): Grid structure (superregions, coordinates): [0][4][0] exterior: [-3.92699081698724,-3.92699081698724,-85.6615384615385] : [1.96349540849362,3.92699081698724,2278.59692307692] : [0.392699081698724,0.392699081698724,17.1323076923077] INFO (Carpet): Grid structure (superregions, grid points): [0][5][0] exterior: [0,0,0] : [20,20,138] ([21,21,139] + PADDING) 61299 INFO (Carpet): Grid structure (superregions, coordinates): [0][5][0] exterior: [-3.92699081698724,-3.92699081698724,-85.6615384615385] : [3.92699081698724,3.92699081698724,2278.59692307692] : [0.392699081698724,0.392699081698724,17.1323076923077] INFO (Carpet): Global grid structure statistics: INFO (Carpet): GF: rhs: 33k active, 118k owned (+261%), 210k total (+78%), 64 steps/time INFO (Carpet): GF: vars: 305, pts: 100M active, 221M owned (+122%), 431M total (+95%), 1.0 comp/proc INFO (Carpet): GA: vars: 1126, pts: 2M active, 2M total (+0%) INFO (Carpet): Total required memory: 3.456 GByte (for GAs and currently active GFs) INFO (Carpet): Load balance: min avg max sdv max/avg-1 INFO (Carpet): Level 0: 16M 16M 16M 0M owned 0% INFO (Carpet): Level 1: 1M 1M 1M 0M owned 0% INFO (Carpet): Level 2: 5M 5M 5M 0M owned 0% INFO (Carpet): Level 3: 26M 26M 26M 0M owned 0% INFO (Carpet): Level 4: 104M 104M 104M 0M owned 0% INFO (Carpet): Level 5: 56M 56M 56M 0M owned 0% INFO (Carpet): Level 6: 14M 14M 14M 0M owned 0% INFO (CartGrid3D): Grid Spacings: INFO (CartGrid3D): dx=>1.7132308e+01 dy=>1.7132308e+01 dz=>1.7132308e+01 INFO (CartGrid3D): Computational Coordinates: INFO (CartGrid3D): x=>[-188.455,188.455] y=>[-188.455,188.455] z=>[-85.662,188.455] INFO (CartGrid3D): Indices of Physical Coordinates: INFO (CartGrid3D): x=>[0,22] y=>[0,22] z=>[0,16] WARNING[L3,P0] (Cactus): CCTK_CoordRange: Range already registered for system 'cart3d' WARNING[L3,P0] (Cactus): CCTK_CoordRegisterRangePhysIndex: Range already registered for system 'cart3d' WARNING[L3,P0] (Cactus): CCTK_CoordRange: Range already registered for system 'cart3d' WARNING[L3,P0] (Cactus): CCTK_CoordRegisterRangePhysIndex: Range already registered for system 'cart3d' WARNING[L3,P0] (Cactus): CCTK_CoordRange: Range already registered for system 'cart3d' WARNING[L3,P0] (Cactus): CCTK_CoordRegisterRangePhysIndex: Range already registered for system 'cart3d' INFO (CartGrid3D): Grid Spacings: INFO (CartGrid3D): dx=>3.9269908e-01 dy=>3.9269908e-01 dz=>1.7132308e+01 INFO (CartGrid3D): Computational Coordinates: INFO (CartGrid3D): x=>[-1.963, 3.927] y=>[-3.927, 3.927] z=>[-85.662,2278.597] INFO (CartGrid3D): Indices of Physical Coordinates: INFO (CartGrid3D): x=>[0,15] y=>[0,20] z=>[0,138] WARNING[L3,P0] (Cactus): CCTK_CoordRange: Range already registered for system 'cart3d' WARNING[L3,P0] (Cactus): CCTK_CoordRegisterRangePhysIndex: Range already registered for system 'cart3d' WARNING[L3,P0] (Cactus): CCTK_CoordRange: Range already registered for system 'cart3d' WARNING[L3,P0] (Cactus): CCTK_CoordRegisterRangePhysIndex: Range already registered for system 'cart3d' WARNING[L3,P0] (Cactus): CCTK_CoordRange: Range already registered for system 'cart3d' WARNING[L3,P0] (Cactus): CCTK_CoordRegisterRangePhysIndex: Range already registered for system 'cart3d' INFO (CartGrid3D): Grid Spacings: INFO (CartGrid3D): dx=>3.9269908e-01 dy=>3.9269908e-01 dz=>1.7132308e+01 INFO (CartGrid3D): Computational Coordinates: INFO (CartGrid3D): x=>[-3.927, 1.963] y=>[-3.927, 3.927] z=>[-85.662,2278.597] INFO (CartGrid3D): Indices of Physical Coordinates: INFO (CartGrid3D): x=>[0,15] y=>[0,20] z=>[0,138] WARNING[L3,P0] (Cactus): CCTK_CoordRange: Range already registered for system 'cart3d' WARNING[L3,P0] (Cactus): CCTK_CoordRegisterRangePhysIndex: Range already registered for system 'cart3d' WARNING[L3,P0] (Cactus): CCTK_CoordRange: Range already registered for system 'cart3d' WARNING[L3,P0] (Cactus): CCTK_CoordRegisterRangePhysIndex: Range already registered for system 'cart3d' WARNING[L3,P0] (Cactus): CCTK_CoordRange: Range already registered for system 'cart3d' WARNING[L3,P0] (Cactus): CCTK_CoordRegisterRangePhysIndex: Range already registered for system 'cart3d' INFO (CartGrid3D): Grid Spacings: INFO (CartGrid3D): dx=>3.9269908e-01 dy=>3.9269908e-01 dz=>1.7132308e+01 INFO (CartGrid3D): Computational Coordinates: INFO (CartGrid3D): x=>[-1.963, 3.927] y=>[-3.927, 3.927] z=>[-85.662,2278.597] INFO (CartGrid3D): Indices of Physical Coordinates: INFO (CartGrid3D): x=>[0,15] y=>[0,20] z=>[0,138] WARNING[L3,P0] (Cactus): CCTK_CoordRange: Range already registered for system 'cart3d' WARNING[L3,P0] (Cactus): CCTK_CoordRegisterRangePhysIndex: Range already registered for system 'cart3d' WARNING[L3,P0] (Cactus): CCTK_CoordRange: Range already registered for system 'cart3d' WARNING[L3,P0] (Cactus): CCTK_CoordRegisterRangePhysIndex: Range already registered for system 'cart3d' WARNING[L3,P0] (Cactus): CCTK_CoordRange: Range already registered for system 'cart3d' WARNING[L3,P0] (Cactus): CCTK_CoordRegisterRangePhysIndex: Range already registered for system 'cart3d' INFO (CartGrid3D): Grid Spacings: INFO (CartGrid3D): dx=>3.9269908e-01 dy=>3.9269908e-01 dz=>1.7132308e+01 INFO (CartGrid3D): Computational Coordinates: INFO (CartGrid3D): x=>[-3.927, 1.963] y=>[-3.927, 3.927] z=>[-85.662,2278.597] INFO (CartGrid3D): Indices of Physical Coordinates: INFO (CartGrid3D): x=>[0,15] y=>[0,20] z=>[0,138] WARNING[L3,P0] (Cactus): CCTK_CoordRange: Range already registered for system 'cart3d' WARNING[L3,P0] (Cactus): CCTK_CoordRegisterRangePhysIndex: Range already registered for system 'cart3d' WARNING[L3,P0] (Cactus): CCTK_CoordRange: Range already registered for system 'cart3d' WARNING[L3,P0] (Cactus): CCTK_CoordRegisterRangePhysIndex: Range already registered for system 'cart3d' WARNING[L3,P0] (Cactus): CCTK_CoordRange: Range already registered for system 'cart3d' WARNING[L3,P0] (Cactus): CCTK_CoordRegisterRangePhysIndex: Range already registered for system 'cart3d' INFO (CartGrid3D): Grid Spacings: INFO (CartGrid3D): dx=>3.9269908e-01 dy=>3.9269908e-01 dz=>1.7132308e+01 INFO (CartGrid3D): Computational Coordinates: INFO (CartGrid3D): x=>[-3.927, 3.927] y=>[-3.927, 3.927] z=>[-85.662,2278.597] INFO (CartGrid3D): Indices of Physical Coordinates: INFO (CartGrid3D): x=>[0,20] y=>[0,20] z=>[0,138] INFO (AHFinderDirect): setting up AHFinderDirect data structures INFO (AHFinderDirect): to search for 3 horizons on 1 processor INFO (AHFinderDirect): constructing full sphere patch system INFO (AHFinderDirect): at origin (4.46154,0,0) INFO (AHFinderDirect): with 18 angular zones per right angle INFO (AHFinderDirect): ==> 2166 nominal, 3174 ghosted angular grid points INFO (AHFinderDirect): constructing full sphere patch system INFO (AHFinderDirect): at origin (-5.53846,0,0) INFO (AHFinderDirect): with 18 angular zones per right angle INFO (AHFinderDirect): ==> 2166 nominal, 3174 ghosted angular grid points INFO (AHFinderDirect): constructing full sphere patch system INFO (AHFinderDirect): at origin (0,0,0) INFO (AHFinderDirect): with 18 angular zones per right angle INFO (AHFinderDirect): ==> 2166 nominal, 3174 ghosted angular grid points WARNING level 0 from host panther process 0 in thorn CoordinatesSymmetry, file /system/user/crangano/einstein_toolkit/Cactus/configs/sim/build/CoordinatesSymmetry/apply.c:665: -> Cannot apply symmetry boundary zones in the lower x direction, since there seem to be more symmetry zones than interior zones Simfactory Done at date: Wed 10 Sep 2025 05:10:09 PM CEST From rhaas at mail.ubc.ca Mon Sep 15 13:16:04 2025 From: rhaas at mail.ubc.ca (Roland Haas) Date: Mon, 15 Sep 2025 11:16:04 -0700 Subject: [Users] Issues Running BNS and GW150914 Example Simulations + storing BSSN variables In-Reply-To: <2dd52b48-05bb-4830-a119-ea83945babcb@ml.jku.at> References: <2dd52b48-05bb-4830-a119-ea83945babcb@ml.jku.at> Message-ID: <20250915111604.52e63d18@haengie2.phas.ubc.ca> Hello Sandeep, > *BNS: *On running the command ./simfactory/bin/sim create-submit bns --parfile bns.par --procs= --num-threads= --walltime=xx:xx:xx > > I encounter the error: WARNING level 1 from host panther process 0 > ? in thorn ML_BSSN_Helper, file /system/user/crangano/einstein_toolkit/BNS/configs/sim/build/ML_BSSN_Helper/Parameters.c:166: > ? -> Parameter ML_BSSN::my_boundary_condition is outdated; please update the parameter file. Do not use this parameter, and set up RHS boundary conditions as usual. The "please update the parameter file" are just warnings that you can ignore. There is no way to write Jacobian or Hessian of the variables on the grid to disk. You would have to introduce an explicit grid function that stores those derivatives and write it to disk. This would however by quite expensive memory and disk storage wise. Usually derivatives (even for posprocessing data after a simulation) are computed on the fly. Since all grid patches are using uniform resolution (in the Berger-Oliger mesh refinement scheme used by Carpet) can be compute easily in postprocessing using just the information stored in the HDF5 files. > cactus_sim: grille3d.C:125: Grille3d::Grille3d(int, int, int, int, int, int, int): Assertion `nr > 0' failed. > Rank 0 with PID 3273101 received signal 6 THis looks like an error from LORENE to me (since it contains French language words). Are you using the version of LORENE included with the Einstein Toolkit or are you using a copy that you have compiled yourself? > For this I have attached the?par file,?err file, the out file (the > initial data file is the one that is already there online: > G2_I12vs12_D4R33T21_45km.resu.xz > ) > that I am currently using. I tried to find the `nr` in the par file, > but didn't find it. I am not sure if this is a MPI/num procs related > problem, or is it something in the .par file that one needs to be > change ? Since this error originates from within LORENE (which has its own grid that it used to solve the initial data constraint problem) that parameter "nr" is internal to LORENE and not handled by the Einstein Toolkit at all. Comparing your parameter file with the one available on the Einstein Toolkit page I notice some more changes that were done to it. For example CoordBase::ymin has been changed (to 0.0) and so has the resolution (coarser). Seeting ymin to 0.0 but keeping the RotatingSymmetry180 thorn active is almost certainly incorrect. Can you confirm that the unmodified bns.par file downloaded directly from https://einsteintoolkit.org/gallery/bns/index.html also fails for you? If not then I would try and slowly change one parameter at a time to isolate the direct cause of the issue. > *GW150914: */Error 1 -- /Here, it is much more complicated to understand for me.?This persists even after disabling |CoordinatesSymmetry::reflection_z|.?Is it likely caused by coarse grid resolution relative to the number of symmetry/ghost zones. > > /Error 2/ -- Another error that I often encounter is File "/system/user/crangano/simulations/GW150914_28/output-0000/GW150914.rpar", line 126, in > sphere_outer_radius = int((outermost_detector + final_time)/(i*hr))*i*hr > ZeroDivisionError: float division by zero > Error: Error while executing parameter file script /system/user/crangano/simulations/GW150914_28/output-0000/GW150914.rpar That is an error that comes out of Simfactory. Since the `rpar` file is a Python script Simfactory will execute it as a Python code to get the par file. Since you are attaching a par file I assume that this worked at least once. Is this all on your workstation? In that case I am somewhat unsure why there would be a difference between running manually and running inside of a job. You only have one instance of Python installed, via your OS's package manager (so no conda or similar additional package manager), no Python virtualenv that you are using? I notice that that the parfile that you are attaching differs from the one produced by the (current) GW150914.rpar on the gallery example page. Namely it changes the `reflection_z` parameter (and nothing else). Similar to the Rotating180 issue above, you most likely have to adjust some other parameters as well (eg the Coordinates::symmetry option). > Aborting Simfactory. -- I guess this occurs in the following line of the *GW150914.rpar* > > sphere_outer_radius=int((outermost_detector+final_time)/(i*hr))*i*hr > sphere_outer_radius=int(sphere_outer_radius/hr) *hr+hr# round up to a multiple of hr > > So, I changed line 78 of the *GW150914.rpar as follows: * > > # Number of cells across finest grid radius > n=int("@N@") if"@N@"[0] !="@"else28 > i=max(int(n/4), 1) > > Is this a valid fix, or does this affect the simulations ? If your "n" was less than 4 so that i was 0 then your simulation will fail due to being way underresolved. The minimum sensible n is somewhere close to 24. Could you provide your GW150915.rpar file, please? > For this too, I attach the err, out and the par file such that you can inspect it. > > *Storing BSSN variables*:We would like to store the BSSN evolved > *3-metric* (|?_ij|), *lapse* (|?|), and *shift* (|?^i|) and also the > corresponding coordinates (t, x^i) of these quantities? at regular > intervals (e.g., every 128 steps) to manage disk usage efficiently. > Moreover, since AMR is used, is there any way we can keep track of > the changes in resolution of the coordinates, since we also aim to do > spatial derivatives via our FD stencils -- Jacobians and Hessians of > these quantities. If there are ways to directly dump the Jacobians > and Hessians of these quantities (by adding lines on the par file), > w/o us implementing (since we are not aware of the AMR being used at > different regions to implement our FD stencils), that would be very > useful too. Unfortunately the only way to write out values of the Jacobian or Hessian would be to actually create grid functions for them, since we only compute those "on the fly" as they are required to compute the RHS of the evolution equations. They would be quite memory and disk space intensive though. If derivatives are required, even during postprocessing then usually they would be re-computed. If you'd like to track when the grid has changed then you could try and monitor Carpet's GetRegriddingEpoch aliased function. Or you could try and schedule a routine at the POSTREGRID bin and use CCTK_OutputVarAsByMethod https://einsteintoolkit.org/referencemanual/ReferenceManual.html#x1-148000doc to output the coordinates when the grid changes. Note that for a Cartesian simulation (not using Llama) one can compute the coordinates of each point using the "origin" and "delta", "ioffset" and "ioffsetdenom" attributes of the HDF5 datasets. > IOHDF5::out_vars = " ML_BSSN::ML_metric ML_BSSN::ML_lapse > ML_BSSN::ML_shift Grid::Coordinates{out_every=1000000000 > refinement_levels={0}} ML_BSSN::ML_log_confac WeylScal4::Psi4r > WeylScal4::Psi4i WeylScal4::curvIr{refinement_levels={3 5}} > WeylScal4::curvIi{refinement_levels={3 5}} > WeylScal4::curvJr{refinement_levels={3 5}} You can add out_every inside of the curly braces (instead of using say CarpetIOHDF5::out_every which would set this globally) the way you see it done for Coordinates. So you already are using that, and likely are ok with this capability? Yours, Roland -- My email is as private as my paper mail. I therefore support encrypting and signing email messages. Get my PGP key from http://pgp.mit.edu . From users at einsteintoolkit.org Mon Sep 15 15:18:02 2025 From: users at einsteintoolkit.org (users at einsteintoolkit.org) Date: Mon, 15 Sep 2025 15:18:02 -0500 Subject: [Users] Agenda for Thursday's Meeting Message-ID: <68c8747a.VSnHKahAB0R21Cf+%users@einsteintoolkit.org> Please update the Wiki with agenda items for Thursday's meeting. Thanks! https://docs.einsteintoolkit.org/et-docs/meeting_agenda --The Maintainers From cranganore at ml.jku.at Mon Sep 15 17:25:31 2025 From: cranganore at ml.jku.at (Sandeep Suresh CrangaNORE) Date: Tue, 16 Sep 2025 00:25:31 +0200 Subject: [Users] Issues Running BNS and GW150914 Example Simulations + storing BSSN variables In-Reply-To: <20250915111604.52e63d18@haengie2.phas.ubc.ca> References: <2dd52b48-05bb-4830-a119-ea83945babcb@ml.jku.at> <20250915111604.52e63d18@haengie2.phas.ubc.ca> Message-ID: <88fd1864-921e-40ab-9eca-70ebdb5e030f@ml.jku.at> Hello Roland, Many thanks for your detailed email and the potential fixes. I shall intersperse through your message: *BNS:* << Comparing your parameter file with the one available on the Einstein << Toolkit page I notice some more changes that were done to it. For << example CoordBase::ymin has been changed (to 0.0) and so has the << resolution (coarser). << Seeting ymin to 0.0 but keeping the RotatingSymmetry180 thorn active is << almost certainly incorrect. << Can you confirm that the unmodified bns.par file downloaded directly from Apologies --the par file I sent was the one on which I tried some changes. For the current runs I am using the unmodified bns.par file, exactly as in https://einsteintoolkit.org/gallery/bns/index.html . Now, I no longer get the cactus_sim: grille3d.C:125: Grille3d::Grille3d(int, int, int, int, int, int, int): Assertion `nr > 0' failed error as earlier, but the following new error (cf. attached err file) fails my runs: ```Rank 0 with PID 3948578 received signal 11 Writing backtrace to bns/backtrace.0.txt /system/user/crangano/simulations/bns/output-0000/SIMFACTORY/RunScript: line 36: 3948578 Segmentation fault (core dumped) /system/user/crangano/simulations/bns/SIMFACTORY/exe/cactus_sim -L 3 /system/user/crangano/simulations/bns/output-0000/bns.par``` Yes, the Lorene codes are the same ones from EinsteinToolkit and not using a copy compiled by me. Pls again find the par, err and the out files for the same. *GW150914: *<< That is an error that comes out of Simfactory. Since the `rpar` file is << a Python script Simfactory will execute it as a Python code to get the << par file. I am not sure what you exactly mean here. I use python3 to run the rpar file and got the corresponding GW150914.par file from that. << Since you are attaching a par file I assume that this worked at least once. Is this all on your workstation? In that case I am somewhat unsure why there would be a difference between running manually and running inside of a job. You only have one instance of Python installed, via your OS's package manager (so no conda or similar additional package manager), no Python virtualenv that you are using? No, for this I am not using an venv or conda environment. The only thing that I export additionally is the correct mpi version on my clusters (check if the correct mpirun exists) and then just run << I notice that that the parfile that you are attaching differs from the << one produced by the (current) GW150914.rpar on the gallery example << page. Namely it changes the `reflection_z` parameter (and nothing << else). Similar to the Rotating180 issue above, you most likely have to << adjust some other parameters as well (eg the Coordinates::symmetry << option). Yes -- again I sent the tweaked one bymistake. The GW150914.rpar is again the same one found on EinsteinToolkit Gallery example -- https://bitbucket.org/einsteintoolkit/einsteinexamples/raw/master/par/GW150914/GW150914.rpar If I don't tweak the rpar file, then I get the following error: ``` Traceback (most recent call last): File "/system/user/crangano/simulations/GW150914_28/output-0000/GW150914.rpar", line 126, in sphere_outer_radius = int((outermost_detector + final_time)/(i*hr))*i*hr ZeroDivisionError: float division by zero Error: Error while executing parameter file script /system/user/crangano/simulations/GW150914_28/output-0000/GW150914.rpar Aborting Simfactory.``` ################################################################################################################################## /The only one *caveat* is that I am not using the following procedure to build and compile this usecase: / Configure SimFactory for your machine: * If you are on a cluster that SimFactory supports, run simfactory/bin/sim setup Hit enter for each question if the default is OK. If you intend to run on a cluster that requires an allocation, make sure to configure this during setup. * If you are not using a cluster supported by SimFactory, see Compiling the Einstein Toolkit for instructions. Compile Cactus (on 4 processes concurrently; you are encouraged to increase this value if you have more cores available and are impatient): simfactory/bin/sim build -j 4 --thornlist manifest/einsteintoolkit.th *But rather this method (suggested by one of the EinsteinToolkit team member in some previous emails): * cd par # this will create a file? GW150914.par, can also use python3 if needed python GW150914.rpar # back to the main Cactus directory cd .. # The following should be all in one line but the email client may # insert a line break utils/Scripts/MakeThornList --master thornlists/einsteintoolkit.th --output thornlists/GW150914.th par/GW150914.par # get rid of partially compiled code rm -r configs/sim ./simfactory/bin/sim build --thornlist thornlists/GW150914.th With this I am able to compile and build, but when I run the simulation it fails with the following error ################################################################################################################################# Again for this I attach the corresponding par, err and out files for the same. *Storing BSSN variables on disk *<< Unfortunately the only way to write out values of the Jacobian or << Hessian would be to actually create grid functions for them, since we << only compute those "on the fly" as they are required to compute the RHS << of the evolution equations. They would be quite memory and disk space << intensive though. << If derivatives are required, even during postprocessing then usually << they would be re-computed. If you'd like to track when the grid has << changed then you could try and monitor Carpet's GetRegriddingEpoch << aliased function. Or you could try and schedule a routine at the << POSTREGRID bin and use CCTK_OutputVarAsByMethod << https://einsteintoolkit.org/referencemanual/ReferenceManual.html#x1-148000doc << to output the coordinates when the grid changes. << Note that for a Cartesian simulation (not using Llama) one can compute << the coordinates of each point using the "origin" and "delta", "ioffset" << and "ioffsetdenom" attributes of the HDF5 datasets. Thanks for showing how we can dump the coordinates when grid changes, this is helpful. The Jacobian and Hessian are optional for us, we can always re-compute the Jacobian and Hessian as a post-processing step with our custom FD stencil implementation. Although, the main question is storing the metric $gamma_ij$ (3-metric), $\alpha$ (lapse function), $\beta^i$ (shift vector), $W$ (conformal factor) and the extrinsic curvature trace $K$, so the code snippet/additions on the par file pasted below does the trick to store on permanent disk storage right? . IOHDF5::out_vars = " ML_BSSN::ML_metric ML_BSSN::ML_lapse ML_BSSN::ML_shift ? (I have avoided the W, K in the below codes). > IOHDF5::out_vars = " ML_BSSN::ML_metric ML_BSSN::ML_lapse > ML_BSSN::ML_shift Grid::Coordinates{out_every=1000000000 > refinement_levels={0}} ML_BSSN::ML_log_confac WeylScal4::Psi4r > WeylScal4::Psi4i WeylScal4::curvIr{refinement_levels={3 5}} > WeylScal4::curvIi{refinement_levels={3 5}} > WeylScal4::curvJr{refinement_levels={3 5}} << You can add out_every inside of the curly braces (instead of using say CarpetIOHDF5::out_every which would set this globally) the way you see it done for Coordinates. So you already are using that, and likely are ok with this capability? Sure, sounds good -- although, I can only gauge ones the simulations run successfully. Many thanks again in advance for your help and happy to hear back from you with the potential fixes. Kind regards , Sandeep On 9/15/25 20:16, Roland Haas wrote: > Hello Sandeep, > >> *BNS: *On running the command ./simfactory/bin/sim create-submit bns --parfile bns.par --procs= --num-threads= --walltime=xx:xx:xx >> >> I encounter the error: WARNING level 1 from host panther process 0 >> ? in thorn ML_BSSN_Helper, file /system/user/crangano/einstein_toolkit/BNS/configs/sim/build/ML_BSSN_Helper/Parameters.c:166: >> ? -> Parameter ML_BSSN::my_boundary_condition is outdated; please update the parameter file. Do not use this parameter, and set up RHS boundary conditions as usual. > The "please update the parameter file" are just warnings that you can > ignore. > > There is no way to write Jacobian or Hessian of the variables on the > grid to disk. You would have to introduce an explicit grid function > that stores those derivatives and write it to disk. This would however > by quite expensive memory and disk storage wise. > > Usually derivatives (even for posprocessing data after a simulation) > are computed on the fly. Since all grid patches are using uniform > resolution (in the Berger-Oliger mesh refinement scheme used by Carpet) > can be compute easily in postprocessing using just the information > stored in the HDF5 files. > >> cactus_sim:grille3d.C:125:Grille3d::Grille3d(int, int, int, int, int, int, int): Assertion `nr > 0' failed. >> Rank 0 with PID 3273101 received signal 6 > THis looks like an error from LORENE to me (since it contains French > language words). Are you using the version of LORENE included with the > Einstein Toolkit or are you using a copy that you have compiled > yourself? > >> For this I have attached the?par file,?err file, the out file (the >> initial data file is the one that is already there online: >> G2_I12vs12_D4R33T21_45km.resu.xz >> ) >> that I am currently using. I tried to find the `nr` in the par file, >> but didn't find it. I am not sure if this is a MPI/num procs related >> problem, or is it something in the .par file that one needs to be >> change ? > Since this error originates from within LORENE (which has its own grid > that it used to solve the initial data constraint problem) that > parameter "nr" is internal to LORENE and not handled by the Einstein > Toolkit at all. > > Comparing your parameter file with the one available on the Einstein > Toolkit page I notice some more changes that were done to it. For > exampleCoordBase::ymin has been changed (to 0.0) and so has the > resolution (coarser). > > Seeting ymin to 0.0 but keeping the RotatingSymmetry180 thorn active is > almost certainly incorrect. > > Can you confirm that the unmodified bns.par file downloaded directly > from > > https://einsteintoolkit.org/gallery/bns/index.html > > also fails for you? If not then I would try and slowly change one > parameter at a time to isolate the direct cause of the issue. > >> *GW150914: */Error 1 -- /Here, it is much more complicated to understand for me.?This persists even after disabling |CoordinatesSymmetry::reflection_z|.?Is it likely caused by coarse grid resolution relative to the number of symmetry/ghost zones. >> >> /Error 2/ -- Another error that I often encounter is File "/system/user/crangano/simulations/GW150914_28/output-0000/GW150914.rpar", line 126, in >> sphere_outer_radius = int((outermost_detector + final_time)/(i*hr))*i*hr >> ZeroDivisionError: float division by zero >> Error: Error while executing parameter file script /system/user/crangano/simulations/GW150914_28/output-0000/GW150914.rpar > That is an error that comes out of Simfactory. Since the `rpar` file is > a Python script Simfactory will execute it as a Python code to get the > par file. > > Since you are attaching a par file I assume that this worked at least > once. Is this all on your workstation? In that case I am somewhat > unsure why there would be a difference between running manually and > running inside of a job. You only have one instance of Python > installed, via your OS's package manager (so no conda or similar > additional package manager), no Python virtualenv that you are using? > > I notice that that the parfile that you are attaching differs from the > one produced by the (current) GW150914.rpar on the gallery example > page. Namely it changes the `reflection_z` parameter (and nothing > else). Similar to the Rotating180 issue above, you most likely have to > adjust some other parameters as well (eg theCoordinates::symmetry > option). > > >> Aborting Simfactory. -- I guess this occurs in the following line of the *GW150914.rpar* >> >> sphere_outer_radius=int((outermost_detector+final_time)/(i*hr))*i*hr >> sphere_outer_radius=int(sphere_outer_radius/hr) *hr+hr# round up to a multiple of hr >> >> So, I changed line 78 of the *GW150914.rpar as follows: * >> >> # Number of cells across finest grid radius >> n=int("@N@") if"@N@"[0] !="@"else28 >> i=max(int(n/4), 1) >> >> Is this a valid fix, or does this affect the simulations ? > If your "n" was less than 4 so that i was 0 then your simulation will > fail due to being way underresolved. The minimum sensible n is > somewhere close to 24. > > Could you provide your GW150915.rpar file, please? > >> For this too, I attach the err, out and the par file such that you can inspect it. >> >> *Storing BSSN variables*:We would like to store the BSSN evolved >> *3-metric* (|?_ij|), *lapse* (|?|), and *shift* (|?^i|) and also the >> corresponding coordinates (t, x^i) of these quantities? at regular >> intervals (e.g., every 128 steps) to manage disk usage efficiently. >> Moreover, since AMR is used, is there any way we can keep track of >> the changes in resolution of the coordinates, since we also aim to do >> spatial derivatives via our FD stencils -- Jacobians and Hessians of >> these quantities. If there are ways to directly dump the Jacobians >> and Hessians of these quantities (by adding lines on the par file), >> w/o us implementing (since we are not aware of the AMR being used at >> different regions to implement our FD stencils), that would be very >> useful too. > Unfortunately the only way to write out values of the Jacobian or > Hessian would be to actually create grid functions for them, since we > only compute those "on the fly" as they are required to compute the RHS > of the evolution equations. They would be quite memory and disk space > intensive though. > > If derivatives are required, even during postprocessing then usually > they would be re-computed. If you'd like to track when the grid has > changed then you could try and monitor Carpet's GetRegriddingEpoch > aliased function. Or you could try and schedule a routine at the > POSTREGRID bin and use CCTK_OutputVarAsByMethod > > https://einsteintoolkit.org/referencemanual/ReferenceManual.html#x1-148000doc > > to output the coordinates when the grid changes. > > Note that for a Cartesian simulation (not using Llama) one can compute > the coordinates of each point using the "origin" and "delta", "ioffset" > and "ioffsetdenom" attributes of the HDF5 datasets. > >> IOHDF5::out_vars = " ML_BSSN::ML_metric ML_BSSN::ML_lapse >> ML_BSSN::ML_shift Grid::Coordinates{out_every=1000000000 >> refinement_levels={0}} ML_BSSN::ML_log_confac WeylScal4::Psi4r >> WeylScal4::Psi4i WeylScal4::curvIr{refinement_levels={3 5}} >> WeylScal4::curvIi{refinement_levels={3 5}} >> WeylScal4::curvJr{refinement_levels={3 5}} > You can add out_every inside of the curly braces (instead of using say > CarpetIOHDF5::out_every which would set this globally) the way you see > it done for Coordinates. So you already are using that, and likely > are ok with this capability? > > Yours, > Roland > -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- # Carpet parameter file for binary Neutron star system # physical ID is LORENE dataset G2_I12vs12_D4R33T21_45km.resu # #------------------------------------------------------------------------------ # Cactus parameters: #------------------------------------------------------------------------------ Cactus::cctk_run_title = "Meudon BNS" Cactus::cctk_full_warnings = "yes" Cactus::highlight_warning_messages = "no" Cactus::terminate = "time" Cactus::cctk_final_time = 2500.0 #------------------------------------------------------------------------------ # Activate all necessary thorns: #------------------------------------------------------------------------------ ActiveThorns = "Boundary CartGrid3D CoordBase Fortran InitBase IOUtil LocalReduce SymBase Time" ActiveThorns = "AEILocalInterp" ActiveThorns = "MoL Slab SpaceMask SphericalSurface" ActiveThorns = "Carpet CarpetInterp CarpetInterp2 CarpetIOASCII CarpetIOHDF5 CarpetIOScalar CarpetLib CarpetIOBasic CarpetReduce CarpetRegrid2 CarpetSlab CarpetTracker CarpetMask LoopControl" ActiveThorns = "Formaline" ActiveThorns = "HTTPD Socket" ActiveThorns = "NaNChecker TerminationTrigger TimerReport" ActiveThorns = "ADMbase ADMcoupling ADMmacros CoordGauge StaticConformal" ActiveThorns = "RotatingSymmetry180 ReflectionSymmetry" ActiveThorns = "Constants TmunuBase HydroBase " ActiveThorns = "QuasiLocalMeasures" ActiveThorns = "EOS_Omni" ActiveThorns = "GRHydro" ActiveThorns = "SummationByParts" ActiveThorns = "GenericFD NewRad" ActiveThorns = "ML_BSSN ML_BSSN_Helper ML_ADMConstraints" ActiveThorns = "Hydro_Analysis NSTracker" ActiveThorns = "Dissipation" ActiveThorns = "SystemStatistics SystemTopology" # Wave extraction (Psi4) ActiveThorns = "WeylScal4 Multipole" #------------------------------------------------------------------------------ # Diagnostic parameters: #------------------------------------------------------------------------------ Carpet::output_timers_every = 0 Carpet::storage_verbose = "no" Carpet::verbose = "no" Carpet::veryverbose = "no" Carpet::grid_structure_filename = "carpet-grid-structure" Carpet::grid_coordinates_filename = "carpet-grid-coordinates" CarpetLib::output_bboxes = "no" CarpetMask::verbose = "no" CarpetReduce::verbose = "no" CarpetRegrid2::verbose = "no" CarpetRegrid2::veryverbose = "no" CarpetTracker::verbose = "no" TimerReport::out_every = 4096 TimerReport::out_filename = "TimerReport" TimerReport::output_all_timers = "yes" TimerReport::output_all_timers_together = "yes" TimerReport::output_all_timers_readable = "yes" TimerReport::n_top_timers = 40 QuasiLocalMeasures::verbose = "no" SphericalSurface::verbose = "no" #------------------------------------------------------------------------------ # Utility parameters: #------------------------------------------------------------------------------ NaNChecker::check_every = 128 # twice for every_coarse NaNChecker::check_vars = " ADMBase::curv ADMBase::metric ADMBase::lapse ADMBase::shift HydroBase::rho HydroBase::eps HydroBase::press HydroBase::vel " NaNChecker::action_if_found = "terminate" #NaNChecker::action_if_found = "just warn" #"terminate", "just warn", "abort" #------------------------------------------------------------------------------ # Run parameters: #------------------------------------------------------------------------------ #------ # Grid: #------ MoL::ODE_Method = "rk4" MoL::MoL_Intermediate_Steps = 4 MoL::MoL_Num_Scratch_Levels = 1 # use dt = 0.4 dx (works for core collapse) Time::dtfac = 0.4 ActiveThorns = "CoordBase" CoordBase::domainsize = "minmax" CoordBase::xmin = 0.00 CoordBase::ymin = -400.00 CoordBase::zmin = 0.00 CoordBase::xmax = +400.00 CoordBase::ymax = +400.00 CoordBase::zmax = +400.00 CoordBase::dx = 8.00 CoordBase::dy = 8.00 CoordBase::dz = 8.00 CoordBase::boundary_size_x_lower = 3 CoordBase::boundary_size_y_lower = 3 CoordBase::boundary_size_z_lower = 3 CoordBase::boundary_size_x_upper = 3 CoordBase::boundary_size_y_upper = 3 CoordBase::boundary_size_z_upper = 3 CoordBase::boundary_shiftout_x_lower = 1 CoordBase::boundary_shiftout_y_lower = 0 CoordBase::boundary_shiftout_z_lower = 1 ReflectionSymmetry::reflection_z = "yes" ReflectionSymmetry::avoid_origin_z = "no" CartGrid3D::type = "coordbase" Carpet::domain_from_coordbase = "yes" Driver::ghost_size = 3 # General Carpet parameters: Carpet::enable_all_storage = "no" Carpet::use_buffer_zones = "yes" Carpet::schedule_barriers = "no" Carpet::poison_new_timelevels = "yes" Carpet::check_for_poison = "no" Carpet::init_3_timelevels = "no" Carpet::init_fill_timelevels = "yes" CarpetLib::poison_new_memory = "yes" CarpetLib::poison_value = 114 CarpetLib::check_bboxes = "no" CarpetLib::interleave_communications = "yes" CarpetLib::combine_sends = "yes" CarpetInterp::tree_search = "yes" CarpetInterp::check_tree_search = "no" CarpetRegrid2::freeze_unaligned_levels = "yes" CarpetRegrid2::freeze_unaligned_parent_levels = "yes" CarpetRegrid2::ensure_proper_nesting = "yes" CarpetRegrid2::snap_to_coarse = "yes" CarpetRegrid2::symmetry_rotating180 = "yes" # System specific Carpet parameters: Carpet::max_refinement_levels = 9 Carpet::prolongation_order_space = 5 Carpet::prolongation_order_time = 2 Carpet::refinement_centering = "vertex" CarpetRegrid2::regrid_every = 2 # as often as required CarpetRegrid2::num_centres = 3 # box sizes are given by: # * the stars seem to puff up to about 13M during the initial phase of the evolution # * I need 12 buffer points (RK4, 3 ghost zones) # * need three coarse points for interpolation onto last fine buffer point # these boxes are minimal in this sense. The coarser grid are completely # covered by the finer grids and their buffers. # add 4 coarse grid points in between to have some leeway against roundoff # grid step sizes are for coarsest anticipated simulation dx = 1.5M CarpetRegrid2::num_levels_1 = 7 CarpetRegrid2::position_x_1 = 15.1875 CarpetRegrid2::radius_1[1] =240.0 CarpetRegrid2::radius_1[2] =120.0 CarpetRegrid2::radius_1[3] = 60.0 CarpetRegrid2::radius_1[4] = 26.125 CarpetRegrid2::radius_1[5] = 17.875 CarpetRegrid2::radius_1[6] = 13 # star seems about 9.5 intially, then puffs up CarpetRegrid2::num_levels_2 = 7 CarpetRegrid2::position_x_2 = -15.1875 CarpetRegrid2::radius_2[1] =240.0 CarpetRegrid2::radius_2[2] =120.0 CarpetRegrid2::radius_2[3] = 60.0 CarpetRegrid2::radius_2[4] = 26.125 CarpetRegrid2::radius_2[5] = 17.875 CarpetRegrid2::radius_2[6] = 13 CarpetRegrid2::num_levels_3 = 1 CarpetRegrid2::radius_3[1] =240.0 CarpetRegrid2::radius_3[2] =120.0 CarpetRegrid2::radius_3[3] = 60.0 CarpetRegrid2::radius_3[4] = 30.0 CarpetRegrid2::radius_3[5] = 15.0 CarpetRegrid2::radius_3[6] = 7.5 CarpetRegrid2::radius_3[7] = 3.75 CarpetMask::excluded_surface [0] = 2 CarpetMask::excluded_surface_factor[0] = 1.0 CarpetTracker::surface_name[0] = "Righthand NS" CarpetTracker::surface_name[1] = "Lefthand NS" #------ # MODEL: #------ ActiveThorns = "Meudon_Bin_NS" HydroBase::initial_hydro = "Meudon_Bin_NS" ADMBase::initial_data = "Meudon_Bin_NS" ADMBase::initial_lapse = "Meudon_Bin_NS" ADMBase::initial_shift = "zero" ADMBase::initial_dtlapse = "Meudon_Bin_NS" ADMBase::initial_dtshift = "zero" # change this to be the full path to he initial data file Meudon_Bin_NS::filename ="/system/user/crangano/einstein_toolkit/BNS/bns.resu" # M_ADM 3.251 # separation 45km # K 123.6 # Gamma 2 EOS_Omni::poly_K = 123.613314525753 # For other (non-Polytropic) EOSs Meudon_Bin_NS::eos_table should be labelled # e.g. ="Hybrid" for hybrid EOS. #---------- # Numerics: #---------- InitBase::initial_data_setup_method = "init_some_levels" TmunuBase::stress_energy_storage = "yes" TmunuBase::stress_energy_at_RHS = "yes" TmunuBase::timelevels = 1 TmunuBase::prolongation_type = "none" TmunuBase::support_old_CalcTmunu_mechanism = "no" HydroBase::timelevels = 3 SpaceMask::use_mask = "yes" SphericalSurface::nsurfaces = 5 SphericalSurface::maxntheta = 39 SphericalSurface::maxnphi = 76 SphericalSurface::ntheta [0] = 39 SphericalSurface::nphi [0] = 76 SphericalSurface::nghoststheta[0] = 2 SphericalSurface::nghostsphi [0] = 2 SphericalSurface::name [0] = "Righthand NS" SphericalSurface::ntheta [1] = 39 SphericalSurface::nphi [1] = 76 SphericalSurface::nghoststheta[1] = 2 SphericalSurface::nghostsphi [1] = 2 SphericalSurface::name [1] = "Lefthand NS" SphericalSurface::ntheta [3] = 39 SphericalSurface::nphi [3] = 76 SphericalSurface::nghoststheta[3] = 2 SphericalSurface::nghostsphi [3] = 2 SphericalSurface::set_spherical[3] = yes SphericalSurface::radius [3] = 100 SphericalSurface::name [3] = "waveextract surface at 100" SphericalSurface::ntheta [4] = 39 SphericalSurface::nphi [4] = 76 SphericalSurface::nghoststheta[4] = 2 SphericalSurface::nghostsphi [4] = 2 SphericalSurface::set_spherical[4] = yes SphericalSurface::radius [4] = 250 SphericalSurface::name [4] = "waveextract surface at 250" #----------- # Evolution: #----------- HydroBase::evolution_method = "GRHydro" ADMMacros::spatial_order = 4 GRHydro::sources_spatial_order = 4 GRHydro::riemann_solver = "HLLE" # Marquina is currently not supported by MP GRHydro::recon_method = "ppm" GRHydro::GRHydro_stencil = 3 GRHydro::bound = "flat" GRHydro::rho_abs_min = 1.e-11 GRHydro::GRHydro_atmo_tolerance = 0.01 GRHydro::c2p_reset_pressure = "yes" GRHydro::GRHydro_eos_type = "General" GRHydro::GRHydro_eos_table = "Ideal_Fluid" # these can save some memory since they prevent MoL from allocating unnecessary # scratch space for saveandrestore variables GRHydro::GRHydro_MaxNumSandRVars = 0 GRHydro::use_enhanced_ppm = "yes" # Parameters are defaults, which in turn are from Colella & Sekora 2008 and # McCorquodale & Colella 2011 GRHydro::sync_conserved_only = "yes" GRHydro::reconstruct_Wv = "yes" GRHydro::c2p_resort_to_bisection = "yes" GRHydro::use_cxx_code = "yes" # MacLachlan evolution parameters ADMBase::metric_type = physical ADMBase::evolution_method = ML_BSSN ADMBase::lapse_evolution_method = ML_BSSN ADMBase::shift_evolution_method = ML_BSSN ADMBase::dtlapse_evolution_method = ML_BSSN ADMBase::dtshift_evolution_method = ML_BSSN ML_BSSN::timelevels = 3 ML_BSSN::my_initial_data = "ADMBase" ML_BSSN::my_initial_boundary_condition = "extrapolate-gammas" ML_BSSN::my_boundary_condition = "none" ML_BSSN::my_rhs_boundary_condition = "NewRad" Boundary::radpower = 2 # not really needed I think but who knows what NewRad uses ML_BSSN::harmonicN = 1 # 1+log ML_BSSN::harmonicF = 2.0 # 1+log ML_BSSN::ShiftGammaCoeff = 0.75 ML_BSSN::AlphaDriver = 0.0 ML_BSSN::BetaDriver = 1.0 ML_BSSN::LapseAdvectionCoeff = 1.0 ML_BSSN::ShiftAdvectionCoeff = 1.0 ML_BSSN::MinimumLapse = 1.0e-8 ML_BSSN::ML_log_confac_bound = "none" ML_BSSN::ML_metric_bound = "none" ML_BSSN::ML_Gamma_bound = "none" ML_BSSN::ML_trace_curv_bound = "none" ML_BSSN::ML_curv_bound = "none" ML_BSSN::ML_lapse_bound = "none" ML_BSSN::ML_dtlapse_bound = "none" ML_BSSN::ML_shift_bound = "none" ML_BSSN::ML_dtshift_bound = "none" ML_BSSN::UseSpatialBetaDriver = 1 ML_BSSN::SpatialBetaDriverRadius = 50 ML_BSSN::apply_dissipation = "never" Dissipation::epsdis = 0.1 Dissipation::order = 5 Dissipation::vars = " ML_BSSN::ML_log_confac ML_BSSN::ML_metric ML_BSSN::ML_trace_curv ML_BSSN::ML_curv ML_BSSN::ML_Gamma ML_BSSN::ML_lapse ML_BSSN::ML_shift ML_BSSN::ML_dtlapse ML_BSSN::ML_dtshift " #------------------------------------------------------------------------------ # Output: #------------------------------------------------------------------------------ IO::out_dir = $parfile IOBasic::outInfo_every = 1 IOBasic::outInfo_reductions = "maximum" IOBasic::outInfo_vars = " Carpet::physical_time_per_hour HydroBase::rho ML_ADMConstraints::ML_Ham SystemStatistics::maxrss_mb GRHydro::dens{reductions = 'sum maximum'} HydroBase::w_lorentz " IOScalar::outScalar_every = 256 # every_coarse IOScalar::all_reductions_in_one_file = "yes" IOScalar::one_file_per_group = "yes" IOScalar::outScalar_reductions = "minimum maximum average norm1 norm2" IOScalar::outScalar_vars = " ADMBase::lapse ADMBase::shift ADMBase::metric ADMBase::curv HydroBase::rho HydroBase::vel HydroBase::w_lorentz GRHydro::dens{reductions = 'minimum maximum average norm1 norm2 sum'} SystemStatistics::process_memory_mb SphericalSurface::sf_radius ML_ADMConstraints::ML_Ham " IOASCII::one_file_per_group = "yes" IOASCII::compact_format = "yes" IOASCII::out0D_every = 256 # every_coarse IOASCII::out0D_vars = " Carpet::timing QuasiLocalMeasures::qlm_scalars SphericalSurface::sf_active SphericalSurface::sf_valid SphericalSurface::sf_info SphericalSurface::sf_radius SphericalSurface::sf_origin SphericalSurface::sf_coordinate_descriptors Hydro_Analysis::Hydro_Analysis_rho_max_loc Hydro_Analysis::Hydro_Analysis_rho_max_origin_distance " #Set these IOASCII options for initial data only: IOASCII::out1D_every = 0 IOASCII::out1D_d = "no" IOASCII::out1D_vars = " HydroBase::rho HydroBase::vel ADMBase::lapse ADMBase::shift ADMBase::metric ADMBase::curv ML_ADMConstraints::ML_Ham " CarpetIOHDF5::one_file_per_group = "no" # this is required by multipatch CarpetIOHDF5::open_one_input_file_at_a_time = "yes" CarpetIOHDF5::out2D_every = 1536 # 6*every coarse CarpetIOHDF5::out2D_xy = "yes" CarpetIOHDF5::out2D_xz = "no" CarpetIOHDF5::out2D_yz = "no" CarpetIOHDF5::out2D_xyplane_z = 0.0 CarpetIOHDF5::out2D_vars = " CarpetReduce::weight Grid::coordinates HydroBase::rho HydroBase::vel HydroBase::eps ADMBase::lapse ADMBase::shift ADMBase::metric ML_ADMConstraints::ML_Ham # Hydro_Analysis::Hydro_Analysis_Temperature " IOHDF5::out3D_every = 8192 # = 32*every_coarse IOHDF5::out3D_vars = " CarpetReduce::weight HydroBase::rho HydroBase::vel HydroBase::eps ADMBase::lapse ADMBase::shift ML_ADMConstraints::ML_Ham grid::coordinates " #------------------------------------------------------------------------------ # Analysis: #------------------------------------------------------------------------------ Hydro_Analysis::Hydro_Analysis_comp_rho_max = "true" Hydro_Analysis::Hydro_Analysis_rho_max_loc_only_positive_x = "true" Hydro_Analysis::Hydro_Analysis_comp_rho_max_origin_distance = "yes" Hydro_Analysis::Hydro_Analysis_average_multiple_maxima_locations = "yes" Hydro_Analysis::Hydro_Analysis_interpolator_name = "Lagrange polynomial interpolation (tensor product)" NSTracker::NSTracker_SF_Name = "Righthand NS" NSTracker::NSTracker_SF_Name_Opposite = "Lefthand NS" NSTracker::NSTracker_max_distance = 3 NSTracker::NSTracker_verbose = "yes" NSTracker::NSTracker_tracked_location = "Hydro_Analysis::Hydro_Analysis_rho_max_loc" QuasiLocalMeasures::num_surfaces = 2 QuasiLocalMeasures::spatial_order = 4 QuasiLocalMeasures::interpolator = "Lagrange polynomial interpolation" QuasiLocalMeasures::interpolator_options = "order=4" QuasiLocalMeasures::surface_name[0] = "waveextract surface at 100" QuasiLocalMeasures::surface_name[1] = "waveextract surface at 250" ################################################################################ ################################################################################ # Wave extraction ################################################################################ ################################################################################ WeylScal4::offset = 1e-8 WeylScal4::fd_order = "4th" WeylScal4::verbose = 0 Multipole::nradii = 8 Multipole::out_every = 128 Multipole::radius[0] = 45 Multipole::radius[1] = 70 Multipole::radius[2] = 100 Multipole::radius[3] = 125 Multipole::radius[4] = 150 Multipole::radius[5] = 200 Multipole::radius[6] = 250 Multipole::radius[7] = 300 Multipole::variables = "WeylScal4::Psi4r{sw=-2 cmplx='WeylScal4::Psi4i' name='Psi4'}" Multipole::l_max = 6 #------------------------------------------------------------------------------ # Checkpoint/Recovery: #------------------------------------------------------------------------------ IOHDF5::checkpoint = "yes" IO::checkpoint_dir = $parfile IO::checkpoint_ID = "yes" IO::checkpoint_on_terminate = "yes" # disable extra con2prim in Post_Recover_Variables to ensure bit-identical # recovery from checkpoints MoL::run_MoL_PostStep_in_Post_Recover_Variables = "no" IO::recover = "autoprobe" IO::recover_dir = $parfile #------------------------------------------------------------------------------ # Control #------------------------------------------------------------------------------ HTTPD::user = "cactus" HTTPD::password = "einstein" ActiveThorns = "TerminationTrigger" TerminationTrigger::max_walltime = 0.3333333333333333 # Trigger termination 30 minutes before the walltime is reached TerminationTrigger::on_remaining_walltime = 30 TerminationTrigger::termination_from_file = yes TerminationTrigger::termination_file = "terminate.txt" TerminationTrigger::create_termination_file = yes ActiveThorns = "Trigger" Trigger::Trigger_Number = 4 Trigger::Trigger_Checked_Variable[0]="Hydro_Analysis::Hydro_Analysis_rho_max_origin_distance" Trigger::Trigger_Reduction [0]="" Trigger::Trigger_Relation [0]="<" Trigger::Trigger_Checked_Value [0]=10 Trigger::Trigger_Reaction [0]="steerscalar" Trigger::Trigger_Steered_Scalar [0] = "CarpetRegrid2::num_levels[2]" # == num_levels_3 Trigger::Trigger_Steered_Scalar_Value[0] = "7" Trigger::Trigger_Checked_Variable[1]="ADMBase::alp" Trigger::Trigger_Reduction [1]="minimum" Trigger::Trigger_Relation [1]="<" Trigger::Trigger_Checked_Value [1]=0.1 Trigger::Trigger_Reaction [1]="steerscalar" Trigger::Trigger_Steered_Scalar [1] = "CarpetRegrid2::num_levels[2]" # == num_levels_3 Trigger::Trigger_Steered_Scalar_Value[1] = "8" Trigger::Trigger_Checked_Variable[2]="SphericalSurface::sf_valid[2]" Trigger::Trigger_Reduction [2]="" Trigger::Trigger_Relation [2]=">" Trigger::Trigger_Checked_Value [2]=0. Trigger::Trigger_Reaction [2]="steerscalar" Trigger::Trigger_Steered_Scalar [2] = "CarpetRegrid2::num_levels[0]" # == num_levels_1 Trigger::Trigger_Steered_Scalar_Value[2] = "1" Trigger::Trigger_Checked_Variable[3]="SphericalSurface::sf_valid[2]" Trigger::Trigger_Reduction [3]="" Trigger::Trigger_Relation [3]=">" Trigger::Trigger_Checked_Value [3]=0. Trigger::Trigger_Reaction [3]="steerscalar" Trigger::Trigger_Steered_Scalar [3] = "CarpetRegrid2::num_levels[1]" # == num_levels_2 Trigger::Trigger_Steered_Scalar_Value[3] = "1" -------------- next part -------------- + set -e + cd /system/user/crangano/simulations/bns/output-0000-active + echo Checking: + pwd + hostname + date + echo Environment: + export CACTUS_NUM_PROCS=1 + CACTUS_NUM_PROCS=1 + export CACTUS_NUM_THREADS=1 + CACTUS_NUM_THREADS=1 + export GMON_OUT_PREFIX=gmon.out + GMON_OUT_PREFIX=gmon.out + export OMP_NUM_THREADS=1 + OMP_NUM_THREADS=1 + sort + env + echo Starting: ++ date +%s + export CACTUS_STARTTIME=1757968321 + CACTUS_STARTTIME=1757968321 + '[' 1 = 1 ']' + '[' 0 -eq 0 ']' + /system/user/crangano/simulations/bns/SIMFACTORY/exe/cactus_sim -L 3 /system/user/crangano/simulations/bns/output-0000/bns.par WARNING level 1 from host panther process 0 in thorn ML_BSSN_Helper, file /system/user/crangano/einstein_toolkit/BNS/configs/sim/build/ML_BSSN_Helper/Parameters.c:10: -> Forcing ML_BSSN::initial_boundary_condition="extrapolate-gammas" because ML_BSSN::my_initial_boundary_condition="extrapolate-gammas" WARNING level 1 from host panther process 0 in thorn ML_BSSN_Helper, file /system/user/crangano/einstein_toolkit/BNS/configs/sim/build/ML_BSSN_Helper/Parameters.c:25: -> Forcing ML_BSSN::rhs_boundary_condition="NewRad" because ML_BSSN::my_rhs_boundary_condition="NewRad" WARNING level 1 from host panther process 0 in thorn ML_BSSN_Helper, file /system/user/crangano/einstein_toolkit/BNS/configs/sim/build/ML_BSSN_Helper/Parameters.c:45: -> Forcing ML_BSSN::epsDiss=0.0 because ML_BSSN::apply_dissipation="never" WARNING level 1 from host panther process 0 in thorn ML_BSSN_Helper, file /system/user/crangano/einstein_toolkit/BNS/configs/sim/build/ML_BSSN_Helper/Parameters.c:114: -> Forcing ML_BSSN::advectLapse=1 because ML_BSSN::LapseAdvectionCoeff=1.0 WARNING level 1 from host panther process 0 in thorn ML_BSSN_Helper, file /system/user/crangano/einstein_toolkit/BNS/configs/sim/build/ML_BSSN_Helper/Parameters.c:132: -> Forcing ML_BSSN::advectShift=1 because ML_BSSN::ShiftAdvectionCoeff=1.0 WARNING level 1 from host panther process 0 in thorn ML_BSSN_Helper, file /system/user/crangano/einstein_toolkit/BNS/configs/sim/build/ML_BSSN_Helper/Parameters.c:145: -> Parameter ML_BSSN::my_initial_data is outdated; please update the parameter file. Do not use this parameter, and set up initial conditions via ADMBase as usual. WARNING level 1 from host panther process 0 in thorn ML_BSSN_Helper, file /system/user/crangano/einstein_toolkit/BNS/configs/sim/build/ML_BSSN_Helper/Parameters.c:152: -> Parameter ML_BSSN::my_initial_boundary_condition is outdated; please update the parameter file. Do not use this parameter, and set up initial boundary conditions as usual. WARNING level 1 from host panther process 0 in thorn ML_BSSN_Helper, file /system/user/crangano/einstein_toolkit/BNS/configs/sim/build/ML_BSSN_Helper/Parameters.c:159: -> Parameter ML_BSSN::my_rhs_boundary_condition is outdated; please update the parameter file. Do not use this parameter, and set up RHS boundary conditions as usual. WARNING level 1 from host panther process 0 in thorn ML_BSSN_Helper, file /system/user/crangano/einstein_toolkit/BNS/configs/sim/build/ML_BSSN_Helper/Parameters.c:166: -> Parameter ML_BSSN::my_boundary_condition is outdated; please update the parameter file. Do not use this parameter, and set up RHS boundary conditions as usual. WARNING level 1 from host panther process 0 in thorn ML_BSSN_Helper, file /system/user/crangano/einstein_toolkit/BNS/configs/sim/build/ML_BSSN_Helper/Parameters.c:193: -> Parameter ML_BSSN::LapseAdvectionCoeff is outdated; please update the parameter file. Instead of using this parameter, you should set ML_BSSN::advectLapse. WARNING level 1 from host panther process 0 in thorn ML_BSSN_Helper, file /system/user/crangano/einstein_toolkit/BNS/configs/sim/build/ML_BSSN_Helper/Parameters.c:199: -> Parameter ML_BSSN::ShiftAdvectionCoeff is outdated; please update the parameter file. Instead of using this parameter, you should set ML_BSSN::advectShift. Rank 0 with PID 3948578 received signal 11 Writing backtrace to bns/backtrace.0.txt /system/user/crangano/simulations/bns/output-0000/SIMFACTORY/RunScript: line 36: 3948578 Segmentation fault (core dumped) /system/user/crangano/simulations/bns/SIMFACTORY/exe/cactus_sim -L 3 /system/user/crangano/simulations/bns/output-0000/bns.par -------------- next part -------------- Simulation name: bns Running simulation bns Preparing: Checking: /system/user/crangano/simulations/bns/output-0000-active panther Mon Sep 15 10:32:01 PM CEST 2025 Environment: Starting: INFO (Cactus): Increased logging level from 0 to 3 -------------------------------------------------------------------------------- 10 1 0101 ************************ 01 1010 10 The Cactus Code V4.18.0 1010 1101 011 www.cactuscode.org 1001 100101 ************************ 00010101 100011 (c) Copyright The Authors 0100 GNU Licensed. No Warranty 0101 -------------------------------------------------------------------------------- Cactus version: 4.18.0 Compile date: Sep 15 2025 (21:44:45) Run date: Sep 15 2025 (22:32:02+0200) Run host: panther (pid=3948578) Working directory: /system/user/crangano/simulations/bns/output-0000 Executable: /system/user/crangano/simulations/bns/SIMFACTORY/exe/cactus_sim Parameter file: /system/user/crangano/simulations/bns/output-0000/bns.par -------------------------------------------------------------------------------- Activating thorn Cactus...Success -> active implementation Cactus Activation requested for --->Boundary CartGrid3D CoordBase Fortran InitBase IOUtil LocalReduce SymBase Time AEILocalInterp MoL Slab SpaceMask SphericalSurface Carpet CarpetInterp CarpetInterp2 CarpetIOASCII CarpetIOHDF5 CarpetIOScalar CarpetLib CarpetIOBasic CarpetReduce CarpetRegrid2 CarpetSlab CarpetTracker CarpetMask LoopControl Formaline HTTPD Socket NaNChecker TerminationTrigger TimerReport ADMbase ADMcoupling ADMmacros CoordGauge StaticConformal RotatingSymmetry180 ReflectionSymmetry Constants TmunuBase HydroBase QuasiLocalMeasures EOS_Omni GRHydro SummationByParts GenericFD NewRad ML_BSSN ML_BSSN_Helper ML_ADMConstraints Hydro_Analysis NSTracker Dissipation SystemStatistics SystemTopology WeylScal4 Multipole CoordBase Meudon_Bin_NS TerminationTrigger Trigger<--- Warning: thorn CoordBase already scheduled for activation Warning: thorn TerminationTrigger already scheduled for activation Thorn Carpet requests automatic activation of MPI Thorn Carpet requests automatic activation of Timers Thorn CarpetIOHDF5 requests automatic activation of HDF5 Thorn CarpetLib requests automatic activation of Vectors Thorn CarpetLib requests automatic activation of CycleClock Thorn Formaline requests automatic activation of pthreads Thorn GRHydro requests automatic activation of EOS_Polytrope Thorn LoopControl requests automatic activation of hwloc Thorn Meudon_Bin_NS requests automatic activation of LORENE Thorn QuasiLocalMeasures requests automatic activation of LAPACK Thorn QuasiLocalMeasures requests automatic activation of TGRtensor Thorn EOS_Polytrope requests automatic activation of EOS_Base Thorn HDF5 requests automatic activation of zlib Thorn LAPACK requests automatic activation of BLAS Thorn LORENE requests automatic activation of GSL Activating thorn ADMbase...Success -> active implementation ADMBase Activating thorn ADMcoupling...Success -> active implementation ADMCoupling Activating thorn ADMmacros...Success -> active implementation ADMMacros Activating thorn AEILocalInterp...Success -> active implementation AEILocalInterp Activating thorn BLAS...Success -> active implementation BLAS Activating thorn Boundary...Success -> active implementation boundary Activating thorn Carpet...Success -> active implementation Driver Activating thorn CarpetInterp...Success -> active implementation interp Activating thorn CarpetInterp2...Success -> active implementation interp2 Activating thorn CarpetIOASCII...Success -> active implementation IOASCII Activating thorn CarpetIOBasic...Success -> active implementation IOBasic Activating thorn CarpetIOHDF5...Success -> active implementation IOHDF5 Activating thorn CarpetIOScalar...Success -> active implementation IOScalar Activating thorn CarpetLib...Success -> active implementation CarpetLib Activating thorn CarpetMask...Success -> active implementation CarpetMask Activating thorn CarpetReduce...Success -> active implementation reduce Activating thorn CarpetRegrid2...Success -> active implementation CarpetRegrid2 Activating thorn CarpetSlab...Success -> active implementation Hyperslab Activating thorn CarpetTracker...Success -> active implementation CarpetTracker Activating thorn CartGrid3D...Success -> active implementation grid Activating thorn Constants...Success -> active implementation Constants Activating thorn CoordBase...Success -> active implementation CoordBase Activating thorn CoordGauge...Success -> active implementation CoordGauge Activating thorn CycleClock...Success -> active implementation CycleClock Activating thorn Dissipation...Success -> active implementation Dissipation Activating thorn EOS_Base...Success -> active implementation EOS_Base Activating thorn EOS_Omni...Success -> active implementation EOS_Omni Activating thorn EOS_Polytrope...Success -> active implementation EOS_2d_Polytrope Activating thorn Formaline...Success -> active implementation Formaline Activating thorn Fortran...Success -> active implementation Fortran Activating thorn GenericFD...Success -> active implementation GenericFD Activating thorn GRHydro...Success -> active implementation GRHydro Activating thorn GSL...Success -> active implementation GSL Activating thorn HDF5...Success -> active implementation HDF5 Activating thorn HTTPD...Success -> active implementation HTTPD Activating thorn hwloc...Success -> active implementation hwloc Activating thorn Hydro_Analysis...Success -> active implementation Hydro_Analysis Activating thorn HydroBase...Success -> active implementation HydroBase Activating thorn InitBase...Success -> active implementation InitBase Activating thorn IOUtil...Success -> active implementation IO Activating thorn LAPACK...Success -> active implementation LAPACK Activating thorn LocalReduce...Success -> active implementation LocalReduce Activating thorn LoopControl...Success -> active implementation LoopControl Activating thorn LORENE...Success -> active implementation LORENE Activating thorn Meudon_Bin_NS...Success -> active implementation Meudon_Bin_NS Activating thorn ML_ADMConstraints...Success -> active implementation ML_ADMConstraints Activating thorn ML_BSSN...Success -> active implementation ML_BSSN Activating thorn ML_BSSN_Helper...Success -> active implementation ML_BSSN_Helper Activating thorn MoL...Success -> active implementation MethodOfLines Activating thorn MPI...Success -> active implementation MPI Activating thorn Multipole...Success -> active implementation multipole Activating thorn NaNChecker...Success -> active implementation NaNChecker Activating thorn NewRad...Success -> active implementation NewRad Activating thorn NSTracker...Success -> active implementation NSTracker Activating thorn pthreads...Success -> active implementation PTHREADS Activating thorn QuasiLocalMeasures...Success -> active implementation QuasiLocalMeasures Activating thorn ReflectionSymmetry...Success -> active implementation ReflectionSymmetry Activating thorn RotatingSymmetry180...Success -> active implementation RotatingSymmetry180 Activating thorn Slab...Success -> active implementation Slab Activating thorn Socket...Success -> active implementation Socket Activating thorn SpaceMask...Success -> active implementation SpaceMask Activating thorn SphericalSurface...Success -> active implementation SphericalSurface Activating thorn StaticConformal...Success -> active implementation StaticConformal Activating thorn SummationByParts...Success -> active implementation SummationByParts Activating thorn SymBase...Success -> active implementation SymBase Activating thorn SystemStatistics...Success -> active implementation SystemStatistics Activating thorn SystemTopology...Success -> active implementation SystemTopology Activating thorn TerminationTrigger...Success -> active implementation TerminationTrigger Activating thorn TGRtensor...Success -> active implementation TGRtensor Activating thorn Time...Success -> active implementation time Activating thorn TimerReport...Success -> active implementation timerreport Activating thorn Timers...Success -> active implementation Timers Activating thorn TmunuBase...Success -> active implementation TmunuBase Activating thorn Trigger...Success -> active implementation trigger Activating thorn Vectors...Success -> active implementation Vectors Activating thorn WeylScal4...Success -> active implementation WeylScal4 Activating thorn zlib...Success -> active implementation zlib WARNING level 3 from host panther process 0 in thorn IOUtil, file /system/user/crangano/einstein_toolkit/BNS/configs/sim/build/IOUtil/CheckpointRecovery.cc:748: -> No HDF5 checkpoint files with basefilename 'checkpoint.chkpt' and file extension '.h5' found in recovery directory 'bns' -------------------------------------------------------------------------------- if (recover initial data) Recover parameters endif Startup routines [CCTK_STARTUP] Carpet::MultiModel_Startup: Multi-model Startup routine CycleClock::CycleClock_Setup: Set up CycleClock LoopControl::LC_setup: Set up LoopControl ML_BSSN_Helper::ML_BSSN_SetGroupTags: [meta] Set checkpointing and prolongation group tags GROUP hwloc_startup: hwloc startup group hwloc::hwloc_version: Output hwloc version Timers::Timer_Startup: Prepare hierarchical timers SystemTopology::ST_system_topology: Output and/or modify system topology and hardware locality CarpetInterp::CarpetInterpStartup: Startup routine CarpetReduce::CarpetReduceStartup: Startup routine CartGrid3D::SymmetryStartup: Register GH Extension for GridSymmetry CoordBase::CoordBase_Startup: Register a GH extension to store the coordinate system handles Carpet::Driver_Startup: Startup routine EOS_Omni::EOS_Omni_Startup: [global] Set up conversion factors and other fun stuff EOS_Polytrope::EOS_Polytrope_Startup: Setup the polytropic EOS Formaline::Formaline_PrintIDs: [meta] Print the build and simulation ids GRHydro::GRHydro_Startup: Startup banner GRHydro::GRHydro_RegisterMask: Register the hydro masks GROUP HTTP_Startup: HTTP daemon startup group HTTPD::HTTP_StartServer: [global] Start HTTP server GROUP HTTP_SetupPages: Group to setup stuff which needs to be done between starting the server and the first time it serves pages HTTPD::HTTP_FirstServ: [global] Serve first pages at startup HydroBase::HydroBase_StartUp: Startup banner IOUtil::IOUtil_Startup: Startup routine LocalReduce::LocalReduce_Startup: Startup routine CarpetIOBasic::CarpetIOBasicStartup: [global] Startup routine ML_ADMConstraints::ML_ADMConstraints_Startup: [meta] create banner ML_BSSN::ML_BSSN_Startup: [meta] create banner ML_BSSN_Helper::ML_BSSN_RegisterSlicing: [meta] Register slicing CarpetIOHDF5::CarpetIOHDF5_Startup: Startup routine MoL::MoL_Startup: Startup banner Slab::Slab_InitMPIDatatypes: Create MPI datatypes for complex variables in C SymBase::SymBase_Startup: Register GH Extension for SymBase CarpetIOScalar::CarpetIOScalarStartup: [global] Startup routine TerminationTrigger::TerminationTrigger_StartSignalHandler: Start signal handler CarpetIOASCII::CarpetIOASCIIStartup: [global] Startup routine Trigger::Trigger_Startup: Startup Routine Vectors::Vectors_Startup: Print startup message WeylScal4::WeylScal4_Startup: [meta] create banner AEILocalInterp::AEILocalInterp_U_Startup: register CCTK_InterpLocalUniform() interpolation operators Startup routines which need an existing grid hierarchy [CCTK_WRAGH] ADMBase::Einstein_InitSymBound: [global] Set up GF symmetries Boundary::Boundary_RegisterBCs: [global] Register boundary conditions that this thorn provides CarpetRegrid2::CarpetRegrid2_Initialise: [global] Initialise locations of refined regions CartGrid3D::RegisterCartGrid3DCoords: [meta] Register coordinates for the Cartesian grid CoordGauge::Einstein_ActivateSlicing: Initialize slicing, setup priorities for mixed slicings CoordGauge::Einstein_SetNextSlicing: Identify the slicing for the next iteration Formaline::Formaline_OutputSource: [meta] Output Cactus source tree Formaline::Formaline_RegisterWarnings: [meta] Register to receive warnings and info messages from the flesh Formaline::Formaline_AnnounceInitial: [global] Put some meta information about the current run into permanent storage GRHydro::GRHydro_ClearLastMoLPostStep: [global] Initialize InLastMoLPostStep to zero ML_BSSN_Helper::ML_BSSN_ParamCompat: [meta] Handle parameter backward compatibility MoL::MoL_SetupIndexArrays: Set up the MoL bookkeeping index arrays MoL::MoL_SetScheduleStatus: [global] Set the flag so it is ok to register with MoL TmunuBase::TmunuBase_SetStressEnergyState: [global] Set the stress_energy_state variable GROUP MoL_Register: The group where physics thorns register variables with MoL GRHydro::GRHydro_Register: Register variables for MoL ML_ADMConstraints::ML_ADMConstraints_RegisterVars: [meta] Register Variables for MoL ML_BSSN::ML_BSSN_RegisterVars: [meta] Register Variables for MoL ML_BSSN_Helper::ML_BSSN_RegisterConstrained: [meta] Register ADMBase variables as constrained WeylScal4::WeylScal4_RegisterVars: [meta] Register Variables for MoL Slab::Slab_InitTimers: Initialise timers SpaceMask::MaskSym: [global] Set grid symmetries for mask SpaceMask::MaskSym_emask: [global] Set grid symmetries for emask (compatibility mode) GROUP SymBase_Wrapper: Wrapper group for SymBase GROUP SymmetryRegister: Register your symmetries here CartGrid3D::CartGrid3D_RegisterSymmetryBoundaries: [meta] Register symmetry boundaries ML_ADMConstraints::ML_ADMConstraints_RegisterSymmetries: [meta] register symmetries ML_BSSN::ML_BSSN_RegisterSymmetries: [meta] register symmetries ReflectionSymmetry::ReflectionSymmetry_Register: Register reflection symmetry boundaries RotatingSymmetry180::Rot180_RegisterSymmetry: Register symmetry boundaries WeylScal4::WeylScal4_RegisterSymmetries: [meta] register symmetries SymBase::SymBase_Statistics: Print symmetry boundary face descriptions MoL::MoL_ReportNumberVariables: [meta] Report how many of each type of variable there are Parameter checking routines [CCTK_PARAMCHECK] ADMBase::ADMBase_ParamCheck: [global] Check consistency of parameters Boundary::Boundary_Check: Check dimension of grid variables Carpet::CarpetParamCheck: Parameter checking routine CarpetLib::CarpetLib_test_prolongate_3d_rf2: [global] Test prolongation operators CarpetMask::CarpetSurfaceParamCheck: [global] Check parameters CarpetRegrid2::CarpetRegrid2_ParamCheck: Check parameters CartGrid3D::ParamCheck_CartGrid3D: Check coordinates for CartGrid3D Dissipation::dissipation_paramcheck: Check dissipation parameters for consistency Fortran::CheckFortranParameters: Test whether Fortran parameters work correctly GRHydro::GRHydro_ParamCheck: Check parameters ML_BSSN_Helper::ML_BSSN_ParamCheck: [meta] Check parameters MoL::MoL_ParamCheck: Basic parameter checking Multipole::Multipole_ParamCheck: [global] Check Multipole parameters QuasiLocalMeasures::qlm_paramcheck: [global] Check quasi-local parameter settings RotatingSymmetry180::Rot180_CheckTensorTypes: Check tensor type definitions for consistency SphericalSurface::SphericalSurface_ParamCheck: [global] Check that all surface names are unique TerminationTrigger::TerminationTrigger_ParamCheck: Check consitency of parameters TmunuBase::TmunuBase_ParamCheck: [global] Check that no deprecated parameters are used. Trigger::Trigger_ParamCheck: Parameter checking Vectors::Vectors_Test: Run correctness tests. Initialisation if (NOT (recover initial data AND recovery_mode is 'strict')) [CCTK_PREREGRIDINITIAL] Set up grid hierarchy [CCTK_POSTREGRIDINITIAL] CartGrid3D::SpatialCoordinates: Set Coordinates after regridding GROUP MaskBase_SetupMask: Set up the weight function GROUP MaskBase_SetupMaskAll: Set up the weight function CarpetReduce::MaskBase_AllocateMask: [global] Allocate the weight function CarpetReduce::MaskBase_InitMask: [global] [loop-local] Initialise the weight function GROUP SetupIMaskInternal: Set up the integer weight function (schedule other routines in here) CarpetReduce::CoordBase_SetupMask: [global] [loop-local] Set up the outer boundaries of the weight function CarpetReduce::CarpetMaskSetup: [global] [loop-singlemap] Set up the weight function for the restriction regions GROUP SetupIMask: Set up the integer weight function (schedule other routines in here) CarpetMask::CarpetExcludedSetup: [global] [loop-local] Set up the weight function for the excluded regions CarpetMask::CarpetSurfaceSetup: [global] [loop-local] Set up the weight function for the excluded spherical surfaces CarpetReduce::MaskBase_SetMask: [global] [loop-local] Set the weight function GROUP SetupMask: Set up the real weight function (schedule other routines in here) CarpetReduce::MaskBase_TestMask: [global] Test the weight function Dissipation::setup_epsdis: Setup spatially varying dissipation SpaceMask::MaskZero: Initialise mask to zero GRHydro::GRHydro_RefinementLevel: Calculate current refinement level GROUP GRHydro_PrimitiveBoundaries: Apply boundary conditions to all primitives GRHydro::GRHydro_SelectPrimitiveBoundaries: [level] Select primitive variables for boundary conditions GROUP GRHydro_ApplyPrimitiveBCs: Apply boundary conditions to all primitive variables GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP HydroBase_ExcisionMaskSetup: Set up hydro excision mask HydroBase::HydroBase_InitExcisionMask: Initialize hydro excision mask to 'no excision everywhere' GROUP MoL_PseudoEvolutionBoundaries: Apply boundary conditions to pseudo-evolved quantities GROUP ML_ADMConstraints_evaluate_bc_group: ML_ADMConstraints_evaluate ML_ADMConstraints::ML_ADMConstraints_evaluate_SelectBCs: [level] ML_ADMConstraints_evaluate_SelectBCs GROUP ML_ADMConstraints_evaluate_ApplyBCs: Apply BCs for groups set in ML_ADMConstraints_evaluate GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP ML_BSSN_ConstraintsEverywhere_bc_group: ML_BSSN_ConstraintsEverywhere ML_BSSN::ML_BSSN_ConstraintsEverywhere_SelectBCs: [level] ML_BSSN_ConstraintsEverywhere_SelectBCs GROUP ML_BSSN_ConstraintsEverywhere_ApplyBCs: Apply BCs for groups set in ML_BSSN_ConstraintsEverywhere GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP ML_BSSN_ConstraintsInterior_bc_group: ML_BSSN_ConstraintsInterior ML_BSSN::ML_BSSN_ConstraintsInterior_SelectBCs: [level] ML_BSSN_ConstraintsInterior_SelectBCs GROUP ML_BSSN_ConstraintsInterior_ApplyBCs: Apply BCs for groups set in ML_BSSN_ConstraintsInterior GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP WeylScal4_psi4_calc_4th_bc_group: WeylScal4_psi4_calc_4th WeylScal4::WeylScal4_psi4_calc_4th_SelectBCs: [level] WeylScal4_psi4_calc_4th_SelectBCs GROUP WeylScal4_psi4_calc_4th_ApplyBCs: Apply BCs for groups set in WeylScal4_psi4_calc_4th GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions SpaceMask::MaskOne: Set mask to one GRHydro::GRHydro_SetupMask: Initialize the atmosphere mask [CCTK_BASEGRID] ADMBase::ADMBase_SetShiftStateOn: Set the shift_state variable to 1 ADMBase::ADMBase_SetDtLapseStateOn: Set the dtlapse_state variable to 1 ADMBase::ADMBase_SetDtShiftStateOn: Set the dtshift_state variable to 1 ADMMacros::ADMMacros_SetLocalSpatialOrder: Initialize the local_spatial_order CartGrid3D::SpatialSpacings: Set up ranges for spatial 3D Cartesian coordinates (on all grids) CartGrid3D::SpatialCoordinates: Set up spatial 3D Cartesian coordinates on the GH SphericalSurface::SphericalSurface_SetupRes: [global] [loop-local] Set surface resolution automatically Dissipation::dissipation_basegrid: Ensure that there are enough ghost zones GRHydro::GRHydro_Reset_Execution_Flags: [global] Initially set execution flags to 'YEAH, Execute'! GRHydro::GRHydro_InitSymBound: Schedule symmetries and check shift state GRHydro::reset_GRHydro_C2P_failed: Initialise the mask function that contains the points where C2P has failed (at BASEGRID) Hydro_Analysis::Hydro_Analysis_Init: [global] Initialize variables ML_ADMConstraints::ML_ADMConstraints_CheckBoundaries: [meta] check boundaries treatment ML_BSSN::ML_BSSN_CheckBoundaries: [meta] check boundaries treatment NaNChecker::NaNChecker_ResetCounter: [global] Reset the NaNChecker::NaNsFound counter SpaceMask::MaskZero: Initialise mask to zero SpaceMask::MaskOne: Set old style mask to one SphericalSurface::SphericalSurface_Setup: [global] Calculate surface coordinate descriptors GROUP MaskBase_SetupMask: Set up the weight function GROUP MaskBase_SetupMaskAll: Set up the weight function CarpetReduce::MaskBase_AllocateMask: [global] Allocate the weight function CarpetReduce::MaskBase_InitMask: [global] [loop-local] Initialise the weight function GROUP SetupIMaskInternal: Set up the integer weight function (schedule other routines in here) CarpetReduce::CoordBase_SetupMask: [global] [loop-local] Set up the outer boundaries of the weight function CarpetReduce::CarpetMaskSetup: [global] [loop-singlemap] Set up the weight function for the restriction regions GROUP SetupIMask: Set up the integer weight function (schedule other routines in here) CarpetMask::CarpetExcludedSetup: [global] [loop-local] Set up the weight function for the excluded regions CarpetMask::CarpetSurfaceSetup: [global] [loop-local] Set up the weight function for the excluded spherical surfaces CarpetReduce::MaskBase_SetMask: [global] [loop-local] Set the weight function GROUP SetupMask: Set up the real weight function (schedule other routines in here) CarpetReduce::MaskBase_TestMask: [global] Test the weight function SphericalSurface::SphericalSurface_Set: [global] Set surface radii to be used for initial setup in other thorns GROUP SphericalSurface_HasBeenSet: Set the spherical surfaces before this group, and use it afterwards SphericalSurface::SphericalSurface_CheckState: [global] Test the state of the spherical surfaces SummationByParts::SBP_SetNormMask: Setup the mask for the calculation of the norm SymBase::SymBase_Check: Check whether the driver set up the grid consistently TerminationTrigger::TerminationTrigger_ResetTrigger: Clear trigger state TerminationTrigger::TerminationTrigger_StartTimer: Start timer TerminationTrigger::TerminationTrigger_CreateFile: Create termination file Time::Time_Initialise: [global] Initialise Time variables Time::TemporalSpacings: [singlemap] Set timestep based on Courant condition (courant_static) WeylScal4::WeylScal4_CheckBoundaries: [meta] check boundaries treatment [CCTK_INITIAL] StaticConformal::StaticConformal_InitialiseState: Set the conformal_state variable to 0 GROUP ADMBase_InitialData: Schedule group for calculating ADM initial data GRHydro::GRHydro_EOSHandle: [global] Set the EOS number CarpetIOASCII::CarpetIOASCIIInit: [global] Initialisation routine CarpetIOBasic::CarpetIOBasicInit: [global] Initialisation routine CarpetIOHDF5::CarpetIOHDF5_Init: [global] Initialisation routine CarpetIOScalar::CarpetIOScalarInit: [global] Initialisation routine GRHydro::GRHydro_Rho_Minima_Setup: Set up minimum for the rest-mass density in the atmosphere (before intial data) GRHydro::GRHydro_SetupMask: Initialize the atmosphere mask GRHydro::GRHydro_RefinementLevel: Calculate current refinement level GROUP ADMBase_InitialGauge: Schedule group for the ADM initial gauge condition ADMBase::ADMBase_ShiftZero: Set the shift to 0 at all points ADMBase::ADMBase_DtShiftZero: Set the dtshift to 0 at all points GROUP HydroBase_Initial: HydroBase initial data group GROUP GRHydro_Initial: GRHydro initial data group GROUP HydroBase_ExcisionMaskSetup: Set up hydro excision mask HydroBase::HydroBase_InitExcisionMask: Initialize hydro excision mask to 'no excision everywhere' Meudon_Bin_NS::Meudon_Bin_NS_initialise: Set up binary neutron star initial data GROUP ADMBase_PostInitial: Schedule group for modifying the ADM initial data, such as e.g. adding noise GRHydro::GRHydro_SqrtSpatialDeterminant: Calculate sdetg GRHydro::GRHydro_InitialAtmosphereReset: Use mask to enforce atmosphere at initial time ML_BSSN::ML_BSSN_InitialADMBase1Everywhere: ML_BSSN_InitialADMBase1Everywhere ML_BSSN::ML_BSSN_InitialADMBase2Interior: ML_BSSN_InitialADMBase2Interior ML_BSSN::ML_BSSN_InitialADMBase2BoundaryScalar: ML_BSSN_InitialADMBase2BoundaryScalar ML_BSSN_Helper::ML_BSSN_ExtrapolateGammas: Extrapolate Gammas and time derivatives of lapse and shift MoL::MoL_StartLoop: [level] Initialise the step size control QuasiLocalMeasures::qlm_init: [global] Initialise quasi-local calculations GROUP HydroBase_Prim2ConInitial: Recover the conservative variables from the primitive variables GRHydro::Primitive2ConservativeCells: Convert initial data given in primive variables to conserved variables [CCTK_POSTINITIAL] CarpetIOHDF5::CarpetIOHDF5_CloseFiles: [global] Close all filereader input files GRHydro::GRHydro_Scalar_Setup: Set up and check scalars for efficiency GROUP MoL_PostStepModify: The group for physics thorns to schedule enforcing constraints ML_BSSN::ML_BSSN_EnforceEverywhere: ML_BSSN_EnforceEverywhere GROUP MoL_PostStep: Ensure that everything is correct after the initial data have been set up ML_BSSN::ML_BSSN_SelectBoundConds: [level] select boundary conditions GRHydro::GRHydro_RefinementLevel: Calculate current refinement level GRHydro::GRHydro_SetLastMoLPostStep: [level] Set grid scalar InLastMoLPostStep if this is the last MoL PostStep call GROUP ML_BSSN_ApplyBCs: Apply boundary conditions controlled by thorn Boundary GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions ML_ADMConstraints::ML_ADMConstraints_SelectBoundConds: [level] select boundary conditions GROUP ML_ADMConstraints_ApplyBCs: Apply boundary conditions controlled by thorn Boundary GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions ML_BSSN::ML_BSSN_ADMBaseInterior: ML_BSSN_ADMBaseInterior ML_BSSN::ML_BSSN_ADMBaseBoundaryScalar: ML_BSSN_ADMBaseBoundaryScalar ML_BSSN::ML_BSSN_ADMBaseEverywhere: ML_BSSN_ADMBaseEverywhere ML_BSSN_Helper::ML_BSSN_ADMBase_SelectBCs: [level] Select boundary conditions for ADMBase variables GROUP ML_BSSN_ADMBase_ApplyBCs: Apply boundary conditions to ADMBase variables GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP ADMBase_SetADMVars: Set the ADM variables before this group, and use them afterwards GROUP HydroBase_PostStep: Post step tasks for hydro thorns GROUP GRHydro_PostStep: Post step tasks for GRHydro GROUP GRHydro_AtmosphereMaskBoundaries: Apply boundary conditions to primitives GRHydro::GRHydro_SelectAtmosphereMaskBoundaries: [level] Select atmosphere mask for boundary conditions GROUP GRHydro_ApplyAtmosphereMaskBCs: Apply boundary conditions to real-valued atmosphere mask GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GRHydro::GRHydroPostSyncAtmosphereMask: Set integer atmosphere mask from synchronized real atmosphere mask if (GRHydro::InLastMoLPostStep) GRHydro::GRHydro_AtmosphereReset: Reset the atmosphere if (GRHydro::InLastMoLPostStep) GROUP GRHydro_PrimitiveInitialGuessesBoundaries: Apply boundary conditions to those primitives used as initial guesses GRHydro::GRHydro_SelectPrimitiveInitialGuessesBoundaries: [level] Select initial guess primitive variables for boudary conditions GROUP GRHydro_ApplyPrimitiveInitialGuessBCs: Apply boundary conditions to initial guess primitive variables GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP HydroBase_Boundaries: HydroBase-internal Boundary conditions group GROUP Do_GRHydro_Boundaries: GRHydro Boundary conditions group GROUP HydroBase_Select_Boundaries: Group to schedule the boundary condition functions if (GRHydro::execute_MoL_PostStep) GRHydro::GRHydro_Bound: [level] Select GRHydro boundary conditions GROUP HydroBase_ApplyBCs: Apply the boundary conditions of HydroBase GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP HydroBase_Con2Prim: Convert from conservative to primitive variables if (GRHydro::execute_MoL_Step) GRHydro::GRHydro_SqrtSpatialDeterminant: Calculate sdetg if (GRHydro::execute_MoL_PostStep) GRHydro::Con2Prim: Convert back to primitive variables (general) GROUP SetTmunu: Group for calculating the stress-energy tensor TmunuBase::TmunuBase_ZeroTmunu: Initialise the stress-energy tensor to zero GROUP AddToTmunu: Add to the stress-energy tensor here GRHydro::GRHydro_Tmunu: Compute the energy-momentum tensor WeylScal4::WeylScal4_SelectBoundConds: [level] select boundary conditions GROUP WeylScal4_ApplyBCs: Apply boundary conditions controlled by thorn Boundary GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP MoL_PseudoEvolution: Calculate pseudo-evolved quantities GROUP ADMBase_SetADMVars: Set the ADM variables before this group, and use them afterwards GROUP ML_ADMConstraints_evaluate_group: ML_ADMConstraints_evaluate ML_ADMConstraints::ML_ADMConstraints_evaluate: ML_ADMConstraints_evaluate GROUP ML_ADMConstraints_evaluate_bc_group: ML_ADMConstraints_evaluate ML_ADMConstraints::ML_ADMConstraints_evaluate_SelectBCs: [level] ML_ADMConstraints_evaluate_SelectBCs GROUP ML_ADMConstraints_evaluate_ApplyBCs: Apply BCs for groups set in ML_ADMConstraints_evaluate GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP ML_BSSN_ConstraintsEverywhere_group: ML_BSSN_ConstraintsEverywhere ML_BSSN::ML_BSSN_ConstraintsEverywhere: ML_BSSN_ConstraintsEverywhere GROUP ML_BSSN_ConstraintsEverywhere_bc_group: ML_BSSN_ConstraintsEverywhere ML_BSSN::ML_BSSN_ConstraintsEverywhere_SelectBCs: [level] ML_BSSN_ConstraintsEverywhere_SelectBCs GROUP ML_BSSN_ConstraintsEverywhere_ApplyBCs: Apply BCs for groups set in ML_BSSN_ConstraintsEverywhere GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP ML_BSSN_ConstraintsInterior_group: ML_BSSN_ConstraintsInterior ML_BSSN::ML_BSSN_ConstraintsInterior: ML_BSSN_ConstraintsInterior GROUP ML_BSSN_ConstraintsInterior_bc_group: ML_BSSN_ConstraintsInterior ML_BSSN::ML_BSSN_ConstraintsInterior_SelectBCs: [level] ML_BSSN_ConstraintsInterior_SelectBCs GROUP ML_BSSN_ConstraintsInterior_ApplyBCs: Apply BCs for groups set in ML_BSSN_ConstraintsInterior GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP WeylScal4_psi4_calc_4th_group: WeylScal4_psi4_calc_4th WeylScal4::WeylScal4_psi4_calc_4th: WeylScal4_psi4_calc_4th GROUP WeylScal4_psi4_calc_4th_bc_group: WeylScal4_psi4_calc_4th WeylScal4::WeylScal4_psi4_calc_4th_SelectBCs: [level] WeylScal4_psi4_calc_4th_SelectBCs GROUP WeylScal4_psi4_calc_4th_ApplyBCs: Apply BCs for groups set in WeylScal4_psi4_calc_4th GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions SummationByParts::SBP_CheckGridSizes: Check grid sizes and ghost zones Initialise finer grids recursively Restrict from finer grids [CCTK_POSTRESTRICTINITIAL] GROUP MoL_PostStep: Ensure that everything is correct after restriction ML_BSSN::ML_BSSN_SelectBoundConds: [level] select boundary conditions GRHydro::GRHydro_RefinementLevel: Calculate current refinement level GRHydro::GRHydro_SetLastMoLPostStep: [level] Set grid scalar InLastMoLPostStep if this is the last MoL PostStep call GROUP ML_BSSN_ApplyBCs: Apply boundary conditions controlled by thorn Boundary GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions ML_ADMConstraints::ML_ADMConstraints_SelectBoundConds: [level] select boundary conditions GROUP ML_ADMConstraints_ApplyBCs: Apply boundary conditions controlled by thorn Boundary GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions ML_BSSN::ML_BSSN_ADMBaseInterior: ML_BSSN_ADMBaseInterior ML_BSSN::ML_BSSN_ADMBaseBoundaryScalar: ML_BSSN_ADMBaseBoundaryScalar ML_BSSN::ML_BSSN_ADMBaseEverywhere: ML_BSSN_ADMBaseEverywhere ML_BSSN_Helper::ML_BSSN_ADMBase_SelectBCs: [level] Select boundary conditions for ADMBase variables GROUP ML_BSSN_ADMBase_ApplyBCs: Apply boundary conditions to ADMBase variables GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP ADMBase_SetADMVars: Set the ADM variables before this group, and use them afterwards GROUP HydroBase_PostStep: Post step tasks for hydro thorns GROUP GRHydro_PostStep: Post step tasks for GRHydro GROUP GRHydro_AtmosphereMaskBoundaries: Apply boundary conditions to primitives GRHydro::GRHydro_SelectAtmosphereMaskBoundaries: [level] Select atmosphere mask for boundary conditions GROUP GRHydro_ApplyAtmosphereMaskBCs: Apply boundary conditions to real-valued atmosphere mask GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GRHydro::GRHydroPostSyncAtmosphereMask: Set integer atmosphere mask from synchronized real atmosphere mask if (GRHydro::InLastMoLPostStep) GRHydro::GRHydro_AtmosphereReset: Reset the atmosphere if (GRHydro::InLastMoLPostStep) GROUP GRHydro_PrimitiveInitialGuessesBoundaries: Apply boundary conditions to those primitives used as initial guesses GRHydro::GRHydro_SelectPrimitiveInitialGuessesBoundaries: [level] Select initial guess primitive variables for boudary conditions GROUP GRHydro_ApplyPrimitiveInitialGuessBCs: Apply boundary conditions to initial guess primitive variables GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP HydroBase_Boundaries: HydroBase-internal Boundary conditions group GROUP Do_GRHydro_Boundaries: GRHydro Boundary conditions group GROUP HydroBase_Select_Boundaries: Group to schedule the boundary condition functions if (GRHydro::execute_MoL_PostStep) GRHydro::GRHydro_Bound: [level] Select GRHydro boundary conditions GROUP HydroBase_ApplyBCs: Apply the boundary conditions of HydroBase GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP HydroBase_Con2Prim: Convert from conservative to primitive variables if (GRHydro::execute_MoL_Step) GRHydro::GRHydro_SqrtSpatialDeterminant: Calculate sdetg if (GRHydro::execute_MoL_PostStep) GRHydro::Con2Prim: Convert back to primitive variables (general) GROUP SetTmunu: Group for calculating the stress-energy tensor TmunuBase::TmunuBase_ZeroTmunu: Initialise the stress-energy tensor to zero GROUP AddToTmunu: Add to the stress-energy tensor here GRHydro::GRHydro_Tmunu: Compute the energy-momentum tensor WeylScal4::WeylScal4_SelectBoundConds: [level] select boundary conditions GROUP WeylScal4_ApplyBCs: Apply boundary conditions controlled by thorn Boundary GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP MoL_PseudoEvolutionBoundaries: Apply boundary conditions to pseudo-evolved quantities GROUP ML_ADMConstraints_evaluate_bc_group: ML_ADMConstraints_evaluate ML_ADMConstraints::ML_ADMConstraints_evaluate_SelectBCs: [level] ML_ADMConstraints_evaluate_SelectBCs GROUP ML_ADMConstraints_evaluate_ApplyBCs: Apply BCs for groups set in ML_ADMConstraints_evaluate GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP ML_BSSN_ConstraintsEverywhere_bc_group: ML_BSSN_ConstraintsEverywhere ML_BSSN::ML_BSSN_ConstraintsEverywhere_SelectBCs: [level] ML_BSSN_ConstraintsEverywhere_SelectBCs GROUP ML_BSSN_ConstraintsEverywhere_ApplyBCs: Apply BCs for groups set in ML_BSSN_ConstraintsEverywhere GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP ML_BSSN_ConstraintsInterior_bc_group: ML_BSSN_ConstraintsInterior ML_BSSN::ML_BSSN_ConstraintsInterior_SelectBCs: [level] ML_BSSN_ConstraintsInterior_SelectBCs GROUP ML_BSSN_ConstraintsInterior_ApplyBCs: Apply BCs for groups set in ML_BSSN_ConstraintsInterior GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP WeylScal4_psi4_calc_4th_bc_group: WeylScal4_psi4_calc_4th WeylScal4::WeylScal4_psi4_calc_4th_SelectBCs: [level] WeylScal4_psi4_calc_4th_SelectBCs GROUP WeylScal4_psi4_calc_4th_ApplyBCs: Apply BCs for groups set in WeylScal4_psi4_calc_4th GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions [CCTK_POSTPOSTINITIAL] GRHydro::GRHydro_Rho_Minima_Setup_Final: Set the value of the rest-mass density of the atmosphere which will be used during the evolution GRHydro::GRHydro_InitialAtmosphereReset: Use mask to enforce atmosphere at initial time GROUP Con2Prim: Convert from conservative to primitive variables (might be redundant) if (GRHydro::execute_MoL_Step) GRHydro::GRHydro_SqrtSpatialDeterminant: Calculate sdetg if (GRHydro::execute_MoL_PostStep) GRHydro::Con2Prim: Convert back to primitive variables (general) GROUP SetTmunu: Calculate the stress-energy tensor TmunuBase::TmunuBase_ZeroTmunu: Initialise the stress-energy tensor to zero GROUP AddToTmunu: Add to the stress-energy tensor here GRHydro::GRHydro_Tmunu: Compute the energy-momentum tensor GROUP ADMConstraintsGroup: Evaluate ADM constraints, and perform symmetry boundary conditions [CCTK_POSTSTEP] SphericalSurface::SphericalSurface_Set: [global] Set surface radii GROUP SphericalSurface_HasBeenSet: Set the spherical surfaces before this group, and use it afterwards SphericalSurface::SphericalSurface_CheckState: [global] Test the state of the spherical surfaces GRHydro::GRHydro_RefinementLevel: Calculate current refinement level (for the check of the C2P mask) GRHydro::check_GRHydro_C2P_failed: Check the mask function that contains the points where C2P has failed and report an error in case a failure is found HTTPD::HTTP_Work: [global] Working routine HTTPD::HTTP_ContentWork: [global] Content Working routine GROUP HydroBase_ExcisionHasBeenSet: Group to schedule thorns changing the mask before and thorns using the mask after GROUP Hydro_Analysis: Group for Hydro_Analysis routines Hydro_Analysis::Hydro_Analysis_PrepareReduction: [global] [loop-local] Compute the local reduction results Hydro_Analysis::Hydro_Analysis_Reduction: [global] Compute the global reduction results GROUP Hydro_Analysis_LocationSearch: Look for the location of the maximum density Hydro_Analysis::Hydro_Analysis_LocationSearch_Setup: [global] Prepare data structures for search Hydro_Analysis::Hydro_Analysis_LocationSearch_Search: [global] [loop-local] Search for the location of the maximum density Hydro_Analysis::Hydro_Analysis_LocationSearch_Combine: [global] Communicate and verify the location of the maximum density Hydro_Analysis::Hydro_Analysis_FindSeparation: [global] Compute the proper distance between the maximum of the density and the origin (along a straight coordinate line) NSTracker::NSTracker: [global] Look for the star and set grid accordingly GROUP zzz_NaNChecker_NaNCheck: Check for NaNs and count them in NaNChecker::NaNsFound NaNChecker::NaNChecker_NaNCheck_Prepare: [level] Prepare data structures to check for NaNs NaNChecker::NaNChecker_NaNCheck_Check: [local] Check for NaNs NaNChecker::NaNChecker_NaNCheck_Finish: [level] Count NaNs in NaNChecker::NaNsFound NaNChecker::NaNChecker_TakeAction: [global] [loop-level] Output NaNChecker::NaNmask and take action according to NaNChecker::action_if_found SpaceMask::CheckMask: Ensure that all mask values are legal Dissipation::setup_epsdis: Setup spatially varying dissipation GROUP MaskBase_SetupMask: Set up the weight function GROUP MaskBase_SetupMaskAll: Set up the weight function CarpetReduce::MaskBase_AllocateMask: [global] Allocate the weight function CarpetReduce::MaskBase_InitMask: [global] [loop-local] Initialise the weight function GROUP SetupIMaskInternal: Set up the integer weight function (schedule other routines in here) CarpetReduce::CoordBase_SetupMask: [global] [loop-local] Set up the outer boundaries of the weight function CarpetReduce::CarpetMaskSetup: [global] [loop-singlemap] Set up the weight function for the restriction regions GROUP SetupIMask: Set up the integer weight function (schedule other routines in here) CarpetMask::CarpetExcludedSetup: [global] [loop-local] Set up the weight function for the excluded regions CarpetMask::CarpetSurfaceSetup: [global] [loop-local] Set up the weight function for the excluded spherical surfaces CarpetReduce::MaskBase_SetMask: [global] [loop-local] Set the weight function GROUP SetupMask: Set up the real weight function (schedule other routines in here) CarpetReduce::MaskBase_TestMask: [global] Test the weight function endif if (recover initial data) [CCTK_BASEGRID] ADMBase::ADMBase_SetShiftStateOn: Set the shift_state variable to 1 ADMBase::ADMBase_SetDtLapseStateOn: Set the dtlapse_state variable to 1 ADMBase::ADMBase_SetDtShiftStateOn: Set the dtshift_state variable to 1 ADMMacros::ADMMacros_SetLocalSpatialOrder: Initialize the local_spatial_order CartGrid3D::SpatialSpacings: Set up ranges for spatial 3D Cartesian coordinates (on all grids) CartGrid3D::SpatialCoordinates: Set up spatial 3D Cartesian coordinates on the GH SphericalSurface::SphericalSurface_SetupRes: [global] [loop-local] Set surface resolution automatically Dissipation::dissipation_basegrid: Ensure that there are enough ghost zones GRHydro::GRHydro_Reset_Execution_Flags: [global] Initially set execution flags to 'YEAH, Execute'! GRHydro::GRHydro_InitSymBound: Schedule symmetries and check shift state GRHydro::reset_GRHydro_C2P_failed: Initialise the mask function that contains the points where C2P has failed (at BASEGRID) Hydro_Analysis::Hydro_Analysis_Init: [global] Initialize variables ML_ADMConstraints::ML_ADMConstraints_CheckBoundaries: [meta] check boundaries treatment ML_BSSN::ML_BSSN_CheckBoundaries: [meta] check boundaries treatment NaNChecker::NaNChecker_ResetCounter: [global] Reset the NaNChecker::NaNsFound counter SpaceMask::MaskZero: Initialise mask to zero SpaceMask::MaskOne: Set old style mask to one SphericalSurface::SphericalSurface_Setup: [global] Calculate surface coordinate descriptors GROUP MaskBase_SetupMask: Set up the weight function GROUP MaskBase_SetupMaskAll: Set up the weight function CarpetReduce::MaskBase_AllocateMask: [global] Allocate the weight function CarpetReduce::MaskBase_InitMask: [global] [loop-local] Initialise the weight function GROUP SetupIMaskInternal: Set up the integer weight function (schedule other routines in here) CarpetReduce::CoordBase_SetupMask: [global] [loop-local] Set up the outer boundaries of the weight function CarpetReduce::CarpetMaskSetup: [global] [loop-singlemap] Set up the weight function for the restriction regions GROUP SetupIMask: Set up the integer weight function (schedule other routines in here) CarpetMask::CarpetExcludedSetup: [global] [loop-local] Set up the weight function for the excluded regions CarpetMask::CarpetSurfaceSetup: [global] [loop-local] Set up the weight function for the excluded spherical surfaces CarpetReduce::MaskBase_SetMask: [global] [loop-local] Set the weight function GROUP SetupMask: Set up the real weight function (schedule other routines in here) CarpetReduce::MaskBase_TestMask: [global] Test the weight function SphericalSurface::SphericalSurface_Set: [global] Set surface radii to be used for initial setup in other thorns GROUP SphericalSurface_HasBeenSet: Set the spherical surfaces before this group, and use it afterwards SphericalSurface::SphericalSurface_CheckState: [global] Test the state of the spherical surfaces SummationByParts::SBP_SetNormMask: Setup the mask for the calculation of the norm SymBase::SymBase_Check: Check whether the driver set up the grid consistently TerminationTrigger::TerminationTrigger_ResetTrigger: Clear trigger state TerminationTrigger::TerminationTrigger_StartTimer: Start timer TerminationTrigger::TerminationTrigger_CreateFile: Create termination file Time::Time_Initialise: [global] Initialise Time variables Time::TemporalSpacings: [singlemap] Set timestep based on Courant condition (courant_static) WeylScal4::WeylScal4_CheckBoundaries: [meta] check boundaries treatment [CCTK_RECOVER_VARIABLES] IOUtil::IOUtil_RecoverGH: [level] Checkpoint recovery routine [CCTK_POST_RECOVER_VARIABLES] CarpetIOHDF5::CarpetIOHDF5_InitCheckpointingIntervals: [global] Initialisation of checkpointing intervals after recovery CarpetIOHDF5::CarpetIOHDF5_CloseFiles: [meta] Close all initial data checkpoint files after recovery GROUP MaskBase_SetupMask: Set up the weight function GROUP MaskBase_SetupMaskAll: Set up the weight function CarpetReduce::MaskBase_AllocateMask: [global] Allocate the weight function CarpetReduce::MaskBase_InitMask: [global] [loop-local] Initialise the weight function GROUP SetupIMaskInternal: Set up the integer weight function (schedule other routines in here) CarpetReduce::CoordBase_SetupMask: [global] [loop-local] Set up the outer boundaries of the weight function CarpetReduce::CarpetMaskSetup: [global] [loop-singlemap] Set up the weight function for the restriction regions GROUP SetupIMask: Set up the integer weight function (schedule other routines in here) CarpetMask::CarpetExcludedSetup: [global] [loop-local] Set up the weight function for the excluded regions CarpetMask::CarpetSurfaceSetup: [global] [loop-local] Set up the weight function for the excluded spherical surfaces CarpetReduce::MaskBase_SetMask: [global] [loop-local] Set the weight function GROUP SetupMask: Set up the real weight function (schedule other routines in here) CarpetReduce::MaskBase_TestMask: [global] Test the weight function GRHydro::GRHydro_EOSHandle: [global] Set the EOS number GRHydro::GRHydroCopyIntegerMask: Initialize the real valued atmosphere mask after checkpoint recovery GRHydro::GRHydro_SqrtSpatialDeterminant: Calculate sdetg GROUP HydroBase_ExcisionMaskSetup: Set up hydro excision mask HydroBase::HydroBase_InitExcisionMask: Initialize hydro excision mask to 'no excision everywhere' GROUP zzz_NaNChecker_NaNCheck: Check for NaNs and count them in NaNChecker::NaNsFound NaNChecker::NaNChecker_NaNCheck_Prepare: [level] Prepare data structures to check for NaNs NaNChecker::NaNChecker_NaNCheck_Check: [local] Check for NaNs NaNChecker::NaNChecker_NaNCheck_Finish: [level] Count NaNs in NaNChecker::NaNsFound NaNChecker::NaNChecker_TakeAction: [global] [loop-level] Output NaNChecker::NaNmask and take action according to NaNChecker::action_if_found TerminationTrigger::TerminationTrigger_ResetMinutes: [global] Reset Watchtime endif if (checkpoint initial data) [CCTK_CPINITIAL] CarpetIOHDF5::CarpetIOHDF5_InitialDataCheckpoint: [meta] Initial data checkpoint routine endif if (analysis) [CCTK_ANALYSIS] CarpetLib::CarpetLib_printtimestats: [global] Print timing statistics if desired CarpetLib::CarpetLib_printmemstats: [global] Print memory statistics if desired Formaline::Formaline_AnnounceUpdate: [global] Put some meta information about the current run into permanent storage LoopControl::LC_statistics_analysis: [meta] Output LoopControl statistics GROUP ML_BSSN_EvolutionAnalysis: Calculate RHS at analysis ML_BSSN::ML_BSSN_EvolutionAnalysisInit: ML_BSSN_EvolutionAnalysisInit ML_BSSN::ML_BSSN_EvolutionAnalysisInterior: ML_BSSN_EvolutionAnalysisInterior ML_BSSN_Helper::ML_BSSN_NewRad: Apply NewRad boundary conditions to RHS Multipole::Multipole_Calc: [global] Calculate Multipoles QuasiLocalMeasures::qlm_calculate: [global] Calculate quasi-local quantities SystemStatistics::SystemStatistics_Collect: [global] Collect system statistics TerminationTrigger::TerminationTrigger_CheckWalltime: Check elapsed job walltime TerminationTrigger::TerminationTrigger_CheckSignal: Check if we received a termination signal TerminationTrigger::TerminationTrigger_CheckFile: Check termination file TimerReport::zzz_TimerReport_Output: [global] Print the timer report Trigger::Trigger_Check: [global] test endif Output grid variables do loop over timesteps [CCTK_PREREGRID] CarpetTracker::CarpetTracker_SetPositions: [global] Set positions of refined regions Change grid hierarchy [CCTK_POSTREGRID] CartGrid3D::SpatialCoordinates: Set Coordinates after regridding GROUP MaskBase_SetupMask: Set up the weight function GROUP MaskBase_SetupMaskAll: Set up the weight function CarpetReduce::MaskBase_AllocateMask: [global] Allocate the weight function CarpetReduce::MaskBase_InitMask: [global] [loop-local] Initialise the weight function GROUP SetupIMaskInternal: Set up the integer weight function (schedule other routines in here) CarpetReduce::CoordBase_SetupMask: [global] [loop-local] Set up the outer boundaries of the weight function CarpetReduce::CarpetMaskSetup: [global] [loop-singlemap] Set up the weight function for the restriction regions GROUP SetupIMask: Set up the integer weight function (schedule other routines in here) CarpetMask::CarpetExcludedSetup: [global] [loop-local] Set up the weight function for the excluded regions CarpetMask::CarpetSurfaceSetup: [global] [loop-local] Set up the weight function for the excluded spherical surfaces CarpetReduce::MaskBase_SetMask: [global] [loop-local] Set the weight function GROUP SetupMask: Set up the real weight function (schedule other routines in here) CarpetReduce::MaskBase_TestMask: [global] Test the weight function Dissipation::setup_epsdis: Setup spatially varying dissipation SpaceMask::MaskZero: Initialise mask to zero GROUP GRHydro_PrimitiveBoundaries: Apply boundary conditions to all primitives GRHydro::GRHydro_SelectPrimitiveBoundaries: [level] Select primitive variables for boundary conditions GROUP GRHydro_ApplyPrimitiveBCs: Apply boundary conditions to all primitive variables GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP HydroBase_ExcisionMaskSetup: Set up hydro excision mask HydroBase::HydroBase_InitExcisionMask: Initialize hydro excision mask to 'no excision everywhere' SpaceMask::MaskOne: Set mask to one GRHydro::GRHydro_SetupMask: Initialize the atmosphere mask GROUP MoL_PostStep: Ensure that everything is correct after regridding ML_BSSN::ML_BSSN_SelectBoundConds: [level] select boundary conditions GRHydro::GRHydro_RefinementLevel: Calculate current refinement level GRHydro::GRHydro_SetLastMoLPostStep: [level] Set grid scalar InLastMoLPostStep if this is the last MoL PostStep call GROUP ML_BSSN_ApplyBCs: Apply boundary conditions controlled by thorn Boundary GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions ML_ADMConstraints::ML_ADMConstraints_SelectBoundConds: [level] select boundary conditions GROUP ML_ADMConstraints_ApplyBCs: Apply boundary conditions controlled by thorn Boundary GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions ML_BSSN::ML_BSSN_ADMBaseInterior: ML_BSSN_ADMBaseInterior ML_BSSN::ML_BSSN_ADMBaseBoundaryScalar: ML_BSSN_ADMBaseBoundaryScalar ML_BSSN::ML_BSSN_ADMBaseEverywhere: ML_BSSN_ADMBaseEverywhere ML_BSSN_Helper::ML_BSSN_ADMBase_SelectBCs: [level] Select boundary conditions for ADMBase variables GROUP ML_BSSN_ADMBase_ApplyBCs: Apply boundary conditions to ADMBase variables GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP ADMBase_SetADMVars: Set the ADM variables before this group, and use them afterwards GROUP HydroBase_PostStep: Post step tasks for hydro thorns GROUP GRHydro_PostStep: Post step tasks for GRHydro GROUP GRHydro_AtmosphereMaskBoundaries: Apply boundary conditions to primitives GRHydro::GRHydro_SelectAtmosphereMaskBoundaries: [level] Select atmosphere mask for boundary conditions GROUP GRHydro_ApplyAtmosphereMaskBCs: Apply boundary conditions to real-valued atmosphere mask GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GRHydro::GRHydroPostSyncAtmosphereMask: Set integer atmosphere mask from synchronized real atmosphere mask if (GRHydro::InLastMoLPostStep) GRHydro::GRHydro_AtmosphereReset: Reset the atmosphere if (GRHydro::InLastMoLPostStep) GROUP GRHydro_PrimitiveInitialGuessesBoundaries: Apply boundary conditions to those primitives used as initial guesses GRHydro::GRHydro_SelectPrimitiveInitialGuessesBoundaries: [level] Select initial guess primitive variables for boudary conditions GROUP GRHydro_ApplyPrimitiveInitialGuessBCs: Apply boundary conditions to initial guess primitive variables GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP HydroBase_Boundaries: HydroBase-internal Boundary conditions group GROUP Do_GRHydro_Boundaries: GRHydro Boundary conditions group GROUP HydroBase_Select_Boundaries: Group to schedule the boundary condition functions if (GRHydro::execute_MoL_PostStep) GRHydro::GRHydro_Bound: [level] Select GRHydro boundary conditions GROUP HydroBase_ApplyBCs: Apply the boundary conditions of HydroBase GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP HydroBase_Con2Prim: Convert from conservative to primitive variables if (GRHydro::execute_MoL_Step) GRHydro::GRHydro_SqrtSpatialDeterminant: Calculate sdetg if (GRHydro::execute_MoL_PostStep) GRHydro::Con2Prim: Convert back to primitive variables (general) GROUP SetTmunu: Group for calculating the stress-energy tensor TmunuBase::TmunuBase_ZeroTmunu: Initialise the stress-energy tensor to zero GROUP AddToTmunu: Add to the stress-energy tensor here GRHydro::GRHydro_Tmunu: Compute the energy-momentum tensor WeylScal4::WeylScal4_SelectBoundConds: [level] select boundary conditions GROUP WeylScal4_ApplyBCs: Apply boundary conditions controlled by thorn Boundary GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP MoL_PseudoEvolutionBoundaries: Apply boundary conditions to pseudo-evolved quantities GROUP ML_ADMConstraints_evaluate_bc_group: ML_ADMConstraints_evaluate ML_ADMConstraints::ML_ADMConstraints_evaluate_SelectBCs: [level] ML_ADMConstraints_evaluate_SelectBCs GROUP ML_ADMConstraints_evaluate_ApplyBCs: Apply BCs for groups set in ML_ADMConstraints_evaluate GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP ML_BSSN_ConstraintsEverywhere_bc_group: ML_BSSN_ConstraintsEverywhere ML_BSSN::ML_BSSN_ConstraintsEverywhere_SelectBCs: [level] ML_BSSN_ConstraintsEverywhere_SelectBCs GROUP ML_BSSN_ConstraintsEverywhere_ApplyBCs: Apply BCs for groups set in ML_BSSN_ConstraintsEverywhere GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP ML_BSSN_ConstraintsInterior_bc_group: ML_BSSN_ConstraintsInterior ML_BSSN::ML_BSSN_ConstraintsInterior_SelectBCs: [level] ML_BSSN_ConstraintsInterior_SelectBCs GROUP ML_BSSN_ConstraintsInterior_ApplyBCs: Apply BCs for groups set in ML_BSSN_ConstraintsInterior GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP WeylScal4_psi4_calc_4th_bc_group: WeylScal4_psi4_calc_4th WeylScal4::WeylScal4_psi4_calc_4th_SelectBCs: [level] WeylScal4_psi4_calc_4th_SelectBCs GROUP WeylScal4_psi4_calc_4th_ApplyBCs: Apply BCs for groups set in WeylScal4_psi4_calc_4th GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions Rotate timelevels iteration = iteration+1 t = t+dt [CCTK_PRESTEP] CoordGauge::Einstein_SetNextSlicing: Identify the slicing for the next iteration GRHydro::reset_GRHydro_C2P_failed: Reset the mask function that contains the points where C2P has failed (at PRESTEP) LoopControl::LC_steer: [meta] Update LoopControl algorithm preferences NaNChecker::NaNChecker_ResetCounter: [global] Reset the NaNChecker::NaNsFound counter [CCTK_EVOL] MoL::MoL_StartLoop: [level] Initialise the step size control while (MoL::MoL_Stepsize_Bad) GROUP MoL_Evolution: A single Cactus evolution step using MoL GROUP MoL_StartStep: MoL internal setup for the evolution step MoL::MoL_SetCounter: [level] Set the counter for the ODE method to loop over MoL::MoL_SetTime: [level] Ensure the correct time and timestep are used MoL::MoL_AllocateScratchSpace: [level] Allocate storage for scratch levels GROUP MoL_PreStep: Physics thorns can schedule preloop setup routines in here GRHydro::GRHydro_Scalar_Setup: Set up and check scalars for efficiency MoL::MoL_AllocateScratch: Allocate sufficient space for array scratch variables MoL::MoL_InitialCopy: Ensure the data is in the correct timelevel while (MoL::MoL_Intermediate_Step) GROUP MoL_Step: The loop over the intermediate steps for the ODE integrator MoL::MoL_InitRHS: Initialise the RHS functions GROUP MoL_CalcRHS: Physics thorns schedule the calculation of the discrete spatial operator in here GROUP HydroBase_RHS: Groups for scheduling tasks for calculating RHS of hydro variables if (GRHydro::execute_MoL_Step) GROUP GRHydroRHS: Calculate the update terms GRHydro::SourceTerms: Source term calculation GRHydro::GRHydroStartLoop: [level] Set the flux_direction variable while (GRHydro::flux_direction) GROUP FluxTerms: Calculation of intercell fluxes GRHydro::GRHydro_RefinementLevel: Calculate current refinement level GRHydro::Reconstruct: Reconstruct the functions at the cell boundaries GRHydro::Riemann: Solve the local Riemann problems GRHydro::UpdateCalcul: Calculate the update term from the fluxes GRHydro::GRHydroAdvanceLoop: [level] Decrement the flux_direction variable end while GRHydro::GRHydroUpdateAtmosphereMask: Alter the update terms if inside the atmosphere region ML_BSSN::ML_BSSN_EvolutionBoundaryScalar: ML_BSSN_EvolutionBoundaryScalar GROUP ML_BSSN_EvolutionInteriorSplitBy: ML_BSSN::ML_BSSN_EvolutionInteriorSplitBy1: ML_BSSN_EvolutionInteriorSplitBy1 ML_BSSN::ML_BSSN_EvolutionInteriorSplitBy2: ML_BSSN_EvolutionInteriorSplitBy2 ML_BSSN::ML_BSSN_EvolutionInteriorSplitBy3: ML_BSSN_EvolutionInteriorSplitBy3 ML_BSSN_Helper::ML_BSSN_NewRad: Apply NewRad boundary conditions to RHS GROUP MoL_PostRHS: Modify RHS functions Dissipation::dissipation_add: Add Kreiss-Oliger dissipation to the right hand sides GROUP MoL_RHSBoundaries: Any 'final' modifications to the RHS functions (boundaries etc.) MoL::MoL_Add: Updates calculated with the efficient Runge-Kutta 4 method MoL::MoL_DecrementCounter: [level] Alter the counter number MoL::MoL_ResetTime: [level] If necessary, change the time GROUP MoL_PostStepModify: The group for physics thorns to schedule enforcing constraints ML_BSSN::ML_BSSN_EnforceEverywhere: ML_BSSN_EnforceEverywhere GROUP MoL_PostStep: The group for physics thorns to schedule boundary calls etc. ML_BSSN::ML_BSSN_SelectBoundConds: [level] select boundary conditions GRHydro::GRHydro_RefinementLevel: Calculate current refinement level GRHydro::GRHydro_SetLastMoLPostStep: [level] Set grid scalar InLastMoLPostStep if this is the last MoL PostStep call GROUP ML_BSSN_ApplyBCs: Apply boundary conditions controlled by thorn Boundary GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions ML_ADMConstraints::ML_ADMConstraints_SelectBoundConds: [level] select boundary conditions GROUP ML_ADMConstraints_ApplyBCs: Apply boundary conditions controlled by thorn Boundary GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions ML_BSSN::ML_BSSN_ADMBaseInterior: ML_BSSN_ADMBaseInterior ML_BSSN::ML_BSSN_ADMBaseBoundaryScalar: ML_BSSN_ADMBaseBoundaryScalar ML_BSSN::ML_BSSN_ADMBaseEverywhere: ML_BSSN_ADMBaseEverywhere ML_BSSN_Helper::ML_BSSN_ADMBase_SelectBCs: [level] Select boundary conditions for ADMBase variables GROUP ML_BSSN_ADMBase_ApplyBCs: Apply boundary conditions to ADMBase variables GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP ADMBase_SetADMVars: Set the ADM variables before this group, and use them afterwards GROUP HydroBase_PostStep: Post step tasks for hydro thorns GROUP GRHydro_PostStep: Post step tasks for GRHydro GROUP GRHydro_AtmosphereMaskBoundaries: Apply boundary conditions to primitives GRHydro::GRHydro_SelectAtmosphereMaskBoundaries: [level] Select atmosphere mask for boundary conditions GROUP GRHydro_ApplyAtmosphereMaskBCs: Apply boundary conditions to real-valued atmosphere mask GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GRHydro::GRHydroPostSyncAtmosphereMask: Set integer atmosphere mask from synchronized real atmosphere mask if (GRHydro::InLastMoLPostStep) GRHydro::GRHydro_AtmosphereReset: Reset the atmosphere if (GRHydro::InLastMoLPostStep) GROUP GRHydro_PrimitiveInitialGuessesBoundaries: Apply boundary conditions to those primitives used as initial guesses GRHydro::GRHydro_SelectPrimitiveInitialGuessesBoundaries: [level] Select initial guess primitive variables for boudary conditions GROUP GRHydro_ApplyPrimitiveInitialGuessBCs: Apply boundary conditions to initial guess primitive variables GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP HydroBase_Boundaries: HydroBase-internal Boundary conditions group GROUP Do_GRHydro_Boundaries: GRHydro Boundary conditions group GROUP HydroBase_Select_Boundaries: Group to schedule the boundary condition functions if (GRHydro::execute_MoL_PostStep) GRHydro::GRHydro_Bound: [level] Select GRHydro boundary conditions GROUP HydroBase_ApplyBCs: Apply the boundary conditions of HydroBase GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP HydroBase_Con2Prim: Convert from conservative to primitive variables if (GRHydro::execute_MoL_Step) GRHydro::GRHydro_SqrtSpatialDeterminant: Calculate sdetg if (GRHydro::execute_MoL_PostStep) GRHydro::Con2Prim: Convert back to primitive variables (general) GROUP SetTmunu: Group for calculating the stress-energy tensor TmunuBase::TmunuBase_ZeroTmunu: Initialise the stress-energy tensor to zero GROUP AddToTmunu: Add to the stress-energy tensor here GRHydro::GRHydro_Tmunu: Compute the energy-momentum tensor WeylScal4::WeylScal4_SelectBoundConds: [level] select boundary conditions GROUP WeylScal4_ApplyBCs: Apply boundary conditions controlled by thorn Boundary GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GRHydro::GRHydro_ClearLastMoLPostStep: [level] Reset InLastMoLPostStep to zero MoL::MoL_ResetDeltaTime: [level] If necessary, change the timestep end while MoL::MoL_FinishLoop: [level] Control the step size MoL::MoL_RestoreSandR: Restoring the Save and Restore variables to the original state MoL::MoL_FreeScratchSpace: [level] Free storage for scratch levels end while GRHydro::sync_GRHydro_C2P_failed: Syncronise the mask function that contains the points where C2P has failed GROUP MoL_PseudoEvolution: Calculate pseudo-evolved quantities GROUP ADMBase_SetADMVars: Set the ADM variables before this group, and use them afterwards GROUP ML_ADMConstraints_evaluate_group: ML_ADMConstraints_evaluate ML_ADMConstraints::ML_ADMConstraints_evaluate: ML_ADMConstraints_evaluate GROUP ML_ADMConstraints_evaluate_bc_group: ML_ADMConstraints_evaluate ML_ADMConstraints::ML_ADMConstraints_evaluate_SelectBCs: [level] ML_ADMConstraints_evaluate_SelectBCs GROUP ML_ADMConstraints_evaluate_ApplyBCs: Apply BCs for groups set in ML_ADMConstraints_evaluate GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP ML_BSSN_ConstraintsEverywhere_group: ML_BSSN_ConstraintsEverywhere ML_BSSN::ML_BSSN_ConstraintsEverywhere: ML_BSSN_ConstraintsEverywhere GROUP ML_BSSN_ConstraintsEverywhere_bc_group: ML_BSSN_ConstraintsEverywhere ML_BSSN::ML_BSSN_ConstraintsEverywhere_SelectBCs: [level] ML_BSSN_ConstraintsEverywhere_SelectBCs GROUP ML_BSSN_ConstraintsEverywhere_ApplyBCs: Apply BCs for groups set in ML_BSSN_ConstraintsEverywhere GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP ML_BSSN_ConstraintsInterior_group: ML_BSSN_ConstraintsInterior ML_BSSN::ML_BSSN_ConstraintsInterior: ML_BSSN_ConstraintsInterior GROUP ML_BSSN_ConstraintsInterior_bc_group: ML_BSSN_ConstraintsInterior ML_BSSN::ML_BSSN_ConstraintsInterior_SelectBCs: [level] ML_BSSN_ConstraintsInterior_SelectBCs GROUP ML_BSSN_ConstraintsInterior_ApplyBCs: Apply BCs for groups set in ML_BSSN_ConstraintsInterior GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP WeylScal4_psi4_calc_4th_group: WeylScal4_psi4_calc_4th WeylScal4::WeylScal4_psi4_calc_4th: WeylScal4_psi4_calc_4th GROUP WeylScal4_psi4_calc_4th_bc_group: WeylScal4_psi4_calc_4th WeylScal4::WeylScal4_psi4_calc_4th_SelectBCs: [level] WeylScal4_psi4_calc_4th_SelectBCs GROUP WeylScal4_psi4_calc_4th_ApplyBCs: Apply BCs for groups set in WeylScal4_psi4_calc_4th GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions Evolve finer grids recursively Restrict from finer grids [CCTK_POSTRESTRICT] GROUP MoL_PostStep: Ensure that everything is correct after restriction ML_BSSN::ML_BSSN_SelectBoundConds: [level] select boundary conditions GRHydro::GRHydro_RefinementLevel: Calculate current refinement level GRHydro::GRHydro_SetLastMoLPostStep: [level] Set grid scalar InLastMoLPostStep if this is the last MoL PostStep call GROUP ML_BSSN_ApplyBCs: Apply boundary conditions controlled by thorn Boundary GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions ML_ADMConstraints::ML_ADMConstraints_SelectBoundConds: [level] select boundary conditions GROUP ML_ADMConstraints_ApplyBCs: Apply boundary conditions controlled by thorn Boundary GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions ML_BSSN::ML_BSSN_ADMBaseInterior: ML_BSSN_ADMBaseInterior ML_BSSN::ML_BSSN_ADMBaseBoundaryScalar: ML_BSSN_ADMBaseBoundaryScalar ML_BSSN::ML_BSSN_ADMBaseEverywhere: ML_BSSN_ADMBaseEverywhere ML_BSSN_Helper::ML_BSSN_ADMBase_SelectBCs: [level] Select boundary conditions for ADMBase variables GROUP ML_BSSN_ADMBase_ApplyBCs: Apply boundary conditions to ADMBase variables GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP ADMBase_SetADMVars: Set the ADM variables before this group, and use them afterwards GROUP HydroBase_PostStep: Post step tasks for hydro thorns GROUP GRHydro_PostStep: Post step tasks for GRHydro GROUP GRHydro_AtmosphereMaskBoundaries: Apply boundary conditions to primitives GRHydro::GRHydro_SelectAtmosphereMaskBoundaries: [level] Select atmosphere mask for boundary conditions GROUP GRHydro_ApplyAtmosphereMaskBCs: Apply boundary conditions to real-valued atmosphere mask GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GRHydro::GRHydroPostSyncAtmosphereMask: Set integer atmosphere mask from synchronized real atmosphere mask if (GRHydro::InLastMoLPostStep) GRHydro::GRHydro_AtmosphereReset: Reset the atmosphere if (GRHydro::InLastMoLPostStep) GROUP GRHydro_PrimitiveInitialGuessesBoundaries: Apply boundary conditions to those primitives used as initial guesses GRHydro::GRHydro_SelectPrimitiveInitialGuessesBoundaries: [level] Select initial guess primitive variables for boudary conditions GROUP GRHydro_ApplyPrimitiveInitialGuessBCs: Apply boundary conditions to initial guess primitive variables GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP HydroBase_Boundaries: HydroBase-internal Boundary conditions group GROUP Do_GRHydro_Boundaries: GRHydro Boundary conditions group GROUP HydroBase_Select_Boundaries: Group to schedule the boundary condition functions if (GRHydro::execute_MoL_PostStep) GRHydro::GRHydro_Bound: [level] Select GRHydro boundary conditions GROUP HydroBase_ApplyBCs: Apply the boundary conditions of HydroBase GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP HydroBase_Con2Prim: Convert from conservative to primitive variables if (GRHydro::execute_MoL_Step) GRHydro::GRHydro_SqrtSpatialDeterminant: Calculate sdetg if (GRHydro::execute_MoL_PostStep) GRHydro::Con2Prim: Convert back to primitive variables (general) GROUP SetTmunu: Group for calculating the stress-energy tensor TmunuBase::TmunuBase_ZeroTmunu: Initialise the stress-energy tensor to zero GROUP AddToTmunu: Add to the stress-energy tensor here GRHydro::GRHydro_Tmunu: Compute the energy-momentum tensor WeylScal4::WeylScal4_SelectBoundConds: [level] select boundary conditions GROUP WeylScal4_ApplyBCs: Apply boundary conditions controlled by thorn Boundary GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP MoL_PseudoEvolutionBoundaries: Apply boundary conditions to pseudo-evolved quantities GROUP ML_ADMConstraints_evaluate_bc_group: ML_ADMConstraints_evaluate ML_ADMConstraints::ML_ADMConstraints_evaluate_SelectBCs: [level] ML_ADMConstraints_evaluate_SelectBCs GROUP ML_ADMConstraints_evaluate_ApplyBCs: Apply BCs for groups set in ML_ADMConstraints_evaluate GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP ML_BSSN_ConstraintsEverywhere_bc_group: ML_BSSN_ConstraintsEverywhere ML_BSSN::ML_BSSN_ConstraintsEverywhere_SelectBCs: [level] ML_BSSN_ConstraintsEverywhere_SelectBCs GROUP ML_BSSN_ConstraintsEverywhere_ApplyBCs: Apply BCs for groups set in ML_BSSN_ConstraintsEverywhere GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP ML_BSSN_ConstraintsInterior_bc_group: ML_BSSN_ConstraintsInterior ML_BSSN::ML_BSSN_ConstraintsInterior_SelectBCs: [level] ML_BSSN_ConstraintsInterior_SelectBCs GROUP ML_BSSN_ConstraintsInterior_ApplyBCs: Apply BCs for groups set in ML_BSSN_ConstraintsInterior GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP WeylScal4_psi4_calc_4th_bc_group: WeylScal4_psi4_calc_4th WeylScal4::WeylScal4_psi4_calc_4th_SelectBCs: [level] WeylScal4_psi4_calc_4th_SelectBCs GROUP WeylScal4_psi4_calc_4th_ApplyBCs: Apply BCs for groups set in WeylScal4_psi4_calc_4th GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions [CCTK_POSTSTEP] SphericalSurface::SphericalSurface_Set: [global] Set surface radii GROUP SphericalSurface_HasBeenSet: Set the spherical surfaces before this group, and use it afterwards SphericalSurface::SphericalSurface_CheckState: [global] Test the state of the spherical surfaces GRHydro::GRHydro_RefinementLevel: Calculate current refinement level (for the check of the C2P mask) GRHydro::check_GRHydro_C2P_failed: Check the mask function that contains the points where C2P has failed and report an error in case a failure is found HTTPD::HTTP_Work: [global] Working routine HTTPD::HTTP_ContentWork: [global] Content Working routine GROUP HydroBase_ExcisionHasBeenSet: Group to schedule thorns changing the mask before and thorns using the mask after GROUP Hydro_Analysis: Group for Hydro_Analysis routines Hydro_Analysis::Hydro_Analysis_PrepareReduction: [global] [loop-local] Compute the local reduction results Hydro_Analysis::Hydro_Analysis_Reduction: [global] Compute the global reduction results GROUP Hydro_Analysis_LocationSearch: Look for the location of the maximum density Hydro_Analysis::Hydro_Analysis_LocationSearch_Setup: [global] Prepare data structures for search Hydro_Analysis::Hydro_Analysis_LocationSearch_Search: [global] [loop-local] Search for the location of the maximum density Hydro_Analysis::Hydro_Analysis_LocationSearch_Combine: [global] Communicate and verify the location of the maximum density Hydro_Analysis::Hydro_Analysis_FindSeparation: [global] Compute the proper distance between the maximum of the density and the origin (along a straight coordinate line) NSTracker::NSTracker: [global] Look for the star and set grid accordingly GROUP zzz_NaNChecker_NaNCheck: Check for NaNs and count them in NaNChecker::NaNsFound NaNChecker::NaNChecker_NaNCheck_Prepare: [level] Prepare data structures to check for NaNs NaNChecker::NaNChecker_NaNCheck_Check: [local] Check for NaNs NaNChecker::NaNChecker_NaNCheck_Finish: [level] Count NaNs in NaNChecker::NaNsFound NaNChecker::NaNChecker_TakeAction: [global] [loop-level] Output NaNChecker::NaNmask and take action according to NaNChecker::action_if_found SpaceMask::CheckMask: Ensure that all mask values are legal Dissipation::setup_epsdis: Setup spatially varying dissipation GROUP MaskBase_SetupMask: Set up the weight function GROUP MaskBase_SetupMaskAll: Set up the weight function CarpetReduce::MaskBase_AllocateMask: [global] Allocate the weight function CarpetReduce::MaskBase_InitMask: [global] [loop-local] Initialise the weight function GROUP SetupIMaskInternal: Set up the integer weight function (schedule other routines in here) CarpetReduce::CoordBase_SetupMask: [global] [loop-local] Set up the outer boundaries of the weight function CarpetReduce::CarpetMaskSetup: [global] [loop-singlemap] Set up the weight function for the restriction regions GROUP SetupIMask: Set up the integer weight function (schedule other routines in here) CarpetMask::CarpetExcludedSetup: [global] [loop-local] Set up the weight function for the excluded regions CarpetMask::CarpetSurfaceSetup: [global] [loop-local] Set up the weight function for the excluded spherical surfaces CarpetReduce::MaskBase_SetMask: [global] [loop-local] Set the weight function GROUP SetupMask: Set up the real weight function (schedule other routines in here) CarpetReduce::MaskBase_TestMask: [global] Test the weight function if (checkpoint) [CCTK_CHECKPOINT] CarpetIOHDF5::CarpetIOHDF5_EvolutionCheckpoint: [meta] Evolution checkpoint routine TimerReport::zzz_TimerReport_Checkpoint: [global] Print the timer report endif if (analysis) [CCTK_ANALYSIS] CarpetLib::CarpetLib_printtimestats: [global] Print timing statistics if desired CarpetLib::CarpetLib_printmemstats: [global] Print memory statistics if desired Formaline::Formaline_AnnounceUpdate: [global] Put some meta information about the current run into permanent storage LoopControl::LC_statistics_analysis: [meta] Output LoopControl statistics GROUP ML_BSSN_EvolutionAnalysis: Calculate RHS at analysis ML_BSSN::ML_BSSN_EvolutionAnalysisInit: ML_BSSN_EvolutionAnalysisInit ML_BSSN::ML_BSSN_EvolutionAnalysisInterior: ML_BSSN_EvolutionAnalysisInterior ML_BSSN_Helper::ML_BSSN_NewRad: Apply NewRad boundary conditions to RHS Multipole::Multipole_Calc: [global] Calculate Multipoles QuasiLocalMeasures::qlm_calculate: [global] Calculate quasi-local quantities SystemStatistics::SystemStatistics_Collect: [global] Collect system statistics TerminationTrigger::TerminationTrigger_CheckWalltime: Check elapsed job walltime TerminationTrigger::TerminationTrigger_CheckSignal: Check if we received a termination signal TerminationTrigger::TerminationTrigger_CheckFile: Check termination file TimerReport::zzz_TimerReport_Output: [global] Print the timer report Trigger::Trigger_Check: [global] test endif Output grid variables enddo Termination routines [CCTK_TERMINATE] CarpetIOHDF5::CarpetIOHDF5_TerminationCheckpoint: [meta] Termination checkpoint routine Formaline::Formaline_AnnounceFinal: [global] Put some meta information about the current run into permanent storage LoopControl::LC_statistics_terminate: [meta] Output LoopControl statistics MoL::MoL_FreeIndexArrays: Free the MoL bookkeeping index arrays TimerReport::zzz_TimerReport_Output: [global] Print the timer report Shutdown routines [CCTK_SHUTDOWN] HTTPD::HTTP_Shutdown: [global] HTTP daemon shutdown Timers::Timer_Shutdown: Prepare hierarchical timers Routines run after changing the grid hierarchy: [CCTK_POSTREGRID] CartGrid3D::SpatialCoordinates: Set Coordinates after regridding GROUP MaskBase_SetupMask: Set up the weight function GROUP MaskBase_SetupMaskAll: Set up the weight function CarpetReduce::MaskBase_AllocateMask: [global] Allocate the weight function CarpetReduce::MaskBase_InitMask: [global] [loop-local] Initialise the weight function GROUP SetupIMaskInternal: Set up the integer weight function (schedule other routines in here) CarpetReduce::CoordBase_SetupMask: [global] [loop-local] Set up the outer boundaries of the weight function CarpetReduce::CarpetMaskSetup: [global] [loop-singlemap] Set up the weight function for the restriction regions GROUP SetupIMask: Set up the integer weight function (schedule other routines in here) CarpetMask::CarpetExcludedSetup: [global] [loop-local] Set up the weight function for the excluded regions CarpetMask::CarpetSurfaceSetup: [global] [loop-local] Set up the weight function for the excluded spherical surfaces CarpetReduce::MaskBase_SetMask: [global] [loop-local] Set the weight function GROUP SetupMask: Set up the real weight function (schedule other routines in here) CarpetReduce::MaskBase_TestMask: [global] Test the weight function Dissipation::setup_epsdis: Setup spatially varying dissipation SpaceMask::MaskZero: Initialise mask to zero GROUP GRHydro_PrimitiveBoundaries: Apply boundary conditions to all primitives GRHydro::GRHydro_SelectPrimitiveBoundaries: [level] Select primitive variables for boundary conditions GROUP GRHydro_ApplyPrimitiveBCs: Apply boundary conditions to all primitive variables GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP HydroBase_ExcisionMaskSetup: Set up hydro excision mask HydroBase::HydroBase_InitExcisionMask: Initialize hydro excision mask to 'no excision everywhere' SpaceMask::MaskOne: Set mask to one GRHydro::GRHydro_SetupMask: Initialize the atmosphere mask GROUP MoL_PostStep: Ensure that everything is correct after regridding ML_BSSN::ML_BSSN_SelectBoundConds: [level] select boundary conditions GRHydro::GRHydro_RefinementLevel: Calculate current refinement level GRHydro::GRHydro_SetLastMoLPostStep: [level] Set grid scalar InLastMoLPostStep if this is the last MoL PostStep call GROUP ML_BSSN_ApplyBCs: Apply boundary conditions controlled by thorn Boundary GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions ML_ADMConstraints::ML_ADMConstraints_SelectBoundConds: [level] select boundary conditions GROUP ML_ADMConstraints_ApplyBCs: Apply boundary conditions controlled by thorn Boundary GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions ML_BSSN::ML_BSSN_ADMBaseInterior: ML_BSSN_ADMBaseInterior ML_BSSN::ML_BSSN_ADMBaseBoundaryScalar: ML_BSSN_ADMBaseBoundaryScalar ML_BSSN::ML_BSSN_ADMBaseEverywhere: ML_BSSN_ADMBaseEverywhere ML_BSSN_Helper::ML_BSSN_ADMBase_SelectBCs: [level] Select boundary conditions for ADMBase variables GROUP ML_BSSN_ADMBase_ApplyBCs: Apply boundary conditions to ADMBase variables GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP ADMBase_SetADMVars: Set the ADM variables before this group, and use them afterwards GROUP HydroBase_PostStep: Post step tasks for hydro thorns GROUP GRHydro_PostStep: Post step tasks for GRHydro GROUP GRHydro_AtmosphereMaskBoundaries: Apply boundary conditions to primitives GRHydro::GRHydro_SelectAtmosphereMaskBoundaries: [level] Select atmosphere mask for boundary conditions GROUP GRHydro_ApplyAtmosphereMaskBCs: Apply boundary conditions to real-valued atmosphere mask GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GRHydro::GRHydroPostSyncAtmosphereMask: Set integer atmosphere mask from synchronized real atmosphere mask if (GRHydro::InLastMoLPostStep) GRHydro::GRHydro_AtmosphereReset: Reset the atmosphere if (GRHydro::InLastMoLPostStep) GROUP GRHydro_PrimitiveInitialGuessesBoundaries: Apply boundary conditions to those primitives used as initial guesses GRHydro::GRHydro_SelectPrimitiveInitialGuessesBoundaries: [level] Select initial guess primitive variables for boudary conditions GROUP GRHydro_ApplyPrimitiveInitialGuessBCs: Apply boundary conditions to initial guess primitive variables GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP HydroBase_Boundaries: HydroBase-internal Boundary conditions group GROUP Do_GRHydro_Boundaries: GRHydro Boundary conditions group GROUP HydroBase_Select_Boundaries: Group to schedule the boundary condition functions if (GRHydro::execute_MoL_PostStep) GRHydro::GRHydro_Bound: [level] Select GRHydro boundary conditions GROUP HydroBase_ApplyBCs: Apply the boundary conditions of HydroBase GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP HydroBase_Con2Prim: Convert from conservative to primitive variables if (GRHydro::execute_MoL_Step) GRHydro::GRHydro_SqrtSpatialDeterminant: Calculate sdetg if (GRHydro::execute_MoL_PostStep) GRHydro::Con2Prim: Convert back to primitive variables (general) GROUP SetTmunu: Group for calculating the stress-energy tensor TmunuBase::TmunuBase_ZeroTmunu: Initialise the stress-energy tensor to zero GROUP AddToTmunu: Add to the stress-energy tensor here GRHydro::GRHydro_Tmunu: Compute the energy-momentum tensor WeylScal4::WeylScal4_SelectBoundConds: [level] select boundary conditions GROUP WeylScal4_ApplyBCs: Apply boundary conditions controlled by thorn Boundary GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP MoL_PseudoEvolutionBoundaries: Apply boundary conditions to pseudo-evolved quantities GROUP ML_ADMConstraints_evaluate_bc_group: ML_ADMConstraints_evaluate ML_ADMConstraints::ML_ADMConstraints_evaluate_SelectBCs: [level] ML_ADMConstraints_evaluate_SelectBCs GROUP ML_ADMConstraints_evaluate_ApplyBCs: Apply BCs for groups set in ML_ADMConstraints_evaluate GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP ML_BSSN_ConstraintsEverywhere_bc_group: ML_BSSN_ConstraintsEverywhere ML_BSSN::ML_BSSN_ConstraintsEverywhere_SelectBCs: [level] ML_BSSN_ConstraintsEverywhere_SelectBCs GROUP ML_BSSN_ConstraintsEverywhere_ApplyBCs: Apply BCs for groups set in ML_BSSN_ConstraintsEverywhere GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP ML_BSSN_ConstraintsInterior_bc_group: ML_BSSN_ConstraintsInterior ML_BSSN::ML_BSSN_ConstraintsInterior_SelectBCs: [level] ML_BSSN_ConstraintsInterior_SelectBCs GROUP ML_BSSN_ConstraintsInterior_ApplyBCs: Apply BCs for groups set in ML_BSSN_ConstraintsInterior GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions GROUP WeylScal4_psi4_calc_4th_bc_group: WeylScal4_psi4_calc_4th WeylScal4::WeylScal4_psi4_calc_4th_SelectBCs: [level] WeylScal4_psi4_calc_4th_SelectBCs GROUP WeylScal4_psi4_calc_4th_ApplyBCs: Apply BCs for groups set in WeylScal4_psi4_calc_4th GROUP BoundaryConditions: Execute all boundary conditions Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries RotatingSymmetry180::Rot180_ComputeLevelExtent: [level] Compute extent of level RotatingSymmetry180::Rot180_ApplyBC: Apply 180 degree rotational symmetry boundary condition Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions -------------------------------------------------------------------------------- INFO (Carpet): Multi-Model listing: model 0: "world" INFO (Carpet): Multi-Model process distribution: process 0: model 0 "world" INFO (Carpet): Multi-Model: This is process 0, model 0 "world" INFO (CarpetLib): Process startup time was 1.06 seconds Current core file size limit: hard=[unlimited], soft=[unlimited] Current addres space size limit: hard=[unlimited], soft=[unlimited] Current data segment size limit: hard=[unlimited], soft=[unlimited] Current resident set size limit: hard=[unlimited], soft=[unlimited] INFO (CycleClock): Measuring CycleClock tick via OpenMP... INFO (CycleClock): Calibrated CycleClock: 0.34549 ns per clock tick (2.89444 GHz) INFO (hwloc): library version 2.0.4, API version 0x20000 INFO (SystemTopology): MPI process-to-host mapping: This is MPI process 0 of 1 MPI hosts: 0: panther This MPI process runs on host 0 of 1 On this host, this is MPI process 0 of 1 INFO (SystemTopology): Topology support: Discovery support: discovery->pu : yes CPU binding support: cpubind->set_thisproc_cpubind : yes cpubind->get_thisproc_cpubind : yes cpubind->set_proc_cpubind : yes cpubind->get_proc_cpubind : yes cpubind->set_thisthread_cpubind : yes cpubind->get_thisthread_cpubind : yes cpubind->set_thread_cpubind : yes cpubind->get_thread_cpubind : yes cpubind->get_thisproc_last_cpu_location : yes cpubind->get_proc_last_cpu_location : yes cpubind->get_thisthread_last_cpu_location: yes Memory binding support: membind->set_thisproc_membind : no membind->get_thisproc_membind : no membind->set_proc_membind : no membind->get_proc_membind : no membind->set_thisthread_membind : yes membind->get_thisthread_membind : yes membind->set_area_membind : yes membind->get_area_membind : yes membind->alloc_membind : yes membind->firsttouch_membind : yes membind->bind_membind : yes membind->interleave_membind : yes membind->nexttouch_membind : no membind->migrate_membind : yes INFO (SystemTopology): Hardware objects in this node: Machine L#0: (P#0, total=1043114796KB, DMIProductName="PowerEdge R7515", DMIProductVersion=, DMIBoardVendor="Dell Inc.", DMIBoardName=04F3CJ, DMIBoardVersion=A02, DMIChassisVendor="Dell Inc.", DMIChassisType=23, DMIChassisVersion=, DMIChassisAssetTag=, DMIBIOSVendor="Dell Inc.", DMIBIOSVersion=2.18.1, DMIBIOSDate=02/03/2025, DMISysVendor="Dell Inc.", Backend=Linux, OSName=Linux, OSRelease=5.14.0-570.37.1.el9_6.x86_64, OSVersion="#1 SMP PREEMPT_DYNAMIC Thu Aug 28 10:41:06 UTC 2025", HostName=panther, Architecture=x86_64, hwlocVersion=2.0.4, ProcessName=cactus_sim) Package L#0: (P#0, total=1043114796KB, CPUVendor=AuthenticAMD, CPUFamilyNumber=23, CPUModelNumber=49, CPUModel="AMD EPYC 7542 32-Core Processor ", CPUStepping=0) L3Cache L#0: (P#-1, size=16384KB, linesize=64, ways=16, Inclusive=0) L2Cache L#0: (P#-1, size=512KB, linesize=64, ways=8, Inclusive=1) L1dCache L#0: (P#-1, size=32KB, linesize=64, ways=8, Inclusive=0) Core L#0: (P#0) PU L#0: (P#0) PU L#1: (P#32) L2Cache L#1: (P#-1, size=512KB, linesize=64, ways=8, Inclusive=1) L1dCache L#1: (P#-1, size=32KB, linesize=64, ways=8, Inclusive=0) Core L#1: (P#1) PU L#2: (P#1) PU L#3: (P#33) L2Cache L#2: (P#-1, size=512KB, linesize=64, ways=8, Inclusive=1) L1dCache L#2: (P#-1, size=32KB, linesize=64, ways=8, Inclusive=0) Core L#2: (P#2) PU L#4: (P#2) PU L#5: (P#34) L2Cache L#3: (P#-1, size=512KB, linesize=64, ways=8, Inclusive=1) L1dCache L#3: (P#-1, size=32KB, linesize=64, ways=8, Inclusive=0) Core L#3: (P#3) PU L#6: (P#3) PU L#7: (P#35) L3Cache L#1: (P#-1, size=16384KB, linesize=64, ways=16, Inclusive=0) L2Cache L#4: (P#-1, size=512KB, linesize=64, ways=8, Inclusive=1) L1dCache L#4: (P#-1, size=32KB, linesize=64, ways=8, Inclusive=0) Core L#4: (P#4) PU L#8: (P#4) PU L#9: (P#36) L2Cache L#5: (P#-1, size=512KB, linesize=64, ways=8, Inclusive=1) L1dCache L#5: (P#-1, size=32KB, linesize=64, ways=8, Inclusive=0) Core L#5: (P#5) PU L#10: (P#5) PU L#11: (P#37) L2Cache L#6: (P#-1, size=512KB, linesize=64, ways=8, Inclusive=1) L1dCache L#6: (P#-1, size=32KB, linesize=64, ways=8, Inclusive=0) Core L#6: (P#6) PU L#12: (P#6) PU L#13: (P#38) L2Cache L#7: (P#-1, size=512KB, linesize=64, ways=8, Inclusive=1) L1dCache L#7: (P#-1, size=32KB, linesize=64, ways=8, Inclusive=0) Core L#7: (P#7) PU L#14: (P#7) PU L#15: (P#39) L3Cache L#2: (P#-1, size=16384KB, linesize=64, ways=16, Inclusive=0) L2Cache L#8: (P#-1, size=512KB, linesize=64, ways=8, Inclusive=1) L1dCache L#8: (P#-1, size=32KB, linesize=64, ways=8, Inclusive=0) Core L#8: (P#8) PU L#16: (P#8) PU L#17: (P#40) L2Cache L#9: (P#-1, size=512KB, linesize=64, ways=8, Inclusive=1) L1dCache L#9: (P#-1, size=32KB, linesize=64, ways=8, Inclusive=0) Core L#9: (P#9) PU L#18: (P#9) PU L#19: (P#41) L2Cache L#10: (P#-1, size=512KB, linesize=64, ways=8, Inclusive=1) L1dCache L#10: (P#-1, size=32KB, linesize=64, ways=8, Inclusive=0) Core L#10: (P#10) PU L#20: (P#10) PU L#21: (P#42) L2Cache L#11: (P#-1, size=512KB, linesize=64, ways=8, Inclusive=1) L1dCache L#11: (P#-1, size=32KB, linesize=64, ways=8, Inclusive=0) Core L#11: (P#11) PU L#22: (P#11) PU L#23: (P#43) L3Cache L#3: (P#-1, size=16384KB, linesize=64, ways=16, Inclusive=0) L2Cache L#12: (P#-1, size=512KB, linesize=64, ways=8, Inclusive=1) L1dCache L#12: (P#-1, size=32KB, linesize=64, ways=8, Inclusive=0) Core L#12: (P#12) PU L#24: (P#12) PU L#25: (P#44) L2Cache L#13: (P#-1, size=512KB, linesize=64, ways=8, Inclusive=1) L1dCache L#13: (P#-1, size=32KB, linesize=64, ways=8, Inclusive=0) Core L#13: (P#13) PU L#26: (P#13) PU L#27: (P#45) L2Cache L#14: (P#-1, size=512KB, linesize=64, ways=8, Inclusive=1) L1dCache L#14: (P#-1, size=32KB, linesize=64, ways=8, Inclusive=0) Core L#14: (P#14) PU L#28: (P#14) PU L#29: (P#46) L2Cache L#15: (P#-1, size=512KB, linesize=64, ways=8, Inclusive=1) L1dCache L#15: (P#-1, size=32KB, linesize=64, ways=8, Inclusive=0) Core L#15: (P#15) PU L#30: (P#15) PU L#31: (P#47) L3Cache L#4: (P#-1, size=16384KB, linesize=64, ways=16, Inclusive=0) L2Cache L#16: (P#-1, size=512KB, linesize=64, ways=8, Inclusive=1) L1dCache L#16: (P#-1, size=32KB, linesize=64, ways=8, Inclusive=0) Core L#16: (P#16) PU L#32: (P#16) PU L#33: (P#48) L2Cache L#17: (P#-1, size=512KB, linesize=64, ways=8, Inclusive=1) L1dCache L#17: (P#-1, size=32KB, linesize=64, ways=8, Inclusive=0) Core L#17: (P#17) PU L#34: (P#17) PU L#35: (P#49) L2Cache L#18: (P#-1, size=512KB, linesize=64, ways=8, Inclusive=1) L1dCache L#18: (P#-1, size=32KB, linesize=64, ways=8, Inclusive=0) Core L#18: (P#18) PU L#36: (P#18) PU L#37: (P#50) L2Cache L#19: (P#-1, size=512KB, linesize=64, ways=8, Inclusive=1) L1dCache L#19: (P#-1, size=32KB, linesize=64, ways=8, Inclusive=0) Core L#19: (P#19) PU L#38: (P#19) PU L#39: (P#51) L3Cache L#5: (P#-1, size=16384KB, linesize=64, ways=16, Inclusive=0) L2Cache L#20: (P#-1, size=512KB, linesize=64, ways=8, Inclusive=1) L1dCache L#20: (P#-1, size=32KB, linesize=64, ways=8, Inclusive=0) Core L#20: (P#20) PU L#40: (P#20) PU L#41: (P#52) L2Cache L#21: (P#-1, size=512KB, linesize=64, ways=8, Inclusive=1) L1dCache L#21: (P#-1, size=32KB, linesize=64, ways=8, Inclusive=0) Core L#21: (P#21) PU L#42: (P#21) PU L#43: (P#53) L2Cache L#22: (P#-1, size=512KB, linesize=64, ways=8, Inclusive=1) L1dCache L#22: (P#-1, size=32KB, linesize=64, ways=8, Inclusive=0) Core L#22: (P#22) PU L#44: (P#22) PU L#45: (P#54) L2Cache L#23: (P#-1, size=512KB, linesize=64, ways=8, Inclusive=1) L1dCache L#23: (P#-1, size=32KB, linesize=64, ways=8, Inclusive=0) Core L#23: (P#23) PU L#46: (P#23) PU L#47: (P#55) L3Cache L#6: (P#-1, size=16384KB, linesize=64, ways=16, Inclusive=0) L2Cache L#24: (P#-1, size=512KB, linesize=64, ways=8, Inclusive=1) L1dCache L#24: (P#-1, size=32KB, linesize=64, ways=8, Inclusive=0) Core L#24: (P#24) PU L#48: (P#24) PU L#49: (P#56) L2Cache L#25: (P#-1, size=512KB, linesize=64, ways=8, Inclusive=1) L1dCache L#25: (P#-1, size=32KB, linesize=64, ways=8, Inclusive=0) Core L#25: (P#25) PU L#50: (P#25) PU L#51: (P#57) L2Cache L#26: (P#-1, size=512KB, linesize=64, ways=8, Inclusive=1) L1dCache L#26: (P#-1, size=32KB, linesize=64, ways=8, Inclusive=0) Core L#26: (P#26) PU L#52: (P#26) PU L#53: (P#58) L2Cache L#27: (P#-1, size=512KB, linesize=64, ways=8, Inclusive=1) L1dCache L#27: (P#-1, size=32KB, linesize=64, ways=8, Inclusive=0) Core L#27: (P#27) PU L#54: (P#27) PU L#55: (P#59) L3Cache L#7: (P#-1, size=16384KB, linesize=64, ways=16, Inclusive=0) L2Cache L#28: (P#-1, size=512KB, linesize=64, ways=8, Inclusive=1) L1dCache L#28: (P#-1, size=32KB, linesize=64, ways=8, Inclusive=0) Core L#28: (P#28) PU L#56: (P#28) PU L#57: (P#60) L2Cache L#29: (P#-1, size=512KB, linesize=64, ways=8, Inclusive=1) L1dCache L#29: (P#-1, size=32KB, linesize=64, ways=8, Inclusive=0) Core L#29: (P#29) PU L#58: (P#29) PU L#59: (P#61) L2Cache L#30: (P#-1, size=512KB, linesize=64, ways=8, Inclusive=1) L1dCache L#30: (P#-1, size=32KB, linesize=64, ways=8, Inclusive=0) Core L#30: (P#30) PU L#60: (P#30) PU L#61: (P#62) L2Cache L#31: (P#-1, size=512KB, linesize=64, ways=8, Inclusive=1) L1dCache L#31: (P#-1, size=32KB, linesize=64, ways=8, Inclusive=0) Core L#31: (P#31) PU L#62: (P#31) PU L#63: (P#63) INFO (SystemTopology): Thread CPU bindings: MPI process 0 on host 0 (process 0 of 1 on this host) OpenMP thread 0: PU set L#{0-63} P#{0-63} INFO (SystemTopology): Setting thread CPU bindings: INFO (SystemTopology): Thread CPU bindings: MPI process 0 on host 0 (process 0 of 1 on this host) OpenMP thread 0: PU set L#{0} P#{0} INFO (SystemTopology): Extracting CPU/cache/memory properties: There are 2 PUs per core (aka hardware SMT threads) There are 1 threads per core (aka SMT threads used) Cache (unknown name) has type "data" depth 1 size 32768 linesize 64 associativity 8 stride 4096, for 2 PUs Cache (unknown name) has type "unified" depth 2 size 524288 linesize 64 associativity 8 stride 65536, for 2 PUs Cache (unknown name) has type "unified" depth 3 size 16777216 linesize 64 associativity 16 stride 1048576, for 8 PUs INFO (Formaline): Configuration id: config-sim-panther-system-user-crangano-einstein_toolkit-BNS INFO (Formaline): Build id: build-sim-panther-crangano-2025.09.15-19.43.32-3940620 INFO (Formaline): Simulation id: run-bns-panther-crangano-2025.09.15-20.32.02-3948578 INFO (Formaline): Run id: run-bns-panther-crangano-2025.09.15-20.32.02-3948578 Server started on http://panther:5555/ Not announcing location via Twitter. INFO (Vectors): Using vector size 2 for architecture SSE2+SSE4.1 (64-bit precision) -------------------------------------------------------------------------------- AMR driver provided by Carpet -------------------------------------------------------------------------------- GRHydro: relativistic hydrodynamics, no ice. -------------------------------------------------------------------------------- HydroBase: Let it flow. -------------------------------------------------------------------------------- AMR info I/O provided by CarpetIOBasic -------------------------------------------------------------------------------- ML_ADMConstraints -------------------------------------------------------------------------------- ML_BSSN -------------------------------------------------------------------------------- AMR HDF5 I/O provided by CarpetIOHDF5 -------------------------------------------------------------------------------- AMR 0D HDF5 I/O provided by CarpetIOHDF5 -------------------------------------------------------------------------------- AMR 1D HDF5 I/O provided by CarpetIOHDF5 -------------------------------------------------------------------------------- AMR 2D HDF5 I/O provided by CarpetIOHDF5 -------------------------------------------------------------------------------- AMR 3D HDF5 I/O provided by CarpetIOHDF5 -------------------------------------------------------------------------------- MoL: Generalized time integration. -------------------------------------------------------------------------------- AMR scalar I/O provided by CarpetIOScalar -------------------------------------------------------------------------------- AMR 0D ASCII I/O provided by CarpetIOASCII -------------------------------------------------------------------------------- AMR 1D ASCII I/O provided by CarpetIOASCII -------------------------------------------------------------------------------- AMR 2D ASCII I/O provided by CarpetIOASCII -------------------------------------------------------------------------------- AMR 3D ASCII I/O provided by CarpetIOASCII -------------------------------------------------------------------------------- WeylScal4 -------------------------------------------------------------------------------- INFO (Carpet): MPI is enabled INFO (Carpet): Carpet is running on 1 processes INFO (Carpet): This is process 0 INFO (Carpet): OpenMP is enabled INFO (Carpet): This process contains 1 threads, this is thread 0 INFO (Carpet): There are 1 threads in total INFO (Carpet): There are 1 threads per process INFO (Carpet): This process runs on host panther, pid=3948578 INFO (Carpet): This process runs on 1 core: 0 INFO (Carpet): Thread 0 runs on 1 core: 0 INFO (Carpet): This simulation is running in 3 dimensions INFO (Carpet): Boundary specification for map 0: nboundaryzones: [[3,3,3],[3,3,3]] is_internal : [[0,0,0],[0,0,0]] is_staggered : [[0,0,0],[0,0,0]] shiftout : [[1,0,1],[0,0,0]] INFO (Carpet): CoordBase domain specification for map 0: physical extent: [0,-400,0] : [400,400,400] ([400,800,400]) interior extent: [0,-392,0] : [392,392,392] ([392,784,392]) exterior extent: [-24,-416,-24] : [416,416,416] ([440,832,440]) base_spacing : [8,8,8] INFO (Carpet): Adapted domain specification for map 0: convergence factor: 2 convergence level : 0 physical extent : [0,-400,0] : [400,400,400] ([400,800,400]) interior extent : [0,-392,0] : [392,392,392] ([392,784,392]) exterior extent : [-24,-416,-24] : [416,416,416] ([440,832,440]) spacing : [8,8,8] INFO (Carpet): Base grid specification for map 0: number of grid points : [56,105,56] number of coarse grid ghost points: [[3,3,3],[3,3,3]] INFO (Carpet): Buffer zone counts (excluding ghosts): [0]: [[0,0,0],[0,0,0]] [1]: [[9,9,9],[9,9,9]] [2]: [[9,9,9],[9,9,9]] [3]: [[9,9,9],[9,9,9]] [4]: [[9,9,9],[9,9,9]] [5]: [[9,9,9],[9,9,9]] [6]: [[9,9,9],[9,9,9]] [7]: [[9,9,9],[9,9,9]] [8]: [[9,9,9],[9,9,9]] INFO (Carpet): Overlap zone counts: [0]: [[0,0,0],[0,0,0]] [1]: [[0,0,0],[0,0,0]] [2]: [[0,0,0],[0,0,0]] [3]: [[0,0,0],[0,0,0]] [4]: [[0,0,0],[0,0,0]] [5]: [[0,0,0],[0,0,0]] [6]: [[0,0,0],[0,0,0]] [7]: [[0,0,0],[0,0,0]] [8]: [[0,0,0],[0,0,0]] INFO (Carpet): Group and variable statistics: INFO (Carpet): There are 1162 grid functions in 159 groups INFO (Carpet): There are 604 grid scalars in 96 groups INFO (Carpet): There are 130 1-dimensional grid arrays in 13 groups INFO (Carpet): There are 143 2-dimensional grid arrays in 18 groups INFO (Carpet): There are 0 3-dimensional grid arrays in 0 groups INFO (Carpet): (The number of variables counts all time levels) INFO (CarpetIOHDF5): I/O Method 'IOHDF5' registered: AMR output of grid variables to HDF5 files INFO (CarpetIOHDF5): I/O Method 'IOHDF5_0D' registered: 0D AMR output of grid variables to HDF5 files INFO (CarpetIOHDF5): I/O Method 'IOHDF5_1D' registered: 1D AMR output of grid variables to HDF5 files INFO (CarpetIOHDF5): I/O Method 'IOHDF5_2D' registered: 2D AMR output of grid variables to HDF5 files INFO (CarpetIOHDF5): Periodic 2D AMR output requested for: ADMBASE::gxx ADMBASE::gxy ADMBASE::gxz ADMBASE::gyy ADMBASE::gyz ADMBASE::gzz ADMBASE::alp ADMBASE::betax ADMBASE::betay ADMBASE::betaz CARPETREDUCE::weight GRID::x GRID::y GRID::z GRID::r HYDROBASE::rho HYDROBASE::eps HYDROBASE::vel[0] HYDROBASE::vel[1] HYDROBASE::vel[2] ML_ADMCONSTRAINTS::H INFO (CarpetIOHDF5): I/O Method 'IOHDF5_3D' registered: 3D AMR output of grid variables to HDF5 files INFO (CarpetIOHDF5): Periodic 3D AMR output requested for: ADMBASE::alp ADMBASE::betax ADMBASE::betay ADMBASE::betaz CARPETREDUCE::weight GRID::x GRID::y GRID::z GRID::r HYDROBASE::rho HYDROBASE::eps HYDROBASE::vel[0] HYDROBASE::vel[1] HYDROBASE::vel[2] ML_ADMCONSTRAINTS::H INFO (CarpetIOScalar): Periodic scalar output requested for: ADMBASE::gxx ADMBASE::gxy ADMBASE::gxz ADMBASE::gyy ADMBASE::gyz ADMBASE::gzz ADMBASE::kxx ADMBASE::kxy ADMBASE::kxz ADMBASE::kyy ADMBASE::kyz ADMBASE::kzz ADMBASE::alp ADMBASE::betax ADMBASE::betay ADMBASE::betaz GRHYDRO::dens HYDROBASE::rho HYDROBASE::vel[0] HYDROBASE::vel[1] HYDROBASE::vel[2] HYDROBASE::w_lorentz ML_ADMCONSTRAINTS::H SPHERICALSURFACE::sf_radius[0] SPHERICALSURFACE::sf_radius[1] SPHERICALSURFACE::sf_radius[2] SPHERICALSURFACE::sf_radius[3] SPHERICALSURFACE::sf_radius[4] SYSTEMSTATISTICS::maxrss_mb SYSTEMSTATISTICS::majflt_mb SYSTEMSTATISTICS::arena_mb SYSTEMSTATISTICS::ordblks_mb SYSTEMSTATISTICS::hblks_mb SYSTEMSTATISTICS::hblkhd_mb SYSTEMSTATISTICS::uordblks_mb SYSTEMSTATISTICS::fordblks_mb SYSTEMSTATISTICS::keepcost_mb SYSTEMSTATISTICS::swap_used_mb INFO (CarpetIOASCII): I/O Method 'IOASCII_0D' registered: 0D AMR output of grid variables to ASCII files INFO (CarpetIOASCII): Periodic 0D AMR output requested for: CARPET::physical_time_per_hour CARPET::current_physical_time_per_hour CARPET::time_total CARPET::time_evolution CARPET::time_computing CARPET::time_communicating CARPET::time_io CARPET::evolution_steps_count CARPET::local_grid_points_per_second CARPET::total_grid_points_per_second CARPET::local_grid_point_updates_count CARPET::total_grid_point_updates_count CARPET::local_interior_points_per_second CARPET::total_interior_points_per_second CARPET::local_interior_point_updates_count CARPET::total_interior_point_updates_count CARPET::io_per_second CARPET::io_bytes_per_second CARPET::io_bytes_ascii_per_second CARPET::io_bytes_binary_per_second CARPET::io_count CARPET::io_bytes_count CARPET::io_bytes_ascii_count CARPET::io_bytes_binary_count CARPET::comm_per_second CARPET::comm_bytes_per_second CARPET::comm_count CARPET::comm_bytes_count CARPET::time_levels CARPET::current_walltime CARPET::syncs_count HYDRO_ANALYSIS::Hydro_Analysis_rho_max_loc[0] HYDRO_ANALYSIS::Hydro_Analysis_rho_max_loc[1] HYDRO_ANALYSIS::Hydro_Analysis_rho_max_loc[2] HYDRO_ANALYSIS::Hydro_Analysis_rho_max_origin_distance QUASILOCALMEASURES::qlm_time[0] QUASILOCALMEASURES::qlm_time[1] QUASILOCALMEASURES::qlm_equatorial_circumference[0] QUASILOCALMEASURES::qlm_equatorial_circumference[1] QUASILOCALMEASURES::qlm_polar_circumference_0[0] QUASILOCALMEASURES::qlm_polar_circumference_0[1] QUASILOCALMEASURES::qlm_polar_circumference_pi_2[0] QUASILOCALMEASURES::qlm_polar_circumference_pi_2[1] QUASILOCALMEASURES::qlm_area[0] QUASILOCALMEASURES::qlm_area[1] QUASILOCALMEASURES::qlm_irreducible_mass[0] QUASILOCALMEASURES::qlm_irreducible_mass[1] QUASILOCALMEASURES::qlm_radius[0] QUASILOCALMEASURES::qlm_radius[1] QUASILOCALMEASURES::qlm_spin_guess[0] QUASILOCALMEASURES::qlm_spin_guess[1] QUASILOCALMEASURES::qlm_mass_guess[0] QUASILOCALMEASURES::qlm_mass_guess[1] QUASILOCALMEASURES::qlm_killing_eigenvalue_re[0] QUASILOCALMEASURES::qlm_killing_eigenvalue_re[1] QUASILOCALMEASURES::qlm_killing_eigenvalue_im[0] QUASILOCALMEASURES::qlm_killing_eigenvalue_im[1] QUASILOCALMEASURES::qlm_spin[0] QUASILOCALMEASURES::qlm_spin[1] QUASILOCALMEASURES::qlm_npspin[0] QUASILOCALMEASURES::qlm_npspin[1] QUASILOCALMEASURES::qlm_wsspin[0] QUASILOCALMEASURES::qlm_wsspin[1] QUASILOCALMEASURES::qlm_cvspin[0] QUASILOCALMEASURES::qlm_cvspin[1] QUASILOCALMEASURES::qlm_coordspinx[0] QUASILOCALMEASURES::qlm_coordspinx[1] QUASILOCALMEASURES::qlm_coordspiny[0] QUASILOCALMEASURES::qlm_coordspiny[1] QUASILOCALMEASURES::qlm_coordspinz[0] QUASILOCALMEASURES::qlm_coordspinz[1] QUASILOCALMEASURES::qlm_mass[0] QUASILOCALMEASURES::qlm_mass[1] QUASILOCALMEASURES::qlm_adm_energy[0] QUASILOCALMEASURES::qlm_adm_energy[1] QUASILOCALMEASURES::qlm_adm_momentum_x[0] QUASILOCALMEASURES::qlm_adm_momentum_x[1] QUASILOCALMEASURES::qlm_adm_momentum_y[0] QUASILOCALMEASURES::qlm_adm_momentum_y[1] QUASILOCALMEASURES::qlm_adm_momentum_z[0] QUASILOCALMEASURES::qlm_adm_momentum_z[1] QUASILOCALMEASURES::qlm_adm_angular_momentum_x[0] QUASILOCALMEASURES::qlm_adm_angular_momentum_x[1] QUASILOCALMEASURES::qlm_adm_angular_momentum_y[0] QUASILOCALMEASURES::qlm_adm_angular_momentum_y[1] QUASILOCALMEASURES::qlm_adm_angular_momentum_z[0] QUASILOCALMEASURES::qlm_adm_angular_momentum_z[1] QUASILOCALMEASURES::qlm_w_energy[0] QUASILOCALMEASURES::qlm_w_energy[1] QUASILOCALMEASURES::qlm_w_momentum_x[0] QUASILOCALMEASURES::qlm_w_momentum_x[1] QUASILOCALMEASURES::qlm_w_momentum_y[0] QUASILOCALMEASURES::qlm_w_momentum_y[1] QUASILOCALMEASURES::qlm_w_momentum_z[0] QUASILOCALMEASURES::qlm_w_momentum_z[1] QUASILOCALMEASURES::qlm_w_angular_momentum_x[0] QUASILOCALMEASURES::qlm_w_angular_momentum_x[1] QUASILOCALMEASURES::qlm_w_angular_momentum_y[0] QUASILOCALMEASURES::qlm_w_angular_momentum_y[1] QUASILOCALMEASURES::qlm_w_angular_momentum_z[0] QUASILOCALMEASURES::qlm_w_angular_momentum_z[1] SPHERICALSURFACE::sf_active[0] SPHERICALSURFACE::sf_active[1] SPHERICALSURFACE::sf_active[2] SPHERICALSURFACE::sf_active[3] SPHERICALSURFACE::sf_active[4] SPHERICALSURFACE::sf_valid[0] SPHERICALSURFACE::sf_valid[1] SPHERICALSURFACE::sf_valid[2] SPHERICALSURFACE::sf_valid[3] SPHERICALSURFACE::sf_valid[4] SPHERICALSURFACE::sf_area[0] SPHERICALSURFACE::sf_area[1] SPHERICALSURFACE::sf_area[2] SPHERICALSURFACE::sf_area[3] SPHERICALSURFACE::sf_area[4] SPHERICALSURFACE::sf_mean_radius[0] SPHERICALSURFACE::sf_mean_radius[1] SPHERICALSURFACE::sf_mean_radius[2] SPHERICALSURFACE::sf_mean_radius[3] SPHERICALSURFACE::sf_mean_radius[4] SPHERICALSURFACE::sf_centroid_x[0] SPHERICALSURFACE::sf_centroid_x[1] SPHERICALSURFACE::sf_centroid_x[2] SPHERICALSURFACE::sf_centroid_x[3] SPHERICALSURFACE::sf_centroid_x[4] SPHERICALSURFACE::sf_centroid_y[0] SPHERICALSURFACE::sf_centroid_y[1] SPHERICALSURFACE::sf_centroid_y[2] SPHERICALSURFACE::sf_centroid_y[3] SPHERICALSURFACE::sf_centroid_y[4] SPHERICALSURFACE::sf_centroid_z[0] SPHERICALSURFACE::sf_centroid_z[1] SPHERICALSURFACE::sf_centroid_z[2] SPHERICALSURFACE::sf_centroid_z[3] SPHERICALSURFACE::sf_centroid_z[4] SPHERICALSURFACE::sf_quadrupole_xx[0] SPHERICALSURFACE::sf_quadrupole_xx[1] SPHERICALSURFACE::sf_quadrupole_xx[2] SPHERICALSURFACE::sf_quadrupole_xx[3] SPHERICALSURFACE::sf_quadrupole_xx[4] SPHERICALSURFACE::sf_quadrupole_xy[0] SPHERICALSURFACE::sf_quadrupole_xy[1] SPHERICALSURFACE::sf_quadrupole_xy[2] SPHERICALSURFACE::sf_quadrupole_xy[3] SPHERICALSURFACE::sf_quadrupole_xy[4] SPHERICALSURFACE::sf_quadrupole_xz[0] SPHERICALSURFACE::sf_quadrupole_xz[1] SPHERICALSURFACE::sf_quadrupole_xz[2] SPHERICALSURFACE::sf_quadrupole_xz[3] SPHERICALSURFACE::sf_quadrupole_xz[4] SPHERICALSURFACE::sf_quadrupole_yy[0] SPHERICALSURFACE::sf_quadrupole_yy[1] SPHERICALSURFACE::sf_quadrupole_yy[2] SPHERICALSURFACE::sf_quadrupole_yy[3] SPHERICALSURFACE::sf_quadrupole_yy[4] SPHERICALSURFACE::sf_quadrupole_yz[0] SPHERICALSURFACE::sf_quadrupole_yz[1] SPHERICALSURFACE::sf_quadrupole_yz[2] SPHERICALSURFACE::sf_quadrupole_yz[3] SPHERICALSURFACE::sf_quadrupole_yz[4] SPHERICALSURFACE::sf_quadrupole_zz[0] SPHERICALSURFACE::sf_quadrupole_zz[1] SPHERICALSURFACE::sf_quadrupole_zz[2] SPHERICALSURFACE::sf_quadrupole_zz[3] SPHERICALSURFACE::sf_quadrupole_zz[4] SPHERICALSURFACE::sf_min_radius[0] SPHERICALSURFACE::sf_min_radius[1] SPHERICALSURFACE::sf_min_radius[2] SPHERICALSURFACE::sf_min_radius[3] SPHERICALSURFACE::sf_min_radius[4] SPHERICALSURFACE::sf_max_radius[0] SPHERICALSURFACE::sf_max_radius[1] SPHERICALSURFACE::sf_max_radius[2] SPHERICALSURFACE::sf_max_radius[3] SPHERICALSURFACE::sf_max_radius[4] SPHERICALSURFACE::sf_min_x[0] SPHERICALSURFACE::sf_min_x[1] SPHERICALSURFACE::sf_min_x[2] SPHERICALSURFACE::sf_min_x[3] SPHERICALSURFACE::sf_min_x[4] SPHERICALSURFACE::sf_min_y[0] SPHERICALSURFACE::sf_min_y[1] SPHERICALSURFACE::sf_min_y[2] SPHERICALSURFACE::sf_min_y[3] SPHERICALSURFACE::sf_min_y[4] SPHERICALSURFACE::sf_min_z[0] SPHERICALSURFACE::sf_min_z[1] SPHERICALSURFACE::sf_min_z[2] SPHERICALSURFACE::sf_min_z[3] SPHERICALSURFACE::sf_min_z[4] SPHERICALSURFACE::sf_max_x[0] SPHERICALSURFACE::sf_max_x[1] SPHERICALSURFACE::sf_max_x[2] SPHERICALSURFACE::sf_max_x[3] SPHERICALSURFACE::sf_max_x[4] SPHERICALSURFACE::sf_max_y[0] SPHERICALSURFACE::sf_max_y[1] SPHERICALSURFACE::sf_max_y[2] SPHERICALSURFACE::sf_max_y[3] SPHERICALSURFACE::sf_max_y[4] SPHERICALSURFACE::sf_max_z[0] SPHERICALSURFACE::sf_max_z[1] SPHERICALSURFACE::sf_max_z[2] SPHERICALSURFACE::sf_max_z[3] SPHERICALSURFACE::sf_max_z[4] SPHERICALSURFACE::sf_radius[0] SPHERICALSURFACE::sf_radius[1] SPHERICALSURFACE::sf_radius[2] SPHERICALSURFACE::sf_radius[3] SPHERICALSURFACE::sf_radius[4] SPHERICALSURFACE::sf_origin_x[0] SPHERICALSURFACE::sf_origin_x[1] SPHERICALSURFACE::sf_origin_x[2] SPHERICALSURFACE::sf_origin_x[3] SPHERICALSURFACE::sf_origin_x[4] SPHERICALSURFACE::sf_origin_y[0] SPHERICALSURFACE::sf_origin_y[1] SPHERICALSURFACE::sf_origin_y[2] SPHERICALSURFACE::sf_origin_y[3] SPHERICALSURFACE::sf_origin_y[4] SPHERICALSURFACE::sf_origin_z[0] SPHERICALSURFACE::sf_origin_z[1] SPHERICALSURFACE::sf_origin_z[2] SPHERICALSURFACE::sf_origin_z[3] SPHERICALSURFACE::sf_origin_z[4] SPHERICALSURFACE::sf_origin_theta[0] SPHERICALSURFACE::sf_origin_theta[1] SPHERICALSURFACE::sf_origin_theta[2] SPHERICALSURFACE::sf_origin_theta[3] SPHERICALSURFACE::sf_origin_theta[4] SPHERICALSURFACE::sf_origin_phi[0] SPHERICALSURFACE::sf_origin_phi[1] SPHERICALSURFACE::sf_origin_phi[2] SPHERICALSURFACE::sf_origin_phi[3] SPHERICALSURFACE::sf_origin_phi[4] SPHERICALSURFACE::sf_delta_theta[0] SPHERICALSURFACE::sf_delta_theta[1] SPHERICALSURFACE::sf_delta_theta[2] SPHERICALSURFACE::sf_delta_theta[3] SPHERICALSURFACE::sf_delta_theta[4] SPHERICALSURFACE::sf_delta_phi[0] SPHERICALSURFACE::sf_delta_phi[1] SPHERICALSURFACE::sf_delta_phi[2] SPHERICALSURFACE::sf_delta_phi[3] SPHERICALSURFACE::sf_delta_phi[4] INFO (CarpetIOASCII): I/O Method 'IOASCII_1D' registered: 1D AMR output of grid variables to ASCII files INFO (CarpetIOASCII): Periodic 1D AMR output requested for: ADMBASE::gxx ADMBASE::gxy ADMBASE::gxz ADMBASE::gyy ADMBASE::gyz ADMBASE::gzz ADMBASE::kxx ADMBASE::kxy ADMBASE::kxz ADMBASE::kyy ADMBASE::kyz ADMBASE::kzz ADMBASE::alp ADMBASE::betax ADMBASE::betay ADMBASE::betaz HYDROBASE::rho HYDROBASE::vel[0] HYDROBASE::vel[1] HYDROBASE::vel[2] ML_ADMCONSTRAINTS::H INFO (CarpetIOASCII): I/O Method 'IOASCII_2D' registered: 2D AMR output of grid variables to ASCII files INFO (CarpetIOASCII): I/O Method 'IOASCII_3D' registered: 3D AMR output of grid variables to ASCII files -------------------------------------------------------------------------------- INFO (Formaline): Writing tarballs with the Cactus sources into the directory "bns/cactus-source" WARNING level 1 from host panther process 0 in thorn ML_BSSN_Helper, file /system/user/crangano/einstein_toolkit/BNS/configs/sim/build/ML_BSSN_Helper/Parameters.c:10: -> Forcing ML_BSSN::initial_boundary_condition="extrapolate-gammas" because ML_BSSN::my_initial_boundary_condition="extrapolate-gammas" WARNING level 1 from host panther process 0 in thorn ML_BSSN_Helper, file /system/user/crangano/einstein_toolkit/BNS/configs/sim/build/ML_BSSN_Helper/Parameters.c:25: -> Forcing ML_BSSN::rhs_boundary_condition="NewRad" because ML_BSSN::my_rhs_boundary_condition="NewRad" WARNING level 1 from host panther process 0 in thorn ML_BSSN_Helper, file /system/user/crangano/einstein_toolkit/BNS/configs/sim/build/ML_BSSN_Helper/Parameters.c:45: -> Forcing ML_BSSN::epsDiss=0.0 because ML_BSSN::apply_dissipation="never" WARNING level 1 from host panther process 0 in thorn ML_BSSN_Helper, file /system/user/crangano/einstein_toolkit/BNS/configs/sim/build/ML_BSSN_Helper/Parameters.c:114: -> Forcing ML_BSSN::advectLapse=1 because ML_BSSN::LapseAdvectionCoeff=1.0 WARNING level 1 from host panther process 0 in thorn ML_BSSN_Helper, file /system/user/crangano/einstein_toolkit/BNS/configs/sim/build/ML_BSSN_Helper/Parameters.c:132: -> Forcing ML_BSSN::advectShift=1 because ML_BSSN::ShiftAdvectionCoeff=1.0 INFO (MoL): Using Runge-Kutta 4 as the time integrator. INFO (SymBase): Symmetry on lower x-face: rotating_symmetry_180 INFO (SymBase): Symmetry on lower z-face: reflection_symmetry INFO (MoL): The maximum number of evolved variables is 1246. 30 are registered. INFO (MoL): The maximum number of slow evolved variables is 1246. 0 are registered. INFO (MoL): The maximum number of constrained variables is 1246. 37 are registered. INFO (MoL): The maximum number of SandR variables is 1246. 0 are registered. INFO (MoL): The maximum number of evolved array variables is 1246. 0 are registered. INFO (MoL): The maximum number of constrained array variables is 1246. 0 are registered. INFO (MoL): The maximum number of SandR array variables is 1246. 0 are registered. INFO (MoL): The maximum size of any array variables is 0. WARNING level 1 from host panther process 0 in thorn ML_BSSN_Helper, file /system/user/crangano/einstein_toolkit/BNS/configs/sim/build/ML_BSSN_Helper/Parameters.c:145: -> Parameter ML_BSSN::my_initial_data is outdated; please update the parameter file. Do not use this parameter, and set up initial conditions via ADMBase as usual. WARNING level 1 from host panther process 0 in thorn ML_BSSN_Helper, file /system/user/crangano/einstein_toolkit/BNS/configs/sim/build/ML_BSSN_Helper/Parameters.c:152: -> Parameter ML_BSSN::my_initial_boundary_condition is outdated; please update the parameter file. Do not use this parameter, and set up initial boundary conditions as usual. WARNING level 1 from host panther process 0 in thorn ML_BSSN_Helper, file /system/user/crangano/einstein_toolkit/BNS/configs/sim/build/ML_BSSN_Helper/Parameters.c:159: -> Parameter ML_BSSN::my_rhs_boundary_condition is outdated; please update the parameter file. Do not use this parameter, and set up RHS boundary conditions as usual. WARNING level 1 from host panther process 0 in thorn ML_BSSN_Helper, file /system/user/crangano/einstein_toolkit/BNS/configs/sim/build/ML_BSSN_Helper/Parameters.c:166: -> Parameter ML_BSSN::my_boundary_condition is outdated; please update the parameter file. Do not use this parameter, and set up RHS boundary conditions as usual. WARNING level 1 from host panther process 0 in thorn ML_BSSN_Helper, file /system/user/crangano/einstein_toolkit/BNS/configs/sim/build/ML_BSSN_Helper/Parameters.c:193: -> Parameter ML_BSSN::LapseAdvectionCoeff is outdated; please update the parameter file. Instead of using this parameter, you should set ML_BSSN::advectLapse. WARNING level 1 from host panther process 0 in thorn ML_BSSN_Helper, file /system/user/crangano/einstein_toolkit/BNS/configs/sim/build/ML_BSSN_Helper/Parameters.c:199: -> Parameter ML_BSSN::ShiftAdvectionCoeff is outdated; please update the parameter file. Instead of using this parameter, you should set ML_BSSN::advectShift. WARNING level 2 from host panther process 0 in thorn TerminationTrigger, file /system/user/crangano/einstein_toolkit/BNS/configs/sim/build/TerminationTrigger/paramcheck.c:10: -> on_remaining_walltime (30 minutes) is more than max_walltime (0.333333 h) and will be reset to 0.0 to avoid an immediate termination INFO (Vectors): Testing vectorisation... [errors may result in segfaults] INFO (Vectors): 375/375 tests passed INFO (CarpetRegrid2): Enforcing grid structure properties, iteration 0 INFO (CarpetRegrid2): Enforcing grid structure properties, iteration 1 INFO (CarpetRegrid2): Enforcing grid structure properties, iteration 0 INFO (CarpetRegrid2): Enforcing grid structure properties, iteration 1 INFO (CarpetRegrid2): Enforcing grid structure properties, iteration 0 INFO (CarpetRegrid2): Enforcing grid structure properties, iteration 1 INFO (CarpetRegrid2): Enforcing grid structure properties, iteration 0 INFO (CarpetRegrid2): Enforcing grid structure properties, iteration 1 INFO (CarpetRegrid2): Enforcing grid structure properties, iteration 0 INFO (CarpetRegrid2): Enforcing grid structure properties, iteration 1 INFO (CarpetRegrid2): Enforcing grid structure properties, iteration 0 INFO (CarpetRegrid2): Enforcing grid structure properties, iteration 1 INFO (Carpet): Grid structure (superregions, grid points): [0][0][0] exterior: [0,0,0] : [55,104,55] ([56,105,56] + PADDING) 329280 [1][0][0] exterior: [3,35,3] : [79,173,75] ([77,139,73] + PADDING) 781319 [2][0][0] exterior: [9,139,9] : [89,277,81] ([81,139,73] + PADDING) 821907 [3][0][0] exterior: [21,347,21] : [109,485,93] ([89,139,73] + PADDING) 903083 [4][0][0] exterior: [45,771,45] : [141,893,109] ([97,123,65] + PADDING) 775515 [5][0][0] exterior: [93,1583,93] : [237,1745,177] ([145,163,85] + PADDING) 2008975 [6][0][0] exterior: [189,3215,189] : [427,3441,305] ([239,227,117] + PADDING) 6347601 INFO (Carpet): Grid structure (superregions, coordinates): [0][0][0] exterior: [-24,-416,-24] : [416,416,416] : [8,8,8] [1][0][0] exterior: [-12,-276,-12] : [292,276,276] : [4,4,4] [2][0][0] exterior: [-6,-138,-6] : [154,138,138] : [2,2,2] [3][0][0] exterior: [-3,-69,-3] : [85,69,69] : [1,1,1] [4][0][0] exterior: [-1.5,-30.5,-1.5] : [46.5,30.5,30.5] : [0.5,0.5,0.5] [5][0][0] exterior: [-0.75,-20.25,-0.75] : [35.25,20.25,20.25] : [0.25,0.25,0.25] [6][0][0] exterior: [-0.375,-14.125,-0.375] : [29.375,14.125,14.125] : [0.125,0.125,0.125] INFO (Carpet): Global grid structure statistics: INFO (Carpet): GF: rhs: 2986k active, 3736k owned (+25%), 4184k total (+12%), 127 steps/time INFO (Carpet): GF: vars: 319, pts: 2765M active, 3600M owned (+30%), 4145M total (+15%), 1.0 comp/proc INFO (Carpet): GA: vars: 808, pts: 0M active, 0M total (+0%) INFO (Carpet): Total required memory: 32.960 GByte (for GAs and currently active GFs) INFO (Carpet): Load balance: min avg max sdv max/avg-1 INFO (Carpet): Level 0: 79M 79M 79M 0M owned 0% INFO (Carpet): Level 1: 230M 230M 230M 0M owned 0% INFO (Carpet): Level 2: 242M 242M 242M 0M owned 0% INFO (Carpet): Level 3: 267M 267M 267M 0M owned 0% INFO (Carpet): Level 4: 229M 229M 229M 0M owned 0% INFO (Carpet): Level 5: 605M 605M 605M 0M owned 0% INFO (Carpet): Level 6: 1948M 1948M 1948M 0M owned 0% INFO (CartGrid3D): Grid Spacings: INFO (CartGrid3D): dx=>8.0000000e+00 dy=>8.0000000e+00 dz=>8.0000000e+00 INFO (CartGrid3D): Computational Coordinates: INFO (CartGrid3D): x=>[-24.000,416.000] y=>[-416.000,416.000] z=>[-24.000,416.000] INFO (CartGrid3D): Indices of Physical Coordinates: INFO (CartGrid3D): x=>[0,55] y=>[0,104] z=>[0,55] INFO (TerminationTrigger): Reminding you every 60 minutes about remaining walltime INFO (Time): Timestep set to 3.2 (courant_static) INFO (GRHydro): Trying to get EOS handles INFO (GRHydro): Trying to get EOS handles INFO (GRHydro): GRHydro will use the Ideal_Fluid equation of state. INFO (GRHydro): Setting up the atmosphere mask: all points are not_atmosphere INFO (Meudon_Bin_NS): Setting up LORENE Bin_NS initial data INFO (Meudon_Bin_NS): Setting up coordinates INFO (Meudon_Bin_NS): Meudon_Bin_NS will use the 2D_Polytrope equation of state. INFO (Meudon_Bin_NS): Meudon_Bin_NS will use the 1 eos handle INFO (Meudon_Bin_NS): Reading from file "/system/user/crangano/einstein_toolkit/BNS/bns.resu" Simfactory Done at date: Mon 15 Sep 2025 10:32:12 PM CEST -------------- next part -------------- #!/usr/bin/env python # Copyright Barry Wardell, Ian Hinder, Eloisa Bentivegna # We ask that if you make use of the parameter file or the example # data, then please cite # Simulation of GW150914 binary black hole merger using the # Einstein Toolkit - https://doi.org/10.5281/zenodo.155394 # as well as the Einstein Toolkit, the Llama multi-block # infrastructure, the Carpet mesh-refinement driver, the apparent # horizon finder AHFinderDirect, the TwoPunctures initial data code, # QuasiLocalMeasures, Cactus, and the McLachlan spacetime evolution # code, the Kranc code generation package, and the Simulation Factory. # An appropriate bibtex file, etgw150914.bib, is provided with this # parameter file. from math import * import sys import re from string import Template ################################################################################ # Binary black hole configuration ################################################################################ # BHs labeled as '+' and '-' ('p' and 'm') for their initial position # on the x axis. The more massive BH is '+'. D = 10.0 # Separation q = 36.0/29.0 # Mass ratio: q = mp/mm >= 1 M = 1.0 # Total mass chip = 0.31 # Dimensionsless z component of spin of + BH (x0 > 0, more massive) chim = -0.46 # Dimensionsless z component of spin of - BH (x0 < 0, less massive) Pr = -0.00084541526517121; # Radial linear momentum Pphi = 0.09530152296974252; # Azimuthal linear momentum ################################################################################ # Initial data ################################################################################ mp = M * q/(1+q) # Heavier, larger BH, AH1, SS 0 mm = M * 1/(1+q) # Lighter, smaller BH, AH2, SS 1 xp = D * mm xm = -D * mp half_D = D/2.0 center_offset = xp - half_D Spx = 0.; Spy = 0.; Spz = chip * mp**2; Smx = 0.; Smy = 0.; Smz = chim * mm**2; Ppx = Pr Ppy = Pphi Ppz = 0 Pmx = -Ppx Pmy = -Ppy Pmz = -Ppz ################################################################################ # Grid structure ################################################################################ sphere_inner_radius = 45 expected_merger = 1000.0 # Number of cells across finest grid radius n = int("2") if "2"[0] != "@" else 28 i = int(n/4) n_angular = 2*i*2 # Estimated eventual AH radii ahrp = mp * 1.0 ahrm = mm * 1.0 # Coordinate size of finest boxes around each BH rp = ahrp * 1.2 rm = ahrm * 1.2 # Minimum acceptable radial grid spacing hr_min = 2*0.96*M maxrls = 9 # Essentially determines iteration counting # Smaller '-' BH # Decisions are made independent of n, to avoid roundoff problems. # This is achieved by using nmin for the calculations and scaling by # n/nmin at the end. n_min = 24 # Cells across AHm radius hfm_min = rm/n_min # Fine grid spacing for '-' BH rlsm = 1 + int(log(hr_min/hfm_min,2)) # Number of refinements to attain hr_min h0_min = hfm_min * 2**(rlsm-1) # RL0 Cartesian spacing # Radii of each level for the centre around the '-' BH levelsm = "[0," + ",".join(["%f" %(rm*2**(rlsm-l-1)) for l in range(1,rlsm)])+"]" # '+' BH # Number of levels for '+' BH estimated to keep error in angular # velocity the same as for the '-' BH, assuming vErr \propto v * # (h0/2**(rls-1)/rAH)**8 rlsp = rlsm + log(ahrm/ahrp * (xp/-xm)**(1./8))/log(2) rlsp = int(round(rlsp)) levelsp = "[0," + ",".join(["%f" %(rp*2**(rlsp-l-1)) for l in range(1,rlsp)])+"]" hr = h0_min * float(n_min)/float(n) # This could be changed (h0_min -> # hr_min) to get the minimum # radial resolution for efficiency h0 = h0_min * float(n_min)/float(n) time_after_merger = 200.0 waveform_length = expected_merger + time_after_merger outermost_detector = 500.0 final_time = waveform_length + outermost_detector sphere_outer_radius = int((outermost_detector + final_time)/(i*hr))*i*hr sphere_outer_radius = int(sphere_outer_radius / hr) * hr + hr # round up to a multiple of hr sphere_inner_radius = int(ceil(sphere_inner_radius/(i*h0))) * h0 * i ################################################################################ # Frequencies ################################################################################ dtdx = 0.45 # Courant factor dtdx0 = dtdx * 0.5 # RL0 is evolved with the same frequency as RL1 dt0 = dtdx0 * h0 # Time step on RL0 rl0_every = 2**(maxrls-2) rl1_every = rl0_every rl_max = maxrls-1 dt_it = dt0/2.0**(rl_max-1) # Time step of one iteration find_cah = max(int((expected_merger - 200)/dt_it), 0) wave_extract_every = rl0_every * 2 # Every other coarse grid step # (TODO: should this be every # coarse grid step?) horizon_every = rl0_every * 2 out_every = rl0_every # out3d_every = rl0_every * 2 out3d_every = 0 out2d_every = rl0_every * 8 ################################################################################ # Sanity checks ################################################################################ # this example uses # Coordinates::symmetry = "+z bitant" # thus all data must by mirror symmetric accross the z=0 plane requiring 0 # values for z components of all polar vectors at the z=0 plane and 0 values # for the x,y components of axial vectors if (Spx != 0. or Spy != 0. or Smx != 0. or Smy != 0.): sys.stderr.write("This parameter file employs z-symmetry via the 'Coordinates::symmetry=\"+z bitant\"' setting. Therefore all spins must be in the z-direction only.\n"); sys.exit(1) if (Ppz != 0. or Pmz != 0.): sys.stderr.write("This parameter file employs z-symmetry via the 'Coordinates::symmetry=\"+z bitant\"' setting. Therefore no momentum may have a non-zero z-component.\n"); sys.exit(1) ################################################################################ # Record all script variables in generated parameter file ################################################################################ local_vars = locals() col_width = 0 for key,val in sorted(local_vars.items()): if isinstance(val, (int, float, complex, str)): col_width = max(len(str(key)), col_width) var_settings = [] for key,val in sorted(local_vars.items()): if isinstance(val, (int, float, complex, str)): var_settings = var_settings + ["# {0}{1} = {2}".format(key," "*(col_width-len(key)),val)] var_settings_str = "\n".join(var_settings) lines = """ ################################################################################ # Script variables ################################################################################ $var_settings_str ################################################################################ # Active thorns ################################################################################ ActiveThorns = " ADMBase ML_ADMConstraints ADMCoupling ADMMacros AEILocalInterp AHFinderDirect Boundary Carpet CarpetIOASCII CarpetIOBasic CarpetIOHDF5 CarpetIOScalar CarpetInterp CarpetInterp2 CarpetLib CarpetReduce CarpetRegrid2 CarpetTracker CartGrid3D CoordBase CoordGauge Coordinates CoordinatesSymmetry Formaline GlobalDerivative hwloc IOUtil InitBase Interpolate2 QuasiLocalMeasures LocalInterp LoopControl MoL NaNChecker PunctureTracker Slab SpaceMask SphericalSurface StaticConformal SummationByParts SymBase SystemStatistics SystemTopology TerminationTrigger TensorTypes Time TmunuBase TwoPunctures Vectors ML_BSSN ML_BSSN_Helper NewRad GenericFD WeylScal4 Multipole # WaveExtractCPM ADMDerivatives " ################################################################################ # Grid structure ################################################################################ Carpet::domain_from_multipatch = yes CartGrid3D::type = "multipatch" CartGrid3D::set_coordinate_ranges_on = "all maps" Coordinates::coordinate_system = "Thornburg04" Coordinates::h_cartesian = $h0 Coordinates::h_radial = $hr Coordinates::sphere_inner_radius = $sphere_inner_radius Coordinates::sphere_outer_radius = $sphere_outer_radius Coordinates::n_angular = $n_angular Driver::ghost_size = 5 Coordinates::patch_boundary_size = 5 Coordinates::additional_overlap_size = 3 Coordinates::outer_boundary_size = 5 # change these (to their defaults) to disable z-zymmetry CoordinatesSymmetry::reflection_z = yes CoordinatesSymmetry::stagger = no Coordinates::symmetry = "+z bitant" Coordinates::additional_symmetry_size = 1 Coordinates::verbose = no Time::timestep_method = "given" Time::timestep = $dt0 Carpet::time_refinement_factors = "[1,1,2,4,8,16,32,64,128,256]" ################################################################################ # Mesh refinement ################################################################################ Carpet::max_refinement_levels = $maxrls CarpetRegrid2::num_centres = 2 CarpetRegrid2::num_levels_1 = $rlsp CarpetRegrid2::position_x_1 = $xp CarpetRegrid2::radius_1 = $levelsp CarpetRegrid2::num_levels_2 = $rlsm CarpetRegrid2::position_x_2 = $xm CarpetRegrid2::radius_2 = $levelsm Carpet::use_buffer_zones = yes Carpet::prolongation_order_space = 5 Carpet::prolongation_order_time = 2 CarpetRegrid2::regrid_every = $rl1_every CarpetRegrid2::verbose = no Carpet::grid_coordinates_filename = "carpet-grid.asc" ################################################################################ # Initial data ################################################################################ ADMBase::initial_data = "twopunctures" ADMBase::initial_lapse = "twopunctures-averaged" ADMBase::initial_shift = "zero" ADMBase::initial_dtlapse = "zero" ADMBase::initial_dtshift = "zero" TwoPunctures::target_M_plus = $mp TwoPunctures::target_M_minus = $mm # Initial guesses for iterative solve TwoPunctures::par_m_plus = $mp TwoPunctures::par_m_minus = $mm TwoPunctures::par_b = $half_D TwoPunctures::center_offset[0] = $center_offset TwoPunctures::par_P_plus[0] = $Ppx TwoPunctures::par_P_plus[1] = $Ppy TwoPunctures::par_P_plus[2] = $Ppz TwoPunctures::par_P_minus[0] = $Pmx TwoPunctures::par_P_minus[1] = $Pmy TwoPunctures::par_P_minus[2] = $Pmz TwoPunctures::par_S_plus[0] = $Spx TwoPunctures::par_S_plus[1] = $Spy TwoPunctures::par_S_plus[2] = $Spz TwoPunctures::par_S_minus[0] = $Smx TwoPunctures::par_S_minus[1] = $Smy TwoPunctures::par_S_minus[2] = $Smz TwoPunctures::grid_setup_method = "evaluation" TwoPunctures::give_bare_mass = no TwoPunctures::TP_epsilon = 1e-6 Carpet::init_fill_timelevels = yes InitBase::initial_data_setup_method = "init_single_level" ################################################################################ # Evolution and boundary ################################################################################ GenericFD::jacobian_group = "Coordinates::jacobian" GenericFD::jacobian_derivative_group = "Coordinates::jacobian2" GenericFD::jacobian_identity_map = 0 ADMBase::evolution_method = "ML_BSSN" ADMBase::lapse_evolution_method = "ML_BSSN" ADMBase::shift_evolution_method = "ML_BSSN" ADMBase::dtlapse_evolution_method = "ML_BSSN" ADMBase::dtshift_evolution_method = "ML_BSSN" ML_BSSN::fdOrder = 8 # 1+log slicing requires harmonicN = 1 and harmonicF = 2.0 ML_BSSN::harmonicN = 1 ML_BSSN::harmonicF = 2.0 ML_BSSN::ShiftGammaCoeff = 0.75 ML_BSSN::BetaDriver = 1.0 ML_BSSN::advectLapse = 1 ML_BSSN::advectShift = 1 ML_BSSN::MinimumLapse = 1.0e-8 # conformalaMethod = 1 for W, 0 for phi ML_BSSN::conformalMethod = 1 # We apply dissipation using GlobalDerivaitive so disable it here ML_BSSN::epsDiss = 0.0 ML_BSSN::dt_lapse_shift_method = "noLapseShiftAdvection" ML_BSSN::initial_boundary_condition = "extrapolate-gammas" ML_BSSN::rhs_boundary_condition = "scalar" Boundary::radpower = 2 ################################################################################ # BH tracking ################################################################################ CarpetTracker::surface [0] = 0 CarpetTracker::surface [1] = 1 PunctureTracker::track [0] = yes PunctureTracker::initial_x [0] = $xp PunctureTracker::which_surface_to_store_info[0] = 0 PunctureTracker::track [1] = yes PunctureTracker::initial_x [1] = $xm PunctureTracker::which_surface_to_store_info[1] = 1 ################################################################################ # Spatial finite differencing ################################################################################ SummationByParts::order = 8 # Drop order instead of using upwinded stencils, only for advection derivatives SummationByParts::sbp_upwind_deriv = no SummationByParts::sbp_1st_deriv = yes SummationByParts::sbp_2nd_deriv = no SummationByParts::onesided_interpatch_boundaries = no SummationByParts::onesided_outer_boundaries = yes SummationByParts::use_dissipation = no GlobalDerivative::use_dissipation = yes SummationByParts::scale_with_h = yes SummationByParts::dissipation_type = "Kreiss-Oliger" SummationByParts::epsdis = 0.15 # Because dt/dx is half that of the other levels we also need half the dissipation GlobalDerivative::epsdis_for_level [0] = 0.075 # Variables for dissipation SummationByParts::vars = " ML_BSSN::ML_log_confac ML_BSSN::ML_metric ML_BSSN::ML_trace_curv ML_BSSN::ML_curv ML_BSSN::ML_Gamma ML_BSSN::ML_lapse ML_BSSN::ML_shift ML_BSSN::ML_dtlapse ML_BSSN::ML_dtshift " ################################################################################ # Time integration ################################################################################ MoL::ODE_Method = "rk4" MoL::MoL_Intermediate_Steps = 4 MoL::MoL_Num_Scratch_Levels = 1 ################################################################################ # Interpolation ################################################################################ CarpetInterp::check_tree_search = no CarpetInterp::tree_search = yes # Use 5-th order interpatch interpolation on the Llama grid Interpolate::interpolator_order = 5 ################################################################################ # Psi4 computation ################################################################################ WeylScal4::fdOrder = 8 WeylScal4::calc_scalars = "psis" WeylScal4::calc_invariants = "always" ################################################################################ # Psi4 mode decomposition ################################################################################ # Radii are chosen to be evenly spaced in 1/r as that is the variable # extrapolation is performed in Multipole::nradii = 7 Multipole::radius[0] = 100 Multipole::radius[1] = 115 Multipole::radius[2] = 136 Multipole::radius[3] = 167 Multipole::radius[4] = 214 Multipole::radius[5] = 300 Multipole::radius[6] = 500 Multipole::ntheta = 120 Multipole::nphi = 240 Multipole::variables = "WeylScal4::Psi4r{sw=-2 cmplx='WeylScal4::Psi4i' name='psi4'}" Multipole::out_every = $wave_extract_every Multipole::l_max = 8 Multipole::output_hdf5 = yes # Disable ASCII output to avoid creating a large number of files Multipole::output_ascii = no ################################################################################ # Gauge invariant perturbations of Schwarzschild (ZM-CPM variables) ################################################################################ # WaveExtractCPM::out_every = $wave_extract_every # WaveExtractCPM::use_carpetinterp2 = no # WaveExtractCPM::calc_when_necessary = no # WaveExtractCPM::verbose = 0 # WaveExtractCPM::maximum_detector_number = 7 # WaveExtractCPM::switch_output_format = 100 # WaveExtractCPM::rsch2_computation = "average Schwarzschild metric" # WaveExtractCPM::l_mode = 8 # WaveExtractCPM::m_mode = 8 # WaveExtractCPM::detector_radius [0] = 100 # WaveExtractCPM::detector_radius [1] = 115 # WaveExtractCPM::detector_radius [2] = 136 # WaveExtractCPM::detector_radius [3] = 167 # WaveExtractCPM::detector_radius [4] = 214 # WaveExtractCPM::detector_radius [5] = 300 # WaveExtractCPM::detector_radius [6] = 500 # WaveExtractCPM::maxntheta = 120 # WaveExtractCPM::maxnphi = 240 # WaveExtractCPM::output_hdf5 = yes # WaveExtractCPM::output_ascii = no # WaveExtractCPM::output_h = yes # WaveExtractCPM::output_Psi = yes ################################################################################ # Apparent Horizons ################################################################################ AHFinderDirect::N_horizons = 3 AHFinderDirect::find_every = $horizon_every AHFinderDirect::output_h_every = 0 AHFinderDirect::max_Newton_iterations__initial = 50 AHFinderDirect::max_Newton_iterations__subsequent = 50 AHFinderDirect::max_allowable_Theta_growth_iterations = 10 AHFinderDirect::max_allowable_Theta_nonshrink_iterations = 10 AHFinderDirect::geometry_interpolator_name = "Lagrange polynomial interpolation" AHFinderDirect::geometry_interpolator_pars = "order=4" AHFinderDirect::surface_interpolator_name = "Lagrange polynomial interpolation" AHFinderDirect::surface_interpolator_pars = "order=4" AHFinderDirect::verbose_level = "physics details" AHFinderDirect::move_origins = yes AHFinderDirect::origin_x [1] = $xp AHFinderDirect::initial_guess__coord_sphere__x_center[1] = $xp AHFinderDirect::initial_guess__coord_sphere__radius [1] = $rp AHFinderDirect::which_surface_to_store_info [1] = 2 AHFinderDirect::set_mask_for_individual_horizon [1] = no AHFinderDirect::reset_horizon_after_not_finding [1] = no AHFinderDirect::track_origin_from_grid_scalar [1] = yes AHFinderDirect::track_origin_source_x [1] = "PunctureTracker::pt_loc_x[0]" AHFinderDirect::track_origin_source_y [1] = "PunctureTracker::pt_loc_y[0]" AHFinderDirect::track_origin_source_z [1] = "PunctureTracker::pt_loc_z[0]" AHFinderDirect::max_allowable_horizon_radius [1] = 3 AHFinderDirect::origin_x [2] = $xm AHFinderDirect::initial_guess__coord_sphere__x_center[2] = $xm AHFinderDirect::initial_guess__coord_sphere__radius [2] = $rm AHFinderDirect::which_surface_to_store_info [2] = 3 AHFinderDirect::set_mask_for_individual_horizon [2] = no AHFinderDirect::reset_horizon_after_not_finding [2] = no AHFinderDirect::track_origin_from_grid_scalar [2] = yes AHFinderDirect::track_origin_source_x [2] = "PunctureTracker::pt_loc_x[1]" AHFinderDirect::track_origin_source_y [2] = "PunctureTracker::pt_loc_y[1]" AHFinderDirect::track_origin_source_z [2] = "PunctureTracker::pt_loc_z[1]" AHFinderDirect::max_allowable_horizon_radius [2] = 3 AHFinderDirect::origin_x [3] = 0 AHFinderDirect::find_after_individual [3] = $find_cah AHFinderDirect::initial_guess__coord_sphere__x_center[3] = 0 AHFinderDirect::initial_guess__coord_sphere__radius [3] = 1.0 AHFinderDirect::which_surface_to_store_info [3] = 4 AHFinderDirect::set_mask_for_individual_horizon [3] = no AHFinderDirect::max_allowable_horizon_radius [3] = 6 ################################################################################ # Spherical surfaces ################################################################################ SphericalSurface::nsurfaces = 5 SphericalSurface::maxntheta = 66 SphericalSurface::maxnphi = 124 SphericalSurface::verbose = no # Surfaces 0 and 1 are used by PunctureTracker # Horizon 1 SphericalSurface::ntheta [2] = 41 SphericalSurface::nphi [2] = 80 SphericalSurface::nghoststheta [2] = 2 SphericalSurface::nghostsphi [2] = 2 # Horizon 2 SphericalSurface::ntheta [3] = 41 SphericalSurface::nphi [3] = 80 SphericalSurface::nghoststheta [3] = 2 SphericalSurface::nghostsphi [3] = 2 # Common horizon SphericalSurface::ntheta [4] = 41 SphericalSurface::nphi [4] = 80 SphericalSurface::nghoststheta [4] = 2 SphericalSurface::nghostsphi [4] = 2 ################################################################################ # Isolated Horizons ################################################################################ QuasiLocalMeasures::verbose = no QuasiLocalMeasures::veryverbose = no QuasiLocalMeasures::interpolator = "Lagrange polynomial interpolation" QuasiLocalMeasures::interpolator_options = "order=4" QuasiLocalMeasures::spatial_order = 4 QuasiLocalMeasures::num_surfaces = 3 QuasiLocalMeasures::surface_index [0] = 2 QuasiLocalMeasures::surface_index [1] = 3 QuasiLocalMeasures::surface_index [2] = 4 QuasiLocalMeasures::output_vtk_every = $out3d_every ################################################################################ # Correctness checking ################################################################################ Carpet::poison_new_timelevels = no Carpet::check_for_poison = no NaNChecker::check_every = 256 NanChecker::check_after = 0 NaNChecker::report_max = 10 NaNChecker::verbose = "all" NaNChecker::action_if_found = terminate NaNChecker::out_NaNmask = yes NaNChecker::check_vars = " ML_BSSN::ML_log_confac " ################################################################################ # Timers ################################################################################ Carpet::output_timer_tree_every = 1024 Carpet::output_initialise_timer_tree = yes ################################################################################ # Output ################################################################################ IO::out_dir = "@SIMULATION_NAME@" IOScalar::one_file_per_group = yes IOASCII::one_file_per_group = yes IOBasic::outInfo_every = 1 IOBasic::outInfo_reductions = "minimum maximum" IOBasic::outInfo_vars = " ML_BSSN::ML_log_confac Carpet::physical_time_per_hour SystemStatistics::maxrss_mb SystemStatistics::swap_used_mb " IOScalar::outScalar_every = 256 IOScalar::outScalar_reductions = "minimum maximum average" IOScalar::outScalar_vars = "SystemStatistics::process_memory_mb" IOASCII::out0D_every = 256 IOASCII::out0D_vars = " Carpet::timing PunctureTracker::pt_loc QuasiLocalMeasures::qlm_scalars{out_every = $horizon_every} " IOASCII::out1D_every = 0 IOASCII::out1D_d = no IOASCII::out1D_x = yes IOASCII::out1D_y = no IOASCII::out1D_z = yes IOASCII::out1D_vars = " ML_BSSN::ML_log_confac ML_BSSN::ML_trace_curv WeylScal4::Psi4r " IOASCII::out2D_every = 0 IOASCII::out2D_vars = " " IOHDF5::out_every = $out3d_every IOHDF5::out_vars = " Grid::Coordinates{out_every=1000000000 refinement_levels={0}} ML_BSSN::ML_log_confac WeylScal4::Psi4r WeylScal4::Psi4i WeylScal4::curvIr{refinement_levels={3 5}} WeylScal4::curvIi{refinement_levels={3 5}} WeylScal4::curvJr{refinement_levels={3 5}} WeylScal4::curvJi{refinement_levels={3 5}} " IOHDF5::out2D_every = $out2d_every IOHDF5::out2D_vars = " Grid::Coordinates{out_every=1000000000 refinement_levels={0}} ML_BSSN::alpha ML_BSSN::phi ML_BSSN::trK WeylScal4::Psi4r WeylScal4::Psi4i " ################################################################################ # Checkpointing and recovery ################################################################################ CarpetIOHDF5::checkpoint = yes IO::checkpoint_ID = no IO::recover = "autoprobe" IO::out_proc_every = 2 IO::checkpoint_on_terminate = yes IO::checkpoint_dir = "../checkpoints" IO::recover_dir = "../checkpoints" IO::abort_on_io_errors = yes CarpetIOHDF5::open_one_input_file_at_a_time = yes CarpetIOHDF5::compression_level = 0 ################################################################################ # Run termination ################################################################################ TerminationTrigger::max_walltime = @WALLTIME_HOURS@ # Trigger termination 30 minutes before the walltime is reached TerminationTrigger::on_remaining_walltime = 30 TerminationTrigger::output_remtime_every_minutes = 30 TerminationTrigger::termination_from_file = yes TerminationTrigger::termination_file = "terminate.txt" TerminationTrigger::create_termination_file = yes Cactus::terminate = time Cactus::cctk_final_time = $final_time """ open(re.sub(r'(.*)\.rpar$', r'\1.par', sys.argv[0]), 'w').write(re.sub(r'\n *',r'\n',Template(Template(lines).substitute(locals())).substitute(locals()))) -------------- next part -------------- Traceback (most recent call last): File "/system/user/crangano/simulations/GW150914_28/output-0000/GW150914.rpar", line 126, in sphere_outer_radius = int((outermost_detector + final_time)/(i*hr))*i*hr ZeroDivisionError: float division by zero Error: Error while executing parameter file script /system/user/crangano/simulations/GW150914_28/output-0000/GW150914.rpar Aborting Simfactory. -------------- next part -------------- Simulation name: GW150914_28 Running simulation GW150914_28 -------------- next part -------------- ################################################################################ # Script variables ################################################################################ # D = 10.0 # M = 1.0 # Pmx = 0.00084541526517121 # Pmy = -0.09530152296974252 # Pmz = 0 # Pphi = 0.09530152296974252 # Ppx = -0.00084541526517121 # Ppy = 0.09530152296974252 # Ppz = 0 # Pr = -0.00084541526517121 # Smx = 0.0 # Smy = 0.0 # Smz = -0.09156449704142013 # Spx = 0.0 # Spy = 0.0 # Spz = 0.09509112426035504 # __file__ = /system/user/crangano/einstein_toolkit/Cactus/repos/einsteinexamples/par/GW150914/GW150914.rpar # __name__ = __main__ # ahrm = 0.4461538461538462 # ahrp = 0.5538461538461539 # center_offset = -0.5384615384615383 # chim = -0.46 # chip = 0.31 # col_width = 19 # dt0 = 0.2753406593406593 # dt_it = 0.002151098901098901 # dtdx = 0.45 # dtdx0 = 0.225 # e = 2.718281828459045 # expected_merger = 1000.0 # final_time = 1700.0 # find_cah = 371902 # h0 = 1.2237362637362637 # h0_min = 1.4276923076923076 # half_D = 5.0 # hfm_min = 0.022307692307692306 # horizon_every = 256 # hr = 1.2237362637362637 # hr_min = 1.92 # i = 7 # inf = inf # key = xp # levelsm = [0,17.132308,8.566154,4.283077,2.141538,1.070769,0.535385] # levelsp = [0,21.267692,10.633846,5.316923,2.658462,1.329231,0.664615] # maxrls = 9 # mm = 0.4461538461538462 # mp = 0.5538461538461539 # n = 28 # n_angular = 28 # n_min = 24 # nan = nan # out2d_every = 1024 # out3d_every = 0 # out_every = 128 # outermost_detector = 500.0 # pi = 3.141592653589793 # q = 1.2413793103448276 # rl0_every = 128 # rl1_every = 128 # rl_max = 8 # rlsm = 7 # rlsp = 7 # rm = 0.5353846153846153 # rp = 0.6646153846153846 # sphere_inner_radius = 51.39692307692307 # sphere_outer_radius = 2194.1591208791206 # tau = 6.283185307179586 # time_after_merger = 200.0 # val = 4.461538461538462 # wave_extract_every = 256 # waveform_length = 1200.0 # xm = -5.538461538461538 # xp = 4.461538461538462 ################################################################################ # Active thorns ################################################################################ ActiveThorns = " ADMBase ML_ADMConstraints ADMCoupling ADMMacros AEILocalInterp AHFinderDirect Boundary Carpet CarpetIOASCII CarpetIOBasic CarpetIOHDF5 CarpetIOScalar CarpetInterp CarpetInterp2 CarpetLib CarpetReduce CarpetRegrid2 CarpetTracker CartGrid3D CoordBase CoordGauge Coordinates CoordinatesSymmetry Formaline GlobalDerivative hwloc IOUtil InitBase Interpolate2 QuasiLocalMeasures LocalInterp LoopControl MoL NaNChecker PunctureTracker Slab SpaceMask SphericalSurface StaticConformal SummationByParts SymBase SystemStatistics SystemTopology TerminationTrigger TensorTypes Time TmunuBase TwoPunctures Vectors ML_BSSN ML_BSSN_Helper NewRad GenericFD WeylScal4 Multipole # WaveExtractCPM ADMDerivatives " ################################################################################ # Grid structure ################################################################################ Carpet::domain_from_multipatch = yes CartGrid3D::type = "multipatch" CartGrid3D::set_coordinate_ranges_on = "all maps" Coordinates::coordinate_system = "Thornburg04" Coordinates::h_cartesian = 1.2237362637362637 Coordinates::h_radial = 1.2237362637362637 Coordinates::sphere_inner_radius = 51.39692307692307 Coordinates::sphere_outer_radius = 2194.1591208791206 Coordinates::n_angular = 28 Driver::ghost_size = 5 Coordinates::patch_boundary_size = 5 Coordinates::additional_overlap_size = 3 Coordinates::outer_boundary_size = 5 # change these (to their defaults) to disable z-zymmetry CoordinatesSymmetry::reflection_z = yes CoordinatesSymmetry::stagger = no Coordinates::symmetry = "+z bitant" Coordinates::additional_symmetry_size = 1 Coordinates::verbose = no Time::timestep_method = "given" Time::timestep = 0.2753406593406593 Carpet::time_refinement_factors = "[1,1,2,4,8,16,32,64,128,256]" ################################################################################ # Mesh refinement ################################################################################ Carpet::max_refinement_levels = 9 CarpetRegrid2::num_centres = 2 CarpetRegrid2::num_levels_1 = 7 CarpetRegrid2::position_x_1 = 4.461538461538462 CarpetRegrid2::radius_1 = [0,21.267692,10.633846,5.316923,2.658462,1.329231,0.664615] CarpetRegrid2::num_levels_2 = 7 CarpetRegrid2::position_x_2 = -5.538461538461538 CarpetRegrid2::radius_2 = [0,17.132308,8.566154,4.283077,2.141538,1.070769,0.535385] Carpet::use_buffer_zones = yes Carpet::prolongation_order_space = 5 Carpet::prolongation_order_time = 2 CarpetRegrid2::regrid_every = 128 CarpetRegrid2::verbose = no Carpet::grid_coordinates_filename = "carpet-grid.asc" ################################################################################ # Initial data ################################################################################ ADMBase::initial_data = "twopunctures" ADMBase::initial_lapse = "twopunctures-averaged" ADMBase::initial_shift = "zero" ADMBase::initial_dtlapse = "zero" ADMBase::initial_dtshift = "zero" TwoPunctures::target_M_plus = 0.5538461538461539 TwoPunctures::target_M_minus = 0.4461538461538462 # Initial guesses for iterative solve TwoPunctures::par_m_plus = 0.5538461538461539 TwoPunctures::par_m_minus = 0.4461538461538462 TwoPunctures::par_b = 5.0 TwoPunctures::center_offset[0] = -0.5384615384615383 TwoPunctures::par_P_plus[0] = -0.00084541526517121 TwoPunctures::par_P_plus[1] = 0.09530152296974252 TwoPunctures::par_P_plus[2] = 0 TwoPunctures::par_P_minus[0] = 0.00084541526517121 TwoPunctures::par_P_minus[1] = -0.09530152296974252 TwoPunctures::par_P_minus[2] = 0 TwoPunctures::par_S_plus[0] = 0.0 TwoPunctures::par_S_plus[1] = 0.0 TwoPunctures::par_S_plus[2] = 0.09509112426035504 TwoPunctures::par_S_minus[0] = 0.0 TwoPunctures::par_S_minus[1] = 0.0 TwoPunctures::par_S_minus[2] = -0.09156449704142013 TwoPunctures::grid_setup_method = "evaluation" TwoPunctures::give_bare_mass = no TwoPunctures::TP_epsilon = 1e-6 Carpet::init_fill_timelevels = yes InitBase::initial_data_setup_method = "init_single_level" ################################################################################ # Evolution and boundary ################################################################################ GenericFD::jacobian_group = "Coordinates::jacobian" GenericFD::jacobian_derivative_group = "Coordinates::jacobian2" GenericFD::jacobian_identity_map = 0 ADMBase::evolution_method = "ML_BSSN" ADMBase::lapse_evolution_method = "ML_BSSN" ADMBase::shift_evolution_method = "ML_BSSN" ADMBase::dtlapse_evolution_method = "ML_BSSN" ADMBase::dtshift_evolution_method = "ML_BSSN" ML_BSSN::fdOrder = 8 # 1+log slicing requires harmonicN = 1 and harmonicF = 2.0 ML_BSSN::harmonicN = 1 ML_BSSN::harmonicF = 2.0 ML_BSSN::ShiftGammaCoeff = 0.75 ML_BSSN::BetaDriver = 1.0 ML_BSSN::advectLapse = 1 ML_BSSN::advectShift = 1 ML_BSSN::MinimumLapse = 1.0e-8 # conformalaMethod = 1 for W, 0 for phi ML_BSSN::conformalMethod = 1 # We apply dissipation using GlobalDerivaitive so disable it here ML_BSSN::epsDiss = 0.0 ML_BSSN::dt_lapse_shift_method = "noLapseShiftAdvection" ML_BSSN::initial_boundary_condition = "extrapolate-gammas" ML_BSSN::rhs_boundary_condition = "scalar" Boundary::radpower = 2 ################################################################################ # BH tracking ################################################################################ CarpetTracker::surface [0] = 0 CarpetTracker::surface [1] = 1 PunctureTracker::track [0] = yes PunctureTracker::initial_x [0] = 4.461538461538462 PunctureTracker::which_surface_to_store_info[0] = 0 PunctureTracker::track [1] = yes PunctureTracker::initial_x [1] = -5.538461538461538 PunctureTracker::which_surface_to_store_info[1] = 1 ################################################################################ # Spatial finite differencing ################################################################################ SummationByParts::order = 8 # Drop order instead of using upwinded stencils, only for advection derivatives SummationByParts::sbp_upwind_deriv = no SummationByParts::sbp_1st_deriv = yes SummationByParts::sbp_2nd_deriv = no SummationByParts::onesided_interpatch_boundaries = no SummationByParts::onesided_outer_boundaries = yes SummationByParts::use_dissipation = no GlobalDerivative::use_dissipation = yes SummationByParts::scale_with_h = yes SummationByParts::dissipation_type = "Kreiss-Oliger" SummationByParts::epsdis = 0.15 # Because dt/dx is half that of the other levels we also need half the dissipation GlobalDerivative::epsdis_for_level [0] = 0.075 # Variables for dissipation SummationByParts::vars = " ML_BSSN::ML_log_confac ML_BSSN::ML_metric ML_BSSN::ML_trace_curv ML_BSSN::ML_curv ML_BSSN::ML_Gamma ML_BSSN::ML_lapse ML_BSSN::ML_shift ML_BSSN::ML_dtlapse ML_BSSN::ML_dtshift " ################################################################################ # Time integration ################################################################################ MoL::ODE_Method = "rk4" MoL::MoL_Intermediate_Steps = 4 MoL::MoL_Num_Scratch_Levels = 1 ################################################################################ # Interpolation ################################################################################ CarpetInterp::check_tree_search = no CarpetInterp::tree_search = yes # Use 5-th order interpatch interpolation on the Llama grid Interpolate::interpolator_order = 5 ################################################################################ # Psi4 computation ################################################################################ WeylScal4::fdOrder = 8 WeylScal4::calc_scalars = "psis" WeylScal4::calc_invariants = "always" ################################################################################ # Psi4 mode decomposition ################################################################################ # Radii are chosen to be evenly spaced in 1/r as that is the variable # extrapolation is performed in Multipole::nradii = 7 Multipole::radius[0] = 100 Multipole::radius[1] = 115 Multipole::radius[2] = 136 Multipole::radius[3] = 167 Multipole::radius[4] = 214 Multipole::radius[5] = 300 Multipole::radius[6] = 500 Multipole::ntheta = 120 Multipole::nphi = 240 Multipole::variables = "WeylScal4::Psi4r{sw=-2 cmplx='WeylScal4::Psi4i' name='psi4'}" Multipole::out_every = 256 Multipole::l_max = 8 Multipole::output_hdf5 = yes # Disable ASCII output to avoid creating a large number of files Multipole::output_ascii = no ################################################################################ # Gauge invariant perturbations of Schwarzschild (ZM-CPM variables) ################################################################################ # WaveExtractCPM::out_every = 256 # WaveExtractCPM::use_carpetinterp2 = no # WaveExtractCPM::calc_when_necessary = no # WaveExtractCPM::verbose = 0 # WaveExtractCPM::maximum_detector_number = 7 # WaveExtractCPM::switch_output_format = 100 # WaveExtractCPM::rsch2_computation = "average Schwarzschild metric" # WaveExtractCPM::l_mode = 8 # WaveExtractCPM::m_mode = 8 # WaveExtractCPM::detector_radius [0] = 100 # WaveExtractCPM::detector_radius [1] = 115 # WaveExtractCPM::detector_radius [2] = 136 # WaveExtractCPM::detector_radius [3] = 167 # WaveExtractCPM::detector_radius [4] = 214 # WaveExtractCPM::detector_radius [5] = 300 # WaveExtractCPM::detector_radius [6] = 500 # WaveExtractCPM::maxntheta = 120 # WaveExtractCPM::maxnphi = 240 # WaveExtractCPM::output_hdf5 = yes # WaveExtractCPM::output_ascii = no # WaveExtractCPM::output_h = yes # WaveExtractCPM::output_Psi = yes ################################################################################ # Apparent Horizons ################################################################################ AHFinderDirect::N_horizons = 3 AHFinderDirect::find_every = 256 AHFinderDirect::output_h_every = 0 AHFinderDirect::max_Newton_iterations__initial = 50 AHFinderDirect::max_Newton_iterations__subsequent = 50 AHFinderDirect::max_allowable_Theta_growth_iterations = 10 AHFinderDirect::max_allowable_Theta_nonshrink_iterations = 10 AHFinderDirect::geometry_interpolator_name = "Lagrange polynomial interpolation" AHFinderDirect::geometry_interpolator_pars = "order=4" AHFinderDirect::surface_interpolator_name = "Lagrange polynomial interpolation" AHFinderDirect::surface_interpolator_pars = "order=4" AHFinderDirect::verbose_level = "physics details" AHFinderDirect::move_origins = yes AHFinderDirect::origin_x [1] = 4.461538461538462 AHFinderDirect::initial_guess__coord_sphere__x_center[1] = 4.461538461538462 AHFinderDirect::initial_guess__coord_sphere__radius [1] = 0.6646153846153846 AHFinderDirect::which_surface_to_store_info [1] = 2 AHFinderDirect::set_mask_for_individual_horizon [1] = no AHFinderDirect::reset_horizon_after_not_finding [1] = no AHFinderDirect::track_origin_from_grid_scalar [1] = yes AHFinderDirect::track_origin_source_x [1] = "PunctureTracker::pt_loc_x[0]" AHFinderDirect::track_origin_source_y [1] = "PunctureTracker::pt_loc_y[0]" AHFinderDirect::track_origin_source_z [1] = "PunctureTracker::pt_loc_z[0]" AHFinderDirect::max_allowable_horizon_radius [1] = 3 AHFinderDirect::origin_x [2] = -5.538461538461538 AHFinderDirect::initial_guess__coord_sphere__x_center[2] = -5.538461538461538 AHFinderDirect::initial_guess__coord_sphere__radius [2] = 0.5353846153846153 AHFinderDirect::which_surface_to_store_info [2] = 3 AHFinderDirect::set_mask_for_individual_horizon [2] = no AHFinderDirect::reset_horizon_after_not_finding [2] = no AHFinderDirect::track_origin_from_grid_scalar [2] = yes AHFinderDirect::track_origin_source_x [2] = "PunctureTracker::pt_loc_x[1]" AHFinderDirect::track_origin_source_y [2] = "PunctureTracker::pt_loc_y[1]" AHFinderDirect::track_origin_source_z [2] = "PunctureTracker::pt_loc_z[1]" AHFinderDirect::max_allowable_horizon_radius [2] = 3 AHFinderDirect::origin_x [3] = 0 AHFinderDirect::find_after_individual [3] = 371902 AHFinderDirect::initial_guess__coord_sphere__x_center[3] = 0 AHFinderDirect::initial_guess__coord_sphere__radius [3] = 1.0 AHFinderDirect::which_surface_to_store_info [3] = 4 AHFinderDirect::set_mask_for_individual_horizon [3] = no AHFinderDirect::max_allowable_horizon_radius [3] = 6 ################################################################################ # Spherical surfaces ################################################################################ SphericalSurface::nsurfaces = 5 SphericalSurface::maxntheta = 66 SphericalSurface::maxnphi = 124 SphericalSurface::verbose = no # Surfaces 0 and 1 are used by PunctureTracker # Horizon 1 SphericalSurface::ntheta [2] = 41 SphericalSurface::nphi [2] = 80 SphericalSurface::nghoststheta [2] = 2 SphericalSurface::nghostsphi [2] = 2 # Horizon 2 SphericalSurface::ntheta [3] = 41 SphericalSurface::nphi [3] = 80 SphericalSurface::nghoststheta [3] = 2 SphericalSurface::nghostsphi [3] = 2 # Common horizon SphericalSurface::ntheta [4] = 41 SphericalSurface::nphi [4] = 80 SphericalSurface::nghoststheta [4] = 2 SphericalSurface::nghostsphi [4] = 2 ################################################################################ # Isolated Horizons ################################################################################ QuasiLocalMeasures::verbose = no QuasiLocalMeasures::veryverbose = no QuasiLocalMeasures::interpolator = "Lagrange polynomial interpolation" QuasiLocalMeasures::interpolator_options = "order=4" QuasiLocalMeasures::spatial_order = 4 QuasiLocalMeasures::num_surfaces = 3 QuasiLocalMeasures::surface_index [0] = 2 QuasiLocalMeasures::surface_index [1] = 3 QuasiLocalMeasures::surface_index [2] = 4 QuasiLocalMeasures::output_vtk_every = 0 ################################################################################ # Correctness checking ################################################################################ Carpet::poison_new_timelevels = no Carpet::check_for_poison = no NaNChecker::check_every = 256 NanChecker::check_after = 0 NaNChecker::report_max = 10 NaNChecker::verbose = "all" NaNChecker::action_if_found = terminate NaNChecker::out_NaNmask = yes NaNChecker::check_vars = " ML_BSSN::ML_log_confac " ################################################################################ # Timers ################################################################################ Carpet::output_timer_tree_every = 1024 Carpet::output_initialise_timer_tree = yes ################################################################################ # Output ################################################################################ IO::out_dir = "@SIMULATION_NAME@" IOScalar::one_file_per_group = yes IOASCII::one_file_per_group = yes IOBasic::outInfo_every = 1 IOBasic::outInfo_reductions = "minimum maximum" IOBasic::outInfo_vars = " ML_BSSN::ML_log_confac Carpet::physical_time_per_hour SystemStatistics::maxrss_mb SystemStatistics::swap_used_mb " IOScalar::outScalar_every = 256 IOScalar::outScalar_reductions = "minimum maximum average" IOScalar::outScalar_vars = "SystemStatistics::process_memory_mb" IOASCII::out0D_every = 256 IOASCII::out0D_vars = " Carpet::timing PunctureTracker::pt_loc QuasiLocalMeasures::qlm_scalars{out_every = 256} " IOASCII::out1D_every = 0 IOASCII::out1D_d = no IOASCII::out1D_x = yes IOASCII::out1D_y = no IOASCII::out1D_z = yes IOASCII::out1D_vars = " ML_BSSN::ML_log_confac ML_BSSN::ML_trace_curv WeylScal4::Psi4r " IOASCII::out2D_every = 0 IOASCII::out2D_vars = " " IOHDF5::out_every = 0 IOHDF5::out_vars = " Grid::Coordinates{out_every=1000000000 refinement_levels={0}} ML_BSSN::ML_log_confac WeylScal4::Psi4r WeylScal4::Psi4i WeylScal4::curvIr{refinement_levels={3 5}} WeylScal4::curvIi{refinement_levels={3 5}} WeylScal4::curvJr{refinement_levels={3 5}} WeylScal4::curvJi{refinement_levels={3 5}} " IOHDF5::out2D_every = 1024 IOHDF5::out2D_vars = " Grid::Coordinates{out_every=1000000000 refinement_levels={0}} ML_BSSN::alpha ML_BSSN::phi ML_BSSN::trK WeylScal4::Psi4r WeylScal4::Psi4i " ################################################################################ # Checkpointing and recovery ################################################################################ CarpetIOHDF5::checkpoint = yes IO::checkpoint_ID = no IO::recover = "autoprobe" IO::out_proc_every = 2 IO::checkpoint_on_terminate = yes IO::checkpoint_dir = "../checkpoints" IO::recover_dir = "../checkpoints" IO::abort_on_io_errors = yes CarpetIOHDF5::open_one_input_file_at_a_time = yes CarpetIOHDF5::compression_level = 0 ################################################################################ # Run termination ################################################################################ TerminationTrigger::max_walltime = @WALLTIME_HOURS@ # Trigger termination 30 minutes before the walltime is reached TerminationTrigger::on_remaining_walltime = 30 TerminationTrigger::output_remtime_every_minutes = 30 TerminationTrigger::termination_from_file = yes TerminationTrigger::termination_file = "terminate.txt" TerminationTrigger::create_termination_file = yes Cactus::terminate = time Cactus::cctk_final_time = 1700.0 From oevelasco at astro.unam.mx Wed Sep 17 12:11:06 2025 From: oevelasco at astro.unam.mx (=?UTF-8?Q?Omar_El=C3=ADas_Velasco_Castillo?=) Date: Wed, 17 Sep 2025 11:11:06 -0600 Subject: [Users] Questions about old ET releases and bundled libraries on clusters Message-ID: Dear Einstein Toolkit team, I hope this message finds you well. I am a beginner with the Einstein Toolkit. On personal workstations I have been able to compile and run tutorial simulations at low resolution, but I am facing problems on two different clusters. I would like to ask two questions: 1. *Are there ET versions prior to 2022_05 (e.g. 2019?2020 releases) that can still be downloaded and compiled successfully?* When I try to fetch them from the website using ./GetComponents, the process fails (CactusSourceJar.git is not created and some components do not download). Since some of the nodes I use have older GCC versions (8 or 10) and limited modules, a stable older release might be more practical. 2. During compilation, I notice that thorns (such as GSL and HDF5, for example) fall back to using the bundled versions because system modules are not found. The build completes successfully, but jobs fail immediately after submission to PBS or SLURM queues. *What is the role of the bundled versions in this case*?* If the build uses bundled GSL/HDF5, do I still need to load corresponding, compatible modules in the runscript?* Could this mismatch explain why jobs die right after submission? Any advice on handling these issues would be very helpful. Thank you very much for your time and support. Greetings, O.V. -------------- next part -------------- An HTML attachment was scrubbed... URL: From academic at perimeterinstitute.ca Wed Sep 17 14:31:51 2025 From: academic at perimeterinstitute.ca (Kendrick Smith) Date: Wed, 17 Sep 2025 14:31:51 -0500 Subject: [Users] Applications Open for Postdoctoral Research Positions Message-ID: <68cb0ca7.H0IJfobExPhBqgoY%academic@perimeterinstitute.ca> Perimeter Institute for Theoretical Physics is inviting applications for Postdoctoral Research positions. Dear Valued Colleague, Perimeter Institute for Theoretical Physics is inviting applications for the Postdoctoral Research Program (https://landing.perimeterinstitute.ca/e3t/Ctc/2I+113/cjL-604/VVCJsX700VFxF2yzFDlrYjjW61z83B5CzGh4N6J4pZ83prCCW8wLKSR6lZ3kXVHDMV37twkN-W3GvCmh12Tt9BW88pSTw2ckYMtW28JypW4LHrZCN3mdQWh99bHzW3t0PBd6YNlRzW2CrCsb4g3NxZW4kKxWS6MvX_VW6ZVcXk2VYkCgW4vQdlj5xq_Q4W5Gbb-61D8CXmW7WTq8d4J6dGrW2VCtfW2xMc3nW6DnWZ_6ZjNVnVHv3R07QwJXHW5SBKk43nYzfXW2fYds54LT7vnW332srL6-4SwRW2x7YvQ6907LdW87Qwwc4f03TMW76Fkpt8xtg1SW43FNCm3wLMYpW9bg2qr5fDB7cW7J8dTy798rwrVNzNnR1W-QKDW1gqPHV97Cvf-W926pMl7H015VW5W_NXM76vb5Tf3Zgs4M04 ) . For more information, please visit the job posting (https://landing.perimeterinstitute.ca/e3t/Ctc/2I+113/cjL-604/VVCJsX700VFxF2yzFDlrYjjW61z83B5CzGh4N6J4pYT3prCCW7Y8-PT6lZ3kFVTjWZk5mRv50N7GFdpgDzmbdW1NlJBY5bgM6DW4nJVf04KyWS3W9g0JrN2rlsl8W2kBw1k7xlL3kW5MLpy0579XH_W5LjJTT29WSjdW4Vmn6N85xG07W2WRsT55g-FZhW1BMDZ22-w8jbW6yhJK79fyDGyW84BNcj89dtJbW8tBrc42PPpvmW6WMcj47tbqYlW69DG5Z1pSgvZW30LFf569cWqzW1KbVl26DB5-QW5hnnlv2WbnpvW7gBNQ27XcFTzW2218mT6D1l3bW4gqdp85B0wSrW12WhqJ79xXnYW3STdbv2-95rpW796Fhj5kyPdGW4vjfGK5JrqbPc-tX604 ) . We hope you will share the news by: 1. Forwarding this information directly to prospective candidates who may be interested in this opportunity 2. Downloading, printing, and hanging the poster (https://landing.perimeterinstitute.ca/e3t/Ctc/2I+113/cjL-604/VVCJsX700VFxF2yzFDlrYjjW61z83B5CzGh4N6J4pZ83prCCW8wLKSR6lZ3ncF6zMzXBDYKLW2n-j878kHCLLW6yB6q035GMTtVX3MwV4qMFh1W5jPX-M3Xzq2gW6-1sLC48P2KPW2bpzK15t2l9KW9ljlsl6jD8PgW4WMdy06fH6C_W1qs0c42w8bQCW74cq1V58tXyBW7jbdWd8zxWK3W8xsbZS9m4YhGN4H6nvbn66tdW3WL5t67mF7bnW6dd9ws5QdBTyN3-kJbR7p2SPW6TpQl86TCMFhW8Z7TMP8Yd3JhW2m01m098sl8GW2ZsH-p1M87tZW7-Myqy65_df5W7XW2Bn2pNWSdW546LST8hhkqbW3n0Cm24lqBnmW3B7q_47bll5VM_NQDy5hngwW2mxPv-1Y0B98f7mRJH404 ) 3. Sharing LinkedIn (https://landing.perimeterinstitute.ca/e3t/Ctc/2I+113/cjL-604/VVCJsX700VFxF2yzFDlrYjjW61z83B5CzGh4N6J4pYg59_KHW6N1X8z6lZ3kCW23Bs3Z6ZVlDLW4029JV8T0SQGW10Zyb_4Cpn4TW7pXrSy8Yy3T7N89NLWM95rYMW6lpfXG4288Q8W23MHX137NfMYW96zhj871f4SqW6Rx0w-4NwBflW5SZz-_4k_WztN4vySmfXs5tGN5w6VWFpLnn3W3bS86q3jvP1nW3rrBpH26W6TqW2wztsV5HwFGpW8J8gF42HhZNVW3Jk4w458FwWMW7fZh9h9h4j4VW3SGnKn8mr3rgW9h54vL7HTK3sW7lNgBS6D60tKV5K8t792SqtnW3ftVp58Q-R9FW4Q_r0z8ZfB7RV-NhSh65wyxXW3d7YBm6NffBwW46vSjV8mYq4DW2Kr3jx4Srq63W6SRWwP49PjRGW1vGy5s4c_PLTVHXBjv2zqS14W3ptbSJ6LLssqW4pWL0M5bRDQVVNfYJG3JDctvVhh-Rg5VCfX3W8dWmVf2C9bvvW1vkkk97rGTnbW1zmPlz6Kmm1tf5-nhgW04 ) and Facebook (https://landing.perimeterinstitute.ca/e3t/Ctc/2I+113/cjL-604/VVCJsX700VFxF2yzFDlrYjjW61z83B5CzGh4N6J4pYg5m_5PW6N1X8z6lZ3ngVrCSGY4VxQzsW1CB4hX99fMlJW2kL8nx8ZcyRwW8SK_mf8xYF1XVcJX0M4XWLtHW1QNhw62xgfwRVLD4Vf56kKtNN29mhPwh1G2XW2YwpjY235QTbW3l5hwQ5CvDf3N9lTrY1qbtctW5QwWZV1_DKkcW7RcM4f81HF49N8dt9VF_lP22W4Xpfnk4GTtQ5W2yMrkB14f6lrW1D08dC2Qs7K-W7mMWGV1lRDVSW69HZ0f5mbRmtW78hLsF8_9g4DW2JMVj25WhnKvW3dTvKB5Ln7qgW3gCVyH3VzsQRW1y0y-95BH3DWN6bLSbnFG_1-W56H9NY35lRBPW1cJQ0X7h81v8VvHTlh20vsqlW24wZ114rZ6wNW5nR5M22wND0ZW7kpgSr6_bWK-N8YMgg8FYL57W4kSdBF8l5zNcW4tsLnx1FShgsV1T3qc532JpKW5MsRYL60WN93W8vvz4L8fxd2kW5xX3gZ5Khh9wf18fSFM04 ) posts with your social networks Perimeter Institute offers a dynamic, multi-disciplinary environment with maximum research freedom and opportunity to collaborate. We welcome all candidates to apply by November 1, 2025 but applications will be considered until all positions are filled. Thank you in advance for forwarding and sharing this information. Sincerely, Kendrick Smith Committee Chair Perimeter Institute for Theoretical Physics, 31 Caroline Street North, Waterloo, Ontario N2L 2Y5, Canada, 519-569-7600 From users at einsteintoolkit.org Wed Sep 17 17:15:02 2025 From: users at einsteintoolkit.org (users at einsteintoolkit.org) Date: Wed, 17 Sep 2025 17:15:02 -0500 Subject: [Users] Einstein Toolkit Meeting Reminder Message-ID: <68cb32e6.0SHNyrF/65GZRX3T%users@einsteintoolkit.org> Hello, Please consider joining the weekly Einstein Toolkit phone call at 9:00 am US central time on Thursdays. For details on how to connect and what agenda items are to be discussed, use the link below. https://docs.einsteintoolkit.org/et-docs/Main_Page#Weekly_Users_Call --The Maintainers From PANAGIOTIS.IOSIF at units.it Thu Sep 18 08:25:39 2025 From: PANAGIOTIS.IOSIF at units.it (IOSIF PANAGIOTIS) Date: Thu, 18 Sep 2025 13:25:39 +0000 Subject: [Users] Inconsistency warnings: cores/threads mismatch [Leonardo cluster] In-Reply-To: <20250910085950.706e31d2@haengie2.phas.ubc.ca> References: <20250910085950.706e31d2@haengie2.phas.ubc.ca> Message-ID: Dear Roland, I am following up on my question about how to correctly request the resources that I need. Here is a reminder of the context: * I have installed ETK on the Leonardo, DCGP partition at CINECA * I want to run a TOV star example requesting a different number of cores each time (e.g. 16, 32, 64 etc) to better understand the timescales. * I use this command to submit the simulation (sim is the typical alias for "./simfactory/bin/sim"): * sim create-submit tov_test --parfile par/tov_ET.par --cores 16 --num-threads 1 --walltime 00:20:00 The config files I use are here, and they are based on Bruno Giacomazzo's originals: * https://github.com/piosif/einstein-toolkit-config-leonardo/ I get a weird error from the CarpetLib thorn, which complains that: * The grid structure is inconsistent. It is impossible to continue. I have attached the simulation .err and .out files. FWIW, inspecting the very beginning of the two files, it seems that the overall setup I used (config files and submit command) results in: * MPI processes: 16 * OpenMP threads per core: 0.14285714285714285 This is 16 divided by 112 (the total cores in each node), and it does not seem to be what I want. I would expect that each OpenMP thread should be assigned to one core. Furthermore, for that simple test and if my understanding of the basics of MPI and OpenMP is correct, it would make more sense to have just one MPI process and 16 OpenMP threads. Could you help me understand what is wrong with my submit command above in connection to the options specified in my config files? Best, Panayotis ________________________________ From: Roland Haas Sent: Wednesday, September 10, 2025 5:59 PM To: IOSIF PANAGIOTIS Cc: Einstein Toolkit Users Subject: Re: [Users] Inconsistency warnings: cores/threads mismatch [Leonardo cluster] Hello Panayotis, "procs" as used by simfactory is (for historical reasons) a misleading term (https://simfactory.bitbucket.io/simfactory2/userguide/processterminology.html). Really "procs" is the total number of threads that will be started, which usually will be the same as total number of (logical) cpus being used, and again usually the same as the number of (phyisal) cores used. If using --cores 16 you are asking for a total of 16 threads to be created which should be assigned to 16 cores. You should then select a number of threads-per-rank using the `--num-threads` option that is good for your setup. For 16 cores most likely `--num-threads 1` (ie one thread per MPI rank) is best. This will most likely fix your issue. Note that this will leave some cores unused and a number of simfactory machine description files will not handle that case gracefully since they assume that you will always use at least one full node. For you questions: * this is usually more of an issue with the SubmitScript and RunScript rather than the ini file and the simfactory command line, though in your case the init script has `num-threads = 56` which sets the default number of for `--num-threads` to 56, which is a bit high (but is overruled by a --num-threads option on your command line) * yes, specifying --num-threads on the command line will override the value in the ini file Yours, Roland > [CAUTION: Non-UBC Email] > > Hi all, > > After having installed the toolkit on Leonardo, CINECA, I wanted to try the TOV examples. > > However, I am getting the following warnings: > Warning: Too many threads per process specified: specified num-threads=56 (ppn-used is 112) > Warning: Total number of threads and number of threads per process are inconsistent: procs=16, num-threads=56 (procs*num-smt must be an integer multiple of num-threads) > Warning: Total number of threads and number of cores per node are inconsistent: procs=16, ppn-used=112 (procs must be an integer multiple of ppn-used) > From those, it seems that I am doing something wrong either in my machine.ini file or in the resources request when submitting the job. > > Here is the command I use to run the TOV example: > ./simfactory/bin/sim submit tov_ET --parfile par/tov_ET.par --cores 16 --walltime 00:20:00 > > And here is the ini file I am using (the "Cluster characteristics" section is identical to Bruno Giacomazzo's file: > > https://github.com/piosif/einstein-toolkit-config-leonardo/blob/main/leonardo-dcgp1.ini > > The job ran and completed fine, but from what I understand, it ran on 56 cores instead of the 16 that I requested. > > From what I understand, the problem is that I am requesting just 16 cores for this simple test, but due to the cluster specs in the ini file, SimFactory expects me to request a multiple of 56. > Is that so? > > So, my question is the following: > > * > I want to do some tests, running a single star e.g. on a single node using 16, 32, 64 cores (to understand the timescale required). Do I need to change my ini file? > * > Or is there some option to specify when submitting the job that will override the cluster specs? > > I read the simfactory's terminology explanation on nodes, threads etc, but it is not clear to me how to request the cores I need. > > Best, > Panayotis -- My email is as private as my paper mail. I therefore support encrypting and signing email messages. Get my PGP key from http://pgp.mit.edu . -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: tov_test.out Type: application/octet-stream Size: 144874 bytes Desc: tov_test.out URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: tov_test.err Type: application/octet-stream Size: 75260 bytes Desc: tov_test.err URL: From PANAGIOTIS.IOSIF at units.it Thu Sep 18 09:05:10 2025 From: PANAGIOTIS.IOSIF at units.it (IOSIF PANAGIOTIS) Date: Thu, 18 Sep 2025 14:05:10 +0000 Subject: [Users] clarification on folders output-xxxx Message-ID: Hi all, I am not sure I understand what the output-xxxx folders do under each individual simulation folder. Suppose we "create-submit" a new simulation named test. The results will appear in the output-0000 folder under the folder "home/username/simulations/test/". OK, great. Suppose now we want to re-run that simulation. And suppose we want to re-run it from scratch and not continue from a previous checkpoint (e.g. maybe we forgot to save some output we needed). The skeleton is already created, so we can just use the command submit this time. The results will appear in the output-0001 folder. Is the above understanding correct? Questions: * When we submit again the simulation for the re-run, do we have to specify the same computational resources? * I.e. is this info something already encoded in the simulation's skeleton and therefore we should keep it fixed for consistency? * Does the skeleton of the simulation also fix what quantities are saved and therefore if we forgot to save some quantity we have to create a new simulation skeleton? Additionally, from what I understand, the output-xxxx folders mainly handle different checkpoints of a larger simulation that could take weeks or months. * Does the user have to do anything to combine all the information from the checkpoints, or is this handled automatically by ETK / SimFactory? * Is there any advice on best practices regarding this scenario? Best, Panayotis ------------------- Panagiotis Iosif postdoctoral researcher Department of Physics, University of Trieste Via Alfonso Valerio 2, Trieste 34127 Italy ------------------- -------------- next part -------------- An HTML attachment was scrubbed... URL: From diener at cct.lsu.edu Thu Sep 18 10:45:33 2025 From: diener at cct.lsu.edu (Peter Diener) Date: Thu, 18 Sep 2025 10:45:33 -0500 (CDT) Subject: [Users] clarification on folders output-xxxx In-Reply-To: References: Message-ID: <240fe41e-f59a-4ec3-43b2-fecf9a91f797@cct.lsu.edu> Hi Iosif, The different output-xxxx folders are mostly intended to be used when recovering from checkpoint files for run that is longer than the queue time on a machine. However, the recovery from checkpoint files have to be activated by parameter settings in the parameter file. You can see an example of that in the Checkpoint/Recovery: section of the binary neutron star gallery parameter file (see https://www.einsteintoolkit.org/gallery/bns/bns.par). As you say using create-submit will create a new simulation and the output from that run will be in output-0000. If you have checkpoint/recovery activated in your parameter file (here the IO::recover = "autoprobe" is crucial) if you then does a simfactory submit, a new run will be started starting from the last checkpoint file in output-0000 with the output being placed in output-0001. Another simfactory submit will then create a new run that starts from the last checkpoint file in output-0001 and produces output in output-0002. If you instead want to rerun from the beginning I would rather create a new simulation either with a new name or deleting the old simulation first (simfactory has a delete with create-submit. If you rerun using simfactory submit, you don't have to specify the same number of processors. Cactus supports recovering on a different number of processors than the run that created the checkpoint files. Cactus supports changing some (not all) parameters on recovery. It depends on whether the parameters are declared (in the param.ccl file) as being steerable. If you look at the param.ccl file for CarpetIOHDF5 you can see that the parameters that chooses the output variables all have STEERABLE = ALWAYS, so yes, you can change what variables are output on recovery. It's been a while since I have done this, but I suspect you have to modify your parameter file and then specify --parfile= when you do the simfactory submit command to restart. Finally in order to look at the data in from multiple output-xxxx directories, Cactus itself does nothing to combine the data from different output directories. However, Kuibit and SimulationTools, have been developed to handle stuff like this automatically for you. Kuibit is written in python, while SimulationTools is written in Mathematica. You can find these at: https://sbozzolo.github.io/kuibit/ https://simulationtools.org/ I hope this answered your questions. Cheers, Peter Diener Assistant Professor Research Center for Computation & Technology and Department of Physics & Astronomy Louisiana State University On Thu, 18 Sep 2025, IOSIF PANAGIOTIS wrote: > Hi all, > > I am not sure I understand what the output-xxxx?folders do under each > individual simulation folder. > > Suppose we "create-submit"?a new simulation named test. > The results will appear in the output-0000?folder under the folder > "home/username/simulations/test/". > OK, great. > > Suppose now we want to re-run that simulation. > And suppose we want to re-run it from scratch and not continue from a > previous checkpoint (e.g. maybe we forgot to save some output we needed).? > The skeleton is already created, so we can just use the command submit?this > time. > The results will appear in the output-0001?folder. > Is the above understanding correct? > > Questions: > * When we submit again the simulation for the re-run, do we have to > specify the same computational resources? > * I.e. is this info something already encoded in the simulation's skeleton > and therefore we should keep it fixed for consistency? > * Does the skeleton of the simulation also fix what quantities are saved > and therefore if we forgot to save some quantity we have to create a new > simulation skeleton? > > Additionally, from what I understand, the output-xxxx?folders mainly handle > different checkpoints?of a larger simulation that could take weeks or > months. > * Does the user have to do anything to combine all the information from > the checkpoints, or is this handled automatically by ETK / SimFactory? > * Is there any advice on best practices regarding this scenario? > > > Best, > Panayotis > > > ------------------- > Panagiotis Iosif > postdoctoral researcher > Department of Physics, University of Trieste? > Via Alfonso Valerio 2, Trieste 34127 > Italy > ------------------- > > > From sbrandt at cct.lsu.edu Thu Sep 18 15:04:36 2025 From: sbrandt at cct.lsu.edu (Steven Brandt) Date: Thu, 18 Sep 2025 15:04:36 -0500 Subject: [Users] Questions about old ET releases and bundled libraries on clusters In-Reply-To: References: Message-ID: On 9/17/2025 12:11 PM, Omar El?as Velasco Castillo wrote: > Dear Einstein Toolkit team, > > I hope this message finds you well. I am a beginner with the Einstein > Toolkit. On personal workstations I have been able to compile and run > tutorial simulations at low resolution, but I am facing problems on > two different clusters. I would like to ask two questions: > > 1. *Are there ET versions prior to 2022_05 (e.g. 2019?2020 releases) > that can still be downloaded and compiled successfully?* When I try to > fetch them from the website using ./GetComponents, the process fails > (CactusSourceJar.git is not created and some components do not > download). Since some of the nodes I use have older GCC versions (8 or > 10) and limited modules, a stable older release might be more practical. > > 2. During compilation, I notice that thorns (such as GSL and HDF5, for > example) fall back to using the bundled versions because system > modules are not found. The build completes successfully, but jobs fail > immediately after submission to PBS or SLURM queues. Can you show us what the error message(s) are? > > *What is the role of the bundled versions in this case*?*If the build > uses bundled GSL/HDF5, do I still need to load corresponding, > compatible modules in the runscript?* > > Could this mismatch explain why jobs die right after submission? Maybe. I'm not 100% sure what you are doing. Can you be clearer about how you are running the ET? --Steve > > Any advice on handling these issues would be very helpful. Thank you > very much for your time and support. > > Greetings, > > O.V. > > > > > > _______________________________________________ > Users mailing list > Users at einsteintoolkit.org > http://lists.einsteintoolkit.org/mailman/listinfo/users -------------- next part -------------- An HTML attachment was scrubbed... URL: From oevelasco at astro.unam.mx Thu Sep 18 23:11:43 2025 From: oevelasco at astro.unam.mx (=?UTF-8?Q?Omar_El=C3=ADas_Velasco_Castillo?=) Date: Thu, 18 Sep 2025 22:11:43 -0600 Subject: [Users] Questions about old ET releases and bundled libraries on clusters In-Reply-To: References: Message-ID: Hi Steven, thank you very much for your attention. *Can you show us what the error message(s) are? * Yes, my error messages displayed in the .err file of a simulation are: -------------------------------------------------------------------------- Primary job terminated normally, but 1 process returned a non-zero exit code. Per user-direction, the job has been aborted. -------------------------------------------------------------------------- ERROR: ld.so: object '/lib64/libpapi.so.5.2.0.0' from LD_PRELOAD cannot be preloaded: ignored. /home/ia/ovelasco/simulations/tov_ET_decisiva/SIMFACTORY/exe/cactus_sim: error while loading shared libraries: libpapi.so.5.2.0.0: cannot open shared object file: No such file or directory =>> PBS: job killed: walltime 864033 exceeded limit 864000 mpirun: abort is already in progress...hit ctrl-c again to forcibly terminate I attach below the configurations file, runscript and submitscript I used for this job, together with the .err and .out files generated for that case. *Maybe. I'm not 100% sure what you are doing. Can you be clearer about how > you are running the ET?* Yes, my intention is to run the ET in the queue of a remote machine using Simfactory and a proper configurations file for the machine, as well as a suitable runscript and submitscript for either PBS or SLURM nodes, whatever I need in one node or another. The key point is that, I notice that while the sim build is being done, it seems that some lines printed on the shell indicate that when a module is not found in that machine, the ET compiler "builds a bundle" from a thorn, for those modules that weren't found: ******************************************************************************** Running configuration script for thorn GSL: GSL selected, but GSL_DIR not set. Checking pkg-config ... GSL not found. Checking standard paths ... GSL not found. Using bundled GSL... Finished running configuration script for thorn GSL. ******************************************************************************** Running configuration script for thorn HDF5: Additional requested language support: Fortran HDF5 selected, but HDF5_DIR not set. Checking pkg-config ... HDF5 not found. Checking standard paths ... HDF5 not found. Using bundled HDF5... Finished running configuration script for thorn HDF5. I attach here too as an example a logfile, called "build.log", produced with the prints on the terminal I refer to. So then, my question is, why can't I still run simulations in a queue if the sim build instruction is completed and done successfully? And I get errors like, for example, the one I showed you above. The bundle itself doesn't replace the lack of recent versions of modules or the availability of those modules in those remote machines/nodes? Sending you my best regards, El jue, 18 sept 2025 a las 14:04, Steven Brandt () escribi?: > > On 9/17/2025 12:11 PM, Omar El?as Velasco Castillo wrote: > > Dear Einstein Toolkit team, > > I hope this message finds you well. I am a beginner with the Einstein > Toolkit. On personal workstations I have been able to compile and run > tutorial simulations at low resolution, but I am facing problems on two > different clusters. I would like to ask two questions: > > 1. *Are there ET versions prior to 2022_05 (e.g. 2019?2020 releases) that > can still be downloaded and compiled successfully?* When I try to fetch > them from the website using ./GetComponents, the process fails > (CactusSourceJar.git is not created and some components do not download). > Since some of the nodes I use have older GCC versions (8 or 10) and limited > modules, a stable older release might be more practical. > > 2. During compilation, I notice that thorns (such as GSL and HDF5, for > example) fall back to using the bundled versions because system modules are > not found. The build completes successfully, but jobs fail immediately > after submission to PBS or SLURM queues. > > Can you show us what the error message(s) are? > > > *What is the role of the bundled versions in this case*?* If the build > uses bundled GSL/HDF5, do I still need to load corresponding, compatible > modules in the runscript?* > > Could this mismatch explain why jobs die right after submission? > > Maybe. I'm not 100% sure what you are doing. Can you be clearer about how > you are running the ET? > > --Steve > > > Any advice on handling these issues would be very helpful. Thank you very > much for your time and support. > > Greetings, > > O.V. > > > > > > _______________________________________________ > Users mailing listUsers at einsteintoolkit.orghttp://lists.einsteintoolkit.org/mailman/listinfo/users > > -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: flaw.cfg Type: application/octet-stream Size: 2537 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: tov.err Type: application/octet-stream Size: 18692 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: flaw.sub Type: text/x-microdvd Size: 761 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: flaw.run Type: application/octet-stream Size: 863 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: tov.out Type: application/octet-stream Size: 13710 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: build_NEW.log Type: text/x-log Size: 1986650 bytes Desc: not available URL: From mewesv at ornl.gov Mon Sep 22 09:30:39 2025 From: mewesv at ornl.gov (Mewes, Vassilios) Date: Mon, 22 Sep 2025 14:30:39 +0000 Subject: [Users] Assistant Professor - Computational Astrophysics, The University of Tennessee, Knoxville Message-ID: Hi all, See the attached pdf document for an Assistant Professor in Computational Astrophysics opening at the nearby University of Tennessee in Knoxville. Please share this with any interested candidates. Best wishes, Vassili -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: Univ of Tennessee - Astrophysics 2025.pdf Type: application/pdf Size: 108138 bytes Desc: Univ of Tennessee - Astrophysics 2025.pdf URL: From users at einsteintoolkit.org Mon Sep 22 15:18:02 2025 From: users at einsteintoolkit.org (users at einsteintoolkit.org) Date: Mon, 22 Sep 2025 15:18:02 -0500 Subject: [Users] Agenda for Thursday's Meeting Message-ID: <68d1aefa.N2TAJp1IZm5Djoxj%users@einsteintoolkit.org> Please update the Wiki with agenda items for Thursday's meeting. Thanks! https://docs.einsteintoolkit.org/et-docs/meeting_agenda --The Maintainers From users at einsteintoolkit.org Wed Sep 24 17:15:02 2025 From: users at einsteintoolkit.org (users at einsteintoolkit.org) Date: Wed, 24 Sep 2025 17:15:02 -0500 Subject: [Users] Einstein Toolkit Meeting Reminder Message-ID: <68d46d66.4HpbbsvXO7y63qkC%users@einsteintoolkit.org> Hello, Please consider joining the weekly Einstein Toolkit phone call at 9:00 am US central time on Thursdays. For details on how to connect and what agenda items are to be discussed, use the link below. https://docs.einsteintoolkit.org/et-docs/Main_Page#Weekly_Users_Call --The Maintainers From lucas.t.s.carneiro at gmail.com Thu Sep 25 10:20:14 2025 From: lucas.t.s.carneiro at gmail.com (Lucas Timotheo Sanches) Date: Thu, 25 Sep 2025 10:20:14 -0500 Subject: [Users] Meeting Minutes for 2025-09-18 Message-ID: Present: Lucas(minutes), Peter Diener, Leo Rosa Werneck (chair), Roland Hass, Keith Dow, Beyhan # Agenda items - BBH gallery example: Peter says ran the example with the executable from back then (the Annie Jump Cannon release binary) and it works. He still needs to visualize it. - Move repos to GitHub: Currently, there's a user named CactusCodewe which we could potentially use. We would need to reach out to this person and ask them if they would be interested in releasing the name. Potential issues on the migration includes some loss of functionality like the ability to download large files. Moving issues is also an issue, but we are probably sticking to BitBucket for that. It was decided to vote on this issue again once Steve is back. # Unanswered questions on the mailing list. - "Issues Running BNS and GW150914 Example Simulations + storing BSSN variables": Roland is in contact with the person and trying to help them. - "Questions about old ET releases and bundled libraries on clusters": You can't download old releases. Too bad. No sure if old cold will compile with modern compilers. # Tickets - 2706: Zach says there are updates, but things are moving slowly. Needs more time. - 2886: Roland says this is more of a TODO than a ticket. - 2866: Just comment it out. Still waiting on a response from the reporter. - 2858: Lucas still didn't have time to look into these files. - 2882: Alejandra is still working on testing if there are performance improvements. - 2664: No updates. - 963: Improve accuracy: Peters says testings are running, to early to tell. From PANAGIOTIS.IOSIF at units.it Thu Sep 25 10:29:14 2025 From: PANAGIOTIS.IOSIF at units.it (IOSIF PANAGIOTIS) Date: Thu, 25 Sep 2025 15:29:14 +0000 Subject: [Users] Inconsistency warnings: cores/threads mismatch [Leonardo cluster] In-Reply-To: References: <20250910085950.706e31d2@haengie2.phas.ubc.ca> Message-ID: Hi all, I am bumping this follow-up question regarding inconsistency warnings when requesting resources during a job submit, in case someone has some suggestion. Thanks, Panayotis ________________________________ From: IOSIF PANAGIOTIS Sent: Thursday, September 18, 2025 3:25 PM To: Roland Haas Cc: Einstein Toolkit Users Subject: Re: [Users] Inconsistency warnings: cores/threads mismatch [Leonardo cluster] Dear Roland, I am following up on my question about how to correctly request the resources that I need. Here is a reminder of the context: * I have installed ETK on the Leonardo, DCGP partition at CINECA * I want to run a TOV star example requesting a different number of cores each time (e.g. 16, 32, 64 etc) to better understand the timescales. * I use this command to submit the simulation (sim is the typical alias for "./simfactory/bin/sim"): * sim create-submit tov_test --parfile par/tov_ET.par --cores 16 --num-threads 1 --walltime 00:20:00 The config files I use are here, and they are based on Bruno Giacomazzo's originals: * https://github.com/piosif/einstein-toolkit-config-leonardo/ I get a weird error from the CarpetLib thorn, which complains that: * The grid structure is inconsistent. It is impossible to continue. I have attached the simulation .err and .out files. FWIW, inspecting the very beginning of the two files, it seems that the overall setup I used (config files and submit command) results in: * MPI processes: 16 * OpenMP threads per core: 0.14285714285714285 This is 16 divided by 112 (the total cores in each node), and it does not seem to be what I want. I would expect that each OpenMP thread should be assigned to one core. Furthermore, for that simple test and if my understanding of the basics of MPI and OpenMP is correct, it would make more sense to have just one MPI process and 16 OpenMP threads. Could you help me understand what is wrong with my submit command above in connection to the options specified in my config files? Best, Panayotis ________________________________ From: Roland Haas Sent: Wednesday, September 10, 2025 5:59 PM To: IOSIF PANAGIOTIS Cc: Einstein Toolkit Users Subject: Re: [Users] Inconsistency warnings: cores/threads mismatch [Leonardo cluster] Hello Panayotis, "procs" as used by simfactory is (for historical reasons) a misleading term (https://simfactory.bitbucket.io/simfactory2/userguide/processterminology.html). Really "procs" is the total number of threads that will be started, which usually will be the same as total number of (logical) cpus being used, and again usually the same as the number of (phyisal) cores used. If using --cores 16 you are asking for a total of 16 threads to be created which should be assigned to 16 cores. You should then select a number of threads-per-rank using the `--num-threads` option that is good for your setup. For 16 cores most likely `--num-threads 1` (ie one thread per MPI rank) is best. This will most likely fix your issue. Note that this will leave some cores unused and a number of simfactory machine description files will not handle that case gracefully since they assume that you will always use at least one full node. For you questions: * this is usually more of an issue with the SubmitScript and RunScript rather than the ini file and the simfactory command line, though in your case the init script has `num-threads = 56` which sets the default number of for `--num-threads` to 56, which is a bit high (but is overruled by a --num-threads option on your command line) * yes, specifying --num-threads on the command line will override the value in the ini file Yours, Roland > [CAUTION: Non-UBC Email] > > Hi all, > > After having installed the toolkit on Leonardo, CINECA, I wanted to try the TOV examples. > > However, I am getting the following warnings: > Warning: Too many threads per process specified: specified num-threads=56 (ppn-used is 112) > Warning: Total number of threads and number of threads per process are inconsistent: procs=16, num-threads=56 (procs*num-smt must be an integer multiple of num-threads) > Warning: Total number of threads and number of cores per node are inconsistent: procs=16, ppn-used=112 (procs must be an integer multiple of ppn-used) > From those, it seems that I am doing something wrong either in my machine.ini file or in the resources request when submitting the job. > > Here is the command I use to run the TOV example: > ./simfactory/bin/sim submit tov_ET --parfile par/tov_ET.par --cores 16 --walltime 00:20:00 > > And here is the ini file I am using (the "Cluster characteristics" section is identical to Bruno Giacomazzo's file: > > https://github.com/piosif/einstein-toolkit-config-leonardo/blob/main/leonardo-dcgp1.ini > > The job ran and completed fine, but from what I understand, it ran on 56 cores instead of the 16 that I requested. > > From what I understand, the problem is that I am requesting just 16 cores for this simple test, but due to the cluster specs in the ini file, SimFactory expects me to request a multiple of 56. > Is that so? > > So, my question is the following: > > * > I want to do some tests, running a single star e.g. on a single node using 16, 32, 64 cores (to understand the timescale required). Do I need to change my ini file? > * > Or is there some option to specify when submitting the job that will override the cluster specs? > > I read the simfactory's terminology explanation on nodes, threads etc, but it is not clear to me how to request the cores I need. > > Best, > Panayotis -- My email is as private as my paper mail. I therefore support encrypting and signing email messages. Get my PGP key from http://pgp.mit.edu . -------------- next part -------------- An HTML attachment was scrubbed... URL: From bruno.giacomazzo at unimib.it Fri Sep 26 03:53:21 2025 From: bruno.giacomazzo at unimib.it (Bruno Giacomazzo) Date: Fri, 26 Sep 2025 10:53:21 +0200 Subject: [Users] Inconsistency warnings: cores/threads mismatch [Leonardo cluster] In-Reply-To: References: <20250910085950.706e31d2@haengie2.phas.ubc.ca> Message-ID: Panayotis, Did you try using --procs=16 --num-threads=1? I never used --cores and I don't know the difference with procs. Cheers, Bruno Il giorno gio 25 set 2025 alle ore 17:29 IOSIF PANAGIOTIS < PANAGIOTIS.IOSIF at units.it> ha scritto: > Hi all, > > I am bumping this follow-up question regarding inconsistency warnings when > requesting resources during a job submit, in case someone has some > suggestion. > > Thanks, > Panayotis > > ------------------------------ > *From:* IOSIF PANAGIOTIS > *Sent:* Thursday, September 18, 2025 3:25 PM > *To:* Roland Haas > *Cc:* Einstein Toolkit Users > *Subject:* Re: [Users] Inconsistency warnings: cores/threads mismatch > [Leonardo cluster] > > Dear Roland, > > I am following up on my question about how to correctly request the > resources that I need. > > Here is a reminder of the context: > > - I have installed ETK on the Leonardo, DCGP partition at CINECA > - I want to run a TOV star example requesting a different number of > cores each time (e.g. 16, 32, 64 etc) to better understand the timescales. > > > - > > I use this command to submit the simulation (sim is the typical alias for > "./simfactory/bin/sim"): > > - sim create-submit tov_test --parfile par/tov_ET.par --cores 16 > --num-threads 1 --walltime 00:20:00 > > > The config files I use are here, and they are based on Bruno Giacomazzo's > originals: > > - https://github.com/piosif/einstein-toolkit-config-leonardo/ > > > I get a *weird error from the CarpetLib thorn*, which complains that: > > > - *The grid structure is inconsistent. It is impossible to continue.* > > > I have attached the simulation .err and .out files. > > FWIW, inspecting the very beginning of the two files, it seems that the > overall setup I used (config files and submit command) results in: > > > - MPI processes: 16 > - OpenMP threads per core: 0.14285714285714285 > > > This is 16 divided by 112 (the total cores in each node), and it does not > seem to be what I want. > I would expect that each OpenMP thread should be assigned to one core. > Furthermore, for that simple test and if my understanding of the basics of > MPI and OpenMP is correct, it would make more sense to have just one MPI > process and 16 OpenMP threads. > > Could you help me understand what is wrong with my submit command above in > connection to the options specified in my config files? > > Best, > Panayotis > > ------------------------------ > *From:* Roland Haas > *Sent:* Wednesday, September 10, 2025 5:59 PM > *To:* IOSIF PANAGIOTIS > *Cc:* Einstein Toolkit Users > *Subject:* Re: [Users] Inconsistency warnings: cores/threads mismatch > [Leonardo cluster] > > Hello Panayotis, > > "procs" as used by simfactory is (for historical reasons) a misleading > term > ( > https://simfactory.bitbucket.io/simfactory2/userguide/processterminology.html > ). > > Really "procs" is the total number of threads that will be started, > which usually will be the same as total number of (logical) cpus being > used, and again usually the same as the number of (phyisal) cores used. > > If using --cores 16 you are asking for a total of 16 threads to be > created which should be assigned to 16 cores. You should then select a > number of threads-per-rank using the `--num-threads` option that is > good for your setup. For 16 cores most likely `--num-threads 1` (ie one > thread per MPI rank) is best. This will most likely fix your issue. > > Note that this will leave some cores unused and a number of simfactory > machine description files will not handle that case gracefully since > they assume that you will always use at least one full node. > > For you questions: > > * this is usually more of an issue with the SubmitScript and RunScript > rather than the ini file and the simfactory command line, though > in your case the init script has `num-threads = 56` which sets the > default number of for `--num-threads` to 56, which is a bit high (but > is overruled by a --num-threads option on your command line) > * yes, specifying --num-threads on the command line will override the > value in the ini file > > Yours, > Roland > > > > [CAUTION: Non-UBC Email] > > > > Hi all, > > > > After having installed the toolkit on Leonardo, CINECA, I wanted to try > the TOV examples. > > > > However, I am getting the following warnings: > > Warning: Too many threads per process specified: specified > num-threads=56 (ppn-used is 112) > > Warning: Total number of threads and number of threads per process are > inconsistent: procs=16, num-threads=56 (procs*num-smt must be an integer > multiple of num-threads) > > Warning: Total number of threads and number of cores per node are > inconsistent: procs=16, ppn-used=112 (procs must be an integer multiple of > ppn-used) > > From those, it seems that I am doing something wrong either in my > machine.ini file or in the resources request when submitting the job. > > > > Here is the command I use to run the TOV example: > > ./simfactory/bin/sim submit tov_ET --parfile par/tov_ET.par --cores 16 > --walltime 00:20:00 > > > > And here is the ini file I am using (the "Cluster characteristics" > section is identical to Bruno Giacomazzo's file< > https://bitbucket.org/simfactory/simfactory2/commits/0d4011bf554ee695f9543e048ae9ff5edc5b5c98 > >: > > > > > https://github.com/piosif/einstein-toolkit-config-leonardo/blob/main/leonardo-dcgp1.ini > > > > The job ran and completed fine, but from what I understand, it ran on 56 > cores instead of the 16 that I requested. > > > > From what I understand, the problem is that I am requesting just 16 > cores for this simple test, but due to the cluster specs in the ini file, > SimFactory expects me to request a multiple of 56. > > Is that so? > > > > So, my question is the following: > > > > * > > I want to do some tests, running a single star e.g. on a single node > using 16, 32, 64 cores (to understand the timescale required). Do I need to > change my ini file? > > * > > Or is there some option to specify when submitting the job that will > override the cluster specs? > > > > I read the simfactory's terminology explanation on nodes, threads etc< > https://simfactory.bitbucket.io/simfactory2/userguide/processterminology.html>, > but it is not clear to me how to request the cores I need. > > > > Best, > > Panayotis > > > -- > My email is as private as my paper mail. I therefore support encrypting > and signing email messages. Get my PGP key from http://pgp.mit.edu . > _______________________________________________ > Users mailing list > Users at einsteintoolkit.org > http://lists.einsteintoolkit.org/mailman/listinfo/users > -- Prof. Bruno Giacomazzo Deputy Director of the Department of Physics University of Milano-Bicocca Piazza della Scienza 3 20126 Milano Italy email: bruno.giacomazzo at unimib.it phone: (+39) 02 6448 2321 web: http://www.brunogiacomazzo.org --------------------------------------------------------------------- There are only 10 types of people in the world: Those who understand binary, and those who don't ---------------------------------------------------------------------- -------------- next part -------------- An HTML attachment was scrubbed... URL: From PANAGIOTIS.IOSIF at units.it Fri Sep 26 06:38:25 2025 From: PANAGIOTIS.IOSIF at units.it (IOSIF PANAGIOTIS) Date: Fri, 26 Sep 2025 11:38:25 +0000 Subject: [Users] Inconsistency warnings: cores/threads mismatch [Leonardo cluster] In-Reply-To: References: <20250910085950.706e31d2@haengie2.phas.ubc.ca> Message-ID: Hi Bruno, I tried with --procs instead of --cores, but I got the same error (see .err and .out files attached). Checking the options available via: simfactory/bin/sim --help it seems that these two options have absolutely no difference between them: --cores= Total number of threads to use --procs= See --cores In principle, of course, I could work with all the 112 cores of one node on Leonardo. However, this seems a waste of resources for a simple test. And in any case, it would be good to understand how to properly request resources using the different options of SimFactory, overriding the mdb entries if need be. I will experiment further with the options, but if you have any other suggestion it is welcome. Best, Panayotis ________________________________ From: Bruno Giacomazzo Sent: Friday, September 26, 2025 10:53 AM To: IOSIF PANAGIOTIS Cc: Einstein Toolkit Users Subject: Re: [Users] Inconsistency warnings: cores/threads mismatch [Leonardo cluster] Panayotis, Did you try using --procs=16 --num-threads=1? I never used --cores and I don't know the difference with procs. Cheers, Bruno Il giorno gio 25 set 2025 alle ore 17:29 IOSIF PANAGIOTIS > ha scritto: Hi all, I am bumping this follow-up question regarding inconsistency warnings when requesting resources during a job submit, in case someone has some suggestion. Thanks, Panayotis ________________________________ From: IOSIF PANAGIOTIS > Sent: Thursday, September 18, 2025 3:25 PM To: Roland Haas > Cc: Einstein Toolkit Users > Subject: Re: [Users] Inconsistency warnings: cores/threads mismatch [Leonardo cluster] Dear Roland, I am following up on my question about how to correctly request the resources that I need. Here is a reminder of the context: * I have installed ETK on the Leonardo, DCGP partition at CINECA * I want to run a TOV star example requesting a different number of cores each time (e.g. 16, 32, 64 etc) to better understand the timescales. * I use this command to submit the simulation (sim is the typical alias for "./simfactory/bin/sim"): * sim create-submit tov_test --parfile par/tov_ET.par --cores 16 --num-threads 1 --walltime 00:20:00 The config files I use are here, and they are based on Bruno Giacomazzo's originals: * https://github.com/piosif/einstein-toolkit-config-leonardo/ I get a weird error from the CarpetLib thorn, which complains that: * The grid structure is inconsistent. It is impossible to continue. I have attached the simulation .err and .out files. FWIW, inspecting the very beginning of the two files, it seems that the overall setup I used (config files and submit command) results in: * MPI processes: 16 * OpenMP threads per core: 0.14285714285714285 This is 16 divided by 112 (the total cores in each node), and it does not seem to be what I want. I would expect that each OpenMP thread should be assigned to one core. Furthermore, for that simple test and if my understanding of the basics of MPI and OpenMP is correct, it would make more sense to have just one MPI process and 16 OpenMP threads. Could you help me understand what is wrong with my submit command above in connection to the options specified in my config files? Best, Panayotis ________________________________ From: Roland Haas > Sent: Wednesday, September 10, 2025 5:59 PM To: IOSIF PANAGIOTIS > Cc: Einstein Toolkit Users > Subject: Re: [Users] Inconsistency warnings: cores/threads mismatch [Leonardo cluster] Hello Panayotis, "procs" as used by simfactory is (for historical reasons) a misleading term (https://simfactory.bitbucket.io/simfactory2/userguide/processterminology.html). Really "procs" is the total number of threads that will be started, which usually will be the same as total number of (logical) cpus being used, and again usually the same as the number of (phyisal) cores used. If using --cores 16 you are asking for a total of 16 threads to be created which should be assigned to 16 cores. You should then select a number of threads-per-rank using the `--num-threads` option that is good for your setup. For 16 cores most likely `--num-threads 1` (ie one thread per MPI rank) is best. This will most likely fix your issue. Note that this will leave some cores unused and a number of simfactory machine description files will not handle that case gracefully since they assume that you will always use at least one full node. For you questions: * this is usually more of an issue with the SubmitScript and RunScript rather than the ini file and the simfactory command line, though in your case the init script has `num-threads = 56` which sets the default number of for `--num-threads` to 56, which is a bit high (but is overruled by a --num-threads option on your command line) * yes, specifying --num-threads on the command line will override the value in the ini file Yours, Roland > [CAUTION: Non-UBC Email] > > Hi all, > > After having installed the toolkit on Leonardo, CINECA, I wanted to try the TOV examples. > > However, I am getting the following warnings: > Warning: Too many threads per process specified: specified num-threads=56 (ppn-used is 112) > Warning: Total number of threads and number of threads per process are inconsistent: procs=16, num-threads=56 (procs*num-smt must be an integer multiple of num-threads) > Warning: Total number of threads and number of cores per node are inconsistent: procs=16, ppn-used=112 (procs must be an integer multiple of ppn-used) > From those, it seems that I am doing something wrong either in my machine.ini file or in the resources request when submitting the job. > > Here is the command I use to run the TOV example: > ./simfactory/bin/sim submit tov_ET --parfile par/tov_ET.par --cores 16 --walltime 00:20:00 > > And here is the ini file I am using (the "Cluster characteristics" section is identical to Bruno Giacomazzo's file: > > https://github.com/piosif/einstein-toolkit-config-leonardo/blob/main/leonardo-dcgp1.ini > > The job ran and completed fine, but from what I understand, it ran on 56 cores instead of the 16 that I requested. > > From what I understand, the problem is that I am requesting just 16 cores for this simple test, but due to the cluster specs in the ini file, SimFactory expects me to request a multiple of 56. > Is that so? > > So, my question is the following: > > * > I want to do some tests, running a single star e.g. on a single node using 16, 32, 64 cores (to understand the timescale required). Do I need to change my ini file? > * > Or is there some option to specify when submitting the job that will override the cluster specs? > > I read the simfactory's terminology explanation on nodes, threads etc, but it is not clear to me how to request the cores I need. > > Best, > Panayotis -- My email is as private as my paper mail. I therefore support encrypting and signing email messages. Get my PGP key from http://pgp.mit.edu . _______________________________________________ Users mailing list Users at einsteintoolkit.org http://lists.einsteintoolkit.org/mailman/listinfo/users -- Prof. Bruno Giacomazzo Deputy Director of the Department of Physics University of Milano-Bicocca Piazza della Scienza 3 20126 Milano Italy email: bruno.giacomazzo at unimib.it phone: (+39) 02 6448 2321 web: http://www.brunogiacomazzo.org --------------------------------------------------------------------- There are only 10 types of people in the world: Those who understand binary, and those who don't ---------------------------------------------------------------------- -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: tov_test2.out Type: application/octet-stream Size: 144880 bytes Desc: tov_test2.out URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: tov_test2.err Type: application/octet-stream Size: 75443 bytes Desc: tov_test2.err URL: From rhaas at mail.ubc.ca Fri Sep 26 09:31:45 2025 From: rhaas at mail.ubc.ca (Roland Haas) Date: Fri, 26 Sep 2025 07:31:45 -0700 Subject: [Users] Inconsistency warnings: cores/threads mismatch [Leonardo cluster] In-Reply-To: References: <20250910085950.706e31d2@haengie2.phas.ubc.ca> Message-ID: <20250926073145.70ee6853@haengie2.phas.ubc.ca> Hello all, > I never used --cores and I don't know the difference with procs. --cores is a synonym for --procs in simfactory. The hope was to avoid the confusion of "procs" being "Processes" or "Processors". Though it has been pointed out that the best name would actually be "--threads" since that is what simfactory actually starts, which then collides with "--num-threads" (threads per process). Does Leonardo actually charge you for partial nodes if you do no use a full one? Simfactory is mostly written under the assumption (true at the time) that HPC systems would give you full nodes all the time, so if you use 1 core or 112 cores of a node, the charge would be the same (though shared node systems are becoming more common for HPC now [or again]). Yours, Roland -- My email is as private as my paper mail. I therefore support encrypting and signing email messages. Get my PGP key from http://pgp.mit.edu . From wsun3 at nd.edu Fri Sep 26 09:58:30 2025 From: wsun3 at nd.edu (Wei Sun) Date: Fri, 26 Sep 2025 10:58:30 -0400 Subject: [Users] Inconsistency warnings: cores/threads mismatch [Leonardo cluster] Message-ID: Hi Panayotis, Can you try specifying --procs, --num-threads, and --ppn-used at the same time? It looks like your machine defaulted to --ppn-used=112, which is not what you want, so you may need to overwrite it. You can check your allocation details in /output-000x/SIMFACTORY/properties.ini. A useful strategy is to verify whether the allocation matches these relations: ppn-used = numthreads * nodeprocsprocs = numthreads * numprocs = nodes * ppn-used Below is my understanding (I personally use numsmt=1, so threads = cores in my case). Please correct me if I?m wrong on them: nodes = number of nodes procsrequested = total number of cores requested (may not all be used, but you still occupy them on the HPC) ppn = number of cores per node requested (may not all be used) numprocs = total number of MPI ranks actually used nodeprocs = number of MPI ranks per node actually used procs = total number of cores actually used numthreads = number of OpenMP threads per MPI rank actually used ppnused = number of cores per node actually used numsmt = number of threads per physical core Best, Wei -------------- next part -------------- An HTML attachment was scrubbed... URL: From PANAGIOTIS.IOSIF at units.it Mon Sep 29 05:24:48 2025 From: PANAGIOTIS.IOSIF at units.it (IOSIF PANAGIOTIS) Date: Mon, 29 Sep 2025 10:24:48 +0000 Subject: [Users] Inconsistency warnings: cores/threads mismatch [Leonardo cluster] In-Reply-To: <20250926073145.70ee6853@haengie2.phas.ubc.ca> References: <20250910085950.706e31d2@haengie2.phas.ubc.ca> <20250926073145.70ee6853@haengie2.phas.ubc.ca> Message-ID: Hi Roland, Thanks for your reply. You touch on an important point, i.e the cluster's billing policy, that hadn't crossed my mind. >From the billing policy of Leonardo, it seems that it is possible to use only a fraction of a node's total CPUs. https://docs.hpc.cineca.it/hpc/hpc_intro.html#billing-policy However, the documentation also stresses that: ...if a job reserves all of a node?s RAM ? even without utilizing all its CPUs ? the node becomes unusable for other jobs and is therefore billed accordingly. So, apart from the cores requested, should I also try to calculate the RAM requirements? For example, I see that Bruno's "leonardo-dcgp.ini" file specifies: memory = 494000 And the respective submitscript also has this line: #SBATCH --mem 494000MB I note that each node in Leonardo has 512GB of RAM, so that means that the script requests ~94.2% of the RAM. I am not sure I follow the reasoning behind this. What is the default behavior of SimFactory if I were to remove the above specifications from the config files? Because, if by default Simfactory requests/uses all the RAM available in a node, then as far as I understand, it does not make sense to request fewer cores than a full node. Let me know what you think. Best, Panayotis ________________________________ From: Roland Haas Sent: Friday, September 26, 2025 4:31 PM To: Bruno Giacomazzo Cc: IOSIF PANAGIOTIS ; Einstein Toolkit Users Subject: Re: [Users] Inconsistency warnings: cores/threads mismatch [Leonardo cluster] Hello all, > I never used --cores and I don't know the difference with procs. --cores is a synonym for --procs in simfactory. The hope was to avoid the confusion of "procs" being "Processes" or "Processors". Though it has been pointed out that the best name would actually be "--threads" since that is what simfactory actually starts, which then collides with "--num-threads" (threads per process). Does Leonardo actually charge you for partial nodes if you do no use a full one? Simfactory is mostly written under the assumption (true at the time) that HPC systems would give you full nodes all the time, so if you use 1 core or 112 cores of a node, the charge would be the same (though shared node systems are becoming more common for HPC now [or again]). Yours, Roland -- My email is as private as my paper mail. I therefore support encrypting and signing email messages. Get my PGP key from http://pgp.mit.edu . -------------- next part -------------- An HTML attachment was scrubbed... URL: From PANAGIOTIS.IOSIF at units.it Mon Sep 29 08:47:36 2025 From: PANAGIOTIS.IOSIF at units.it (IOSIF PANAGIOTIS) Date: Mon, 29 Sep 2025 13:47:36 +0000 Subject: [Users] Inconsistency warnings: cores/threads mismatch [Leonardo cluster] In-Reply-To: References: Message-ID: Hi Wei and all, Thank you for your suggestion. The properties.ini file and the two relations you mentioned are very useful to check that what I specify as options when submitting the job is actually what I want. Indeed, specifying at the same time e.g. --procs 32 --num-threads 32 --ppn-used 32 I can force the use of 32 cores, overriding the settings in the machine.ini file, without getting inconsistency warnings. Therefore, it seems that if I am to leave the .ini file unchanged, all 3 options above are needed. Now, taking also into account the cluster's billing policy (that Roland Haas flagged in his answer), I am trying to understand how best to formulate my resource requests. So, I created a new simulation, not making any specific resource request via the command line, to see what happens with the default values in , i.e: sim create-submit tov_test3 --parfile par/tov_ET.par --walltime 00:20:00 I have attached the respective properties.ini, .err and .out files. Inspecting the files, do I understand correctly that only 56 threads/cores were used in this scenario? Given that in the Leonardo cluster there are 112 cores per node (namely 2 x Intel? Xeon? Platinum 8480+ Processor , each of which has 56 cores) does this mean that this way I am using only half the node? Should I specify: --procs 112 --num-threads 112 to use one whole node in my case? For reference, I have also attached the ini file that I am using (note that there we have the value num-threads=56). Does this make sense in the light of the above discussion? Best regards, Panayotis ________________________________ From: Wei Sun Sent: Friday, September 26, 2025 4:58 PM To: IOSIF PANAGIOTIS Cc: users at einsteintoolkit.org Subject: Re: Inconsistency warnings: cores/threads mismatch [Leonardo cluster] Hi Panayotis, Can you try specifying --procs, --num-threads, and --ppn-used at the same time? It looks like your machine defaulted to --ppn-used=112, which is not what you want, so you may need to overwrite it. You can check your allocation details in /output-000x/SIMFACTORY/properties.ini. A useful strategy is to verify whether the allocation matches these relations: ppn-used = numthreads * nodeprocs procs = numthreads * numprocs = nodes * ppn-used Below is my understanding (I personally use numsmt=1, so threads = cores in my case). Please correct me if I?m wrong on them: nodes = number of nodes procsrequested = total number of cores requested (may not all be used, but you still occupy them on the HPC) ppn = number of cores per node requested (may not all be used) numprocs = total number of MPI ranks actually used nodeprocs = number of MPI ranks per node actually used procs = total number of cores actually used numthreads = number of OpenMP threads per MPI rank actually used ppnused = number of cores per node actually used numsmt = number of threads per physical core Best, Wei -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: properties_tov_test3.ini Type: application/octet-stream Size: 1462 bytes Desc: properties_tov_test3.ini URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: tov_test3.out Type: application/octet-stream Size: 150035 bytes Desc: tov_test3.out URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: tov_test3.err Type: application/octet-stream Size: 2308 bytes Desc: tov_test3.err URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: leonardo-dcgp1.ini Type: application/octet-stream Size: 6708 bytes Desc: leonardo-dcgp1.ini URL: From users at einsteintoolkit.org Mon Sep 29 15:18:02 2025 From: users at einsteintoolkit.org (users at einsteintoolkit.org) Date: Mon, 29 Sep 2025 15:18:02 -0500 Subject: [Users] Agenda for Thursday's Meeting Message-ID: <68dae97a.W3HnXC0QCf9Q0hM6%users@einsteintoolkit.org> Please update the Wiki with agenda items for Thursday's meeting. Thanks! https://docs.einsteintoolkit.org/et-docs/meeting_agenda --The Maintainers