From hayleyjmacpherson at gmail.com Fri Sep 1 15:08:57 2023 From: hayleyjmacpherson at gmail.com (Hayley Macpherson) Date: Fri, 1 Sep 2023 10:08:57 -1000 Subject: [Users] Loss of precision after postintial? In-Reply-To: <20230831105408.2d3944e2@fdea4908.spdns.org> References:

<47C899DA-E45A-486E-987B-4F8ADD3E2EE1@gmail.com> <20230824163926.7ff601f9@fdea4908.spdns.org> <20230831105408.2d3944e2@fdea4908.spdns.org> Message-ID: Hi Roland, It?s no worries at all. Thanks for discussing it in the call. > One question that came up was if the difference you see in say 3-Ricci > is present everywhere or only at say refinement boundaries? I?m not using any refinement. And I did check my calculation of 3-Ricci vs the 3-Ricci output from ADMAnalysis and these do disagree at the same order as the violation after POSTINITIAL. I?m still confused because in my separate calculation I am using fourth-order derivatives and I also checked that in ADMAnalysis the order of stencil is also fourth order, so they should be the same precision. I am confident there isn?t a bug in my code because I?ve been using the same one for years, but of course this is always a possibility. Though, because I am also seeing a difference in the momentum constraint, and that this difference occurs in the derivatives of K_ij, my guess would be it?s most likely down to a difference in precision/accuracy of derivatives somewhere and not directly a problem with 3-Ricci itself. > If you have not already done so (if you have, then sorry, I missed > them) could you possible send a (cut down, does not have to include the > analysis code or private codes) parameter file that would show the > problem? I could do this, though it includes some additional codes to the flrwsolver thorn which are not in the ET repository. These are the codes that calculate the constraints and show the change in satisfaction of the constraints at these two steps. Let me know what would be the best way to get these to you. > Is the issue present only in quantities that are being > re-computed (3-Ricci) by the analysis code or already in quantities > that are in the HDF5 files (gij, kij in ADMBase and gammaij, Aij and > the traces in the BSSN code)? The issue is only in quantities which contain derivatives of gij or Kij, so I only see a change in the 3-Ricci term and D_i K^i_j and D_i K using the same codes at the post initial stage and then again using the HDF5 initial dump. Thanks for all your help on this! Best, Hayley ---- Hayley Macpherson | NASA Einstein Fellow Email: hjmacpherson at uchicago.edu Pronouns: she/her/hers Office: ERC 479 Kavli Institute for Cosmological Physics Eckhardt Research Center 5640 South Ellis Avenue Chicago, IL, 60637 > On 31 Aug 2023, at 04:54, Roland Haas wrote: > > Hello Hayley, > > Sorry for the delay. I am currently out of office and slow with emails. > > GRHydro was really my only guess, and I ma pretty stumped. We discussed > this in the ET call today but could not come up with a good answer. > > One question that came up was if the difference you see in say 3-Ricci > is present everywhere or only at say refinement boundaries? > > If you have not already done so (if you have, then sorry, I missed > them) could you possible send a (cut down, does not have to include the > analysis code or private codes) parameter file that would show the > problem? Is the issue present only in quantities that are being > re-computed (3-Ricci) by the analysis code or already in quantities > that are in the HDF5 files (gij, kij in ADMBase and gammaij, Aij and > the traces in the BSSN code)? > > Yours, > Roland > >> Hi Roland, >> >> Yes, there is a fluid and I am using GRHydro. But I have checked individual terms, and the matter terms are identical between my checks in HydroBase_Initial and my analysis of the HDF5 output. >> >> I have pinpointed the difference to be coming from the terms with spatial derivatives of metric quantities, e.g., D_i trK and 3-Ricci tensor. The codes I use for these two calculations use the same routines, so all I can think of now is a possible precision issue somewhere. >> >> I?ll keep investigating, but if you have any other ideas please let me know! >> >> Best, >> Hayley >> >>> On 24 Aug 2023, at 15:39, Roland Haas wrote: >>> >>> Hello all, >>> >>> just to be sure: is there a fluid present (and GRHydro)? Then there is >>> also the issue of con2prim to consider (and Tmunu). >>> >>> Yours, >>> Roland >>> >>>> Hayley >>>> >>>> No, there should not be any loss of precision. The difference in the >>>> ADMBase variables should be at round-off level. >>>> >>>> -erik >>>> >>>> On Wed, Aug 23, 2023 at 12:37?PM Hayley Macpherson >>>> wrote: >>>>> >>>>> Hi Erik, >>>>> Thanks for the quick response! >>>>> >>>>> I?m not using mesh refinement, and yes I ignore the ghost zones in my calculations. I use periodicity within the original (no ghosts) grid for derivatives. >>>>> >>>>> And yes I?m using the ADMBase metric and extrinsic curvature. I get the same result whether I do the calculation in the postpostinitial stage or using the 3D dumps in post-processing. Would there be a loss of precision in ADMBase gij and kij when translating back and forth to BSSN metric/kij, which could then be propagated to the output files? >>>>> >>>>> Best, >>>>> Hayley >>>>> >>>>> On 23 Aug 2023, at 11:31, Erik Schnetter wrote: >>>>> >>>>> Hayley >>>>> >>>>> Are you using mesh refinement in your calculations? If so, the ET >>>>> would fill the ghost and the buffer zones with interpolated data. >>>>> These are generally less accurate than properly calculated data. If >>>>> so, are you ignoring ghost and buffer zones? These should not be used >>>>> for visualization and post-processing. >>>>> >>>>> Which metric variables are you accessing? I assume you are looking at >>>>> ADMBase for the metric and extrinsic curvature? These should change, >>>>> unless you look at them too early: During initialization, we set up >>>>> the ADMBase variables, then define the BSSN variables from these, and >>>>> then recalculate the ADMBase variables. This would change them >>>>> slightly. Analysis tools should only look at the ADMBase variables >>>>> after they have been reset. >>>>> >>>>> -erik >>>>> >>>>> >>>>> On Wed, Aug 23, 2023 at 11:56?AM Hayley Macpherson >>>>> wrote: >>>>> >>>>> >>>>> Hi all, >>>>> >>>>> I?m the author of a cosmological initial data thorn in the ET; FLRWSolver. I?m currently working on improving the initial data to solve the constraint equations exactly (instead of previously using an approximation), for a given metric and Kij. The way this works is to calculate the metric terms on the LHS of both the constraints and solve for the relevant rest mass density and peculiar 3-velocities from the matter terms. >>>>> >>>>> I have my own routines from a post-processing analysis code to calculate the metric terms, and I?ve incorporated these into the ET for both generating the initial data but also then double checking the constraint violation after the initial data is set up. >>>>> >>>>> I noticed something strange: my checks immediately after FLRWSolver is called in the ET show the constraints are satisfied essentially to roundoff error (i.e. the momentum constraint violation has magnitude ~ 1.e-15), but when I take the 3D dump of the initial time slice and run this through my analysis code (which uses the same routines as I use to set up initial data), I see the momentum constraint is violated at the ~ 1e-7 level. >>>>> >>>>> I thought this might be my post processing code, so I added a second call to check the constraints using my routines after the full initial process is finished. The first call which immediately follows my FLRWSolver routine is placed in group HydroBase_Initial, and I added another call in CCTK_POSTINITIAL which gives the same result, however, if I instead schedule this call in POSTPOSTINITIAL I see the momentum constraint violation is identical to what I see when post processing the initial dump, at ~ 1.e-7. I can see the specific terms which are causing this difference are the terms which use finite-difference derivatives (the curvature terms in the momentum constraint), while all others are identical. >>>>> >>>>> So my question is the following: is there any way that the metric and curvature variables could suffer a loss of precision between the POSTINITIAL and POSTPOSTINITIAL phases of the run? Especially, a loss in precision which is then translated to the 3D dumps. >>>>> >>>>> The momentum constraint violation I have is satisfactory, but I am trying to pinpoint why this jump happens to make sure it?s not a bug in my separate code somewhere (also to explain why the constraints aren?t satisfied to roundoff level when they should be, by construction of my initial data :) ). >>>>> >>>>> Any help is much appreciated! >>>>> Best wishes, >>>>> Hayley >>>>> >>>>> ---- >>>>> >>>>> Hayley Macpherson | NASA Einstein Fellow >>>>> >>>>> Email: hjmacpherson at uchicago.edu >>>>> Pronouns: she/her/hers >>>>> >>>>> Office: ERC 479 >>>>> Kavli Institute for Cosmological Physics >>>>> Eckhardt Research Center >>>>> 5640 South Ellis Avenue >>>>> Chicago, IL, 60637 >>>>> >>>>> _______________________________________________ >>>>> Users mailing list >>>>> Users at einsteintoolkit.org >>>>> https://urldefense.com/v3/__http://lists.einsteintoolkit.org/mailman/listinfo/users__;!!DZ3fjg!6h4G9d13wz3CvnZuXtOUn3T5cMOIpEMAdnA3o2nTka8JLWDxyoWPws-NTWy-99dGdNk6LmRxzBagF6wcfg$ > >>>>> >>>>> >>>>> >>>>> >>>>> -- >>>>> Erik Schnetter >> >>>>> https://urldefense.com/v3/__http://www.perimeterinstitute.ca/personal/eschnetter/__;!!DZ3fjg!6h4G9d13wz3CvnZuXtOUn3T5cMOIpEMAdnA3o2nTka8JLWDxyoWPws-NTWy-99dGdNk6LmRxzBbMw-NSHw$ > >>>>> >>>>> >>>> >>>> >>> >>> >>> >>> -- >>> My email is as private as my paper mail. I therefore support encrypting >>> and signing email messages. Get my PGP key from https://urldefense.com/v3/__http://keys.gnupg.net__;!!DZ3fjg!7tqgOCho0o2qv2M3wbGJgaL-UlExqMms_TgqHTqS06axk1z1N7PN7DZ-xRG7RjL4tcMN8BrEgpF0V6guM-_619mXG18$ >. >> > > > > -- > My email is as private as my paper mail. I therefore support encrypting > and signing email messages. Get my PGP key from http://keys.gnupg.net . -------------- next part -------------- An HTML attachment was scrubbed... URL: From rhaas at illinois.edu Sun Sep 3 09:46:32 2023 From: rhaas at illinois.edu (Roland Haas) Date: Sun, 3 Sep 2023 10:46:32 -0400 Subject: [Users] Problems running BBH example In-Reply-To: <20230824174049.64b6b36d@fdea4908.spdns.org> References: <20230824174049.64b6b36d@fdea4908.spdns.org> Message-ID: <20230903104559.066c9d78@fdea4908.spdns.org> Hello Krishna, Turns out I am getting a similar issue myself now when running on my (old, well tested) laptop: cactus_sim: configs/sim/build/SystemTopology/system_topology.cc:471: void {anonymous}::set_bindings(hwloc_topology_t, const mpi_host_mapping_t&): Assertion `num_pus % num_cores == 0' failed. so something may have changed in hwloc even for standard machines. I will see if this is something that can be fixed in code. Yours, Roland > Hello Krishna, > > The abort is triggered by the SystemTopology / HWLOC thorns having > encountered a processor / core / PU combination that they did not > expect. Is there anything in the stdout file? > > I am not quite familiar enough with hwloc in particular to really make > a good suggestion. > > You can certainly try and remove > > SystemTopology > > and > > hwloc > > from your parameter file's ActiveThorn line(s). This will avoid the > issue but may, depending on your cluster (mis-)configuration, yield > non-ideal speeds (basically you must ensure manually that there is at > most one thread per CPU core). Carpet has an option > > Carpet::set_cpu_affinity = "yes" > > that probably gives you almost all you would get from SystemTopology so > you can try that one instead. > > Can you provide details on the cluster / CPU of the cluster? > > Yours, > Roland > > > Hello, > > > > I've downloaded and configured the Einstein Toolkit on my desktop, > > but I'm having an issue when I try to run the BBH example provided > > here > > > > and the BNS simulation from here > > > > > >. The job finishes in an instant, producing the attached error > > > >file. (I'm using the > > command "simfactory/bin/sim create-submit GW150914_28 --define N 28 > > --parfile par/GW150914/GW150914.rpar --procs 24 --walltime 24:00:00") > > > > There seems to be an issue with the 'num_pus' to 'num_cores' ratio in > > the "system_topology.cc" file, but the error oddly persists even when > > I manually equate them, or cut out line 471 altogether. This error > > does not appear when I run the tov_ET example or the HelloWorld one. > > I'm also able to run this mini BNS tutorial > > > > without any issues, so I'm a bit baffled. I've attached my .ini > > > file and the .cc file along with the error file in case it helps. > > Please take a look and let me know if I'm missing something obvious. > > > > PS: My PC has 128GB of memory, a 13900K(24C/32T) processor and 1TB of > > storage. > > > > Regards, > > Praveer > > > -- My email is as private as my paper mail. I therefore support encrypting and signing email messages. Get my PGP key from http://keys.gnupg.net. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From rhaas at illinois.edu Sun Sep 3 10:25:47 2023 From: rhaas at illinois.edu (Roland Haas) Date: Sun, 3 Sep 2023 11:25:47 -0400 Subject: [Users] Problems running BBH example In-Reply-To: <20230903104559.066c9d78@fdea4908.spdns.org> References: <20230824174049.64b6b36d@fdea4908.spdns.org> <20230903104559.066c9d78@fdea4908.spdns.org> Message-ID: <20230903112547.1193d3a5@fdea4908.spdns.org> Hello Krishna, all, Correction. This is happening on my brand new workstation with 12th Gen Intel(R) Core(TM) i7-12700 https://en.wikipedia.org/wiki/List_of_Intel_Core_i7_processors#Golden_Cove_+_Gracemont_microarchitecture_(12th_generation) with performance (16) and efficiency (4) logical cpus. With this hwloc reports: core_depth 5 num_cores 12 pu_depth 6 num_pus 20 and the assert num_pus % num_cores fails (b/c only the 8 performance cores have hyperthreads). So an newly developed issue in systemtopology's assumptions due to changing hardware, really. I have created a ticket for this (https://bitbucket.org/einsteintoolkit/tickets/issues/2756) since this needs to be fixed since one would expect the new Intel CPUs with mixed cores to become more common at least on laptops and workstations. Yours, Roland > Hello Krishna, > > Turns out I am getting a similar issue myself now when running on my > (old, well tested) laptop: > > cactus_sim: configs/sim/build/SystemTopology/system_topology.cc:471: > void {anonymous}::set_bindings(hwloc_topology_t, const > mpi_host_mapping_t&): Assertion `num_pus % num_cores == 0' failed. > > so something may have changed in hwloc even for standard machines. I > will see if this is something that can be fixed in code. > > Yours, > Roland > > > Hello Krishna, > > > > The abort is triggered by the SystemTopology / HWLOC thorns having > > encountered a processor / core / PU combination that they did not > > expect. Is there anything in the stdout file? > > > > I am not quite familiar enough with hwloc in particular to really > > make a good suggestion. > > > > You can certainly try and remove > > > > SystemTopology > > > > and > > > > hwloc > > > > from your parameter file's ActiveThorn line(s). This will avoid the > > issue but may, depending on your cluster (mis-)configuration, yield > > non-ideal speeds (basically you must ensure manually that there is > > at most one thread per CPU core). Carpet has an option > > > > Carpet::set_cpu_affinity = "yes" > > > > that probably gives you almost all you would get from > > SystemTopology so you can try that one instead. > > > > Can you provide details on the cluster / CPU of the cluster? > > > > Yours, > > Roland > > > > > Hello, > > > > > > I've downloaded and configured the Einstein Toolkit on my desktop, > > > but I'm having an issue when I try to run the BBH example provided > > > here > > > > > > > > > and the BNS simulation from here > > > > > > > > > > > >. The job finishes in an instant, producing the attached error > > > > >file. (I'm using the > > > command "simfactory/bin/sim create-submit GW150914_28 --define N > > > 28 --parfile par/GW150914/GW150914.rpar --procs 24 --walltime > > > 24:00:00") > > > > > > There seems to be an issue with the 'num_pus' to 'num_cores' > > > ratio in the "system_topology.cc" file, but the error oddly > > > persists even when I manually equate them, or cut out line 471 > > > altogether. This error does not appear when I run the tov_ET > > > example or the HelloWorld one. I'm also able to run this mini BNS > > > tutorial > > > > > > > > > without any issues, so I'm a bit baffled. I've attached my .ini > > > > file and the .cc file along with the error file in case it > > > > helps. > > > Please take a look and let me know if I'm missing something > > > obvious. > > > > > > PS: My PC has 128GB of memory, a 13900K(24C/32T) processor and > > > 1TB of storage. > > > > > > Regards, > > > Praveer > > > > > > > > > -- My email is as private as my paper mail. I therefore support encrypting and signing email messages. Get my PGP key from http://keys.gnupg.net. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From rhaas at illinois.edu Mon Sep 4 15:18:02 2023 From: rhaas at illinois.edu (rhaas at illinois.edu) Date: Mon, 04 Sep 2023 15:18:02 -0500 Subject: [Users] Agenda for Thursday's Meeting Message-ID: Please update the Wiki with agenda items for Thursday's meeting. Thanks! https://docs.einsteintoolkit.org/et-docs/meeting_agenda --The Maintainers From rhaas at illinois.edu Wed Sep 6 17:15:01 2023 From: rhaas at illinois.edu (rhaas at illinois.edu) Date: Wed, 06 Sep 2023 17:15:01 -0500 Subject: [Users] Einstein Toolkit Meeting Reminder Message-ID: Hello, Please consider joining the weekly Einstein Toolkit phone call at 9:00 am US central time on Thursdays. For details on how to connect and what agenda items are to be discussed, use the link below. https://docs.einsteintoolkit.org/et-docs/Main_Page#Weekly_Users_Call --The Maintainers From rhaas at illinois.edu Thu Sep 7 09:59:26 2023 From: rhaas at illinois.edu (Roland Haas) Date: Thu, 7 Sep 2023 10:59:26 -0400 Subject: [Users] meeting minutes for 2023-09-07 Message-ID: <20230907105926.6c274b91@fdea4908.spdns.org> Present: Roland, Steve, Peter, Yosef, Sam, Johnny, Leo, Zach ET release: * deadline to include thorns in thornlist was yesterday, none have been added ** Sam asks for a reviewer to look at GRHayl thornlist, Steve will look at pull request ** CarpetX still has issues, Roland will fix LoopControl problem ** no updates from SGRID Tickets: * merging SeedMagneticField in https://bitbucket.org/einsteintoolkit/tickets/issues/2746/make-seed_magnetic_fields-independent-of Sam has made some changes to push things forward * https://bitbucket.org/einsteintoolkit/tickets/issues/2745 Zach and Sam on fence on whether to push for GRHayL in this release or the next to replace IllinoisGRMHD * https://bitbucket.org/einsteintoolkit/tickets/issues/2756 Roland described issue being caused by E and P cores on Intel cpus and unexpected number of logical CPUs reported by the OS. Steve asked if there is an easy way to to identify them. Roland is not aware. Yosef asks if it is possible to limit to only one sort of cores. The ET currently tries to avoid HT but the issue here is really the uneven number of logical cpus compared to cores. Currenlty lscpu --extended can be used to identify the cores based on their maximum frequency, but it is not nice. * https://bitbucket.org/einsteintoolkit/tickets/issues/2752/parameter-file-parser-generates-illegal-c causes issues since we copy the string context verbatim to C, then LaTeX then HTML code. Had a discussion about allowing everything and escaping, or listing forbidden characters, or listing allowed characters. Discussion favored escaping for C, LaTeX and HTML output. * https://bitbucket.org/einsteintoolkit/tickets/issues/2749/wvuthorns-sprintf-use-violates-standard Zach will fix this * https://bitbucket.org/einsteintoolkit/tickets/issues/2751/ehfinder-array-reference-out-of-bounds Peter will fix * https://bitbucket.org/einsteintoolkit/tickets/issues/2750/httpd-array-access-out-of-bounds Roland will fix * https://bitbucket.org/einsteintoolkit/tickets/issues/2739/change-admmacros-spatial_order-to-grhydro Steve will review * https://bitbucket.org/einsteintoolkit/tickets/issues/2754/inconsistent-definition-of-b-in-evolution Sam will add a parameter Email: * https://lists.einsteintoolkit.org/pipermail/users/2023-September/009059.html "Loss of precision after postintial?". Yosef suggested that init_3_tl could also cause changes in POSTPOSTINITIAL. Peter is not sure if anything more can be said without having seen the parameter file. Peter will try and contact Hayley. Chair next week: Steve Minutes next week: Sam Yours, Roland -- My email is as private as my paper mail. I therefore support encrypting and signing email messages. Get my PGP key from http://keys.gnupg.net. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From rhaas at illinois.edu Mon Sep 11 15:18:01 2023 From: rhaas at illinois.edu (rhaas at illinois.edu) Date: Mon, 11 Sep 2023 15:18:01 -0500 Subject: [Users] Agenda for Thursday's Meeting Message-ID: Please update the Wiki with agenda items for Thursday's meeting. Thanks! https://docs.einsteintoolkit.org/et-docs/meeting_agenda --The Maintainers From rhaas at illinois.edu Wed Sep 13 17:15:01 2023 From: rhaas at illinois.edu (rhaas at illinois.edu) Date: Wed, 13 Sep 2023 17:15:01 -0500 Subject: [Users] Einstein Toolkit Meeting Reminder Message-ID: Hello, Please consider joining the weekly Einstein Toolkit phone call at 9:00 am US central time on Thursdays. For details on how to connect and what agenda items are to be discussed, use the link below. https://docs.einsteintoolkit.org/et-docs/Main_Page#Weekly_Users_Call --The Maintainers From scupp1 at my.apsu.edu Thu Sep 14 09:58:34 2023 From: scupp1 at my.apsu.edu (Cupp, Samuel D.) Date: Thu, 14 Sep 2023 14:58:34 +0000 Subject: [Users] Meeting Minutes for 2023-9-14 Message-ID: Present: Steve, Sam, Roland, Leo, Jose, Peter, Yosef ET_2023_11: CarpetX review Still has a severe lack of documentation, and it is much harder to build successfully than Carpet Roland: compilation failed on Expanse because the NSIMD external library thorn tries to download sleef (Issue #2757). This should be resolvable by including sleef in the thorn. Steve: I think another thing that should be done before inclusion is cleaning up the dead code, which it has a lot of. There is some room to push back release into December, and we should consider extending the release window to get CarpetX into the release. CarpetX thorns should be added to master thornlist to see what breaks on the different clusters. ------------------------------ LORENE fails to build with spack, and spack doesn't reliably work across multiple clusters with the same setup ------------------------------ GRHayL needs more documentation on the wiki; add some more discussion to the thorn doc GRHayLHD test failed on intel; Peter is looking into it test also failed in automated testing due to misconfigured test.ccl ------------------------------ sgrid Roland: Wolfgang and Michal are still working on getting the code working as a standalone thorn that can be directly checked out; not sure how far along they are or what dependencies their code has ------------------------------ Unanswered emails: Peter will follow up with unanswered email from Hayley. Next meeting (9/21): Chair - Peter Minute taker - Steve Samuel Cupp Postdoctoral Researcher Department of Physics University of Idaho -------------- next part -------------- An HTML attachment was scrubbed... URL: From rhaas at illinois.edu Mon Sep 18 15:18:02 2023 From: rhaas at illinois.edu (rhaas at illinois.edu) Date: Mon, 18 Sep 2023 15:18:02 -0500 Subject: [Users] Agenda for Thursday's Meeting Message-ID: Please update the Wiki with agenda items for Thursday's meeting. Thanks! https://docs.einsteintoolkit.org/et-docs/meeting_agenda --The Maintainers From rhaas at illinois.edu Wed Sep 20 17:15:01 2023 From: rhaas at illinois.edu (rhaas at illinois.edu) Date: Wed, 20 Sep 2023 17:15:01 -0500 Subject: [Users] Einstein Toolkit Meeting Reminder Message-ID: Hello, Please consider joining the weekly Einstein Toolkit phone call at 9:00 am US central time on Thursdays. For details on how to connect and what agenda items are to be discussed, use the link below. https://docs.einsteintoolkit.org/et-docs/Main_Page#Weekly_Users_Call --The Maintainers From sbrandt at cct.lsu.edu Thu Sep 21 09:52:32 2023 From: sbrandt at cct.lsu.edu (Steven Brandt) Date: Thu, 21 Sep 2023 09:52:32 -0500 Subject: [Users] Meeting minutes for 2023-09-21 Message-ID: Present: Peter, Steve, Roland, Erik, Leo, Sam, Zach Minutes: Steve Chair: Peter The release date is pushed back to Dec 6. Update on the review of Grail (Peter looked at the test suite). There is an issue with initial data. Shock is exactly at a grid point which gives a rounding issue with intel vs. gnu. Lucas will be doing the documentation for CarpetX in October. The tests are not running in the test suite yet. They do get run once in a while (pull request). Sameer Sende (friend of Steve's) offers a hardware lab with lots of different versions. We could use this for testing. Intel 19.1.3.304 Sep 2020 on Deep Bayou fails to compile AMReX. sgrid... is it going in? Steve asks if we should use the MakeThornList script on the download page? Roland thinks we should add it to the new user's tutorial. Update on Seed Magnetic Fields? Zach asks to be reminded in a week. Test suite server has unreliable results? Still the case. Something is glitchy in detecting which tests fail. Balsara is still failing. Email is fixed. Should we email maintainers of thorns that are failing? Unanswered questions: Peter forgot to reply to Haleigh McPherson. Zach has been working on constraint damping and improving BSSN. Peter wants to use it to close out the ticket he is working on for McLachlan. He also wants to use it for Sphynx BSSN. Leo fixed several WVU thorns following Roland's suggestion. Ticket should be closed. NSIMD is not required by CarpetX, but is nice to have. NSIMD's install downloads something using requests. 2754 can probably be closed. Do we still need HTTPD? Does it work at all? Can have tiers of support, tier 1 is actively used, tier 3 moves to unsupported. How do people show active support, etc. McLachlin has incorrect reads declaration. Ian doesn't have time to look at it. Roland has a patch for Kranc. Zach says NRPy 2.0 is pip installable that works with Python 3.6 and up. Next Chair: Leo Next Minute Taker: Roland From rhaas at illinois.edu Mon Sep 25 15:18:02 2023 From: rhaas at illinois.edu (rhaas at illinois.edu) Date: Mon, 25 Sep 2023 15:18:02 -0500 Subject: [Users] Agenda for Thursday's Meeting Message-ID: Please update the Wiki with agenda items for Thursday's meeting. Thanks! https://docs.einsteintoolkit.org/et-docs/meeting_agenda --The Maintainers From bruno.giacomazzo at unimib.it Tue Sep 26 06:49:13 2023 From: bruno.giacomazzo at unimib.it (Bruno Giacomazzo) Date: Tue, 26 Sep 2023 13:49:13 +0200 Subject: [Users] 2-year postdoc position in numerical relativity in Milano-Bicocca In-Reply-To: References: Message-ID: [sorry if you already received this] The astrophysics group at the University of Milano-Bicocca has an opening for a 2-year postdoc position in the field of numerical relativity sponsored by the National Institute for Nuclear Physics (INFN). The postdoc will work in the Physics department of the University of Milano-Bicocca under the supervision of Prof. Bruno Giacomazzo (www.brunogiacomazzo.org). The research topic is ?Numerical Relativity simulations of neutron star binary mergers?. The deadline for applications is November 10 2023 (11:59 a.m. CET) and the starting date for the position should be no later than November 1st 2024. Candidates planning to finish their PhD before November 1st 2024 are also welcome to apply. The annual gross salary is 31308 EUR (the net yearly salary is ~28000 EUR). Candidates from outside Milan may also be eligible to get an additional 5000 EUR per year. Funds for attending conferences and HPC resources are also available via INFN. Applications are only accepted online via the following link: https://reclutamento.dsi.infn.it/ (search for job number 25864) Eligible candidates may be: - Non-Italian citizens, or - Italian citizens who, at the submission deadline, hold a position in a foreign institution and have been continuously abroad for at least three years. Please note also that applicants with a PhD should have obtained it no more than 8 years prior to the call deadline, i.e. on or after November 10th, 2015 (this time limit may be extended in case of maternity, paternity, national service, or long-term illness). The application must include: - a curriculum vitae; - a publication list; - the names and e-mail addresses of three referees, each of them may upload a reference letter not later than November 13th, 2023 (11:59 a.m. CET). In the application the candidate will be requested to indicate the INFN site (Milano Bicocca) and the research topic (Numerical Relativity simulations of neutron star binary mergers) since the call includes other 14 topics in different INFN sites (candidates may also indicate a second option if they wish to). Full instructions in English can be found on the above page (pdf file ?Pdf Bando?). The astrophysics unit at Milano-Bicocca provides a vibrant environment with expertise covering all aspects of gravitational-wave astronomy, relativistic astrophysics, and numerical relativity, as well as a wider astronomical context including observational and experimental activities. The gravitational wave group is composed of professors Monica Colpi, Massimo Dotti, Davide Gerosa, Bruno Giacomazzo, and Alberto Sesana. The group has tight connections with the LISA Consortium, the Virgo Collaboration, the Einstein Telescope Observational Science Board, the European Pulsar Timing Array, the Italian National Institute for Nuclear Physics (INFN), and the Italian Center for Supercomputing (ICSC). Milan is a beautiful, international city in the north of Italy with history, art, and outstanding food. Mountains and lakes are just around the corner. -- Prof. Bruno Giacomazzo Department of Physics University of Milano-Bicocca Piazza della Scienza 3 20126 Milano Italy email: bruno.giacomazzo at unimib.it phone: (+39) 02 6448 2321 web: http://www.brunogiacomazzo.org --------------------------------------------------------------------- There are only 10 types of people in the world: Those who understand binary, and those who don't ---------------------------------------------------------------------- -------------- next part -------------- An HTML attachment was scrubbed... URL: From emost at caltech.edu Tue Sep 26 12:39:01 2023 From: emost at caltech.edu (Most, Elias R.) Date: Tue, 26 Sep 2023 17:39:01 +0000 Subject: [Users] =?windows-1252?q?Postdoctoral_position_at_Caltech_=96_Co?= =?windows-1252?q?mputational_relativistic_astrophysics_and_numerical_rela?= =?windows-1252?q?tivity?= Message-ID: Dear all, We invite applications for a postdoctoral scholar to work in computational relativistic astrophysics and numerical relativity at Caltech for a position starting in September 2024 (or sooner). The postdoctoral scholar will be working with Elias R. Most on nuclear and plasma physics aspects of multi-messenger gravitational wave events, including associated high-energy transients. The postdoctoral scholar will also have the opportunity to participate in Caltech?s numerical relativity efforts, which are part of the Simulating Extreme Spacetimes Collaboration. This position is based at the Theoretical Astrophysics Including Relativity (TAPIR) group. There will be frequent opportunity to interact and collaborate with other TAPIR faculty members (Katerina Chatziioannou, Yanbei Chen, Jim Fuller, Phil Hopkins, Sterl Phinney, Mark Scheel and Saul Teukolsky), as well as members of LIGO Lab and Caltech?s Astronomy department. The initial appointment will be for two years with an expected renewal for a third year, subject to performance and availability of funds. Applicants should submit (1) a CV, including a publication list, (2) contact information of at least three professional references, (3) summary of previous, current, and anticipated research relevant to this opening (limited to three pages). Application materials should be submitted through AcademicJobsOnline. Applicants must hold a PhD degree in astrophysics, physics or a related field by the starting date (September 2024 or sooner). Complete applications submitted by December 1, 2023 will receive full consideration, but review of applications will continue until a suitable candidate has been identified. The minimum salary for the position is $68k per year. The salary of the finalist(s) selected for this role will be set based on a variety of factors, including but not limited to, experience, qualifications, skills, education, specialty, training, and internal equity. Applications may be submitted here: https://academicjobsonline.org/ajo/jobs/25498 As one of the largest employers in Pasadena, CA, Caltech is committed to providing comprehensive benefits to eligible employees and their eligible dependents. Our benefits package includes competitive compensation, health, dental, and vision insurance, retirement savings plans, generous paid time off, parental leave, tuition reimbursement, and more. Learn more about our benefits and who is eligible. We are an equal opportunity employer and all qualified applicants will receive consideration for employment without regard to age, race, color, religion, sex, sexual orientation, gender identity, or national origin, disability status, protected veteran status, or any other characteristic protected by law. Caltech is a VEVRAA Federal Contractor. If you would like to request an accommodation in completing this application, interviewing, or otherwise participating in the employee selection process, please direct your inquiries to the contact on the job posting. -- Elias R. Most Assistant Professor Theoretical Astrophysics (TAPIR) California Institute of Technology (Caltech) Pasadena, CA 91125, USA -------------- next part -------------- An HTML attachment was scrubbed... URL: From nils.vu at black-holes.org Tue Sep 5 16:53:34 2023 From: nils.vu at black-holes.org (Nils Vu) Date: Tue, 05 Sep 2023 21:53:34 -0000 Subject: [Users] Next NR community call: Monday, Sep 11 Message-ID: <5CA51C38-DE16-4C36-8838-CCFB509E0AEE@black-holes.org> Dear Numerical Relativists, A new season of NR community calls begins next week! Our first community call of the season will be on Monday, September 11 (9am Pacific, noon NYC, 6pm Berlin). Our speakers will be Deborah Ferguson (UT Austin / UIUC) and Jaime Redondo Yuste (NBI). You can find the schedule in the wiki: https://github.com/sxs-collaboration/nr-community-call/wiki As always: - Feel free to circulate this information within the NR community. - Please subscribe to our mailing list to receive info on the community calls in the future (or send a short message to nr-community-call+subscribe at black-holes.org to subscribe) and join our Slack for discussions (channel: #nr-community-call ). - Please sign up for a slot in the schedule to speak at one of the community calls: https://github.com/sxs-collaboration/nr-community-call/wiki Best, Nils ? Nils Vu, Ph.D. (he/him) Sherman Fairchild Postdoctoral Fellow in Theoretical Astrophysics California Institute of Technology -------------- next part -------------- An HTML attachment was scrubbed... URL: From sabdolrahimi at cpp.edu Sun Sep 10 13:55:32 2023 From: sabdolrahimi at cpp.edu (Shohreh Abdolrahimi) Date: Sun, 10 Sep 2023 18:55:32 -0000 Subject: [Users] Einstein toolkit access Message-ID: Dear Einstein Toolkit, I am an associate professor at Cal State Pomona. I am trying to get access to the tutorials of the Einstein Toolkit. I do have an ORCID account. However, I received an error when submitting the form that I should enter a correct email address. However, there is no place to enter the email address. I've attached the file for you to look over. -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: Screenshot 2023-09-09 at 9.18.03 AM.png Type: image/png Size: 225242 bytes Desc: Screenshot 2023-09-09 at 9.18.03 AM.png URL: From nils.vu at black-holes.org Mon Sep 11 09:51:47 2023 From: nils.vu at black-holes.org (Nils Vu) Date: Mon, 11 Sep 2023 14:51:47 -0000 Subject: [Users] Today: NR community call at 9am Pacific / noon NYC / 5pm London / 6pm Berlin Message-ID: <42ED8340-ED86-442E-9231-591AF9B2B619@black-holes.org> ??Dear Numerical Relativists, We?ll have our NR community call today, Monday Sep 11, 9am LA / noon NYC / 5pm London / 6pm Berlin in this Zoom room: https://caltech.zoom.us/j/87114806411 Our speakers today will be Deborah Ferguson (UT Austin / UIUC) and Jaime Redondo Yuste (NBI). - Feel free to circulate this information within the NR community. - Please subscribe to our mailing list to receive info on the community calls in the future (or send a short message to nr-community-call+subscribe at black-holes.org to subscribe) and join our Slack for discussions (channel: #nr-community-call ). - Please sign up for a slot in the schedule to speak at one of the community calls: https://github.com/sxs-collaboration/nr-community-call/wiki Looking forward to seeing you all later, Nils ? Nils Vu, Ph.D. (he/him) Sherman Fairchild Postdoctoral Fellow in Theoretical Astrophysics California Institute of Technology -------------- next part -------------- An HTML attachment was scrubbed... URL: From rhaas at illinois.edu Wed Sep 27 17:15:01 2023 From: rhaas at illinois.edu (rhaas at illinois.edu) Date: Wed, 27 Sep 2023 17:15:01 -0500 Subject: [Users] Einstein Toolkit Meeting Reminder Message-ID: Hello, Please consider joining the weekly Einstein Toolkit phone call at 9:00 am US central time on Thursdays. For details on how to connect and what agenda items are to be discussed, use the link below. https://docs.einsteintoolkit.org/et-docs/Main_Page#Weekly_Users_Call --The Maintainers From rhaas at illinois.edu Thu Sep 28 13:39:51 2023 From: rhaas at illinois.edu (Roland Haas) Date: Thu, 28 Sep 2023 14:39:51 -0400 Subject: [Users] meeting minutes for 2023-09-28 Message-ID: <20230928143951.18a092d1@fdea4908.spdns.org> Present: Roland, Steve, Peter, Leo, Sam, Zach New thorns in release: * deadline for review is Oct 11th * Sam and Roland added mising GRHayL thorns to autoamted tests SGRID importer: * Sam eventually heard back from authors * tried out provided thorn, fails since it is missing SGRID itself * Roland mentioned new ExternalLibraries-SGRID thorn GRHayL MHD thorns: * Zach is looking will need some spare seconds CarpetX review: * Steve looked at some of the code, not much progress yet ET release name: * possible options are listed on https://docs.einsteintoolkit.org/et-docs/Release_Details#Release_names * suggested name: Lise Meitner Unanswered questions: * no new emails Open tickets: * no new non-release related tickets * icpc 19 cannot compile AMReX https://bitbucket.org/einsteintoolkit/tickets/issues/2758 Roland reports that this is still happening and a workaround may be possible but large. Liwei has tried a modern OneAPI based Intel compiler (clang based) on Fronterra and seems to have had success * Peter and Zach will interface about https://bitbucket.org/einsteintoolkit/tickets/issues/963 improved ML accuracy * no progress on https://bitbucket.org/einsteintoolkit/tickets/issues/2647/incorrect-weno-coefficient-in-grhydro-weno chair next week: Peter minutes next week: Leo Yours, Roland -- My email is as private as my paper mail. I therefore support encrypting and signing email messages. Get my PGP key from http://keys.gnupg.net. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: