September 2023 - Users - lists.einsteintoolkit.org

meeting minutes for 2023-09-28
by Roland Haas 28 Sep '23

28 Sep '23

Present: Roland, Steve, Peter, Leo, Sam, Zach New thorns in release: * deadline for review is Oct 11th * Sam and Roland added mising GRHayL thorns to autoamted tests SGRID importer: * Sam eventually heard back from authors * tried out provided thorn, fails since it is missing SGRID itself * Roland mentioned new ExternalLibraries-SGRID thorn GRHayL MHD thorns: * Zach is looking will need some spare seconds CarpetX review: * Steve looked at some of the code, not much progress yet ET release name: * possible options are listed on https://docs.einsteintoolkit.org/et-docs/Release_Details#Release_names * suggested name: Lise Meitner Unanswered questions: * no new emails Open tickets: * no new non-release related tickets * icpc 19 cannot compile AMReX https://bitbucket.org/einsteintoolkit/tickets/issues/2758 Roland reports that this is still happening and a workaround may be possible but large. Liwei has tried a modern OneAPI based Intel compiler (clang based) on Fronterra and seems to have had success * Peter and Zach will interface about https://bitbucket.org/einsteintoolkit/tickets/issues/963 improved ML accuracy * no progress on https://bitbucket.org/einsteintoolkit/tickets/issues/2647/incorrect-weno-co… chair next week: Peter minutes next week: Leo Yours, Roland -- My email is as private as my paper mail. I therefore support encrypting and signing email messages. Get my PGP key from http://keys.gnupg.net.

1 0

Einstein Toolkit Meeting Reminder
by rhaas＠illinois.edu 27 Sep '23

27 Sep '23

Hello, Please consider joining the weekly Einstein Toolkit phone call at 9:00 am US central time on Thursdays. For details on how to connect and what agenda items are to be discussed, use the link below. https://docs.einsteintoolkit.org/et-docs/Main_Page#Weekly_Users_Call --The Maintainers

1 0

Postdoctoral position at Caltech – Computational relativistic astrophysics and numerical relativity
by Most, Elias R. 26 Sep '23

26 Sep '23

Dear all, We invite applications for a postdoctoral scholar to work in computational relativistic astrophysics and numerical relativity at Caltech for a position starting in September 2024 (or sooner). The postdoctoral scholar will be working with Elias R. Most on nuclear and plasma physics aspects of multi-messenger gravitational wave events, including associated high-energy transients. The postdoctoral scholar will also have the opportunity to participate in Caltech’s numerical relativity efforts, which are part of the Simulating Extreme Spacetimes Collaboration. This position is based at the Theoretical Astrophysics Including Relativity (TAPIR) group. There will be frequent opportunity to interact and collaborate with other TAPIR faculty members (Katerina Chatziioannou, Yanbei Chen, Jim Fuller, Phil Hopkins, Sterl Phinney, Mark Scheel and Saul Teukolsky), as well as members of LIGO Lab and Caltech’s Astronomy department. The initial appointment will be for two years with an expected renewal for a third year, subject to performance and availability of funds. Applicants should submit (1) a CV, including a publication list, (2) contact information of at least three professional references, (3) summary of previous, current, and anticipated research relevant to this opening (limited to three pages). Application materials should be submitted through AcademicJobsOnline. Applicants must hold a PhD degree in astrophysics, physics or a related field by the starting date (September 2024 or sooner). Complete applications submitted by December 1, 2023 will receive full consideration, but review of applications will continue until a suitable candidate has been identified. The minimum salary for the position is $68k per year. The salary of the finalist(s) selected for this role will be set based on a variety of factors, including but not limited to, experience, qualifications, skills, education, specialty, training, and internal equity. Applications may be submitted here: https://academicjobsonline.org/ajo/jobs/25498 As one of the largest employers in Pasadena, CA, Caltech is committed to providing comprehensive benefits to eligible employees and their eligible dependents. Our benefits package includes competitive compensation, health, dental, and vision insurance, retirement savings plans, generous paid time off, parental leave, tuition reimbursement, and more. Learn more about our benefits and who is eligible. We are an equal opportunity employer and all qualified applicants will receive consideration for employment without regard to age, race, color, religion, sex, sexual orientation, gender identity, or national origin, disability status, protected veteran status, or any other characteristic protected by law. Caltech is a VEVRAA Federal Contractor. If you would like to request an accommodation in completing this application, interviewing, or otherwise participating in the employee selection process, please direct your inquiries to the contact on the job posting. -- Elias R. Most Assistant Professor Theoretical Astrophysics (TAPIR) California Institute of Technology (Caltech) Pasadena, CA 91125, USA

1 0

2-year postdoc position in numerical relativity in Milano-Bicocca
by Bruno Giacomazzo 26 Sep '23

26 Sep '23

[sorry if you already received this] The astrophysics group <https://astro.fisica.unimib.it/> at the University of Milano-Bicocca <https://en.unimib.it/> has an opening for a 2-year postdoc position in the field of numerical relativity sponsored by the National Institute for Nuclear Physics (INFN). The postdoc will work in the Physics department of the University of Milano-Bicocca under the supervision of Prof. Bruno Giacomazzo (www.brunogiacomazzo.org). The research topic is “Numerical Relativity simulations of neutron star binary mergers”. The deadline for applications is November 10 2023 (11:59 a.m. CET) and the starting date for the position should be no later than November 1st 2024. Candidates planning to finish their PhD before November 1st 2024 are also welcome to apply. The annual gross salary is 31308 EUR (the net yearly salary is ~28000 EUR). Candidates from outside Milan may also be eligible to get an additional 5000 EUR per year. Funds for attending conferences and HPC resources are also available via INFN. Applications are only accepted online via the following link: https://reclutamento.dsi.infn.it/ (search for job number 25864) Eligible candidates may be: - Non-Italian citizens, or - Italian citizens who, at the submission deadline, hold a position in a foreign institution and have been continuously abroad for at least three years. Please note also that applicants with a PhD should have obtained it no more than 8 years prior to the call deadline, i.e. on or after November 10th, 2015 (this time limit may be extended in case of maternity, paternity, national service, or long-term illness). The application must include: - a curriculum vitae; - a publication list; - the names and e-mail addresses of three referees, each of them may upload a reference letter not later than November 13th, 2023 (11:59 a.m. CET). In the application the candidate will be requested to indicate the INFN site (Milano Bicocca) and the research topic (Numerical Relativity simulations of neutron star binary mergers) since the call includes other 14 topics in different INFN sites (candidates may also indicate a second option if they wish to). Full instructions in English can be found on the above page (pdf file “Pdf Bando”). The astrophysics unit at Milano-Bicocca provides a vibrant environment with expertise covering all aspects of gravitational-wave astronomy, relativistic astrophysics, and numerical relativity, as well as a wider astronomical context including observational and experimental activities. The gravitational wave group is composed of professors Monica Colpi, Massimo Dotti, Davide Gerosa, Bruno Giacomazzo, and Alberto Sesana. The group has tight connections with the LISA Consortium, the Virgo Collaboration, the Einstein Telescope Observational Science Board, the European Pulsar Timing Array, the Italian National Institute for Nuclear Physics (INFN), and the Italian Center for Supercomputing <https://www.supercomputing-icsc.it/en/icsc-home/> (ICSC). Milan <https://www.yesmilano.it/en> is a beautiful, international city in the north of Italy with history, art, and outstanding food. Mountains and lakes are just around the corner. -- Prof. Bruno Giacomazzo Department of Physics University of Milano-Bicocca Piazza della Scienza 3 20126 Milano Italy email: bruno.giacomazzo(a)unimib.it phone: (+39) 02 6448 2321 web: http://www.brunogiacomazzo.org --------------------------------------------------------------------- There are only 10 types of people in the world: Those who understand binary, and those who don't ----------------------------------------------------------------------

1 0

Agenda for Thursday's Meeting
by rhaas＠illinois.edu 25 Sep '23

25 Sep '23

Please update the Wiki with agenda items for Thursday's meeting. Thanks! https://docs.einsteintoolkit.org/et-docs/meeting_agenda --The Maintainers

1 0

Meeting minutes for 2023-09-21
by Steven Brandt 21 Sep '23

21 Sep '23

Present: Peter, Steve, Roland, Erik, Leo, Sam, Zach Minutes: Steve Chair: Peter The release date is pushed back to Dec 6. Update on the review of Grail (Peter looked at the test suite). There is an issue with initial data. Shock is exactly at a grid point which gives a rounding issue with intel vs. gnu. Lucas will be doing the documentation for CarpetX in October. The tests are not running in the test suite yet. They do get run once in a while (pull request). Sameer Sende (friend of Steve's) offers a hardware lab with lots of different versions. We could use this for testing. Intel 19.1.3.304 Sep 2020 on Deep Bayou fails to compile AMReX. sgrid... is it going in? Steve asks if we should use the MakeThornList script on the download page? Roland thinks we should add it to the new user's tutorial. Update on Seed Magnetic Fields? Zach asks to be reminded in a week. Test suite server has unreliable results? Still the case. Something is glitchy in detecting which tests fail. Balsara is still failing. Email is fixed. Should we email maintainers of thorns that are failing? Unanswered questions: Peter forgot to reply to Haleigh McPherson. Zach has been working on constraint damping and improving BSSN. Peter wants to use it to close out the ticket he is working on for McLachlan. He also wants to use it for Sphynx BSSN. Leo fixed several WVU thorns following Roland's suggestion. Ticket should be closed. NSIMD is not required by CarpetX, but is nice to have. NSIMD's install downloads something using requests. 2754 can probably be closed. Do we still need HTTPD? Does it work at all? Can have tiers of support, tier 1 is actively used, tier 3 moves to unsupported. How do people show active support, etc. McLachlin has incorrect reads declaration. Ian doesn't have time to look at it. Roland has a patch for Kranc. Zach says NRPy 2.0 is pip installable that works with Python 3.6 and up. Next Chair: Leo Next Minute Taker: Roland

1 0

Einstein Toolkit Meeting Reminder
by rhaas＠illinois.edu 20 Sep '23

20 Sep '23

Hello, Please consider joining the weekly Einstein Toolkit phone call at 9:00 am US central time on Thursdays. For details on how to connect and what agenda items are to be discussed, use the link below. https://docs.einsteintoolkit.org/et-docs/Main_Page#Weekly_Users_Call --The Maintainers

1 0

Agenda for Thursday's Meeting
by rhaas＠illinois.edu 18 Sep '23

18 Sep '23

Please update the Wiki with agenda items for Thursday's meeting. Thanks! https://docs.einsteintoolkit.org/et-docs/meeting_agenda --The Maintainers

1 0

Meeting Minutes for 2023-9-14
by Cupp, Samuel D. 14 Sep '23

14 Sep '23

Present: Steve, Sam, Roland, Leo, Jose, Peter, Yosef ET_2023_11: CarpetX review Still has a severe lack of documentation, and it is much harder to build successfully than Carpet Roland: compilation failed on Expanse because the NSIMD external library thorn tries to download sleef (Issue #2757<https://bitbucket.org/einsteintoolkit/tickets/issues/2757/nsimd-tries-to-do…>). This should be resolvable by including sleef in the thorn. Steve: I think another thing that should be done before inclusion is cleaning up the dead code, which it has a lot of. There is some room to push back release into December, and we should consider extending the release window to get CarpetX into the release. CarpetX thorns should be added to master thornlist to see what breaks on the different clusters. ------------------------------ LORENE fails to build with spack, and spack doesn't reliably work across multiple clusters with the same setup ------------------------------ GRHayL needs more documentation on the wiki; add some more discussion to the thorn doc GRHayLHD test failed on intel; Peter is looking into it test also failed in automated testing due to misconfigured test.ccl ------------------------------ sgrid Roland: Wolfgang and Michal are still working on getting the code working as a standalone thorn that can be directly checked out; not sure how far along they are or what dependencies their code has ------------------------------ Unanswered emails: Peter will follow up with unanswered email from Hayley. Next meeting (9/21): Chair - Peter Minute taker - Steve <https://bitbucket.org/einsteintoolkit/tickets/issues/2757/nsimd-tries-to-do…> Samuel Cupp Postdoctoral Researcher Department of Physics University of Idaho

1 0

Einstein Toolkit Meeting Reminder
by rhaas＠illinois.edu 13 Sep '23

13 Sep '23

Hello, Please consider joining the weekly Einstein Toolkit phone call at 9:00 am US central time on Thursdays. For details on how to connect and what agenda items are to be discussed, use the link below. https://docs.einsteintoolkit.org/et-docs/Main_Page#Weekly_Users_Call --The Maintainers

1 0

Agenda for Thursday's Meeting
by rhaas＠illinois.edu 11 Sep '23

11 Sep '23

Please update the Wiki with agenda items for Thursday's meeting. Thanks! https://docs.einsteintoolkit.org/et-docs/meeting_agenda --The Maintainers

1 0

Today: NR community call at 9am Pacific / noon NYC / 5pm London / 6pm Berlin
by Nils Vu 11 Sep '23

11 Sep '23

Dear Numerical Relativists, We’ll have our NR community call today, Monday Sep 11, 9am LA / noon NYC / 5pm London / 6pm Berlin in this Zoom room: https://caltech.zoom.us/j/87114806411 Our speakers today will be Deborah Ferguson (UT Austin / UIUC) and Jaime Redondo Yuste (NBI). - Feel free to circulate this information within the NR community. - Please subscribe to our mailing list <https://groups.google.com/u/1/a/black-holes.org/g/nr-community-call/> to receive info on the community calls in the future (or send a short message to nr-community-call+subscribe(a)black-holes.org <mailto:nr-community-call+subscribe@black-holes.org> to subscribe) and join our Slack <https://join.slack.com/t/generalrelativity/shared_invite/zt-dl4gqb6v-aAuw~p…> for discussions (channel: #nr-community-call <https://generalrelativity.slack.com/archives/C021LLTLK37>). - Please sign up for a slot <https://docs.google.com/forms/d/e/1FAIpQLSfoCnnCJve7J7Jzrpdlqw7nAbwWS_iuX_e…> in the schedule to speak at one of the community calls: https://github.com/sxs-collaboration/nr-community-call/wiki Looking forward to seeing you all later, Nils — Nils Vu, Ph.D. (he/him) Sherman Fairchild Postdoctoral Fellow in Theoretical Astrophysics California Institute of Technology

1 0

Einstein toolkit access
by Shohreh Abdolrahimi 10 Sep '23

10 Sep '23

Dear Einstein Toolkit, I am an associate professor at Cal State Pomona. I am trying to get access to the tutorials of the Einstein Toolkit. I do have an ORCID account. However, I received an error when submitting the form that I should enter a correct email address. However, there is no place to enter the email address. I've attached the file for you to look over.

1 0

meeting minutes for 2023-09-07
by Roland Haas 07 Sep '23

07 Sep '23

Present: Roland, Steve, Peter, Yosef, Sam, Johnny, Leo, Zach ET release: * deadline to include thorns in thornlist was yesterday, none have been added ** Sam asks for a reviewer to look at GRHayl thornlist, Steve will look at pull request ** CarpetX still has issues, Roland will fix LoopControl problem ** no updates from SGRID Tickets: * merging SeedMagneticField in https://bitbucket.org/einsteintoolkit/tickets/issues/2746/make-seed_magneti… Sam has made some changes to push things forward * https://bitbucket.org/einsteintoolkit/tickets/issues/2745 Zach and Sam on fence on whether to push for GRHayL in this release or the next to replace IllinoisGRMHD * https://bitbucket.org/einsteintoolkit/tickets/issues/2756 Roland described issue being caused by E and P cores on Intel cpus and unexpected number of logical CPUs reported by the OS. Steve asked if there is an easy way to to identify them. Roland is not aware. Yosef asks if it is possible to limit to only one sort of cores. The ET currently tries to avoid HT but the issue here is really the uneven number of logical cpus compared to cores. Currenlty lscpu --extended can be used to identify the cores based on their maximum frequency, but it is not nice. * https://bitbucket.org/einsteintoolkit/tickets/issues/2752/parameter-file-pa… causes issues since we copy the string context verbatim to C, then LaTeX then HTML code. Had a discussion about allowing everything and escaping, or listing forbidden characters, or listing allowed characters. Discussion favored escaping for C, LaTeX and HTML output. * https://bitbucket.org/einsteintoolkit/tickets/issues/2749/wvuthorns-sprintf… Zach will fix this * https://bitbucket.org/einsteintoolkit/tickets/issues/2751/ehfinder-array-re… Peter will fix * https://bitbucket.org/einsteintoolkit/tickets/issues/2750/httpd-array-acces… Roland will fix * https://bitbucket.org/einsteintoolkit/tickets/issues/2739/change-admmacros-… Steve will review * https://bitbucket.org/einsteintoolkit/tickets/issues/2754/inconsistent-defi… Sam will add a parameter Email: * https://lists.einsteintoolkit.org/pipermail/users/2023-September/009059.html "Loss of precision after postintial?". Yosef suggested that init_3_tl could also cause changes in POSTPOSTINITIAL. Peter is not sure if anything more can be said without having seen the parameter file. Peter will try and contact Hayley. Chair next week: Steve Minutes next week: Sam Yours, Roland -- My email is as private as my paper mail. I therefore support encrypting and signing email messages. Get my PGP key from http://keys.gnupg.net.

1 0

Einstein Toolkit Meeting Reminder
by rhaas＠illinois.edu 06 Sep '23

06 Sep '23

Hello, Please consider joining the weekly Einstein Toolkit phone call at 9:00 am US central time on Thursdays. For details on how to connect and what agenda items are to be discussed, use the link below. https://docs.einsteintoolkit.org/et-docs/Main_Page#Weekly_Users_Call --The Maintainers

1 0

Next NR community call: Monday, Sep 11
by Nils Vu 05 Sep '23

05 Sep '23

Dear Numerical Relativists, A new season of NR community calls begins next week! Our first community call of the season will be on Monday, September 11 (9am Pacific, noon NYC, 6pm Berlin). Our speakers will be Deborah Ferguson (UT Austin / UIUC) and Jaime Redondo Yuste (NBI). You can find the schedule in the wiki: https://github.com/sxs-collaboration/nr-community-call/wiki As always: - Feel free to circulate this information within the NR community. - Please subscribe to our mailing list <https://groups.google.com/u/1/a/black-holes.org/g/nr-community-call/> to receive info on the community calls in the future (or send a short message to nr-community-call+subscribe(a)black-holes.org <mailto:nr-community-call+subscribe@black-holes.org> to subscribe) and join our Slack <https://join.slack.com/t/generalrelativity/shared_invite/zt-dl4gqb6v-aAuw~p…> for discussions (channel: #nr-community-call <https://generalrelativity.slack.com/archives/C021LLTLK37>). - Please sign up for a slot <https://docs.google.com/forms/d/e/1FAIpQLSfoCnnCJve7J7Jzrpdlqw7nAbwWS_iuX_e…> in the schedule to speak at one of the community calls: https://github.com/sxs-collaboration/nr-community-call/wiki Best, Nils — Nils Vu, Ph.D. (he/him) Sherman Fairchild Postdoctoral Fellow in Theoretical Astrophysics California Institute of Technology

1 0

Agenda for Thursday's Meeting
by rhaas＠illinois.edu 04 Sep '23

04 Sep '23

Please update the Wiki with agenda items for Thursday's meeting. Thanks! https://docs.einsteintoolkit.org/et-docs/meeting_agenda --The Maintainers

1 0

Problems running BBH example
by Praveer Krishna 03 Sep '23

03 Sep '23

Hello, I've downloaded and configured the Einstein Toolkit on my desktop, but I'm having an issue when I try to run the BBH example provided here <https://einsteintoolkit.org/gallery/bbh/index.html> and the BNS simulation from here <https://einsteintoolkit.org/gallery/bns/index.html>. The job finishes in an instant, producing the attached error file. (I'm using the command "simfactory/bin/sim create-submit GW150914_28 --define N 28 --parfile par/GW150914/GW150914.rpar --procs 24 --walltime 24:00:00") There seems to be an issue with the 'num_pus' to 'num_cores' ratio in the "system_topology.cc" file, but the error oddly persists even when I manually equate them, or cut out line 471 altogether. This error does not appear when I run the tov_ET example or the HelloWorld one. I'm also able to run this mini BNS tutorial <https://www.youtube.com/watch?v=3euS5yMBFmI> without any issues, so I'm a bit baffled. I've attached my .ini file and the .cc file along with the error file in case it helps. Please take a look and let me know if I'm missing something obvious. PS: My PC has 128GB of memory, a 13900K(24C/32T) processor and 1TB of storage. Regards, Praveer

2 3

Loss of precision after postintial?
by Hayley Macpherson 01 Sep '23

01 Sep '23

Hi all, I’m the author of a cosmological initial data thorn in the ET; FLRWSolver. I’m currently working on improving the initial data to solve the constraint equations exactly (instead of previously using an approximation), for a given metric and Kij. The way this works is to calculate the metric terms on the LHS of both the constraints and solve for the relevant rest mass density and peculiar 3-velocities from the matter terms. I have my own routines from a post-processing analysis code to calculate the metric terms, and I’ve incorporated these into the ET for both generating the initial data but also then double checking the constraint violation after the initial data is set up. I noticed something strange: my checks immediately after FLRWSolver is called in the ET show the constraints are satisfied essentially to roundoff error (i.e. the momentum constraint violation has magnitude ~ 1.e-15), but when I take the 3D dump of the initial time slice and run this through my analysis code (which uses the same routines as I use to set up initial data), I see the momentum constraint is violated at the ~ 1e-7 level. I thought this might be my post processing code, so I added a second call to check the constraints using my routines after the full initial process is finished. The first call which immediately follows my FLRWSolver routine is placed in group HydroBase_Initial, and I added another call in CCTK_POSTINITIAL which gives the same result, however, if I instead schedule this call in POSTPOSTINITIAL I see the momentum constraint violation is identical to what I see when post processing the initial dump, at ~ 1.e-7. I can see the specific terms which are causing this difference are the terms which use finite-difference derivatives (the curvature terms in the momentum constraint), while all others are identical. So my question is the following: is there any way that the metric and curvature variables could suffer a loss of precision between the POSTINITIAL and POSTPOSTINITIAL phases of the run? Especially, a loss in precision which is then translated to the 3D dumps. The momentum constraint violation I have is satisfactory, but I am trying to pinpoint why this jump happens to make sure it’s not a bug in my separate code somewhere (also to explain why the constraints aren’t satisfied to roundoff level when they should be, by construction of my initial data :) ). Any help is much appreciated! Best wishes, Hayley ---- Hayley Macpherson | NASA Einstein Fellow Email: hjmacpherson(a)uchicago.edu Pronouns: she/her/hers Office: ERC 479 Kavli Institute for Cosmological Physics Eckhardt Research Center 5640 South Ellis Avenue Chicago, IL, 60637

3 7

2026

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

Users September 2023