Users October 2022

users@lists.einsteintoolkit.org

12 participants
26 discussions

Avoiding writing one checkpoint file per MPI process
by Lorenzo Ennoggi 01 Nov '22

01 Nov '22

Hello, In order to avoid stressing the filesystem on the cluster I'm running on, I was suggested to avoid writing one output/checkpoint file per MPI process and instead collecting data from multiple processes before outputting/checkpointing happens. I found the combination of parameters IO::out_mode = "np" IO::out_proc_every = 8 does the job for output files, but I still have one checkpoint file per process. Is there a similar parameter, or combination of parameters, which can be used for checkpoint files? Thank you very much, Lorenzo Ennoggi

3 5

Agenda for Thursday's Meeting
by reminders＠einsteintoolkit.org 31 Oct '22

31 Oct '22

Please update the Wiki with agenda items for Thursday's meeting. Thanks! https://docs.einsteintoolkit.org/et-docs/meeting_agenda --The Maintainers

1 0

Meeting minutes for 2022-10-27
by Bing-Jyun Tsao 27 Oct '22

27 Oct '22

Present: Leo, Johnny, Roland, Steve, Peter, Keith, Beyhan, Samuel, Yosef - ET release - testing has started: <http://einsteintoolkit.org/testsuite_results/index.php> http://einsteintoolkit.org/testsuite_results/index.php - M1 and generic macOS <https://bitbucket.org/einsteintoolkit/tickets/issues/2659/externallibrary-a…> issues [RH], clang doesn’t support openMP - homebrew moved [RH] - test run on sunrise [PD] - release name variation "Sophie Kowalevski" which is (say wikipedia) how she transcribed her name in her papers [RH], agreed in meeting - release teams definition update / [ <https://docs.einsteintoolkit.org/et-docs/Meeting_agenda#2020-05-21recap> https://docs.einsteintoolkit.org/et-docs/Meeting_agenda#2020-05-21recap]: - release coordinator(s) - test runner(s) - gallery example runner(s) - anyone putting in work specifically for the release - reviewer(s): be members to ET, test runners and gallery example runners who will look at the contribution for the new release add as co-authors - Gallery runs: - BNS example <https://bitbucket.org/einsteintoolkit/tickets/issues/2176/test-binary-neutr…> -- Johnny: ran on expanse - BBH example <https://bitbucket.org/einsteintoolkit/tickets/issues/2172/test-binary-black…> -- Allen Wen (need email address): already ran without simfactory - Multipatch <https://bitbucket.org/einsteintoolkit/tickets/issues/2174/test-multi-patch-…> -- Hrishikesh Kalyanaraman, will be followed up by RH - Poisson <https://bitbucket.org/einsteintoolkit/tickets/issues/2173/test-poisson-equa…> -- Nadine Kuo - Single NS <https://bitbucket.org/einsteintoolkit/tickets/issues/2175/test-single-stabl…> -- Lisa Leung - Tickets: - #2644 <https://bitbucket.org/einsteintoolkit/tickets/issues/2644>: issue with later version of ET - Other: - Switching to CarpetX [ES], need gallery example, maybe GW150914 [SB], no sub-cycling in time with CarpetX might not be sufficient for BBH, will need to check walltime for an example run [YZ], need documentation [LW], symmetry boundary condition and periodic boundary exist, but no radiated boundary condition yet [RH] - Citation of ET [YZ] - Next week - Chair: Peter - Minute: Steve

1 0

Einstein Toolkit Meeting Reminder
by reminders＠einsteintoolkit.org 26 Oct '22

26 Oct '22

Hello, Please consider joining the weekly Einstein Toolkit phone call at 9:00 am US central time on Thursdays. For details on how to connect and what agenda items are to be discussed, use the link below. https://docs.einsteintoolkit.org/et-docs/Main_Page#Weekly_Users_Call --The Maintainers

1 0

Postdoc position at UIUC on strong field tests of gravity
by Helvi Witek 26 Oct '22

26 Oct '22

Dear all, I am happy to announce a postdoc position to work in my group at the University of Illinois. Interested candidates should submit there application to https://my.physics.illinois.edu/join/ and select the "Strong field test of gravity" opening. Please submit your application until 1 December 2022 for full consideration. The successful candidate will join Asst. Prof. Helvi Witek and her team to work on gravitational wave signatures in extensions of General Relativity, including quadratic gravity, or signatures of extra degrees of freedom to probe for beyond-standard model particles and dark matter candidates. The candidate should have strengths in one or more of the following topics: numerical relativity in and beyond General Relativity, Post-Newtonian theory, inspiral-merger-ringdown waveform modelling or gravitational wave data analysis. The expected start date in Summer 2023. Please find the full announcement below and on https://hyperspace.uni-frankfurt.de/2022/10/25/postdoctoral-position-in-str… Best wishes Helvi --- The Illinois Relativity Group in the Department of Physics at the University of Illinois at Urbana-Champaign invites applications for a postdoctoral research position with focus on strong-field tests of gravity. The successful candidate will join Asst. Prof. Helvi Witek and her team to work on gravitational wave signatures in extensions of General Relativity, including quadratic gravity, or signatures of extra degrees of freedom to probe for beyond-standard model particles and dark matter candidates. The candidate should have strengths in one or more of the following topics: numerical relativity in and beyond General Relativity, Post-Newtonian theory, inspiral-merger-ringdown waveform modelling or gravitational wave data analysis. In addition to Asst. Prof. Helvi Witek, the Illinois Relativity Group is composed of Prof. N. Yunes, Prof. S. Shapiro and Research Professor A. Tsokaros, and their research postdocs, graduate and undergraduate students. The Illinois Relativity Group maintains close interactions with UIUC colleagues in the Department of Physics, the National Center for Supercomputing Applications (NCSA; in particular Dr. R. Haas), and in the Department of Astronomy. Furthermore, the successful candidate will become part of a vibrant research community in the Illinois Center for Advanced Study of the University (ICASU; https://icasu.illinois.edu/) that bridges the Departments of Physics, Astronomy, Computer Science, Mathematics and NCSA. ICASU faculty working in closely related areas include Profs. Peter Adshead, Charles Gammie, Gil Holder, Yoni Kahn, Jaki Noronha-Hostler, Jorge Noronha, and Jessie Shelton. The successful candidate will also have opportunities to join the LISA Consortium (https://lisamission.org) and the Einstein Toolkit Consortium (https://einsteintoolkit.org). Interested candidates must have completed a Ph.D. in astronomy, physics or a related field before the start date. They must submit their application package online here: https://my.physics.illinois.edu/join/ under “Postdoctoral Fellowships in Strong-field Tests of Gravity 2023.” Applicants are requested to submit (1) a curriculum vitae, including a list of publications (2) a research statement (limited to 3 pages, 1 inch margin, 11pt) containing a summary of previous and current research, and future plans (3) contact information for three referees who will be asked to write letters of recommendation The deadline for applications and all letters of recommendation is December 1, 2022 for full consideration. The expected starting date of the position is 15 August 2022. Please contact Helvi Witek (hwitek[AT]illinois.edu) if you have any questions about the position itself, or Brandy Koebbe (bkoebbe[AT]illinois.edu) if you have any questions about the application form. Candidates from underrepresented groups in STEM are particularly encouraged to apply. Any offer for this position is contingent upon the successful completion of a criminal background check process in accordance with the University of Illinois Background Check Policy. The University of Illinois is an equal opportunity employer and all qualified applicants will receive consideration for employment without regard to race, religion, color, national origin, sex, sexual orientation, gender identity age, status as a protected veteran, or status as a qualified individual with a disability, or criminal conviction history. Illinois welcomes individuals with diverse backgrounds, experiences, and ideas who embrace and value diversity and inclusivity. (www.inclusiveillinois.illinois.edu). --------------------------------------------- Dr. Helvi Witek Assistant Professor Department of Physics University of Illinois at Urbana-Champaign 247 Loomis Lab 1110 W Green St Urbana, IL 61801 ---------------------------------------------

1 0

Re: [Users] create-submit AttributeError
by Traver, Bradley 26 Oct '22

26 Oct '22

Roland, Thanks for getting back to me! The backtrace is below: Warning: Importing properties from /home/brad.traver/et_runs/NeutronStarTest/output-0000/SIMFACTORY/properties.ini failed. Traceback (most recent call last): File "/home/brad.traver/einsteintoolkit/Cactus/repos/simfactory2/bin/../lib/sim.py", line 149, in <module> main() File "/home/brad.traver/einsteintoolkit/Cactus/repos/simfactory2/bin/../lib/sim.py", line 145, in main CommandDispatch() File "/home/brad.traver/einsteintoolkit/Cactus/repos/simfactory2/bin/../lib/sim.py", line 107, in CommandDispatch module.main() File "/home/brad.traver/einsteintoolkit/Cactus/repos/simfactory2/lib/sim-manage.py", line 398, in main CommandDispatch() File "/home/brad.traver/einsteintoolkit/Cactus/repos/simfactory2/lib/sim-manage.py", line 377, in CommandDispatch exec("command_%s()" % command) File "<string>", line 1, in <module> File "/home/brad.traver/einsteintoolkit/Cactus/repos/simfactory2/lib/sim-manage.py", line 220, in command_run restart.submitRun(simulationName, restart_id) File "/home/brad.traver/einsteintoolkit/Cactus/repos/simfactory2/lib/simrestart.py", line 1075, in submitRun self.run() File "/home/brad.traver/einsteintoolkit/Cactus/repos/simfactory2/lib/simrestart.py", line 1096, in run testsuite = self.Properties.testsuite == "True" AttributeError: 'SimProperties' object has no attribute 'testsuite' Bradley Traver Cyberinfrastructure Engineer Computational Research Center Chester Fritz Library, Room 334A University of North Dakota https://und.edu/research/computational-research-center/ Office: 701-777-6514 -----Original Message----- From: Roland Haas <rhaas(a)illinois.edu> Sent: Friday, October 14, 2022 10:04 AM To: Traver, Bradley <brad.traver(a)und.edu> Cc: users(a)einsteintoolkit.org Subject: Re: [Users] create-submit AttributeError Hello Bradley, Sorry to hear that simfactory failed for you and thank you for reporting this. Do you have the Python backtrace (if there was one) at hand and could attach it (as a text file) to an email, please? That way we would know exactly which line of code produces the error message. Yours, Roland > Greetings! > > I am trying to get Einstein Toolkit up and running on our cluster. > All of the configuration files seem to be correct. As a test, I am > running the following command with a parameter file provided by one of > our physics professors. ./simfactory/bin/sim create-submit > NeutronStarTest tov_gallery_polytrop.par 1 24:00:00 > > When the job tries to run, the following Python error occurs: > > AttributeError: 'SimProperties' object has no attribute 'testsuite' > > Looking at the properties.ini file, I can see "testsuite = False" but > I'm not sure where that's being pulled in from. Any assistance would > be greatly appreciated! > > Bradley Traver > Cyberinfrastructure Engineer > Computational Research Center > University of North Dakota > Chester Fritz Library, Room 334A > 3051 University Ave, Stop 9000 > https://urldefense.com/v3/__https://und.edu/research/computational-res > earch-center/__;!!DZ3fjg!7duwwcdFJDSSCR3jpx0w6vnWFBDb_0d5u4CSvdODw6-go > VSC9ltcBcVDyDsmT0gLRc38tJmuHm-rcf7-tTfhMQ$ > Office: 701-777-6514 > -- My email is as private as my paper mail. I therefore support encrypting and signing email messages. Get my PGP key from http://keys.gnupg.net.

2 1

Postdoctoral Positions in Computational Astrophysics and Numerical Relativity
by Zach Etienne 26 Oct '22

26 Oct '22

Dear Einstein Toolkit community, The Department of Physics at the University of Idaho (Principal Investigator: Zachariah Etienne) is seeking applications for two Postdoctoral Fellows in the field of Computational Astrophysics/Numerical Relativity. The successful candidate will: * Perform state-of-the-art simulations of important astrophysical/multi-messenger phenomena, and analyze and visualize simulation data. * Work with multi-institution networks of collaborators on next-generation numerical relativity projects such as: 1. BlackHoles@Home (https://blackholesathome.net), which is a proposed, BOINC-based volunteer-computing / public-outreach project called aimed at unlocking the consumer-grade desktop computer as a core tool for numerical relativity simulations of black hole binaries (BHBs). Our goal is to generate more than 10,000 NR-based waveform predictions per year, validating all existing waveform catalogs and greatly improving our coverage of BHB parameter space. 2. superB, a supercomputer-ready, highly-scalable fork of BlackHoles@Home aimed at both rapid follow-up simulations of binary black hole gravitational wave detections, and simulating extreme binary black hole mergers (super-high spins and mass ratios). 3. NRPy+-enhanced Einstein Toolkit, which involves advancing NRPy+-based code generation (http://nrpyplus.net/) within the Einstein Toolkit ( https://einsteintoolkit.org/) framework, lowering the barrier to entry for new users. 4. IllinoisGRMHD (https://illinoisgrmhd.net): Supercomputer simulations of magnetized binary neutron star mergers with IllinoisGRMHD, coupling dynamical spacetimes, GRMHD, tabulated equations of state, and neutrino physics. 5. ChaNGa/NRPy+: Advancing the state-of-the-art in supercomputer simulations of binary neutron star mergers, through the marriage of the ChaNGa moving-mesh ALE code (for GRHD/GRMHD) and the NRPy+-based BlackHoles@Home/superB infrastructure (for the spacetime). 6. NRPyElliptic: A fast new hyperbolic relaxation elliptic solver for generating initial data for numerical relativity. * Write scientific papers summarizing work and present work at scientific conferences. -={ Position Details }=- * Application deadline: The screening process will begin on December 15, 2022 and will continue until the position is filled. * Preferred start date: August 2023, though start dates earlier in the year are acceptable. * Duration: These positions may be renewed in annualized increments for a duration of at least three years, contingent upon performance, need, and funding. * Remote work option: Available, considered on a case-by-case basis * Salary range: $55-60k annually -={ Qualifications }=- * All Ph.D. requirements fulfilled in Physics, Mathematics, Astronomy, Computer Science, or a related field. * Strong programming skills (C and Python, or closely related languages). * Experience in high-performance computing and large-scale data analysis. * Excellent ability to communicate research to both experts and non-experts. -={ How to Apply }=- * Step 1. Visit the site https://uidaho.peopleadmin.com/postings/38351 and click "Apply for this Job". The site will request the following documents be uploaded: - Resume or CV - Statement of research * Step 2. Please have three letters of recommendation sent directly to zetienne(a)uidaho.edu -={ University of Idaho }=- The University of Idaho (UI) is located in the idyllic Palouse region, on the Washington state border. It experiences a Mediterranean-like climate with warm, dry summers, and mild winters. UI is situated in Moscow, the most walkable city in Idaho (walkscore.com), and is only 15km from Pullman, WA, another college town. The University of Idaho is an Equal Opportunity/Affirmative Action/Veterans/Disability Employer. * * * Zachariah Etienne Assoc. Prof. of Physics, U. of Idaho Adjunct Assoc. Prof. of Physics & Astronomy, West Virginia U. https://etienneresearch.com https://blackholesathome.net

1 0

Agenda for Thursday's Meeting
by reminders＠einsteintoolkit.org 24 Oct '22

24 Oct '22

Please update the Wiki with agenda items for Thursday's meeting. Thanks! https://docs.einsteintoolkit.org/et-docs/meeting_agenda --The Maintainers

1 0

Meeting minutes for 2022-10-20
by Leonardo Rosa Werneck 20 Oct '22

20 Oct '22

Hi all, Here are the minutes from today’s meeting. Chair: Sam Minutes: Leo Attending: Sam, Leo, Peter, Cheng-Hsin, Johnny, Keith, Roland, Zach, Yosef ET Release - Progress is being made, fixing failures in clusters that are not frequently used. - Odd compilation error in Summit cluster that hasn't been fixed yet. - Roland will contact contributors. - All galery examples are running. - Peter will re-run tests on Sunrise. Extension of PreSync's Capabilities with Carpet - PreSync is used to fill in ghostzones and apply BCs automatically whenever needed. - Extend to fill in "auxiliary" quantities automatically when access is requested. For example, if a thorn evolves the BSSN variables, then it provides a function to compute the ADMBase variables from the BSSN variables and PreSync would schedule this function appropriately whenever another function needs to access the ADMase variables. - CarpetX has some support for gridfunction dependencies, which could potentially guide this process. - There are subtleties that could lead to circular logic, so care must be taken when doing this. - First idea is to not deal with more complicated cases where e.g., a function adds something to a gridfunction (like KO dissipation). Instead, the new feature would fill in gridfunctions for which a "READ" was requested but the gridfunctions are not valid anywhere. Open Tickets - No warning printed by the Perl scripts for ambiguous READ/WRITES since it is unlikely users will attempt to use the old PreSync standard. Zach suggests: would be nice to have the option of being more verbose instead of completely removing the warning, if possible. Roland says: maybe we can give the Perl script another option, e.g. "Picky" (Verbose is already taken) so that it can print the warnings if requested. - There is a ticket ready for review which prevents PreSync from syncing coarser refinement levels if those have already been synced previously. - There's an interface error in some of the ADMBase functions where read-only gridfunctions are maked as "OUT". Sam says: changing "OUT" to "IN" fixes the interface. Fix is working on local machines but needs more testing. - cctk_time and cctk_deltas are wrong sometimes and a fix has been implemented that needs further review. - With PreSync, CarpetRegrid does not change the validity of the gridfunctions after a regrid, which leads to problems. The logic should be: after a regrid all gridfunctions are valid nowhere. Then the gridfunctions that support prolongation can be flagged as valid in the interior after prolongation. Everything else should remain invalid everywhere until something else sets it. Unanswered Questions in the Mailing List None. Next Chair: Leo Next Minutes: Johnny Cheers, Leo ------ Leonardo R. Werneck, Ph.D. Postdoctoral researcher Office EP 314 | Department of Physics | University of Idaho 875 Perimeter Dr. MS 0903 Moscow, ID 83844-0903, USA leonardo(a)uidaho.edu <mailto:leonardo@uidaho.edu> https://leowerneck.github.io <https://leowerneck.github.io/>

1 0

upcoming Einstein Toolkit release ET_2022_11
by Roland Haas 20 Oct '22

20 Oct '22

Hello all, The upcoming Einstein Toolkit release ET_2022_11 is scheduled for: Thursday 2022-11-10 The release will be named after Sofya Kovalevskaya https://en.wikipedia.org/wiki/Sofya_Kovalevskaya The toolkit has now entered a feature freeze period during which no new features allowed anymore, and extensive testing begins. Yours, Roland -- My email is as private as my paper mail. I therefore support encrypting and signing email messages. Get my PGP key from http://pgp.mit.edu .

1 0

2026

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

Users October 2022