Users January 2022

users@lists.einsteintoolkit.org

15 participants
22 discussions

Meeting Minutes 2022-01-20
by Gabella, William E 20 Jan '22

20 Jan '22

Present: Peter D, Roland H, Yosef Z, Bill G, Miguel G, Atul K, Leo RW, Steve B, Maria H, Zach E Chair: Peter D, Minutes: Bill G * Any First Business None * search for next ET release manager [RH] Steve, Yosef is the next release manager. And next release manager is his deputy, who is Roland. * FLRWSolver inclusion [RH] Roland, have not heard from Helvi, and need help reviewing this---last time not enough time for the review. Helvi and Hayley need to be poked and check on progress---Peter will do that. Ticket https://bitbucket.org/einsteintoolkit/tickets/issues/2549/inlcude-flrwsolve… and GitHub https://github.com/hayleyjm/FLRWSolver_public . * ET hackathon [RH] Schedule the hackathon? Steve, requests that we do not have it on a Tues or Thurs. Steve, we have a good idea of what the bugs are and should go ahead with it. Roland, we should have a pre-selection of tickets and bugs for people to work on. Steve, we have done that. Steve and Roland added to some tickets "hackathon". Fixing a ticket miggt be 10 minutes but the testing might be an hour or so. It will be on Wednesday, Feb 2nd, start at 11am CST. Think of it as most of a day with people coming in and out. * ET presentation series [RH] We should keep it up. See page https://docs.einsteintoolkit.org/et-docs/Einstein_Toolkit_Seminar_Proposals Simfactory Setup and Kuibit tutorial seem to be next on the list. Steve, would like to see another NRPY tutorial but he wants time to work on that. Llama presentation would be good; unsure if the 13 patch is working. Roland, 2D visualization is covered by Kuibit, so will remove those. Turn the Mulitpatch into a TBD. In 3-4 weeks, Roland can do the Simfactory tutorial, time to organize it. Simfactory Setup scheduled for Thursday, Feb 10th, during the meeting/telecon. * ET Jenkins server retirement [RH] Roland, status update, it ran on a virtual machine on NCSA and that has been retired. So we do not have Jenkins server running, but there is sort of a continuous testing on GitHub. Looked at yesterday and looked "wonky" so we may not have as nice a continuous integration testing server. May not want a Jenkins server though because it requires such extra effort. Zach, does GitHub actions do all the tests? Roland, it does, and takes 1/2 - 1 hour. What is wonky is it claims 250 commits since the last release. It is still triggering, and have a lot of computer time because of the educational discount at GitHub. See the students work at https://mojamil.github.io/einsteintoolkit/ , and can contribute or comment at https://github.com/mojamil/einsteintoolkit/issues . Would be good for someone to help out making the GitHub actions better here. * Suggestions for new inclusions to consider for the ET [RH] ** Self force code [RH] Peter, we can have something for the next release. Samuel Cupp has completed evolution in Schwarzschild. Need to make sure the code is documented. ** Full reprimand code / thorn [RH] *** tabulated hot EOS in Reprimand? Roland was reviewing it. Roland right now we have a RePriMand library that is not used by anything. We should make sure that something uses it. The build process requeres changes / fixes, Wolfgang, the Meson build system being used will always assume you can find the external lib by pgkconfig whtihi is not true on clusters. Though it has tabulate EOS is for the cold matter, and not hot, so hybrid EOS. Not godo for micro-physics. Need to keep up-to-date what they are doing. It si Con2Prim but has its own EOS framework. The Con2Prim routing is somewhat indep of the EOS, but you stell need to make sure the EOS gives the guarantees that the Con2Prim requires ** Major update to Baikal Zach, mostly a modernization of how the thorn is constructed in NRPy. Functionally identical (maybe one bug fix) to current thorn. [https://www.einsteintoolkit.org/tools/unanswered.php unanswered question on mailing list] Current link shows all the emails in the last month, a bug right now. [https://bitbucket.org/einsteintoolkit/tickets/issues?status=open&status=new… open tickets sorted by update time] Can put "hackathon" in the Search, next to Advanced Search and it pulls up 10 tickets right now. [https://bitbucket.org/einsteintoolkit/tickets/issues?status=new&status=open… tickets ready for review] * Chair and minute taker For Jan 27 meeting, Chair Maria and Minutes Miguel G * Any other business None -- ===================================== William Gabella Research Assistant Professor Department of Physics and Astronomy Vanderbilt University Nashville, TN USA b.gabella(a)vanderbilt.edu (o) 615-343-2713

1 0

Einstein Toolkit Meeting Reminder
by users＠einsteintoolkit.org 19 Jan '22

19 Jan '22

Hello, Please consider joining the weekly Einstein Toolkit phone call at 9:00 am US central time on Thursdays. For details on how to connect and what agenda items are to be discussed, use the link below. https://docs.einsteintoolkit.org/et-docs/Main_Page#Weekly_Users_Call --The Maintainers

1 0

2+1 year postdoc position on numerical simulations for supermassive black holes
by dschleicher＠astro-udec.cl 18 Jan '22

18 Jan '22

Dear colleagues, please help to distribute the advertisement for a 2+1 year postdoctoral position for numerical simulations on the formation, accretion or growth of supermassive black holes. Cheers, Dominik https://jobregister.aas.org/ad/8e41ba3e Postdoctoral position for simulations on supermassive black hole formation and growth The "Millennium Nucleus for Transversal Research and Technology to explore Supermassive Black Holes" (TITANs, http://www.astro.udec.cl/titans/) announces a call for a postdoctoral position on computational modeling in the context of black hole formation, growth or accretion. The position will be initially for two years, including the possibility of a one-year extension based on a performance evaluation. We are particularly interested in candidates with strong experience in 3D (magneto-)hydrodynamical and/or N-body simulations, including codes such as Gizmo and AMUSE, and/or experience in cosmological simulations of galaxy formation/evolution. Candidates for the position should also be interested in working with students and ideally have some experience with the supervision of students already. Computational modeling in TITANS is led by Profs. S. Bovino, N. Leigh, D. Schleicher (U. de Concepción) and Prof. A. Escala (U. de Chile). The Nucleo provides a competitive salary, travel support, as well as a stimulating research environment, currently including 10 main researchers, 6 postdocs, and 25 Ph.D. and Masters students. Computational facilities for the simulations are both available in Concepción as well as in the National Laboratory for Supercomputing. The Nucleo actively seeks to balance its gender and other minority presence. Applicants requiring part time employment to share time with caring responsibilities are also encouraged. Flexible working hours, and partial work-at-home or tele-commuting are also open to discussion. Please contact S. Bovino (stefanobovino(a)udec.cl) for details. Interested applicants should submit here (https://form.jotform.com/220102712488650) the following materials as a single pdf file: a CV with publication list, statements of previous (1-2 pages) and proposed (1-2 pages) research, contact information for three recommendation letter writers as well as a one-page document detailing their experience and/or interest in the supervision of students. We encourage applicants to contact team members ahead of submission. The deadline for applications is February 25; applications will be accepted until the position is filled. Included Benefits: The benefits of the position include travel support and logistic support for the arrival in Chile. ------------------------------------------------- Dr. rer. nat. Dominik Schleicher associate professor Department of Astronomy Universidad de Concepción Barrio Universitario Av. Esteban Iturra, Casilla 160-C Concepción, Chile TEL: +56 9 76550770 http://theory-starformation-group.cl/ds/index.html -------------------------------------------------

1 0

Agenda for Thursday's Meeting
by users＠einsteintoolkit.org 17 Jan '22

17 Jan '22

Please update the Wiki with agenda items for Thursday's meeting. Thanks! https://docs.einsteintoolkit.org/et-docs/meeting_agenda --The Maintainers

1 0

Fwd: PreSync and recovery from checkpoint
by Roland Haas 14 Jan '22

14 Jan '22

Hello Gabriele, I seem to have trouble sending email to the ET mailing list server using my regular UIUC email address, so maybe using GMail will work better. Yours, Roland Begin forwarded message: Date: Sat, 1 Jan 2022 10:36:00 -0500 From: Roland Haas <rhaas(a)illinois.edu> To: Gabriele Bozzola <bozzola.gabriele(a)gmail.com> Cc: Einstein Toolkit Users <users(a)einsteintoolkit.org> Subject: Re: [Users] PreSync and recovery from checkpoint Hello Gabriele, > I have a fork of ProcaEvolve ( > https://urldefense.com/v3/__https://bitbucket.org/Sbozzolo/proca/src/master… ) > where I added the READS/WRITES statements for > PreSync and other features. One of such features > is computing the constraint, but I found out that there > is some problem when PreSync is running with > `mixed-error`. hmm. > I have a write statement for chi in CCTK_INITIAL, which > is not called when restarting a simulation. Nonetheless, > chi should be available everywhere because it is read > from a checkpoint. What is the correct thing to do in > this scenario? When grid functions are read in from a checkpoint then they are marked as being valid everywhere. This is done in CarpetIOHDF5/src/Input.cc in the Recover function in the block that starts with // check that all variables have been read completely on this mglevel/reflevel specifically in line 793 of that file: gf->set_valid(mglevel, reflevel, tl, CCTK_VALID_EVERYWHERE); If these variables are not checkpointed (likely, given that they are only constraints and thus not usually considered "important" to checkpoint) then you would have to add code to say POST_RECOVER_VARIABLES to initialize them (eg to 0). This is a good idea to do even without considering PreSync since it avoids having random junk data (or poison) in the constraint output for a couple of steps (depending on mesh refinement details) after recovery. Something like: schedule Proca_ClearConstraints AT POST_RECOVER_VARIABLES { LANG: Fortran } "Initialize constraint values after recovery" You can also try to to Proca_Constraints in post-recover variables but, since the constraints require derivatives and those cannot be (correctly) computed in post-recover-variables, the constraints for the checkpointed timestep will (possibly I would have to think about exactly how checkpointing, prolongation and time stepping interact) differ before and after recovery. Also POST_RECOVER_VARIABLES only runs the scheduled function on timelevel 0 (ie current time) and not any past timelevels so if there are multiple time levels to the constraint variables, only tl=0 will be initialized. Yours, Roland -- My email is as private as my paper mail. I therefore support encrypting and signing email messages. Get my PGP key from http://keys.gnupg.net. -- My email is as private as my paper mail. I therefore support encrypting and signing email messages. Get my PGP key from http://keys.gnupg.net.

3 2

meeting minutes for 2022-01-13
by Steven R. Brandt 13 Jan '22

13 Jan '22

Einstein Toolkit Meeting Notes Present: Yosef, Steve, Peter, Atul Leonardo Rosa Werneck, Bill, Miguel Gabriele, Federico, Zach Chair: Yosef Minute Taker: Steve Visualization page on the website: Gabriele made pull request with a visualization page. Also cleaned up some typos. Unanswered questions on the mailing list: Gabriele's question about reconstructing past time levels. I told him I believe it needs to be done manually. Tickets: Some tickets flagged for the hackathon. What new things do we want for a release? FLRW solver Additional docs for visualization is desirable. The tutorial server viz should be updated. Next week: 1-20-2022 Chair: Peter Minutes: Bill

1 0

Einstein Toolkit Meeting Reminder
by users＠einsteintoolkit.org 12 Jan '22

12 Jan '22

1 0

Unable to compile ET with OpenMPI 2.1.0 in cluster
by Shamim Haque 11 Jan '22

11 Jan '22

Hello all, I am trying to install ET with OpenMPI 2.1.0 but it seems to be causing issues. I have done it successfully once with OpenMPI-x86_64 in our cluster, but it turns out this MPI is only available in the login node and not available on computing nodes, hence, I am not able to use it once a job is submitted in the queue. I could find OpenMPI 2.1.0 being available on computing nodes, so I was trying to compile with this version. I created a module file to load this module before installation. I could see the following in the terminal, which I guess, ET could recognise OpenMPI 2.1.0: Running configuration script for thorn MPI: MPI selected, but MPI_DIR is not set. Computing settings... Found MPI compiler wrapper at /opt/mpi/openmpi/2.1.0/bin/mpic++! Successfully configured MPI. Finished running configuration script for thorn MPI. Later, the progress stops with the following error: /home2/mallick/ET3/Cactus/configs/sim/lib/libthorn_CT_MultiLevel.a(CT_MultiLevel.cc.o): In function `MPI::Op::Init(void (*)(void const*, void*, int, MPI::Datatype const&), bool)': /usr/include/openmpi-x86_64/openmpi/ompi/mpi/cxx/op_inln.h:121: undefined reference to `ompi_mpi_cxx_op_intercept' /home2/mallick/ET3/Cactus/configs/sim/lib/libthorn_CT_MultiLevel.a(CT_MultiLevel.cc.o): In function `Intracomm': /usr/include/openmpi-x86_64/openmpi/ompi/mpi/cxx/intracomm_inln.h:23: undefined reference to `MPI::Comm::Comm()' /home2/mallick/ET3/Cactus/configs/sim/lib/libthorn_CT_MultiLevel.a(CT_MultiLevel.cc.o): In function `Intracomm': /usr/include/openmpi-x86_64/openmpi/ompi/mpi/cxx/intracomm.h:25: undefined reference to `MPI::Comm::Comm()' /home2/mallick/ET3/Cactus/configs/sim/lib/libthorn_CT_MultiLevel.a(CT_MultiLevel.cc.o): In function `Cartcomm': /usr/include/openmpi-x86_64/openmpi/ompi/mpi/cxx/intracomm.h:25: undefined reference to `MPI::Comm::Comm()' /home2/mallick/ET3/Cactus/configs/sim/lib/libthorn_CT_MultiLevel.a(CT_MultiLevel.cc.o): In function `Graphcomm': /usr/include/openmpi-x86_64/openmpi/ompi/mpi/cxx/intracomm.h:25: undefined reference to `MPI::Comm::Comm()' /home2/mallick/ET3/Cactus/configs/sim/lib/libthorn_CT_MultiLevel.a(CT_MultiLevel.cc.o): In function `Intracomm': /usr/include/openmpi-x86_64/openmpi/ompi/mpi/cxx/intracomm.h:25: undefined reference to `MPI::Comm::Comm()' /home2/mallick/ET3/Cactus/configs/sim/lib/libthorn_CT_MultiLevel.a(CT_MultiLevel.cc.o):/usr/include/openmpi-x86_64/openmpi/ompi/mpi/cxx/intracomm.h:25: more undefined references to `MPI::Comm::Comm()' follow /home2/mallick/ET3/Cactus/configs/sim/lib/libthorn_CT_MultiLevel.a(CT_MultiLevel.cc.o):(.rodata._ZTVN3MPI8DatatypeE[_ZTVN3MPI8DatatypeE]+0x78): undefined reference to `MPI::Datatype::Free()' /home2/mallick/ET3/Cactus/configs/sim/lib/libthorn_CT_MultiLevel.a(CT_MultiLevel.cc.o):(.rodata._ZTVN3MPI3WinE[_ZTVN3MPI3WinE]+0x48): undefined reference to `MPI::Win::Free()' /home2/mallick/ET3/Cactus/configs/sim/lib/libthorn_CarpetLib.a(backtrace.cc.o): In function `MPI::Intracomm::Intracomm()': /usr/include/openmpi-x86_64/openmpi/ompi/mpi/cxx/intracomm.h:25: undefined reference to `MPI::Comm::Comm()' /home2/mallick/ET3/Cactus/configs/sim/lib/libthorn_CarpetLib.a(backtrace.cc.o): In function `MPI::Intracomm::Intracomm(ompi_communicator_t*)': /usr/include/openmpi-x86_64/openmpi/ompi/mpi/cxx/intracomm_inln.h:23: undefined reference to `MPI::Comm::Comm()' collect2: error: ld returned 1 exit status make[1]: *** [/home2/mallick/ET3/Cactus/exe/cactus_sim] Error 1 make: *** [sim] Error 2 I could not figure out the issue here. It seems */usr/include/openmpi-x86_64/openmpi/ompi/mpi *is being taken by ET from somewhere and it is not taking OpenMPI 2.1.0. Some suggestions would be very helpful. Thank you. Regards Shamim Haque Senior Research Fellow (SRF) Department of Physics IISER Bhopal

2 2

Agenda for Thursday's Meeting
by users＠einsteintoolkit.org 10 Jan '22

10 Jan '22

Please update the Wiki with agenda items for Thursday's meeting. Thanks! https://docs.einsteintoolkit.org/et-docs/meeting_agenda --The Maintainers

1 0

meeting minutes for 2022-01-06
by Haas, Roland 06 Jan '22

06 Jan '22

Present: Roland H, Yosef Z, Miguel G, Steve B, Federico A, Leo W * SPEC benchmark contribution [SB, RH] Steve contact SPEC group Cactus is still in the running, they were able to run the test Seem on track or inclusion * LORENE2 as default LORENE thorn in ET [RH] Roland will change to LORENE2 now that Johnson release is out * Issues regeneration kranc thorns Steve was unable to regenerate Kranc thorns using multiple version of MMA. Similarly for NRPy+ based thorn. Roland could generate code one release past using MMA 12 and a bugfix in Kranc's repo: https://github.com/ianhinder/Kranc/issues/151 Zach redid how codes are expected to be build in NRPy and added a nicer build system. Checked that core kernel code is identical, with some lines changing to different files. At this point this is an update to nrpytutorial. Should be reviewed as a patch to Baikal. Steve will take a look. * thorn to include in next release (May 2022, 4 months from now) ** FLRWSolver find extra volunteer to help review find extra volunteer to help address issues add again next week * ET bug squashing call [BG, RH] add again next week had planned to do so a bit in the past Steve will look at possible easy tickets for people to handle * ET EU workshop were contacted by organizers, might be late August - early September, trying to avoid overlap with workshops in the US US school will likely be June 6th 2022 NR community workshop August 1st 2022 Unanswered questions: Roland to respond to http://lists.einsteintoolkit.org/pipermail/users/2022-January/008342.html Open tickets * https://bitbucket.org/einsteintoolkit/tickets/issues/2554/init_3_timelevels… Yosef will take a look and see if the proposed fix works for him * https://bitbucket.org/einsteintoolkit/tickets/issues/2584/carpet-defines-gl… Yosef approves, Roland will apply chair next week (Jan 13th): Steve minute taker: Yosef Z Yours, Roland

1 0

2026

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

Users January 2022