Users December 2021

users@lists.einsteintoolkit.org

9 participants
17 discussions

Unable to compile ET with OpenMPI 2.1.0 in cluster
by Shamim Haque 11 Jan '22

11 Jan '22

Hello all, I am trying to install ET with OpenMPI 2.1.0 but it seems to be causing issues. I have done it successfully once with OpenMPI-x86_64 in our cluster, but it turns out this MPI is only available in the login node and not available on computing nodes, hence, I am not able to use it once a job is submitted in the queue. I could find OpenMPI 2.1.0 being available on computing nodes, so I was trying to compile with this version. I created a module file to load this module before installation. I could see the following in the terminal, which I guess, ET could recognise OpenMPI 2.1.0: Running configuration script for thorn MPI: MPI selected, but MPI_DIR is not set. Computing settings... Found MPI compiler wrapper at /opt/mpi/openmpi/2.1.0/bin/mpic++! Successfully configured MPI. Finished running configuration script for thorn MPI. Later, the progress stops with the following error: /home2/mallick/ET3/Cactus/configs/sim/lib/libthorn_CT_MultiLevel.a(CT_MultiLevel.cc.o): In function `MPI::Op::Init(void (*)(void const*, void*, int, MPI::Datatype const&), bool)': /usr/include/openmpi-x86_64/openmpi/ompi/mpi/cxx/op_inln.h:121: undefined reference to `ompi_mpi_cxx_op_intercept' /home2/mallick/ET3/Cactus/configs/sim/lib/libthorn_CT_MultiLevel.a(CT_MultiLevel.cc.o): In function `Intracomm': /usr/include/openmpi-x86_64/openmpi/ompi/mpi/cxx/intracomm_inln.h:23: undefined reference to `MPI::Comm::Comm()' /home2/mallick/ET3/Cactus/configs/sim/lib/libthorn_CT_MultiLevel.a(CT_MultiLevel.cc.o): In function `Intracomm': /usr/include/openmpi-x86_64/openmpi/ompi/mpi/cxx/intracomm.h:25: undefined reference to `MPI::Comm::Comm()' /home2/mallick/ET3/Cactus/configs/sim/lib/libthorn_CT_MultiLevel.a(CT_MultiLevel.cc.o): In function `Cartcomm': /usr/include/openmpi-x86_64/openmpi/ompi/mpi/cxx/intracomm.h:25: undefined reference to `MPI::Comm::Comm()' /home2/mallick/ET3/Cactus/configs/sim/lib/libthorn_CT_MultiLevel.a(CT_MultiLevel.cc.o): In function `Graphcomm': /usr/include/openmpi-x86_64/openmpi/ompi/mpi/cxx/intracomm.h:25: undefined reference to `MPI::Comm::Comm()' /home2/mallick/ET3/Cactus/configs/sim/lib/libthorn_CT_MultiLevel.a(CT_MultiLevel.cc.o): In function `Intracomm': /usr/include/openmpi-x86_64/openmpi/ompi/mpi/cxx/intracomm.h:25: undefined reference to `MPI::Comm::Comm()' /home2/mallick/ET3/Cactus/configs/sim/lib/libthorn_CT_MultiLevel.a(CT_MultiLevel.cc.o):/usr/include/openmpi-x86_64/openmpi/ompi/mpi/cxx/intracomm.h:25: more undefined references to `MPI::Comm::Comm()' follow /home2/mallick/ET3/Cactus/configs/sim/lib/libthorn_CT_MultiLevel.a(CT_MultiLevel.cc.o):(.rodata._ZTVN3MPI8DatatypeE[_ZTVN3MPI8DatatypeE]+0x78): undefined reference to `MPI::Datatype::Free()' /home2/mallick/ET3/Cactus/configs/sim/lib/libthorn_CT_MultiLevel.a(CT_MultiLevel.cc.o):(.rodata._ZTVN3MPI3WinE[_ZTVN3MPI3WinE]+0x48): undefined reference to `MPI::Win::Free()' /home2/mallick/ET3/Cactus/configs/sim/lib/libthorn_CarpetLib.a(backtrace.cc.o): In function `MPI::Intracomm::Intracomm()': /usr/include/openmpi-x86_64/openmpi/ompi/mpi/cxx/intracomm.h:25: undefined reference to `MPI::Comm::Comm()' /home2/mallick/ET3/Cactus/configs/sim/lib/libthorn_CarpetLib.a(backtrace.cc.o): In function `MPI::Intracomm::Intracomm(ompi_communicator_t*)': /usr/include/openmpi-x86_64/openmpi/ompi/mpi/cxx/intracomm_inln.h:23: undefined reference to `MPI::Comm::Comm()' collect2: error: ld returned 1 exit status make[1]: *** [/home2/mallick/ET3/Cactus/exe/cactus_sim] Error 1 make: *** [sim] Error 2 I could not figure out the issue here. It seems */usr/include/openmpi-x86_64/openmpi/ompi/mpi *is being taken by ET from somewhere and it is not taking OpenMPI 2.1.0. Some suggestions would be very helpful. Thank you. Regards Shamim Haque Senior Research Fellow (SRF) Department of Physics IISER Bhopal

2 2

PreSync and recovery from checkpoint
by Gabriele Bozzola 30 Dec '21

30 Dec '21

Hello, I have a fork of ProcaEvolve ( https://bitbucket.org/Sbozzolo/proca/src/master/) where I added the READS/WRITES statements for PreSync and other features. One of such features is computing the constraint, but I found out that there is some problem when PreSync is running with `mixed-error`. The relevant scheduler instruction is: schedule GROUP Proca_constraint AT ANALYSIS { LANG: Fortran } "Compute Proca constraint" schedule Proca_constraint IN Proca_constraint { LANG: Fortran READS: ProcaBase::Ei(Everywhere), ProcaBase::Aphi(Everywhere) READS: ProcaEvolve::conf_fac_chi(Everywhere) WRITES: ProcaEvolve::pc(Interior) } "Compute Proca constraint" And the error is WARNING level 0 from host exp-7-40.expanse.sdsc.edu process 0 in thorn Carpet, file /home/sbozzolo/Cactus/arrangements/Carpet/Carpet/src/PreSync.cc:260: -> Required read for PROCAEVOLVE::chi (rl=13) not satisfied. Have Nowhere and require Everywhere missing Everywhere at the start of routine ProcaEvolve::Proca_constraint. Current valid state: Valid entries: PROCAEVOLVE::chi (rl=0) Nowhere PROCAEVOLVE::chi (rl=1) Nowhere PROCAEVOLVE::chi (rl=2) Nowhere PROCAEVOLVE::chi (rl=3) Nowhere PROCAEVOLVE::chi (rl=4) Nowhere PROCAEVOLVE::chi (rl=5) Nowhere PROCAEVOLVE::chi (rl=6) Nowhere PROCAEVOLVE::chi (rl=7) Nowhere PROCAEVOLVE::chi (rl=8) Nowhere PROCAEVOLVE::chi (rl=9) Nowhere PROCAEVOLVE::chi (rl=10) Nowhere PROCAEVOLVE::chi (rl=11) Nowhere PROCAEVOLVE::chi (rl=12) Nowhere PROCAEVOLVE::chi (rl=13) Nowhere. I have a write statement for chi in CCTK_INITIAL, which is not called when restarting a simulation. Nonetheless, chi should be available everywhere because it is read from a checkpoint. What is the correct thing to do in this scenario? Thanks, Gabriele

1 0

Compiling Cactus on M1 Mac
by Garrison, David 22 Dec '21

22 Dec '21

Hello, Has anyone had success getting the Einstein Toolkit to compile on a M1 powered Mac? I get "Bad CPU type in executable” when I try. -DG -- David Garrison, Ph.D. Interim Associate Dean for the College of Science and Engineering, Professor and Former Chair of Physics University of Houston-Clear Lake Bayou 3611 Houston, TX 77058 Tel: 281-283-3796 Fax: 281-283-3709 http://sce.uhcl.edu/garrison http://www.uhcl.edu/physics "If we knew what it was we were doing, it would not be called research, would it?" ‹ Albert Einstein.

2 4

Need help building Einstein Toolkit: make: *** [sim] Error 2
by Maryam Asadnezhad 19 Dec '21

19 Dec '21

Hi All, Thanks so much for making the Einstein Toolkit software available! I'm using the CactusTutorial.ipynb that I got from your site. I'm trying to build ET on my 2017 Macbook Pro running OS 12.0.1 I'm stuck at this cell : %%bash ./simfactory/bin/sim build -j2 --thornlist ../einsteintoolkit.th Running that cell eventually stops with the errors make: *** [sim] Error 2 Can you help? I'm confused by the error message I really appreciate any help you are able to provide! Thank you. maryam

2 3

Postdoc position in numerical relativity joined at Nottingham / UIUC
by Helvi Witek 19 Dec '21

19 Dec '21

Dear all, a brief reminder for our opening of a postdoc position jointly between the University of Nottingham and UIUC, to work with Prof. Thomas Sotiriou (in cc) and myself. The deadline is 21st December 2021. You can find more details on the position and a link to the application website here: https://hyperspace.uni-frankfurt.de/2021/11/25/postdoc-position-in-gravity-… Best wishes Helvi -- --------------------------------------------- Dr. Helvi Witek Assistant Professor Department of Physics University of Illinois at Urbana-Champaign 247 Loomis Lab 1110 W Green St Urbana, IL 61801 ---------------------------------------------

1 0

Need help building Einstein Toolkit: make: *** [sim] Error 2
by Maryam Asadnezhad 16 Dec '21

16 Dec '21

1 0

Meeting Minutes 2021-12-16
by Gabella, William E 16 Dec '21

16 Dec '21

Present: Steve B, Bill G, Roland H, Peter D, Yosef Z, Miguel G, Atul K, Zach E Chair: Steve B, Minutes: Bill G * Last Release (named Johnson, dated Dec 2021), comments Steve, it is done and it was more work than previously thought. Did update the process notes. Yosef, for the announcement upload to Hyperspace in our release notes was a reference to another announcement but could not resolve it. This announcement is currently on Hyperspace at https://hyperspace.uni-frankfurt.de/2021/12/09/new-einstein-toolkit-release… . The contact is not listed correctly, says Yosef and not maintainers. We found the previous release on Hyperspace at https://hyperspace.uni-frankfurt.de/2021/05/31/the-twenty-second-release-of… . Seems to be Wordpress as the host. The other announcement was too long, so Roland asked that Yosef decrease it. Steve, maybe one can also put in the DOI, but Luciano Rezzola (runs the site) does not want links (maybe?). Zenodo reference is https://doi.org/10.5281/zenodo.5770803 . Steve, would like to have Kranc and NRpy build and for this release and they did not. Zach, working on updating and modernizing Baikal, next release will be more consistent and seamless. Improvements to the build system, more in line with NRPy ( http://astro.phys.wvu.edu/bhathome/ ). More code re-use with the NRPy infrastructure, and automatic generation of Par files. That is what is planned. Also will check that you can run the NRPy notebook and create the generated code---Zach is working on it. ** The tutorial server Steve, it has been updated and runs the Cactus tutorial, heat equation, and wave equation notebooks without incident, also the Self Force tutorial. Should be able to get at it in docker, the Docker hub is Einstein Toolkit. And Steve has access to it and can give other people access to the Jupyter ET repo and into the directory and build the images there. He would like to add some documentation---you need to build base, notebook, cilogin (makes the system we have), and CYOL, (create your own login for the tutorial servers). Each uses a "From" the previous step and one can push to the repo and it should all go. Need permissions to push it there. Could make a script on the tutorial server, so that when a new version appears it could install it automatically....otherwise one must deal with LSU's CCT IT system. You can create your own server based on CILogin step. * SPEC Benchmark Need to ping them about the status of ET. Have answered all their email. SPEC at https://www.spec.org/ , last posted results are from 2017. * ET Website Glitch Roland, found a glitch on the ET Website. If you go to the BNS gallery page with http and not https, you can see the movie frame and the play triangle, but it will not play. Need to check out server logs. https://einsteintoolkit.org/gallery/bns/index.html Shows the movie. http://einsteintoolkit.org/gallery/bns/index.html Shows only the movie frame. * Bug Hunt / Hack-a-thon Bill, sometime in early January will post about having a "hack-a-thon" / "bug hunt" to clear as many Open Tickets as possible. Focus of this one would be clear tickets. * 2017 ET Summer school and workshop Some of us were interviewed by a graduate student about using code, and he published his paper, https://arxiv.org/abs/2109.10971 . Roland would like to ask him to talk about it to us. * AOB Peter, talked to Barry Wardell and he will try to have a European ET Workshop in UCD, Dublin, looking for funding. Steve, Supercomputing 2021 ( https://sc21.supercomputing.org/ ) was great. Best Tech Party ever. Was in the St. Louis City Museum that is kind of a fun house, bizarre artwork inside a building, with lots of tunnels and slides weaving through the building, 10 story slide on the outside. Steve, we talked about a regular monthly talk at our telecons. Reminds us that we want to do that. Atul, announcement that he started at RIT...finishing Notre Dame PhD and continuing as an RIT post-doc. He thanks the ET community for all the help and support. * [https://www.einsteintoolkit.org/tools/unanswered.php unanswered question on mailing list] * [https://bitbucket.org/einsteintoolkit/tickets/issues?status=open&status=new… open tickets sorted by update time] Gallery examples should get closed...and then are re-opened for the next release. If it is your ticket, click the Resolve button (upper right) and give it some text. * [https://bitbucket.org/einsteintoolkit/tickets/issues?status=new&status=open… tickets ready for review] The Chair and Minute taker for the next meeting, Jan 6th, chair Yosef, and minutes Roland. ************************************************************************************* *** *** *** Have a great break, Merry Christmas, Happy Holidays, and a great New Year! *** *** *** ************************************************************************************* -- ===================================== William Gabella Research Assistant Professor Department of Physics and Astronomy Vanderbilt University Nashville, TN USA b.gabella(a)vanderbilt.edu (o) 615-343-2713

1 0

GRHydro_c2p_failure_problem
by 白济民 16 Dec '21

16 Dec '21

Hi, I've successfully run the gallery BNS simulation and as a second step, I try to replace the single polytropic EoS with the 7-segment SLyPP one mentioned in the 2021_ETK_Workshop. Mainly I made two substitutions in the original par file, I changed the Thorn EOS_Omni parameters into the hybrid form and I changed the type of the GRHydro_eos_table into "Hybrid". However, the simulation aborted due to the GRHydro_c2p_failure, as can be seen in the output .err file: ---------------------------------------------------------------------- while executing schedule bin CCTK_POSTSTEP, routine GRHydro::check_GRHydro_C2P_failed in thorn GRHydro, file GRHydro_Con2Prim.F90:1787: -> xyz location: 14.06250 -5.625000 1.125000 ---------------------------------------------------------------------- I noticed that the failure point is near the outer layer of the neutron star. For clearance, I uploaded my par file and the err file. I wonder how to address this issue and I'm looking forward to your help. Yours, Jimmy

2 1

Einstein Toolkit Meeting Reminder
by users＠einsteintoolkit.org 15 Dec '21

15 Dec '21

Hello, Please consider joining the weekly Einstein Toolkit phone call at 9:00 am US central time on Thursdays. For details on how to connect and what agenda items are to be discussed, use the link below. https://docs.einsteintoolkit.org/et-docs/Main_Page#Weekly_Users_Call --The Maintainers

1 0

Agenda for Thursday's Meeting
by users＠einsteintoolkit.org 13 Dec '21

13 Dec '21

Please update the Wiki with agenda items for Thursday's meeting. Thanks! https://docs.einsteintoolkit.org/et-docs/meeting_agenda --The Maintainers

1 0

2026

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

Users December 2021