Users December 2019

users@lists.einsteintoolkit.org

8 participants
12 discussions

Boundary issues solving wave equations with MoL
by Severin Frank 22 Feb '20

22 Feb '20

Dear all, I'm trying to solve a wave equation that describes radial oscillations of a TOV star, depending on radius r and t in Llama multipatch coordinates (Thornburg04). The equation has a structure of Ẍ=AX′+BX″+CX\ddot{X} = A X' + B X'' + C X I rewrote it as set of coupled equations to evolve it with the MoL thorn: X˙=Π H˙=Π′\dot{H} = \Pi' Π˙=AH+BH′+CX\dot{\Pi} = AH + B H' +C X or as noted in my code as it is attached: Pidot = A*H + B*drH + C*Xi Hdot = drPi Xidot = Pi For MoL registered as evolved variables are Xi with rhs Xidot, Pi with rhs Pidot and H with rhs Hdot. The spatial derivatives drH and drPi are also calculated during the schedule of MoL_CalcRHS in my thorn. Boundary conditions are applied for r>R and an interval near the TOV radius r=R. if (grid_r(i,j,k) >= (TOV_surface - rprec) .AND. grid_r(i,j,k) <= (TOV_surface + rprec) )then H(i,j,k) = 0 drPi(i,j,k) = 0 else if ( grid_r(i,j,k) > (TOV_surface + rprec) ) then Xi(i,j,k) = 0 Pi(i,j,k) = 0 H(i,j,k) = 0 end if if (grid_r(i,j,k) > (TOV_surface + rprec) )then Xidot(i,j,k) = 0 Hdot(i,j,k) = 0 Pidot(i,j,k) = 0 else if (grid_r(i,j,k) >= (TOV_surface - rprec) .AND. grid_r(i,j,k) <= (TOV_surface + rprec) )then Hdot(i,j,k) = 0 Pidot(i,j,k) = B(i,j,k)*drH(i,j,k) + C(i,j,k)*Xi(i,j,k) end if Problems occur close to the surface of the star. My evolved variables start do diverge close to the surface after a few iterations. Looking at the data it seems that the divergence is founded by the values of H. I tried following things to encircle the issues: * If I put Pidot = B*drH + C*Xi the values seem to be ok, whereas for Pidot = A*H the divergence appears. The A-factor only amplifies this behavior. If I put Pidot = H it behaves the same, but much slower. * If I use drXi instead of H (as it is commented out), it does not make any difference. * If I enlarge the size of the interval (e.g. TOV_surface + 5*rprec) the same divergence appears, but shifted towards grid points next to the interval. Also the divergence appears a few iterations later. * If I change the drXi or H values close to the surface manually (e.g. using a backsided differentiation, or put specific values by hand) it also shifts the divergence (like above). So far I don't know how to remedy this issue, but maybe I'm overlooking something obvious. Does anyone have an idea on what I could try? Thanks a lot! Best regards and merry Christmas, Severin Frank

3 4

Symbol has no IMPLICIT type
by Hamilton, Maria 18 Dec '19

18 Dec '19

Hi, I'm trying to compile nsnstohmns.th on WVU's thorny-flat cluster with the following modules: Currently Loaded Modulefiles: 1) lang/gcc/8.2.0 2) parallel/openmpi/3.1.4_gcc82 3) parallel/hwloc/2.0.3_gcc82 4) lang/python/pypy2.7-7.1.1-portable and the configuration provided by Zach: CC=gcc CFLAGS=-O3 -fopenmp -march=skylake-avx512 -std=gnu99 CPP=cpp CPPFLAGS=-DMPICH_IGNORE_CXX_SEEK CPP_DEBUG_FLAGS=-DCARPET_DEBUG CXX=g++ CXXFLAGS=-O3 -fopenmp -march=skylake-avx512 -std=gnu++11 CXX_DEBUG_FLAGS=-O0 C_DEBUG_FLAGS=-O0 C_LINE_DIRECTIVES=yes DEBUG=no F77=gfortran F77FLAGS=-O3 -fopenmp -march=skylake-avx512 F77_DEBUG_FLAGS=-O0 -check bounds -check format F90=gfortran F90FLAGS=-O3 -fopenmp -march=skylake-avx512 F90_DEBUG_FLAGS=-O0 -check bounds -check format FPP=cpp FPPFLAGS=-traditional FPP_DEBUG_FLAGS=-DCARPET_DEBUG F_LINE_DIRECTIVES=yes HWLOC_DIR=/shared/software/parallel/hwloc/2.0.3_gcc82/ HWLOC_EXTRA_LIBS=open-pal numa LDFLAGS=-rdynamic MPI_INC_DIRS=/shared/software/parallel/openmpi/3.1.4_gcc82/include/ MPI_LIBS=openmpi MPI_LIB_DIRS=/shared/software/parallel/openmpi/3.1.4_gcc82/lib/ PTHREADS_DIR=NO_BUILD THORNLIST=../nsnstohmns.th VECTORISE=yes VECTORISE_ALIGNED_ARRAYS=no VECTORISE_INLINE=no The error I am getting is: COMPILING EinsteinEvolve/GRHydro/src/Utils.F90 /gpfs20/users/mbh0012/Cactus/configs/NSB/build/GRHydro/Utils.f90:92:10: INTEGER cctk_lsh (cctk_dim) 1 Error: Cray pointer declaration at (1) requires -fcray-pointer flag /gpfs20/users/mbh0012/Cactus/configs/NSB/build/GRHydro/Utils.f90:96:8: INTEGER cctk_lbnd (cctk_dim) 1 Error: Symbol ‘pg11’ at (1) has no IMPLICIT type; did you mean ‘g11’? /gpfs20/users/mbh0012/Cactus/configs/NSB/build/GRHydro/Utils.f90:97:8: # 35 1 Error: Symbol ‘pg12’ at (1) has no IMPLICIT type; did you mean ‘g12’? /gpfs20/users/mbh0012/Cactus/configs/NSB/build/GRHydro/Utils.f90:98:8: integer, parameter :: cctki_use_cctk_lbnd = kind(cctk_lbnd) 1 Error: Symbol ‘pg13’ at (1) has no IMPLICIT type; did you mean ‘g13’? /gpfs20/users/mbh0012/Cactus/configs/NSB/build/GRHydro/Utils.f90:99:8: # 35 1 Error: Symbol ‘pg22’ at (1) has no IMPLICIT type; did you mean ‘g22’? /gpfs20/users/mbh0012/Cactus/configs/NSB/build/GRHydro/Utils.f90:100:8: INTEGER cctk_ubnd (cctk_dim) 1 Error: Symbol ‘pg23’ at (1) has no IMPLICIT type; did you mean ‘g23’? /gpfs20/users/mbh0012/Cactus/configs/NSB/build/GRHydro/Utils.f90:101:8: # 35 1 Error: Symbol ‘pg33’ at (1) has no IMPLICIT type; did you mean ‘g33’? make[3]: *** [/gpfs20/users/mbh0012/Cactus/configs/NSB/config-data/make.config.rules:412: Utils.F90.o] Error 1 make[2]: *** [/gpfs20/users/mbh0012/Cactus/lib/make/make.thornlib:113: make.checked] Error 2 make[1]: *** [/gpfs20/users/mbh0012/Cactus/lib/make/make.configuration:181: /gpfs20/users/mbh0012/Cactus/configs/NSB/lib/libthorn_GRHydro.a] Error 2 make: *** [Makefile:260: NSB] Error 2 Is there a work-around this problem? thank you, Maria _______________________ Maria C. Babiuc Hamilton, Ph.D. Professor, Department of Physics College of Science, Marshall University, 1 John Marshall Drive, Huntington, WV, 25755 Room S 257, Phone: (304)696-2754

2 2

Fwd: PhD position at Potsdam University
by Witek, Helvi 18 Dec '19

18 Dec '19

FYI =========================================== Dr. Helvi Witek Royal Society University Research Fellow Theoretical Particle Physics and Cosmology. Department of Physics King's College London =========================================== -------- Forwarded Message -------- Subject: PhD position at Potsdam University Date: Wed, 18 Dec 2019 15:38:25 +0100 From: Tim Dietrich <diet.tim(a)gmail.com><mailto:diet.tim@gmail.com> Dear colleagues, I would be very happy if you could forward this advertisement to interested candidates. Thank you very much and happy holidays, Tim ------------------------------------------------------------------------------------------ PhD position in Theoretical Astrophysics – University of Potsdam In February 2020 a new theoretical astrophysics group led by Tim Dietrich will be established at the University of Potsdam. The group will focus on the simulation of compact binary systems (in particular binary neutron stars), on the construction of gravitational waveform models, and the analysis of electromagnetic signatures connected to compact binary mergers. The group will have a vacant PhD position for which we ask interested candidates to send their expression of interest, which should include a description of previous experience and research topics of interest, together with their CV, and a transcript of records of university courses and grades. Additionally, please provide the contact details of two academics who could provide reference letters. The starting date is flexible but should lie within April and December 2020. Please send your documents by the 14th of February as a single pdf file to Tim Dietrich (diet.tim(a)gmail.com<https://eur03.safelinks.protection.outlook.com/?url=http%3A%2F%2Fgmail.com&…>). Further questions should be sent via email to the above address.

1 0

meeting minutes for 2019-12-12
by Haas, Roland 12 Dec '19

12 Dec '19

Present: Bill, Chris, Helvi, Maria, Peter, Roland, Sam, Steve, Zach Chair: Zach Minutes: Roland status and running of working groups: * working groups have some strings attached and will be terminated for non-activity: https://docs.einsteintoolkit.org/et-docs/ET_Working_Group_Policies_and_Proc… * many working groups have stalled, eg the performance optimization group * more conference calls seem to not be the solution to ensure more activitiy * suggestion to establish a point of contact for the working group, so that there is a person that can speak up when a question relevant to the field are brought up * working groups need to contribute to progress in their area and report on this, should have a well defined goal * amend policy to add "last updated" date to established working groups link * revive regular working group updates in weekly telecons, add round-robin working group updates to telecons * solicit updates from chairs of working groups link cactuscode.org into einsteintoolkit.org: * Steve is working on it but not yet done review policy for code improvements: * Canuda and Lean have updates with new functionality (Llama) ready * IllinoisGRMHD also undergoing updates to use NRPy+ to generate parts of its code, adding more documenation as well, next goal is to add tabulated EOS unreviewed changes: * due to the large number of "minor" and "trivial" fixes and enhancements that are outstanding, Roland is currently adopting the following deviation from the regular ET policy of every change needed review: (1) changes are proposed as pull requests and marked with "Please review" as usual, (2) if a change is trivial or minor and and is not commented on wihtin 2 weeks, add "Unless objected I will push this change after X" with X being 2 weeks in the future. Changes that fix physics bugs or are major will be announced explicitly and not applied without review. This gives 4 weeks of time before an unreviewed change is applied to master. * as a reminder: documentation changes are usually exempt from review future chairs: Steve (Jan 9), Helvi (Jan 16) future minute takers: Bill (Jan 9), Steve (Jan 16) working group update: Optimization (Jan 9), IGRMHD (Jan 16) No unanswered questions, no interesting new tickets. 22 open tickets in review state: https://bitbucket.org/einsteintoolkit/tickets/issues?status=new&status=open… There will be no calls over winter break. The next call will be Jan 9, 2020. Yours, Roland -- My email is as private as my paper mail. I therefore support encrypting and signing email messages. Get my PGP key from http://pgp.mit.edu .

1 0

Agenda for Thursday's Meeting
by users＠einsteintoolkit.org 09 Dec '19

09 Dec '19

Please update the Wiki with agenda items for Thursday's meeting. Thanks! https://docs.einsteintoolkit.org/et-docs/meeting_agenda --The Maintainers

1 0

submiting jobs in a cluster with the pbs queue
by Erik Rodrígo Jiménez Vázquez 06 Dec '19

06 Dec '19

Dear all I need some help for submitting a job in a cluster. In the cluster which I have access, it uses a PBS queue for requesting resources. I want to submit a job in a queue called mpi, but if I execute the simfactory script, it only runs in the node of the current session. Does anyone have the instructions for doing this? Right now I have successfully compiled the Einstein Toolkit in my user account in the cluster (in only one node). Thank you for your help Erik

2 1

Meeting Minutes 2019-12-05
by Witek, Helvi 05 Dec '19

05 Dec '19

* Present: Maria Babiuc-Hamilton, Chris Evans, Peter Diener, Zach Etienne, Antoni Ramos Buades, Bill Gabella, Roland Haas, Steve Brandt, HW * Chair / Minute taker: Peter / Helvi * progress in "NRPy+-fying" IllinoisGRMHD [Zach + student] ** goal: make code more flexible to easily add, e.g., new EoS, extendend field equations, etc ** first successful simulation of BNSs with new implementation * email from (some) GMail servers rejected by mailing list a potential spam [RH] ** spam filter rejects some gmail servers; affects colleagues whose institutions use gmail for university addresses ** possibly add info about this issue on contact website to contact us ** investigate setup of mailing list server * link release doi to inspires [HW] ** Steve added Inspire HEP community to those listed on the ET Zenodo site [https://zenodo.org/record/3522086#.XegxU1lKg5k] so that it appears in inspires database * Peter planning to update the McLachlan documentation shortly * Roland found fix for failing tests on Jenkins server * Roland suggests mechanism to add requested and suggested citations for thorns employed during a simulation at runtime; see [https://bitbucket.org/einsteintoolkit/tickets/issues/2254/let-cactus-thorns…] * Next week: ** discussion on status and running of working groups [ZE] ** link cactuscode.org into einsteintoolkit.org [SB] -- =========================================== Dr. Helvi Witek Royal Society University Research Fellow Theoretical Particle Physics and Cosmology. Department of Physics King's College London ===========================================

1 0

Einstein Toolkit Meeting Reminder
by users＠einsteintoolkit.org 04 Dec '19

04 Dec '19

Hello, Please consider joining the weekly Einstein Toolkit phone call at 9:00 am US central time on Thursdays. For details on how to connect and what agenda items are to be discussed, use the link below. https://docs.einsteintoolkit.org/et-docs/Main_Page#Weekly_Users_Call --The Maintainers

1 0

Communicated region and active part of the domain
by Dominik Guido Rhiem 04 Dec '19

04 Dec '19

Dear all, in order to decrease the necessary calculations for a simulation, I tried to lower the resolution and limit the domain size of the simulation. However, I get the error that the communicated region has to be contained within the active part of the domain; I don't see where the issue is in my parameter file, though. I paid attention that the region size is an integer multiple of the resolution and to not extract e.g. gravitational waves from surfaces at a larger radius than the region size. I have attached both the error file, the output file and the parameter file. (Note that I also use some non-standard thorns, e.g. for seeding magnetic fields.) Sincerely, Dominik Rhiem

3 4

Agenda for Thursday's Meeting
by users＠einsteintoolkit.org 02 Dec '19

02 Dec '19

Please update the Wiki with agenda items for Thursday's meeting. Thanks! https://docs.einsteintoolkit.org/et-docs/meeting_agenda --The Maintainers

1 0

2026

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

Users December 2019