Users January 2020

users@lists.einsteintoolkit.org

15 participants
20 discussions

Boundary issues solving wave equations with MoL
by Severin Frank 22 Feb '20

22 Feb '20

Dear all, I'm trying to solve a wave equation that describes radial oscillations of a TOV star, depending on radius r and t in Llama multipatch coordinates (Thornburg04). The equation has a structure of Ẍ=AX′+BX″+CX\ddot{X} = A X' + B X'' + C X I rewrote it as set of coupled equations to evolve it with the MoL thorn: X˙=Π H˙=Π′\dot{H} = \Pi' Π˙=AH+BH′+CX\dot{\Pi} = AH + B H' +C X or as noted in my code as it is attached: Pidot = A*H + B*drH + C*Xi Hdot = drPi Xidot = Pi For MoL registered as evolved variables are Xi with rhs Xidot, Pi with rhs Pidot and H with rhs Hdot. The spatial derivatives drH and drPi are also calculated during the schedule of MoL_CalcRHS in my thorn. Boundary conditions are applied for r>R and an interval near the TOV radius r=R. if (grid_r(i,j,k) >= (TOV_surface - rprec) .AND. grid_r(i,j,k) <= (TOV_surface + rprec) )then H(i,j,k) = 0 drPi(i,j,k) = 0 else if ( grid_r(i,j,k) > (TOV_surface + rprec) ) then Xi(i,j,k) = 0 Pi(i,j,k) = 0 H(i,j,k) = 0 end if if (grid_r(i,j,k) > (TOV_surface + rprec) )then Xidot(i,j,k) = 0 Hdot(i,j,k) = 0 Pidot(i,j,k) = 0 else if (grid_r(i,j,k) >= (TOV_surface - rprec) .AND. grid_r(i,j,k) <= (TOV_surface + rprec) )then Hdot(i,j,k) = 0 Pidot(i,j,k) = B(i,j,k)*drH(i,j,k) + C(i,j,k)*Xi(i,j,k) end if Problems occur close to the surface of the star. My evolved variables start do diverge close to the surface after a few iterations. Looking at the data it seems that the divergence is founded by the values of H. I tried following things to encircle the issues: * If I put Pidot = B*drH + C*Xi the values seem to be ok, whereas for Pidot = A*H the divergence appears. The A-factor only amplifies this behavior. If I put Pidot = H it behaves the same, but much slower. * If I use drXi instead of H (as it is commented out), it does not make any difference. * If I enlarge the size of the interval (e.g. TOV_surface + 5*rprec) the same divergence appears, but shifted towards grid points next to the interval. Also the divergence appears a few iterations later. * If I change the drXi or H values close to the surface manually (e.g. using a backsided differentiation, or put specific values by hand) it also shifts the divergence (like above). So far I don't know how to remedy this issue, but maybe I'm overlooking something obvious. Does anyone have an idea on what I could try? Thanks a lot! Best regards and merry Christmas, Severin Frank

3 4

NSNSTOHMNS(Gallery example) Plotting issues
by Atul Kedia 17 Feb '20

17 Feb '20

Dear all, I am trying to plot and make movie for the NSNSTOHMNS Gallery example using the support scripts provided. On using the VisIt support scripts as given in the example, I do get some figures named [rho_000000000.png, ... , rho_000000231.png], but they are all usually black screens or look like tv white noise. Details of the simulation: I ran it on a cluster with 8 cores. My run data for mp_Psi4_l2_m2_r300.00.asc and rho.xy.h5 and the sample Results given in the gallery example are of the same order of magnitude(although not identical). So I doubt any problems on the data generation end. I am using VisIt version 3.0.2 and FFMPEG version 4.2.1, and am loading python/3.7.3. I get these when I run the command visit -cli -nowin -s rho.movie.py - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - ... ... VisIt: Message - Rendering window 1... VisIt: Error - The compute engine running on crcfe01.crc.nd.edu has exited abnorm ally. VisIt: Error - Scalable Render Request Failed (VisItException) viewer: Obtained null data reader for rendered image for engine crcfe01.crc.nd.ed u. Running: engine_ser3.0.2 -idle-timeout 480 -host 127.0.0.1 -port 5600 VisIt: Message - Rendering window 1... VisIt: Message - Saving window 1... VisIt: Message - Saved rho_000000000.png ... ... s/nsnstohmns_in_saeng/output-0000/nsnstohmns/rho.xy.h5". VisIt: Message - Closed the compute engine on host crcfe01.crc.nd.edu. ffmpeg version 4.2.1 Copyright (c) 2000-2019 the FFmpeg developers built with gcc 4.8.5 (GCC) 20150623 (Red Hat 4.8.5-39) configuration: --prefix=/afs/crc.nd.edu/group/mathews/tools/ffmpeg/4.2.1 --disable-x86asm libavutil 56. 31.100 / 56. 31.100 libavcodec 58. 54.100 / 58. 54.100 libavformat 58. 29.100 / 58. 29.100 libavdevice 58. 8.100 / 58. 8.100 libavfilter 7. 57.100 / 7. 57.100 libswscale 5. 5.100 / 5. 5.100 libswresample 3. 5.100 / 3. 5.100 Unrecognized option 'crf'. Error splitting the argument list: Option not found ffmpeg version 4.2.1 Copyright (c) 2000-2019 the FFmpeg developers built with gcc 4.8.5 (GCC) 20150623 (Red Hat 4.8.5-39) configuration: --prefix=/afs/crc.nd.edu/group/mathews/tools/ffmpeg/4.2.1 --disable-x86asm libavutil 56. 31.100 / 56. 31.100 libavcodec 58. 54.100 / 58. 54.100 libavformat 58. 29.100 / 58. 29.100 libavdevice 58. 8.100 / 58. 8.100 libavfilter 7. 57.100 / 7. 57.100 libswscale 5. 5.100 / 5. 5.100 libswresample 3. 5.100 / 3. 5.100 Input #0, image2, from 'rho_%09d.png': Duration: 00:00:09.28, start: 0.000000, bitrate: N/A Stream #0:0: Video: png, rgb24(pc), 640x640, 25 fps, 25 tbr, 25 tbn, 25 tbc Unknown encoder 'theora' - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Any help would be greatly appreciated. Thank you, best regards, Atul Kedia Physics Graduate student University of Notre Dame.

4 9

arXiv-1111.3344 collapse example errors
by Justin Verde 17 Feb '20

17 Feb '20

Hi everyone, I'm a new user to the toolkit and have been trying to run the TOV collapse example from arXiv-1111.3344 on my home system with 4 cores. I keep getting the following errors before the job crashes: =============================================== Error: Thorn CartesianCoordinates not found Error: Thorn corecollapsecontrol not found Error: Thorn TriggerTerminationManual not found Activation failed - 3 errors in activation sequence WARNING level 0 from host euler process 0 in thorn Cactus, file /home/verde029/ETK/Cactus/configs/sim/build/Cactus/main/SetParams.c:93: -> CCTKi_SetParameter: Error at line 66 in parameter file /home/verde029/simulations/collapse_crash_test/output-0000/tov_collapse_vc_l3.par while activating thorns -------------------------------------------------------------------------- MPI_ABORT was invoked on rank 0 in communicator MPI_COMM_WORLD with errorcode 1. NOTE: invoking MPI_ABORT causes Open MPI to kill all MPI processes. You may or may not see output from other processes, depending on exactly when Open MPI kills them. ============================================ I can't find anything on these thorns through the thorn documentation guide. Have the thorns or par file been updated, or is there a quick workaround to update the file? I appreciate any help! -- Kindest Regards, Justin Verde

3 3

Getting too many threads per process started on "remote" nodes
by Shoup, Anthony 31 Jan '20

31 Jan '20

Hi all, I am running ETK (2019_10) on a home built cluster consisting of two nodes (8 cores, 16 threads, 64GB 4.3 GHz each). I just finished my second node and am trying to run a simulation (BBHMedRes) over both nodes. For starters I am just running one process (one thread per process) on each node. When I execute my simfactory submit command, I get one process with one thread on the node I submitted the simulation on. However, I get one process with 16 threads on the second node which I don't want. When I run on just the first node, the number of processes and threads per process I get are just what I specify in the simfactory submit command. If I submit the simulation on the second node and just run on the second node I get processs/threads just what I specify in the simfactory submit command. Its only when I run on multiply nodes that don't get the # of processes/threads that I specify. Is there something I am doing wrong? I am using OpenMPI. Thanks for any help, Tony... Relevant data is: 1. RunScript: #!/bin/sh # This runscript is used internally by simfactory as a template during the # sim setup and sim setup-silent commands # Edit at your own risk echo "Preparing:" set -x # Output commands set -e # Abort on errors cd @RUNDIR@-active echo "Checking:" pwd hostname date echo "Environment:" export CACTUS_NUM_PROCS=@NUM_PROCS@ export CACTUS_NUM_THREADS=@NUM_THREADS@ export GMON_OUT_PREFIX=gmon.out export OMP_NUM_THREADS=@NUM_THREADS@ env | sort > SIMFACTORY/ENVIRONMENT echo "Starting:" export CACTUS_STARTTIME=$(date +%s) if [ ${CACTUS_NUM_PROCS} = 1 ]; then if [ @RUNDEBUG@ -eq 0 ]; then @EXECUTABLE@ -L 3 @PARFILE@ else gdb --args @EXECUTABLE@ -L 3 @PARFILE@ fi else mpirun --hostfile /home/mpiuser/mpi-hosts -np @NUM_PROCS@ @EXECUTABLE@ -L 3 @PARFILE@ fi echo "Stopping:" date echo "Done." 2. mpi-hosts file: localhost slots=1 RZNode2 slots=1 3. simfactory submit command: ./simfactory/bin/sim submit BBHMedRes --parfile=par/BBHMedRes.par --procs=2 --num-smt=1 --num-threads=1 --ppn-used=1 --ppn=1 --wallt ime=99:0:0 | cat 4. Machine file on first node (RZNode1): [RZNode1] # This machine description file is used internally by simfactory as a template # during the sim setup and sim setup-silent commands # Edit at your own risk # Machine description nickname = RZNode1 name = RZNode1 location = somewhere description = Whatever status = personal # Access to this machine hostname = RZNode1 aliaspattern = ^generic\.some\.where$ # Source tree management sourcebasedir = /home/Cactus optionlist = generic.cfg submitscript = generic.sub runscript = generic.run make = make -j@MAKEJOBS@ basedir = /home/mpiuser/simulations ppn = 1 # was 16 max-num-threads = 1 # was 16 num-threads = 1 # was 16 nodes = 2 submit = exec nohup @SCRIPTFILE@ < /dev/null > @RUNDIR@/@SIMULATION_NAME@.out 2> @RUNDIR@/@SIMULATION_NAME@.err & echo $! getstatus = ps @JOB_ID@ stop = kill @JOB_ID@ submitpattern = (.*) statuspattern = "^ *@JOB_ID@ " queuedpattern = $^ runningpattern = ^ holdingpattern = $^ exechost = echo localhost exechostpattern = (.*) stdout = cat @SIMULATION_NAME@.out stderr = cat @SIMULATION_NAME@.err stdout-follow = tail -n 100 -f @SIMULATION_NAME@.out @SIMULATION_NAME@.err 5. Machine file on second node (RZNode2): [RZNode2] # This machine description file is used internally by simfactory as a template # during the sim setup and sim setup-silent commands # Edit at your own risk # Machine description nickname = RZNode2 name = RZNode2 location = somewhere description = Whatever status = personal # Access to this machine hostname = RZNode2 aliaspattern = ^generic\.some\.where$ # Source tree management sourcebasedir = /home/ET_2019_10 optionlist = generic.cfg submitscript = generic.sub runscript = generic.run make = make -j@MAKEJOBS@ basedir = /home/mpiuser/simulations ppn = 1 max-num-threads = 1 num-threads = 1 nodes = 1 submit = exec nohup @SCRIPTFILE@ < /dev/null > @RUNDIR@/@SIMULATION_NAME@.out 2> @RUNDIR@/@SIMULATION_NAME@.err & echo $! getstatus = ps @JOB_ID@ stop = kill @JOB_ID@ submitpattern = (.*) statuspattern = "^ *@JOB_ID@ " queuedpattern = $^ runningpattern = ^ holdingpattern = $^ exechost = echo localhost exechostpattern = (.*) stdout = cat @SIMULATION_NAME@.out stderr = cat @SIMULATION_NAME@.err stdout-follow = tail -n 100 -f @SIMULATION_NAME@.out @SIMULATION_NAME@.err [https://email.osu.edu/owa/attachment.ashx?id=RgAAAAAb%2fHy0wVvTSoHQx8OJXAaL…] Anthony Shoup PhD, Senior Lecturer College of Arts & Sciences, College of Engineering Departments of Physics, Astronomy, EEIC 315 Science Bldg. | 4250 Campus Dr. Lima, OH 45807 419-995-8018 Office | 419-516-2257 Mobile shoup.31(a)osu.edu<https://email.osu.edu/owa/redir.aspx?C=j5WpnJiBk0W5oVlCbtvB-xiCkA_lbdEIi9hl…> osu.edu<https://email.osu.edu/owa/redir.aspx?C=j5WpnJiBk0W5oVlCbtvB-xiCkA_lbdEIi9hl…>

2 2

Meeting Minutes for 2020-01-30
by Hamilton, Maria 30 Jan '20

30 Jan '20

Einstein Toolkit Zoom Meeting 10:00 am, Thursday Jan. 30, 2020 Present: Steven, Phillip, Zach, Maria, Peter, Roland, Vasilli, Bruno, Christopher, Eric, Liu, Yosef Call Chair: Peter Minute Taker: Maria Peter opens the meeting. Steven reports on his work on removing the old cacuscode.org web page, and creating a new version on the ETK web page (http://einsteintoolkit.org/cactuscode.html). He requests critiques in order to proceed forward. He is the main organizer of the upcoming US ETK Workshop at LSU and also requests critique on the announcement (https://docs.einsteintoolkit.org/et-docs/LSU_Workshop_2020) Phillips brings news about the EU ETK Workshop, to take place in Amsterdam this year. As main organizer, he requests input and suggestions on the best dates. Bruno, as one of the previous organizers, offers helpful suggestions regarding dates (between June and September), as well as number of participants, recommending as a preferable date when school is in session, to attract students. Roland recommends two dates to avoid: during the LIGO meeting, and the LSU ETK workshop. Phillips reported venue constraints, especially when the school is in session, and will report back next week. Next on the agenda was Vasilli's update on the Spherical NR working group. (link slides?). After his presentation, Bruno asked if is it possible to start evolving the data in cartesian coordinate, and then to transfer to spherical. Vasilli answered that as for now, there is no module to do that, but in the future, if there is need for it, it can be done, similar to the way cartesian Initial Data is read in the spherical code. Maria asked for details on the interpolation, and Vasilli explained. Roland pointed towards a link (address?) that shows how ETK handles the interpolation from cartesian to any grid. Roland asked if the vector-potential implementation in spherical coordinates can be used in plain cartesian coordinates in GRHydro. Vasilli confirmed that it can be done, although the outer boundary conditions need to be changed, and the right hand side correction due to spherical coordinates needs to be zero. He also said that he did not look in detail into the mesh refinement. Next, Roland gave a report on the ETK status, all is updated with the exception of the SelfForce. He requested a1D wave train example to be included in the gallery. Peter recommended to add the request for features as an open issue, then reviewed the agenda and what was discussed regarding the Cactus and the LSU workshop web pages. He recommended to send comments directly to Steve. Lastly, Roland reported that, as at his call nobody stepped forward offering to coordinate for the next ETK release, he will undertake this job. Peter suggested the idea of finding a volunteer to help Roland and to learn, in order to do this next time. At Zach's suggestion, the rota for the next two meetings was established as well. Roland reported that nothing has changed regarding with the tickets, email list and open issues. Peter concluded that no further items are left to discuss, and ended this meeting. _______________________ Maria C. Babiuc Hamilton, Ph.D. Professor, Department of Physics College of Science, Marshall University, 1 John Marshall Drive, Huntington, WV, 25755 Room S 257, Phone: (304)696-2754

1 0

Einstein Toolkit Meeting Reminder
by users＠einsteintoolkit.org 29 Jan '20

29 Jan '20

Hello, Please consider joining the weekly Einstein Toolkit phone call at 9:00 am US central time on Thursdays. For details on how to connect and what agenda items are to be discussed, use the link below. https://docs.einsteintoolkit.org/et-docs/Main_Page#Weekly_Users_Call --The Maintainers

1 0

Agenda for Thursday's Meeting
by users＠einsteintoolkit.org 27 Jan '20

27 Jan '20

Please update the Wiki with agenda items for Thursday's meeting. Thanks! https://docs.einsteintoolkit.org/et-docs/meeting_agenda --The Maintainers

1 0

ET release coordinator elections
by Haas, Roland 23 Jan '20

23 Jan '20

Hello all, for the upcoming ET release ET_2020_04 there is no designated release coordinator yet. In the past Steve has been handling this, though with this release the idea is to spread out the tasks a bit more. If you are interested or know someone who may be interested in coordinating the next release, please add their names to: https://docs.einsteintoolkit.org/et-docs/Release_Process before the next ET call (Jan 23rd 2020). We will hold a vote at the next ET call to select a release coordinator. The release coordinator tasks are listed here: https://docs.einsteintoolkit.org/et-docs/Release_Process and a schedule is usually posted on https://docs.einsteintoolkit.org/et-docs/Release_Details Yours, Roland -- My email is as private as my paper mail. I therefore support encrypting and signing email messages. Get my PGP key from http://pgp.mit.edu .

1 1

Re: [Users] Meeting Minutes for 2020-01-23
by Witek, Helvi 23 Jan '20

23 Jan '20

Einstein Toolkit Meeting on 23 January 2020 * present: Alois Schaffarczyk, Bill Gabella, Christopher Evans, Leo Werneck, Liu Haoyang, Maria Babiuc-Hamilton, Roland Haas, Zach Etienne * status of proposed new codes for the next release: * new TOV solver created with NRPy+ * c- and python-based version * includes a broader range of EoS, including piece-wise polytrops * could possibly replace current TOV solver in ET * Action item: create ticket for code review; ensure that it is backwards compatible with current TOV solver * self-force code -> next week * ET EU meeting status -> next week * elect release coordinator for next release (2020_04) [RH] * Please volunteer to coordinate the next release! * elect coordinator next week * Update on "Cosmology and Particles" Working Group * currently 19 members with diverse scientific background covering cosmology, modified gravity, high energy physics and numerical relativity * Activities: * monthly calls * discussion and presentation of dedicated topics to foster new collaborations * development of new thorns and code revisions; benchmark between different codes * Current status: * group has been dormant * Please step forward if you would like to steps forward as co-chair! * Open tickets * thread safety in Cactus; see ticket https://bitbucket.org/einsteintoolkit/tickets/issues/2326/cctk_vinfo-is-not… * Erik fixed bug in CCTK_TraverseString; see ticket https://bitbucket.org/einsteintoolkit/tickets/issues/2317/cctk_traversestri… -- --------------------------------------------- Dr. Helvi Witek Assistant Professor Department of Physics University of Illinois at Urbana-Champaign ---------------------------------------------

1 0

Einstein Toolkit Meeting Reminder
by users＠einsteintoolkit.org 22 Jan '20

22 Jan '20

1 0

2026

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

Users January 2020