Users February 2017

users@lists.einsteintoolkit.org

23 participants
33 discussions

meeting minutes for 2017-02-13
by Roland Haas 13 Feb '17

13 Feb '17

Present: Eloisa, Frank, Gwyneth, Ian, Yosef, Roland, Steve, Charalampos, Steve CarpetRegrid2: * regrid_every is the first condition checked, it only ever does anything if the iteraton is a multiple of it * movement threshold is an additional condition that prevents grid movement until the grids have moved a minimum distance * there is not well known optimal regridding frequency. One consideration would be to regrid when all timelevels are "in sync" ie with the same frequency that the coarsest level steps so that there is no interpolation in time. Another competing consideration is that the black holes should not have moved too far which gives an upper limit. A reason to use regrid_every is also if the positions are not known at every timestep for example if AHFinderDirect is used. With PunctureTracker is used then the information is known at each time. * the dominant effect is to regrid frequently enough to track the black holes moving * Courant factor is ratio between time step and space step on the coarsest grid, to compute the courant factor on the finest grid involves the time refinement factor * often one sees that the time refinement factors are set 1,1,2,4,8,... instead of 1,2,4,8 which is the default. This is due to an instability in the shift evolution equations. The group in Jena found this to be important (there is a paper by Bernd Bruegmann et al), there is a paper by Erik Schnetter that gives reason for this. It is likely still required for purely Cartesian grids, for Llama (curvilinear) grids one can get away without doing so * metric_timelevels is set to 3 for AHFinderDirect setting the same for lapse an shift does not do harm but there is not directly obvious reason why one would need it * computing the energy carried of by gravitational waves is done by computing the WeylScalar on a sphere and then do the integral yourself. There are a number of analysis packages that help doing so eg SimulationTools (google find it) by Ian Hinder for Mathematica or possibly pyCactusET. The list of tools is available here: https://docs.einsteintoolkit.org/et-docs/Analysis_and_post-processing * some of GW extraction is shown in the gallery: http://einsteintoolkit.org/about/gallery/gw150914 and Ian will send instructions * if seeing NaNs when running on many nodes or other errors this is a bug and should at most lead to an error message. static_tov uses a very small grid and we would suggest using a higher resolution grid. A gut feeling would say that anything above 27 MPI ranks is difficult. The issue seems to happen for 40 MPI ranks. Website: * comments from Ian via email * comments from Roland, who will incorporate them into the website repo: https://knarrff@bitbucket.org/einsteintoolkit/www.git MHD workshop: * https://docs.einsteintoolkit.org/et-docs/2017_MHD_Workshop has the wiki page * Roland will write a summary in an email ET fails to build right now * there is an unsupported OMP pragma in MoL, seems like a compiler specific issue * Roland suggests to check if all compilers on our "usual" clusters support "OMP simd" yet GetComponents: * https://trac.einsteintoolkit.org/ticket/1916 contains a parallelization for GetComponents please give it a try to see if you can make it break Yours, Roland -- My email is as private as my paper mail. I therefore support encrypting and signing email messages. Get my PGP key from http://keys.gnupg.net.

2 1

Einstein Toolkit Meeting Reminder
by Frank Loeffler 13 Feb '17

13 Feb '17

1 1

GRHydro_InitData - magnetic fields - neutron stars
by Daniel Gerlicher 10 Feb '17

10 Feb '17

Hi! I have a question regarding the initial data thorn of GRHydro. The documentation and ccl files didn't really help me. The following was taken from paper [1] and now I am trying to add magnetic fields to the stars. My questions are: 1) What are the units of (A) and (B) below? Is the following conversion factor correct? The star should have a field strength of 10^16 Gauss. Using the following conversion would give "0.00011973228161339154" for (A) and (B). 2) What are reasonable values for (C..F)? Can I just use the default ones given in param.ccl? Is there a reason not to use the default ones? My plan is to analyse the structure of the field after the collapse and the influence on the wave signal. > # constants, in SI > G = 6.673e-11 # m^3/(kg s^2) > c = 299792458 # m/s > M_sun = 1.98892e30 # kg > mu0 = 1.2566370614e-6 # Newton/Ampere^2 > Kb = 1.3806488e-23 # Joule/K > Mparsec = 3.08567758e19 # km > CU_to_Tesla = c**4 / M_sun / G**(1.5)* mu0**(0.5) > CU_to_Gauss = c**4 / M_sun / G**(1.5)* mu0**(0.5) * 10000 > #------------------------------------------------- > # Magentic Fields: > #------------------------------------------------- > (A) GRHydro_InitData::poloidal_A_b_1 = 0.00011973228161339154 > (B) GRHydro_InitData::poloidal_A_b_2 = 0.00011973228161339154 > (C) GRHydro_InitData::poloidal_P_cut_1 = 3.72e-6 #Non è in > percentuale!! > (D) GRHydro_InitData::poloidal_P_cut_2 = 3.72e-6 #Non è in > percentuale!! > (E) GRHydro_InitData::poloidal_n_p = 1 #Esponente di max((rho-rho_cut),0) > (F) GRHydro_InitData::poloidal_P_p = 2 #Esponente di max((P-P_cut),0) > GRHydro_InitData::Xc_1 = -13.5 > GRHydro_InitData::Xc_2 = 13.5 > GRHydro_InitData::Yc_1 = 0.0 > GRHydro_InitData::Yc_2 = 0.0 > GRHydro_InitData::Zc_1 = 0.0 > GRHydro_InitData::Zc_2 = 0.0 3) These should be fine, right? > GRHydro::transport_constraints = "yes" > GRHydro::track_divB = "yes" > GRHydro::calculate_bcom = "yes" > #GRHydro::clean_divergence = "no" > GRHydro::Grhydro_MaxNumConstrainedVars = 33 > GRHydro::GRHydro_MaxNumEvolvedVars = 10 > > HydroBase::initial_Bvec = "bin_ns_poloidalmagfield" The field is evolved and everything seems okay, I am just not sure about the units and (C..F). 4) Is there a good method to visualize the B-field? Visit + Carpet hdf5 reader + Streamlines? (seems to be "buggy") Thank you! Best regards, Daniel References: [1] "R. De Pietri, A. Feo, F. Maione, and F. Löffler, “Modeling Equal and Unequal Mass Binary Neutron Star Mergers Using Public Codes,” Phys. Rev., vol. D93, no. 6, p. 064047, 2016."

3 3

Compile question
by Guy Pitman 10 Feb '17

10 Feb '17

Hi, I am trying to compile a new Kranc thorn on a MacBook Pro (Sierra) and am getting an error COMPILING arrangements/Carpet/CarpetLib/src/prolongate_3d_real8_tvd.F90 gfortran: warning: couldn’t understand kern.osversion ‘16.1.0 /Users/Guy/Documents/Einsteintoolkit/Cactus/configs/kranc/build/CarpetLib/prolongate_3d_real8_tvd.f90:128.11: CHKIDX (i0 ,j0 ,k0 , srciext,srcjext,srckext, "src") 1 Error: Unclassifiable statement at (1) /Users/Guy/Documents/Einsteintoolkit/Cactus/configs/kranc/build/CarpetLib/prolongate_3d_real8_tvd.f90:129.11: CHKIDX (i0+2,j0+2,k0+2, srciext,srcjext,srckext, "src") 1 Error: Unclassifiable statement at (1) /Users/Guy/Documents/Einsteintoolkit/Cactus/configs/kranc/build/CarpetLib/prolongate_3d_real8_tvd.f90:170.11: CHKIDX (dstioff+i+1, dstjoff+j+1, dstkoff+k+1, dstiext,dstjext,dstke 1 Error: Unclassifiable statement at (1) make[4]: *** [prolongate_3d_real8_tvd.F90.o] Error 1 make[3]: *** [make.checked] Error 2 make[2]: *** [/Users/Guy/Documents/Einsteintoolkit/Cactus/configs/kranc/lib/libthorn_CarpetLib.a] Error 2 make[1]: *** [kranc] Error 2 make: *** [kranc-config] Error 2 I previously installed the simple example for OSX and that ran OK. The attached file contains the ThornList that I am using. Do you have any suggestions? Thank you Regards Guy

4 19

two quick question regarding parameter files and systems of coordinates
by Hamilton, Maria 08 Feb '17

08 Feb '17

Hello, I have two few quick questions that are intended to facilitate a user-friendly approach to the EinsteinToolkit in order to be used for an undergraduate project. The first question concerns the availability of out-of-the-box parameter files for the evolution of a binary system. Are the parameter files in repos/einsteinexamples/par/ maintained? If not, would it be possible to have up to date parameter files for the evolution of a binary system: bbh and neutron-bh, with the current version of the toolkit? My second question is about the physical meaning of the coordinate system. My understanding about units is that you get the physical time in seconds by the formula: t[s]=t[code]*M_binary*M_sun*G/c^3 x[m]=x[code]*M_binary*M_sun*G/c^2 Will you please confirm or infirm this? Is this measured in a plain Cartesian coordinate system centered in the center of mass of the system where x=0? Should t=0 be the time of collision or the initial time? Is this the only coordinate system used, or there is a different one in the wave zone? Thank you, Maria

2 1

Parameter file error with EHFinder
by Maitraya Bhattacharyya 08 Feb '17

08 Feb '17

Dear all, I am trying to run the EHFinder thorns on some numerical data stored at hdf files in a directory. However, running the parameter file returns an error: WARNING level 0 from host maitraya-Lenovo-Y50-70 process 0 while executing schedule bin (none), routine (no thorn)::(no routine) in thorn Cactus, file /home/maitraya/Cactus/src/main/SetParams.c:93: -> CCTKi_SetParameter: Error at line 21 in parameter file /home/maitraya/simulations/EHF/output-0000/EHF.par while activating thorns Since I am new, I may have written the parameter file incorrectly. The contents of the parameter file are below: ActiveThorns = "IOUtil ioascii iobasic" # IO parameters # IO::recover_dir = "/home/maitraya/Desktop/David/" ActiveThorns = "cartgrid3d" # grid parameters CartGrid3D::type = "coordbase" CartGrid3D::domain = "full" CartGrid3D::avoid_origin = "no" CoordBase::xmin = -5.9 CoordBase::ymin = -5.9 CoordBase::zmin = -5.9 CoordBase::xmax = 5.9 CoordBase::ymax = 5.9 CoordBase::zmax = 5.9 CoordBase::dx = 0.40689655172413808 CoordBase::dy = 0.40689655172413808 CoordBase::dz = 0.40689655172413808 ActiveThorns = "EHFinder" # EHFinder parameters ehfinder::shell_width = 1.8 ehfinder::mode = normal ehfinder::initial_f[0] = "sphere" ehfinder::initial_rad[0] = 5.0 IOBasic::outScalar_vars = " ehfinder::f ehfinder::eh_mask" IOASCII::out1D_vars = " ehfinder::f ehfinder::eh_mask ehfinder::eh_area ehfinder::eh_centroid_x ehfinder::eh_centroid_y ehfinder::eh_centroid_z ehfinder::eh_circ_eq ehfinder::eh_circ_pol " IOBasic::outInfo_vars = "ehfinder::f " Please tell me what I am doing wrong ? Maitraya.

1 0

new web page design
by Frank Loeffler 07 Feb '17

07 Feb '17

Hi The current Einstein Toolkit web pages at http://einsteintoolkit.org/ are about to be replaced with a newer design. Some content changes might go with that, but a strong emphasis is on the design change. Feel free to suggest content improvements, but please don't mix them with the design change. The new pages can currently be previewed at http://einsteintoolkit.org/new/index.html We would like to get feedback before making the switch. In particular, we are interested in experiences with all kinds of browsers and/or devices and screen sizes. If you think you've encountered a bug that relates to the layout, a screen-shot might be the best way to show what you see. You can view the source behind the web pages within the bitbucket repository https://bitbucket.org/einsteintoolkit/www (and separate, due to its size): https://bitbucket.org/einsteintoolkit/seminars If you are developer, you should have write permissions. Reporting something like a typo probably takes longer than fixing it there. If you like to make larger changes and feel unsure about them, feel free to open a pull request and a ticket. We will discuss received feedback at the next Einstein Toolkit call (Monday next week; 6th of February). Depending how things go, that might be as early as the new version goes live. Frank Löffler PS: Don't report the crappy look of images in Microsoft Edge (probably all IE as well). That's due to the crappy image interpolation MS uses, and there isn't a good way to deal with that other than using a decent browser.

5 5

Cactus Core Usage on HPC Cluster
by Gwyneth Allwright 06 Feb '17

06 Feb '17

Hi All, I'm trying to get the Einstein Toolkit installed on an HPC cluster running SLES. The trouble is that Cactus tries to use all the available processors even when I specify a smaller number (by setting ppn in my PBS script). As a test, I tried running the compilation with a parameter file that required about 5 GB of RAM. In my PBS script, I set nodes=1 and ppn=3, and then ran using openmpi-1.10.1: mpirun -hostfile $PBS_NODEFILE <ET exe> <parameter file> This resulted in the simulation running on all 24 available processors, even though I'd only requested 3. Since PBS and MPI are integrated, I was told that using -np with mpirun wouldn't help. Does anyone know how to address this issue? Thanks, Gwyneth

3 6

meeting minutes for 2017-02-06
by Roland Haas 06 Feb '17

06 Feb '17

Present: Frank, Roland, Josh, Ian, Peter, Gabrielle, Bhavesh Failing master build: * https://trac.einsteintoolkit.org/ticket/1939 * either approve this or revert all changes * ask to review by today Failing tests in release: * due to compiler/cpu issues * not quite clear which versions are affected * happens on both the Jenkins slaves as well as some clusters * on clusters we see multiple intel compiler version failing (15 seems susceptible). Seems a compiler bug since printf() seems to help. * this shows up in the ML constraints in RotatingSymmetry90/180 * Roland to check GRHydro checks * Peter to check on ML constraints Website: * comments must be submitted by next Monday, after which the new website will go online presync work: * currently have a wavetoy working * Steve asks for feedback * https://github.com/stevenrbrandt/PresyncWave which contains instructions Yours, Roland -- My email is as private as my paper mail. I therefore support encrypting and signing email messages. Get my PGP key from http://keys.gnupg.net.

2 2

Re: [Users] Einstein Toolkit Meeting Reminder
by Frank Loeffler 06 Feb '17

06 Feb '17

Hi Please consider joining the weekly Einstein Toolkit phone call at 9:30 am US central time on Mondays. As usual, you can find instructions how to join on the following web site: http://einsteintoolkit.org/community/support/ We use a Google hangout, with an URL announced on this list before the meeting. As reminder, please have a look at http://einsteintoolkit.org/new/ and voice your constructive criticism on the call or by email about these new version of the Einstein Toolkit web pages. Frank Loeffler

4 4

2026

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

Users February 2017