Users March 2016

users@lists.einsteintoolkit.org

10 participants
13 discussions

Einstein Toolkit Meeting Reminder
by Frank Loeffler 06 Mar '16

06 Mar '16

Hi Please consider joining the weekly Einstein Toolkit phone call at 9:30 am US central time on Mondays. As usual, you can find instructions how to join on the following web site: http://einsteintoolkit.org/community/support/ We use a Google hangout, with the URL: https://hangouts.google.com/call/5xt7voshalqmdcztr4zhfeox2ia Frank Löffler

1 0

TwoPunctures data storage
by Alfred Castro 02 Mar '16

02 Mar '16

Hi, I am a master student at UIB, and starting to use the Einstein Toolkit for binary black hole evolutions. I am using the TwoPunctures thorn for solving the constraints. In order to speed up tests of grid structures, I would like to just solve on the spectral grid once, store the data in a file, and reread them for different evolutions (e.g. with different finite difference grids). It looks like this is not supported right now. Has anybody done this already? Thanks, -- Alfred

6 9

Re: [Users] Problems with regridding?
by Geraint Pratten 02 Mar '16

02 Mar '16

Hi, So I've tried to change a few things to see if there was an obvious solution and there does not seem to be any improvement at all. I've added a Carpet::poison_value of 100 to the .par file. I have also tried using various different grid structures (including the example structure used in the /par/arxiv/bbh folders - though adapted to 3D without symmetry). The errors always seem to occur around 75-85M irrespective of the grid structure though. I really am baffled as to what could be causing this problem. The first encounter of a NaN is in TrK *WARNING level 1 from host i24r03c05s08.sm.lrz.de <http://i24r03c05s08.sm.lrz.de> process 225* * while executing schedule bin NaNChecker_NaNCheck, routine NaNChecker::NaNChecker_NaNCheck_Check in thorn NaNChecker, file /gpfs/work/pr92bo/di25maz/Cactus/arrangements/CactusUtils/NaNChecker/src/NaNCheck.cc:567: -> NaN caught in variable 'ML_BSSN::trK' at index (62, 180, 182) level 0 map 0 with coordinates (-63, 55, 57) in iteration 44160* with a subsequent pile of NaNs in all the variables at levels 0 and 1 (map 0) at similar coordinates. E.g. *WARNING level 1 from host i24r01c04s09.sm.lrz.de <http://i24r01c04s09.sm.lrz.de> process 41* * while executing schedule bin NaNChecker_NaNCheck, routine NaNChecker::NaNChecker_NaNCheck_Check in thorn NaNChecker, file /gpfs/work/pr92bo/di25maz/Cactus/arrangements/CactusUtils/NaNChecker/src/NaNCheck.cc:567: -> Inf caught in variable 'ADMBASE::gxy' at index (121, 357, 358) level 1 map 0 with coordinates (-62, 56, 56.5) in iteration 44160* There are plots from ml_admconstraints-ml_ham.file_225.h5 for the first 4 levels (0-3) at iteration 40960 (by this point the Hamiltonian constraint is on its way to blowing up) available at: https://github.com/GeraintPratten/Einstein-Toolkit-Plots You can clearly see that on level 1 something bizarre is happening, it looks like there are some real boundary issues. I have no idea what could be causing this or why. Thanks again for any feedback! Geraint On 26 February 2016 at 07:38, Ian Hinder <ian.hinder(a)aei.mpg.de> wrote: > > On 24 Feb 2016, at 14:51, Geraint Pratten <g.pratten(a)sussex.ac.uk> wrote: > > Hi everyone, > > I've been playing around with grids under the assumption that there is no > reflection symmetry and I seem to be hitting a problem when Carpet goes to > regrid. The system evolves for ~ 80M quite happily but it hits a point > where it attempts to regrid levels 1 and up causing all sorts of havoc, > notably the measure of the Hamiltonian constraint blows up until the > simulation gracefully terminates due to NaNs. > > I've attached a portion of the output for the simulation where it is > initially evolving quite happily and then the simulation regrids (line > 1560+) and becomes unstable (see 4th column of line 1689 for *STRAINTS::H > maximum, this eventually just goes off to ~ 10^{230} and dies). > > > Hi Geraint, > > I also have not seen exactly this before. > > 10^230 looks like a typical poison value, but in the parameter file you > sent, you don't set the poison value, so it should default to NaN. Is the > file you sent exactly the same as the one you ran? Can you look at the 2D > output, for example of H, and check that it is blowing up at this > refinement boundary? > > Have you successfully run similar setups before? If so, what has changed > between a successful run and this one? > > What version of the ET are you using? > > > > I'm presuming that when it tries to regrid levels 1 and up it's > introducing some bad data along the way? > > The other issue that I'm not completely sure on is related to the warning > being thrown up by QuasiLocalMeasures: > > WARNING level 3 from host i23r01c06s07.sm.lrz.de process 0 > while executing schedule bin CCTK_ANALYSIS, routine > QuasiLocalMeasures::qlm_calculate > in thorn QuasiLocalMeasures, file qlm_killing_transport.F90:123: > -> Did not manage to find an eigenvector with the eigenvalue 1 > WARNING level 2 from host i23r01c06s07.sm.lrz.de process 0 > while executing schedule bin CCTK_ANALYSIS, routine > QuasiLocalMeasures::qlm_calculate > in thorn QuasiLocalMeasures, file qlm_killing_normalisation.F90:96: > -> Integration path starts out too steep > WARNING level 2 from host i23r01c06s07.sm.lrz.de process 0 > while executing schedule bin CCTK_ANALYSIS, routine > QuasiLocalMeasures::qlm_calculate > in thorn QuasiLocalMeasures, file qlm_killing_normalisation.F90:96: > -> Integration path starts out too steep > WARNING level 2 from host i23r01c06s07.sm.lrz.de process 0 > while executing schedule bin CCTK_ANALYSIS, routine > QuasiLocalMeasures::qlm_calculate > in thorn QuasiLocalMeasures, file qlm_killing_normalisation.F90:96: > -> Integration path starts out too steep > WARNING level 2 from host i23r01c06s07.sm.lrz.de process 0 > while executing schedule bin CCTK_ANALYSIS, routine > QuasiLocalMeasures::qlm_calculate > in thorn QuasiLocalMeasures, file qlm_killing_normalisation.F90:96: > -> Integration path starts out too steep > WARNING level 2 from host i23r01c06s07.sm.lrz.de process 0 > while executing schedule bin CCTK_ANALYSIS, routine > QuasiLocalMeasures::qlm_calculate > in thorn QuasiLocalMeasures, file qlm_killing_normalisation.F90:96: > -> Integration path starts out too steep > WARNING level 2 from host i23r01c06s07.sm.lrz.de process 0 > while executing schedule bin CCTK_ANALYSIS, routine > QuasiLocalMeasures::qlm_calculate > in thorn QuasiLocalMeasures, file qlm_killing_normalisation.F90:96: > -> Integration path starts out too steep > > Can anyone help with this or has anyone seen similar issues before? > Something is clearly not set up correctly but I can't see where my error > is. If you need any other information, just let me know! > > Thanks in advance! Any help is very much appreciated! > Geraint > > ------------------ > > For reference, the various Carpet settings that I use are as follows: > > #------ > # Grid: > #------ > > Time::dtfac = 0.25 > MoL::ODE_Method = "rk4" > MoL::MoL_Intermediate_Steps = 4 > MoL::MoL_Num_Scratch_Levels = 1 > > #------------------------------ > # Carpet Coord. Grid 3D > #------------------------------ > CartGrid3D::type = "coordbase" > CartGrid3D::domain = "full" > CartGrid3D::avoid_origin = "no" > > #------------------------------ > # Carpet Coord. Base > #------------------------------ > CoordBase::domainsize = "minmax" > CoordBase::spacing = "gridspacing" # "gridspacing" or "numcells" > > CoordBase::xmin = -614.40 > CoordBase::ymin = -614.40 > CoordBase::zmin = -614.40 > CoordBase::xmax = 614.40 > CoordBase::ymax = 614.40 > CoordBase::zmax = 614.40 > CoordBase::dx = 15.36 # dx or ncells_x, nx = 80 > CoordBase::dy = 15.36 # dy or ncells_y, nx = 80 > CoordBase::dz = 15.36 # dz or ncells_z, nx = 80 > > CoordBase::boundary_size_x_lower = 5 > CoordBase::boundary_size_y_lower = 5 > CoordBase::boundary_size_z_lower = 5 > CoordBase::boundary_size_x_upper = 5 > CoordBase::boundary_size_y_upper = 5 > CoordBase::boundary_size_z_upper = 5 > CoordBase::boundary_shiftout_x_lower = 0 > CoordBase::boundary_shiftout_y_lower = 0 > CoordBase::boundary_shiftout_z_lower = 0 > CoordBase::boundary_shiftout_x_upper = 0 > CoordBase::boundary_shiftout_y_upper = 0 > CoordBase::boundary_shiftout_z_upper = 0 > > #------------------------------ > # Driver: Ghost Size > #------------------------------ > Driver::ghost_size = 5 > > #------------------------------ > # Specify Reflection Symmetries > #------------------------------ > ReflectionSymmetry::reflection_x = "no" > ReflectionSymmetry::reflection_y = "no" > ReflectionSymmetry::reflection_z = "no" > ReflectionSymmetry::avoid_origin_x = "no" > ReflectionSymmetry::avoid_origin_y = "no" > ReflectionSymmetry::avoid_origin_z = "no" > > > #------------------------------ > # General Carpet Parameters > #------------------------------ > Carpet::domain_from_coordbase = "yes" > Carpet::enable_all_storage = "no" > Carpet::use_buffer_zones = "yes" > Carpet::schedule_barriers = "no" > > Carpet::poison_new_timelevels = "yes" > Carpet::check_for_poison = "no" > #Carpet::poison_value = 113 # git version only! > > Carpet::init_3_timelevels = "no" > Carpet::init_fill_timelevels = "yes" > > CarpetLib::poison_new_memory = "yes" > CarpetLib::poison_value = 114 > CarpetLib::check_bboxes = "no" > CarpetLib::interleave_communications = "yes" > #CarpetLib::vary_tags = "yes" # git version > only! > CarpetLib::combine_sends = "yes" > #CarpetLib::reduce_mpi_waitall = "yes" # git version only! > > CarpetInterp::tree_search = "yes" > CarpetInterp::check_tree_search = "no" > > CarpetRegrid2::freeze_unaligned_levels = "yes" > #CarpetRegrid2::snap_to_coarse = "yes" > CarpetRegrid2::symmetry_rotating180 = "no" > CarpetRegrid2::min_fraction = 1 > > # System specific Carpet parameters > Carpet::time_refinement_factors = "[1, 1, 2, 4, 8, 16, 32, 64, 128, 256, > 512, 1024]" # Should check this...? > Carpet::max_refinement_levels = 12 > Carpet::prolongation_order_space = 5 > Carpet::prolongation_order_time = 2 > > CarpetRegrid2::regrid_every = 64 > CarpetRegrid2::num_centres = 2 > > CarpetRegrid2::num_levels_1 = 12 > CarpetRegrid2::position_x_1 = 2.00393 > CarpetRegrid2::radius_1[ 1] = 307.2 > CarpetRegrid2::radius_1[ 2] = 230.4 > CarpetRegrid2::radius_1[ 3] = 153.6 > CarpetRegrid2::radius_1[ 4] = 105.6 > CarpetRegrid2::radius_1[ 5] = 38.4 > CarpetRegrid2::radius_1[ 6] = 14.4 > CarpetRegrid2::radius_1[ 7] = 4.8 > CarpetRegrid2::radius_1[ 8] = 2.4 > CarpetRegrid2::radius_1[ 9] = 1.2 > CarpetRegrid2::radius_1[10] = 0.6 > CarpetRegrid2::radius_1[11] = 0.3 > CarpetRegrid2::movement_threshold_1 = 0.075 > > CarpetRegrid2::num_levels_2 = 12 > CarpetRegrid2::position_x_2 = -6.01178 > CarpetRegrid2::radius_2[ 1] = 307.2 > CarpetRegrid2::radius_2[ 2] = 230.4 > CarpetRegrid2::radius_2[ 3] = 153.6 > CarpetRegrid2::radius_2[ 4] = 105.6 > CarpetRegrid2::radius_2[ 5] = 38.4 > CarpetRegrid2::radius_2[ 6] = 14.4 > CarpetRegrid2::radius_2[ 7] = 4.8 > CarpetRegrid2::radius_2[ 8] = 2.4 > CarpetRegrid2::radius_2[ 9] = 1.2 > CarpetRegrid2::radius_2[10] = 0.6 > CarpetRegrid2::radius_2[11] = 0.3 > CarpetRegrid2::movement_threshold_2 = 0.075 > > CarpetMask::excluded_surface [0] = 0 > CarpetMask::excluded_surface_factor[0] = 1.0 > > CarpetMask::excluded_surface [1] = 1 > CarpetMask::excluded_surface_factor[1] = 1.0 > > CarpetMask::excluded_surface [2] = 2 > CarpetMask::excluded_surface_factor[2] = 1.0 > > CarpetTracker::surface[0] = 0 > CarpetTracker::surface[1] = 1 > > -- > Geraint Pratten > Postdoctoral Research Associate > > Mobile: +44(0) 7581709282 > E-mail: G.Pratten(a)sussex.ac.uk > Skype: geraint.pratten > > School of Mathematical and Physical Sciences > Pevensey 3 Building > University of Sussex > Falmer Campus > Brighton > BN1 9QH > United Kingdom > <Output_Cleaned.out>_______________________________________________ > Users mailing list > Users(a)einsteintoolkit.org > http://lists.einsteintoolkit.org/mailman/listinfo/users > > > -- > Ian Hinder > http://members.aei.mpg.de/ianhin > > -- Geraint Pratten Postdoctoral Research Associate Mobile: +44(0) 7581709282 E-mail: G.Pratten(a)sussex.ac.uk Skype: geraint.pratten School of Mathematical and Physical Sciences Pevensey 3 Building University of Sussex Falmer Campus Brighton BN1 9QH United Kingdom

2 1

2026

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

Users March 2016