[Users] Problems with regridding?

Geraint Pratten g.pratten at sussex.ac.uk
Wed Feb 24 07:51:23 CST 2016


Hi everyone,

I've been playing around with grids under the assumption that there is no
reflection symmetry and I seem to be hitting a problem when Carpet goes to
regrid. The system evolves for ~ 80M quite happily but it hits a point
where it attempts to regrid levels 1 and up causing all sorts of havoc,
notably the measure of the Hamiltonian constraint blows up until the
simulation gracefully terminates due to NaNs.

I've attached a portion of the output for the simulation where it is
initially evolving quite happily and then the simulation regrids (line
1560+) and becomes unstable (see 4th column of line 1689 for *STRAINTS::H
 maximum, this eventually just goes off to ~ 10^{230} and dies).

I'm presuming that when it tries to regrid levels 1 and up it's introducing
some bad data along the way?

The other issue that I'm not completely sure on is related to the warning
being thrown up by QuasiLocalMeasures:

WARNING level 3 from host i23r01c06s07.sm.lrz.de process 0
  while executing schedule bin CCTK_ANALYSIS, routine
QuasiLocalMeasures::qlm_calculate
  in thorn QuasiLocalMeasures, file qlm_killing_transport.F90:123:
  -> Did not manage to find an eigenvector with the eigenvalue 1
WARNING level 2 from host i23r01c06s07.sm.lrz.de process 0
  while executing schedule bin CCTK_ANALYSIS, routine
QuasiLocalMeasures::qlm_calculate
  in thorn QuasiLocalMeasures, file qlm_killing_normalisation.F90:96:
  -> Integration path starts out too steep
WARNING level 2 from host i23r01c06s07.sm.lrz.de process 0
  while executing schedule bin CCTK_ANALYSIS, routine
QuasiLocalMeasures::qlm_calculate
  in thorn QuasiLocalMeasures, file qlm_killing_normalisation.F90:96:
  -> Integration path starts out too steep
WARNING level 2 from host i23r01c06s07.sm.lrz.de process 0
  while executing schedule bin CCTK_ANALYSIS, routine
QuasiLocalMeasures::qlm_calculate
  in thorn QuasiLocalMeasures, file qlm_killing_normalisation.F90:96:
  -> Integration path starts out too steep
WARNING level 2 from host i23r01c06s07.sm.lrz.de process 0
  while executing schedule bin CCTK_ANALYSIS, routine
QuasiLocalMeasures::qlm_calculate
  in thorn QuasiLocalMeasures, file qlm_killing_normalisation.F90:96:
  -> Integration path starts out too steep
WARNING level 2 from host i23r01c06s07.sm.lrz.de process 0
  while executing schedule bin CCTK_ANALYSIS, routine
QuasiLocalMeasures::qlm_calculate
  in thorn QuasiLocalMeasures, file qlm_killing_normalisation.F90:96:
  -> Integration path starts out too steep
WARNING level 2 from host i23r01c06s07.sm.lrz.de process 0
  while executing schedule bin CCTK_ANALYSIS, routine
QuasiLocalMeasures::qlm_calculate
  in thorn QuasiLocalMeasures, file qlm_killing_normalisation.F90:96:
  -> Integration path starts out too steep

Can anyone help with this or has anyone seen similar issues before?
Something is clearly not set up correctly but I can't see where my error
is. If you need any other information, just let me know!

Thanks in advance! Any help is very much appreciated!
Geraint

------------------

For reference, the various Carpet settings that I use are as follows:

#------
# Grid:
#------

Time::dtfac                 = 0.25
MoL::ODE_Method             = "rk4"
MoL::MoL_Intermediate_Steps = 4
MoL::MoL_Num_Scratch_Levels = 1

#------------------------------
# Carpet Coord. Grid 3D
#------------------------------
CartGrid3D::type            = "coordbase"
CartGrid3D::domain          = "full"
CartGrid3D::avoid_origin    = "no"

#------------------------------
# Carpet Coord. Base
#------------------------------
CoordBase::domainsize       = "minmax"
CoordBase::spacing          = "gridspacing"  # "gridspacing" or "numcells"

CoordBase::xmin = -614.40
CoordBase::ymin = -614.40
CoordBase::zmin = -614.40
CoordBase::xmax =  614.40
CoordBase::ymax =  614.40
CoordBase::zmax =  614.40
CoordBase::dx   =   15.36  # dx or ncells_x, nx = 80
CoordBase::dy   =   15.36  # dy or ncells_y, nx = 80
CoordBase::dz   =   15.36  # dz or ncells_z, nx = 80

CoordBase::boundary_size_x_lower        = 5
CoordBase::boundary_size_y_lower        = 5
CoordBase::boundary_size_z_lower        = 5
CoordBase::boundary_size_x_upper        = 5
CoordBase::boundary_size_y_upper        = 5
CoordBase::boundary_size_z_upper        = 5
CoordBase::boundary_shiftout_x_lower    = 0
CoordBase::boundary_shiftout_y_lower    = 0
CoordBase::boundary_shiftout_z_lower    = 0
CoordBase::boundary_shiftout_x_upper    = 0
CoordBase::boundary_shiftout_y_upper    = 0
CoordBase::boundary_shiftout_z_upper    = 0

#------------------------------
# Driver: Ghost Size
#------------------------------
Driver::ghost_size                      = 5

#------------------------------
# Specify Reflection Symmetries
#------------------------------
ReflectionSymmetry::reflection_x   = "no"
ReflectionSymmetry::reflection_y   = "no"
ReflectionSymmetry::reflection_z   = "no"
ReflectionSymmetry::avoid_origin_x = "no"
ReflectionSymmetry::avoid_origin_y = "no"
ReflectionSymmetry::avoid_origin_z = "no"


#------------------------------
# General Carpet Parameters
#------------------------------
Carpet::domain_from_coordbase    = "yes"
Carpet::enable_all_storage             = "no"
Carpet::use_buffer_zones               = "yes"
Carpet::schedule_barriers              = "no"

Carpet::poison_new_timelevels    = "yes"
Carpet::check_for_poison             = "no"
#Carpet::poison_value                  = 113 # git version only!

Carpet::init_3_timelevels        = "no"
Carpet::init_fill_timelevels       = "yes"

CarpetLib::poison_new_memory            = "yes"
CarpetLib::poison_value                         = 114
CarpetLib::check_bboxes                       = "no"
CarpetLib::interleave_communications  = "yes"
#CarpetLib::vary_tags                             = "yes" # git version
only!
CarpetLib::combine_sends                     = "yes"
#CarpetLib::reduce_mpi_waitall             = "yes" # git version only!

CarpetInterp::tree_search            = "yes"
CarpetInterp::check_tree_search = "no"

CarpetRegrid2::freeze_unaligned_levels  = "yes"
#CarpetRegrid2::snap_to_coarse             = "yes"
CarpetRegrid2::symmetry_rotating180     = "no"
CarpetRegrid2::min_fraction                     = 1

# System specific Carpet parameters
Carpet::time_refinement_factors  = "[1, 1, 2, 4, 8, 16, 32, 64, 128, 256,
512, 1024]" # Should check this...?
Carpet::max_refinement_levels     = 12
Carpet::prolongation_order_space  = 5
Carpet::prolongation_order_time   = 2

CarpetRegrid2::regrid_every = 64
CarpetRegrid2::num_centres  = 2

CarpetRegrid2::num_levels_1 = 12
CarpetRegrid2::position_x_1 = 2.00393
CarpetRegrid2::radius_1[ 1]         = 307.2
CarpetRegrid2::radius_1[ 2]         = 230.4
CarpetRegrid2::radius_1[ 3]         = 153.6
CarpetRegrid2::radius_1[ 4]         = 105.6
CarpetRegrid2::radius_1[ 5]         =  38.4
CarpetRegrid2::radius_1[ 6]         =  14.4
CarpetRegrid2::radius_1[ 7]         =   4.8
CarpetRegrid2::radius_1[ 8]         =   2.4
CarpetRegrid2::radius_1[ 9]         =   1.2
CarpetRegrid2::radius_1[10]         =   0.6
CarpetRegrid2::radius_1[11]         =   0.3
CarpetRegrid2::movement_threshold_1 =   0.075

CarpetRegrid2::num_levels_2 = 12
CarpetRegrid2::position_x_2 = -6.01178
CarpetRegrid2::radius_2[ 1]         = 307.2
CarpetRegrid2::radius_2[ 2]         = 230.4
CarpetRegrid2::radius_2[ 3]         = 153.6
CarpetRegrid2::radius_2[ 4]         = 105.6
CarpetRegrid2::radius_2[ 5]         =  38.4
CarpetRegrid2::radius_2[ 6]         =  14.4
CarpetRegrid2::radius_2[ 7]         =   4.8
CarpetRegrid2::radius_2[ 8]         =   2.4
CarpetRegrid2::radius_2[ 9]         =   1.2
CarpetRegrid2::radius_2[10]         =   0.6
CarpetRegrid2::radius_2[11]         =   0.3
CarpetRegrid2::movement_threshold_2 =   0.075

CarpetMask::excluded_surface       [0] = 0
CarpetMask::excluded_surface_factor[0] = 1.0

CarpetMask::excluded_surface       [1] = 1
CarpetMask::excluded_surface_factor[1] = 1.0

CarpetMask::excluded_surface       [2] = 2
CarpetMask::excluded_surface_factor[2] = 1.0

CarpetTracker::surface[0] = 0
CarpetTracker::surface[1] = 1

-- 
Geraint Pratten
Postdoctoral Research Associate

Mobile: +44(0) 7581709282
E-mail: G.Pratten at sussex.ac.uk
Skype: geraint.pratten

School of Mathematical and Physical Sciences
Pevensey 3 Building
University of Sussex
Falmer Campus
Brighton
BN1 9QH
United Kingdom
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.einsteintoolkit.org/pipermail/users/attachments/20160224/418cc597/attachment-0001.html 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: Output_Cleaned.out
Type: application/octet-stream
Size: 157136 bytes
Desc: not available
Url : http://lists.einsteintoolkit.org/pipermail/users/attachments/20160224/418cc597/attachment-0001.obj 


More information about the Users mailing list