[Users] Problems with regridding?
Ian Hinder
ian.hinder at aei.mpg.de
Fri Feb 26 01:38:31 CST 2016
On 24 Feb 2016, at 14:51, Geraint Pratten <g.pratten at sussex.ac.uk> wrote:
> Hi everyone,
>
> I've been playing around with grids under the assumption that there is no reflection symmetry and I seem to be hitting a problem when Carpet goes to regrid. The system evolves for ~ 80M quite happily but it hits a point where it attempts to regrid levels 1 and up causing all sorts of havoc, notably the measure of the Hamiltonian constraint blows up until the simulation gracefully terminates due to NaNs.
>
> I've attached a portion of the output for the simulation where it is initially evolving quite happily and then the simulation regrids (line 1560+) and becomes unstable (see 4th column of line 1689 for *STRAINTS::H maximum, this eventually just goes off to ~ 10^{230} and dies).
Hi Geraint,
I also have not seen exactly this before.
10^230 looks like a typical poison value, but in the parameter file you sent, you don't set the poison value, so it should default to NaN. Is the file you sent exactly the same as the one you ran? Can you look at the 2D output, for example of H, and check that it is blowing up at this refinement boundary?
Have you successfully run similar setups before? If so, what has changed between a successful run and this one?
What version of the ET are you using?
>
> I'm presuming that when it tries to regrid levels 1 and up it's introducing some bad data along the way?
>
> The other issue that I'm not completely sure on is related to the warning being thrown up by QuasiLocalMeasures:
>
> WARNING level 3 from host i23r01c06s07.sm.lrz.de process 0
> while executing schedule bin CCTK_ANALYSIS, routine QuasiLocalMeasures::qlm_calculate
> in thorn QuasiLocalMeasures, file qlm_killing_transport.F90:123:
> -> Did not manage to find an eigenvector with the eigenvalue 1
> WARNING level 2 from host i23r01c06s07.sm.lrz.de process 0
> while executing schedule bin CCTK_ANALYSIS, routine QuasiLocalMeasures::qlm_calculate
> in thorn QuasiLocalMeasures, file qlm_killing_normalisation.F90:96:
> -> Integration path starts out too steep
> WARNING level 2 from host i23r01c06s07.sm.lrz.de process 0
> while executing schedule bin CCTK_ANALYSIS, routine QuasiLocalMeasures::qlm_calculate
> in thorn QuasiLocalMeasures, file qlm_killing_normalisation.F90:96:
> -> Integration path starts out too steep
> WARNING level 2 from host i23r01c06s07.sm.lrz.de process 0
> while executing schedule bin CCTK_ANALYSIS, routine QuasiLocalMeasures::qlm_calculate
> in thorn QuasiLocalMeasures, file qlm_killing_normalisation.F90:96:
> -> Integration path starts out too steep
> WARNING level 2 from host i23r01c06s07.sm.lrz.de process 0
> while executing schedule bin CCTK_ANALYSIS, routine QuasiLocalMeasures::qlm_calculate
> in thorn QuasiLocalMeasures, file qlm_killing_normalisation.F90:96:
> -> Integration path starts out too steep
> WARNING level 2 from host i23r01c06s07.sm.lrz.de process 0
> while executing schedule bin CCTK_ANALYSIS, routine QuasiLocalMeasures::qlm_calculate
> in thorn QuasiLocalMeasures, file qlm_killing_normalisation.F90:96:
> -> Integration path starts out too steep
> WARNING level 2 from host i23r01c06s07.sm.lrz.de process 0
> while executing schedule bin CCTK_ANALYSIS, routine QuasiLocalMeasures::qlm_calculate
> in thorn QuasiLocalMeasures, file qlm_killing_normalisation.F90:96:
> -> Integration path starts out too steep
>
> Can anyone help with this or has anyone seen similar issues before? Something is clearly not set up correctly but I can't see where my error is. If you need any other information, just let me know!
>
> Thanks in advance! Any help is very much appreciated!
> Geraint
>
> ------------------
>
> For reference, the various Carpet settings that I use are as follows:
>
> #------
> # Grid:
> #------
>
> Time::dtfac = 0.25
> MoL::ODE_Method = "rk4"
> MoL::MoL_Intermediate_Steps = 4
> MoL::MoL_Num_Scratch_Levels = 1
>
> #------------------------------
> # Carpet Coord. Grid 3D
> #------------------------------
> CartGrid3D::type = "coordbase"
> CartGrid3D::domain = "full"
> CartGrid3D::avoid_origin = "no"
>
> #------------------------------
> # Carpet Coord. Base
> #------------------------------
> CoordBase::domainsize = "minmax"
> CoordBase::spacing = "gridspacing" # "gridspacing" or "numcells"
>
> CoordBase::xmin = -614.40
> CoordBase::ymin = -614.40
> CoordBase::zmin = -614.40
> CoordBase::xmax = 614.40
> CoordBase::ymax = 614.40
> CoordBase::zmax = 614.40
> CoordBase::dx = 15.36 # dx or ncells_x, nx = 80
> CoordBase::dy = 15.36 # dy or ncells_y, nx = 80
> CoordBase::dz = 15.36 # dz or ncells_z, nx = 80
>
> CoordBase::boundary_size_x_lower = 5
> CoordBase::boundary_size_y_lower = 5
> CoordBase::boundary_size_z_lower = 5
> CoordBase::boundary_size_x_upper = 5
> CoordBase::boundary_size_y_upper = 5
> CoordBase::boundary_size_z_upper = 5
> CoordBase::boundary_shiftout_x_lower = 0
> CoordBase::boundary_shiftout_y_lower = 0
> CoordBase::boundary_shiftout_z_lower = 0
> CoordBase::boundary_shiftout_x_upper = 0
> CoordBase::boundary_shiftout_y_upper = 0
> CoordBase::boundary_shiftout_z_upper = 0
>
> #------------------------------
> # Driver: Ghost Size
> #------------------------------
> Driver::ghost_size = 5
>
> #------------------------------
> # Specify Reflection Symmetries
> #------------------------------
> ReflectionSymmetry::reflection_x = "no"
> ReflectionSymmetry::reflection_y = "no"
> ReflectionSymmetry::reflection_z = "no"
> ReflectionSymmetry::avoid_origin_x = "no"
> ReflectionSymmetry::avoid_origin_y = "no"
> ReflectionSymmetry::avoid_origin_z = "no"
>
>
> #------------------------------
> # General Carpet Parameters
> #------------------------------
> Carpet::domain_from_coordbase = "yes"
> Carpet::enable_all_storage = "no"
> Carpet::use_buffer_zones = "yes"
> Carpet::schedule_barriers = "no"
>
> Carpet::poison_new_timelevels = "yes"
> Carpet::check_for_poison = "no"
> #Carpet::poison_value = 113 # git version only!
>
> Carpet::init_3_timelevels = "no"
> Carpet::init_fill_timelevels = "yes"
>
> CarpetLib::poison_new_memory = "yes"
> CarpetLib::poison_value = 114
> CarpetLib::check_bboxes = "no"
> CarpetLib::interleave_communications = "yes"
> #CarpetLib::vary_tags = "yes" # git version only!
> CarpetLib::combine_sends = "yes"
> #CarpetLib::reduce_mpi_waitall = "yes" # git version only!
>
> CarpetInterp::tree_search = "yes"
> CarpetInterp::check_tree_search = "no"
>
> CarpetRegrid2::freeze_unaligned_levels = "yes"
> #CarpetRegrid2::snap_to_coarse = "yes"
> CarpetRegrid2::symmetry_rotating180 = "no"
> CarpetRegrid2::min_fraction = 1
>
> # System specific Carpet parameters
> Carpet::time_refinement_factors = "[1, 1, 2, 4, 8, 16, 32, 64, 128, 256, 512, 1024]" # Should check this...?
> Carpet::max_refinement_levels = 12
> Carpet::prolongation_order_space = 5
> Carpet::prolongation_order_time = 2
>
> CarpetRegrid2::regrid_every = 64
> CarpetRegrid2::num_centres = 2
>
> CarpetRegrid2::num_levels_1 = 12
> CarpetRegrid2::position_x_1 = 2.00393
> CarpetRegrid2::radius_1[ 1] = 307.2
> CarpetRegrid2::radius_1[ 2] = 230.4
> CarpetRegrid2::radius_1[ 3] = 153.6
> CarpetRegrid2::radius_1[ 4] = 105.6
> CarpetRegrid2::radius_1[ 5] = 38.4
> CarpetRegrid2::radius_1[ 6] = 14.4
> CarpetRegrid2::radius_1[ 7] = 4.8
> CarpetRegrid2::radius_1[ 8] = 2.4
> CarpetRegrid2::radius_1[ 9] = 1.2
> CarpetRegrid2::radius_1[10] = 0.6
> CarpetRegrid2::radius_1[11] = 0.3
> CarpetRegrid2::movement_threshold_1 = 0.075
>
> CarpetRegrid2::num_levels_2 = 12
> CarpetRegrid2::position_x_2 = -6.01178
> CarpetRegrid2::radius_2[ 1] = 307.2
> CarpetRegrid2::radius_2[ 2] = 230.4
> CarpetRegrid2::radius_2[ 3] = 153.6
> CarpetRegrid2::radius_2[ 4] = 105.6
> CarpetRegrid2::radius_2[ 5] = 38.4
> CarpetRegrid2::radius_2[ 6] = 14.4
> CarpetRegrid2::radius_2[ 7] = 4.8
> CarpetRegrid2::radius_2[ 8] = 2.4
> CarpetRegrid2::radius_2[ 9] = 1.2
> CarpetRegrid2::radius_2[10] = 0.6
> CarpetRegrid2::radius_2[11] = 0.3
> CarpetRegrid2::movement_threshold_2 = 0.075
>
> CarpetMask::excluded_surface [0] = 0
> CarpetMask::excluded_surface_factor[0] = 1.0
>
> CarpetMask::excluded_surface [1] = 1
> CarpetMask::excluded_surface_factor[1] = 1.0
>
> CarpetMask::excluded_surface [2] = 2
> CarpetMask::excluded_surface_factor[2] = 1.0
>
> CarpetTracker::surface[0] = 0
> CarpetTracker::surface[1] = 1
>
> --
> Geraint Pratten
> Postdoctoral Research Associate
>
> Mobile: +44(0) 7581709282
> E-mail: G.Pratten at sussex.ac.uk
> Skype: geraint.pratten
>
> School of Mathematical and Physical Sciences
> Pevensey 3 Building
> University of Sussex
> Falmer Campus
> Brighton
> BN1 9QH
> United Kingdom
> <Output_Cleaned.out>_______________________________________________
> Users mailing list
> Users at einsteintoolkit.org
> http://lists.einsteintoolkit.org/mailman/listinfo/users
--
Ian Hinder
http://members.aei.mpg.de/ianhin
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.einsteintoolkit.org/pipermail/users/attachments/20160226/3196a4f6/attachment-0001.html
More information about the Users
mailing list