[Users] Carpet: Domain specification fails

dumsani g14n8326 at campus.ru.ac.za
Mon Dec 14 10:14:11 CST 2015


Hi All,

I am working with some piece of code based on a multipatch coordinate 
system
(llama, http://dx.doi.org/10.1103/PhysRevD.83.044045). Currently, I'm 
interested
in doing a test in a cubical kind of domain, that is, without the 
multipatch system.
Closely following the qc0-mclachlan.par parameter file (ET_2014_05), I 
converted
my multipatch parameter file to use the following:
Carpet::domain_from_multipatch          = "no"
Carpet::domain_from_coordbase         = yes
CartGrid3D::type                        = "coordbase"

Unlike when using a multipatch sysytem, I've had to add the 
ReflectionSymmetry
thorn in my thornlist for running on a cubical grid. The simulation is 
able to build
successfully. But when running, it breaks with the following error from 
Carpet:
cactus/arrangements/Carpet/Carpet/src/SetupGH.cc:1744: void 
Carpet::get_domain_specification(const cGH*, int, const ivect&, rvect&, 
rvect&, rvect&): Assertion `not CCTK_IsFunctionAliased 
("MultiPatch_GetDomainSpecification")' failed.

Looking at the function that fails and a few ideas I got from the user 
mailing list, my naive
thought is that this might possibly be an issue of cctk_lsh vs cctk_ash 
in Carpet.
(e.g. 
http://lists.einsteintoolkit.org/pipermail/users/2014-October/003849.html). 
But I'm yet
to test this idea. The code I'm working with uses cctk_lsh.

I would gladly appreciate help from anyone who could have an idea about 
this.

I'm using ET_2014_05 release of the EinsteinToolkit/Cactus. I have 
attached my test
parameter file for viewing in case it will be useful in diagnosing the 
problem(s).

Thank you in advance for your kind assistance.

Regards,
Dumsani
-------------- next part --------------
Cactus::cctk_run_title = "QC0-cubegrid"

Cactus::cctk_full_warnings         = "yes"
Cactus::highlight_warning_messages = "no"

ActiveThorns = "
  ADMBase
  ADMCoupling
  ADMDerivatives
  ADMMacros
  AEILocalInterp
  AHFinderDirect
  Boundary
  CTGBase
  CTGConstraints
  CTGEvolution
  CTGGauge
  CTGRadiativeBC
  Carpet
  CarpetIOASCII
  CarpetIOBasic
  CarpetIOHDF5
  CarpetIOScalar
  CarpetInterp
  CarpetInterp2
  CarpetLib
  CarpetMask
  CarpetReduce
  CarpetRegrid2
  CarpetTracker
  CartGrid3D
  CoordBase
  CoordGauge
  Coordinates
  Formaline
  GlobalDerivative
  IOUtil
  InitBase
  Interpolate2
  LocalInterp
  LoopControl
  MoL
  NaNChecker
  Psiclops
  PunctureTracker
  ReflectionSymmetry
  RotatingSymmetry180
  ShiftCorotation
  Slab
  SpaceMask
  SphericalSlice
  SphericalSurface
  StaticConformal
  SummationByParts
  SymBase
  Time
  TmunuBase
  TwoPunctures
  WorldTube
"

Cactus::terminate                       = "time"
Cactus::cctk_final_time                 = 30.0
# Cactus::terminate                        = "iteration"
# Cactus::cctk_itlast                      = 5120


###################################################
### Grids (No-Multipatch system/grid)
###################################################
Carpet::domain_from_multipatch          = "no"
Carpet::domain_from_coordbase 		= yes
CartGrid3D::type                        = "coordbase"
CartGrid3D::set_coordinate_ranges_on    = "all grids"

CoordBase::domainsize = "minmax"

CoordBase::xmin =    0.00
CoordBase::ymin = -120.00
CoordBase::zmin =    0.00
CoordBase::xmax = +120.00
CoordBase::ymax = +120.00
CoordBase::zmax = +120.00
CoordBase::dx   =    2.00
CoordBase::dy   =    2.00
CoordBase::dz   =    2.00

CoordBase::boundary_size_x_lower     = 3
CoordBase::boundary_size_y_lower     = 3
CoordBase::boundary_size_z_lower     = 3
CoordBase::boundary_size_x_upper     = 3
CoordBase::boundary_size_y_upper     = 3
CoordBase::boundary_size_z_upper     = 3

CoordBase::boundary_shiftout_x_lower = 1
CoordBase::boundary_shiftout_z_lower = 1

ReflectionSymmetry::reflection_z   = yes
ReflectionSymmetry::avoid_origin_z = no

###################################################
### Mesh Refinement (Carpet), interpolation, 
### prolongation, restriction
###################################################
#Coordinates::patch_boundary_size       = 3
#Coordinates::additional_overlap_size   = 3
#Coordinates::outer_boundary_size       = 1
CarpetInterp::check_tree_search         = "no"
CarpetInterp::tree_search               = "yes"

Interpolate::interpolator_order         = 5

Carpet::verbose              		= no
Carpet::veryverbose       		= no
Carpet::schedule_barriers 		= no
Carpet::storage_verbose   		= no
#Carpet::timers_verbose    		= no
CarpetLib::output_bboxes  		= no

Carpet::max_refinement_levels 		= 10
driver::ghost_size                      = 3
Carpet::use_buffer_zones                = "yes"
Carpet::prolongation_order_space        = 5
Carpet::prolongation_order_time         = 2
Carpet::init_fill_timelevels            = "yes"
# needed for AHFinderDirect
CTGBase::timelevels                     = 3
Carpet::enable_all_storage              = "no"
#Carpet::grid_coordinates_filename       = "carpet-grid.asc"

CarpetRegrid2::num_centres              = 1
CarpetRegrid2::num_levels_1             = 7
CarpetRegrid2::position_x_1             = +1.168642873
CarpetRegrid2::radius_1[ 1]         =  64.0
CarpetRegrid2::radius_1[ 2]         =  16.0
CarpetRegrid2::radius_1[ 3]         =   8.0
CarpetRegrid2::radius_1[ 4]         =   4.0
CarpetRegrid2::radius_1[ 5]         =   2.0
CarpetRegrid2::radius_1[ 6]         =   1.0
CarpetRegrid2::movement_threshold_1 =   0.16


###################################################
### Refinement tracking
###################################################
CarpetTracker::surface              [0] = 0
CarpetRegrid2::regrid_every             = 128
CarpetRegrid2::verbose                  = "no"
PunctureTracker::track              [0] = "yes"
PunctureTracker::initial_x          [0] = +1.168642873
PunctureTracker::which_surface_to_store_info[0] = 0

###################################################
############   Initial data 
###################################################
ADMMacros::spatial_order                = 4
ADMBase::metric_type                    = "physical"
ADMBase::initial_data                   = "twopunctures"
ADMBase::initial_lapse                  = "twopunctures-averaged"
ADMBase::initial_shift                  = "zero"


TwoPunctures::par_m_plus                = 0.453
TwoPunctures::par_m_minus               = 0.0

TwoPunctures::par_b                     = 1.168642873

TwoPunctures::par_P_plus[0]             = -0.3331917498
TwoPunctures::par_P_minus[0]            = 0.0

TwoPunctures::grid_setup_method         = "Taylor expansion" 
TwoPunctures::npoints_A                 = 20
TwoPunctures::npoints_B                 = 20
TwoPunctures::npoints_phi               = 8
TwoPunctures::do_residuum_debug_output  = "yes"
TwoPunctures::do_initial_debug_output   = "yes"
TwoPunctures::give_bare_mass            = "yes"
TwoPunctures::TP_epsilon 		= 1.0e-2
TwoPunctures::TP_Tiny    		= 1.0e-2
TwoPunctures::verbose 			= yes

InitBase::initial_data_setup_method     = "init_all_levels"



###################################################
### Main evolution, gauge evolution, BCs
###################################################
ADMBase::evolution_method               = "ctgamma"
ADMBase::lapse_evolution_method         = "1+log"
ADMBase::shift_evolution_method         = "gamma-driver"
ADMBase::dtlapse_evolution_method       = "1+log"
ADMBase::dtshift_evolution_method       = "gamma-driver"
CTGGauge::eta                           = 1.0
CTGBase::conformal_factor_type          = "w"
CTGEvolution::bc                        = "radiative"


###################################################
### Time integration, Method of Lines (MoL)  ######
###################################################
MoL::ODE_Method                         = "rk4"
MoL::MoL_Intermediate_Steps             = 4
MoL::MoL_Num_Scratch_Levels             = 1

Time::timestep_method                   = "courant_static"
Time::dtfac				= 0.05

Carpet::time_refinement_factors    = "[1, 1, 2, 4, 8, 16, 32, 64, 128, 256]"


###################################################
##### Finite differencing (FD coeffs : SBP) #######
###################################################
SummationByParts::order                 = 4
SummationByParts::sbp_upwind_deriv      = "no"
SummationByParts::onesided_outer_boundaries = "yes"
SummationByParts::onesided_interpatch_boundaries     = "no"
SummationByParts::sbp_2nd_deriv         = "no"
SummationByParts::sbp_1st_deriv         = "yes"

SummationByParts::use_dissipation       = "no"
GlobalDerivative::use_dissipation       = "yes"
SummationByParts::scale_with_h          = "yes"
SummationByParts::dissipation_type      = "Kreiss-Oliger"
SummationByParts::epsdis                = 0.1
GlobalDerivative::epsdis_for_level  [0] = 0.3
SummationByParts::vars                  = "
  CTGBase::conformal_factor
  CTGBase::conformal_metric
  CTGBase::curvature_scalar
  CTGBase::curvature_tensor
  CTGBase::Gamma
  ADMBase::lapse
  ADMBase::shift
"

###################################################
######## Spherical surfaces, Horizons           
###################################################
SphericalSurface::nsurfaces             = 1
SphericalSurface::maxntheta             = 66
SphericalSurface::maxnphi               = 124
SphericalSurface::verbose               = no

# Surfaces 0 and 1 are used by PunctureTracker

# Horizon 1
SphericalSurface::ntheta            [0] = 41
SphericalSurface::nphi              [0] = 80
SphericalSurface::nghoststheta      [0] = 2
SphericalSurface::nghostsphi        [0] = 2

### Horizons

AHFinderDirect::N_horizons                               = 1
AHFinderDirect::find_every                               = 64
AHFinderDirect::output_h_every                           = 64
AHFinderDirect::h_directory                              = "$parfile/AHshape"
AHFinderDirect::max_Newton_iterations__initial           = 50
AHFinderDirect::max_Newton_iterations__subsequent        = 50
AHFinderDirect::max_allowable_Theta_growth_iterations    = 10
AHFinderDirect::max_allowable_Theta_nonshrink_iterations = 10
AHFinderDirect::geometry_interpolator_name               = "Lagrange polynomial interpolation"
AHFinderDirect::geometry_interpolator_pars               = "order=4"
AHFinderDirect::surface_interpolator_name                = "Lagrange polynomial interpolation"
AHFinderDirect::surface_interpolator_pars                = "order=4"
AHFinderDirect::verbose_level                            = "physics details"
AHFinderDirect::move_origins                             = "yes"

AHFinderDirect::origin_x                             [1] = +1.168642873
AHFinderDirect::initial_guess__coord_sphere__x_center[1] = +1.168642873
AHFinderDirect::initial_guess__coord_sphere__radius  [1] = 0.2
AHFinderDirect::which_surface_to_store_info          [1] = 0
AHFinderDirect::set_mask_for_individual_horizon      [1] = "no"
AHFinderDirect::reset_horizon_after_not_finding      [1] = "no"
AHFinderDirect::track_origin_from_grid_scalar        [1] = "yes"
AHFinderDirect::track_origin_source_x [1] = "PunctureTracker::pt_loc_x[0]"
AHFinderDirect::track_origin_source_y [1] = "PunctureTracker::pt_loc_y[0]"
AHFinderDirect::track_origin_source_z [1] = "PunctureTracker::pt_loc_z[0]"


###################################################
####### Constraint measurements
###################################################
CTGConstraints::constraints_persist     = "yes"
CarpetMask::excluded_surface        [0] = 0
CarpetMask::excluded_surface_factor [0] = 1.



###################################################
####### Gravitational Wave Extraction.
####### Mode decompositions for Psi4 and 
####### characteristic extraction
###################################################
SphericalSlice::nslices                 = 1
SphericalSlice::precalc_sYlms           = "no"

SphericalSlice::set_spherical       [0] = "yes"
SphericalSlice::radius              [0] = 60
SphericalSlice::type                [0] = "1patch"
SphericalSlice::use_Llama           [0] = "no"
SphericalSlice::ntheta              [0] = 80
SphericalSlice::nphi                [0] = 160


### Psi4 measurements

Psiclops::store_psi4                    = "yes"
Psiclops::angular_grid_only             = "no"

Psiclops::nslices                       = 1
Psiclops::lmax                          = 8
Psiclops::which_slice_to_take       [0] = 0


###################################################
###### Checking NaNs (NanCheck)
###################################################
Carpet::poison_new_timelevels           = "yes"
Carpet::poison_value                    = 0
CarpetLib::poison_new_memory            = "yes"
CarpetLib::poison_value                 = 0
Carpet::check_for_poison                = "no"

NaNChecker::check_every                 = 1  #16 
NanChecker::check_after                 = 0
NaNChecker::action_if_found             = "terminate"
NaNChecker::out_NaNmask                 = "yes"
NaNChecker::check_vars                  = "
  CTGBase::conformal_factor
  CTGBase::conformal_metric
  CTGBase::curvature_scalar
  CTGBase::curvature_tensor
  CTGBase::Gamma
  ADMBase::lapse
  ADMBase::shift
"


###################################################
###### Performance timers
###################################################
Cactus::cctk_timer_output               = "full"
Carpet::output_timers_every             = 64000000
CarpetLib::print_timestats_every        = 64000000
CarpetLib::print_memstats_every         = 64000000


###################################################
########## Output (I/O)
###################################################

### output directory (out_dir)
IO::out_dir                             = $parfile   

IOBasic::outInfo_every                  = 32
IOBasic::outInfo_reductions             = "norm2 minimum maximum"
IOBasic::outInfo_vars                   = "
  CTGConstraints::H
  Carpet::physical_time_per_hour
"

IOScalar::one_file_per_group            = "no"
IOScalar::outScalar_every               = 32
IOScalar::outScalar_reductions          = "norm_inf norm2 inorm2 maximum minimum"
IOScalar::outScalar_vars                = "
 CTGConstraints::Hamiltonian_constraint
 CTGConstraints::momentum_constraint
"

IOASCII::one_file_per_group 		= "no"
IOASCII::out0D_every                    = 256
IOASCII::out0D_vars                     = "
  Carpet::timing
  PunctureTracker::pt_loc
"

IOASCII::out1D_every     = 256
IOASCII::out1D_x         = "yes"
IOASCII::out1D_y         = "no"
IOASCII::out1D_z         = "no"
IOASCII::out1D_d         = "no"
IOASCII::out1D_vars                     = "
 CTGConstraints::Hamiltonian_constraint		
 CTGBase::conformal_factor 
 CTGBase::Gamma
 ADMBase::lapse	
 ADMBase::betax	
"




More information about the Users mailing list