Hello Luciano,
We quickly discussed this in last week's ET call.
Unfortunately not one present had any experience using Llaman with GRHydro and a non-trivial topology.
Oleg Korobkin did some similar work on accretion disks using a Cactus based code with a central cutout (but not Llama based but using an earlier version of Cactus/Carpet based multiblock):
https://arxiv.org/abs/1011.3010
You may be a able to reach out to him (at LANL).
My own recollection is that hydro_excision is not used very frequently in GRHydro, to it may or may not actually work at all.
Note when you say "origin" since this is the Llama grid with a hole in the center, you mean "inner cutout" (ie not r=0 but instead r=3.0)?
Yours, Roland
Hi everyone,
I started testing the Thornburg04nc coordinates (spherical patches without Cartesian box) from Llama with (a modified version) of GRHydro. I started with just an atmosphere around a Kerr-Schild metric evolving only hydro. I'm seeing instabilities near the origin, so I was wondering if someone already encountered this issue. In particular, I wanted to know if someone has a good idea of how to treat the inner boundary. Right now, I'm putting the inner boundary inside the horizon, above the ring singularity, and applying hydro excision. I've tried a combination of things playing with those parameters but without success. Has anyone tried a similar thing evolving hydro? Just want to double check if there's something obvious in the grid setup I'm missing.
Below I attach the relevant part of the par file I'm using. Main difference with the public version of GRHydro is that I'm using Siegel+2017 c2p which works just fine with the same setup in Cartesian coordinates.
Thanks a lot!!
######
ActiveThorns = "Time MoL" ActiveThorns = "Coordbase CartGrid3d Boundary StaticConformal" ActiveThorns = "SymBase ADMBase TmunuBase HydroBase InitBase ADMCoupling ADMMacros" ActiveThorns = "IOUtil Formaline" ActiveThorns = "SpaceMask CoordGauge Constants LocalReduce aeilocalinterp LoopControl" ActiveThorns = "Carpet CarpetLib CarpetReduce CarpetRegrid2 " ActiveThorns = "Coordinates GenericFD Interpolate2 CarpetInterp2 CarpetInterp" ActiveThorns = "CarpetIOASCII CarpetIOScalar CarpetIOHDF5 CarpetIOBasic"
################################################################################ # Grid setup and general parameters ################################################################################
# Grid
CartGrid3D::type = "multipatch" Coordinates::coordinate_system = "Thornburg04nc" CartGrid3D::set_coordinate_ranges_on = "all maps"
Coordinates::h_radial = 0.35 Coordinates::sphere_inner_radius = 3.0 Coordinates::sphere_outer_radius = 90.0 Coordinates::n_angular = 25
#Coordinates::radial_stretch = yes #Coordinates::stretch_rmin_1 = 10.0 #Coordinates::stretch_rmax_1 = 90.0
Coordinates::stagger_patch_boundaries = "no" Coordinates::stagger_outer_boundaries = "no"
Interpolate::interpolator_order = 4 Interpolate2::shift_edges = no
Coordinates::store_jacobian = yes # I need to Coordinates::store_inverse_jacobian = yes # Check for runtime errors #Coordinates::store_jacobian_derivative = yes Coordinates::store_volume_form = yes
#GenericFD::jacobian_group = "Coordinates::jacobian" #GenericFD::jacobian_derivative_group = "Coordinates::jacobian2"
# General Carpet parameters: Carpet::enable_all_storage = "no" Carpet::use_buffer_zones = "yes" Carpet::use_overlap_zones = "no" Carpet::schedule_barriers = "no" #Carpet::processor_topology = "recursive" # recursive breaks multipatch
Carpet::check_for_poison = "no" Carpet::init_3_timelevels = "no" Carpet::init_fill_timelevels = "yes"
CarpetLib::poison_new_memory = "yes" CarpetLib::poison_value = 114 CarpetLib::check_bboxes = "no" CarpetLib::interleave_communications = "yes" CarpetLib::combine_sends = "no" # Erik says this is faster and does not use more memory than yes CarpetLib::combine_recompose = "no" # yes is the default, set to no if we run out of memory during recomposing #CarpetLib::restriction_order_space = 0
CarpetInterp::tree_search = "yes" CarpetInterp::check_tree_search = "no"
# OVERLAP Coordinates::additional_overlap_size = 3
# BOUNDARY ZONES (needs to be set to three) Coordinates::patch_boundary_size = 3 Coordinates::outer_boundary_size = 3 #Carpet::prolongation_order_space = 5 #Carpet::prolongation_order_time = 2 Carpet::refinement_centering = "vertex"
#==================================== # TmunuBase & HydroBase #==================================== TmunuBase::stress_energy_storage = yes TmunuBase::stress_energy_at_RHS = yes TmunuBase::timelevels = 1 TmunuBase::prolongation_type = none HydroBase::timelevels = 3
############################################################# # Carpet #############################################################
Carpet::domain_from_multipatch = yes Carpet::ghost_size = 3 # not for FD #Carpet::granularity = 5 # (order+1)*m #Carpet::granularity_boundary = 1 #Carpet::poison_new_timelevels = "yes" # does not work with persistent RHS
#--------CarpetMask-------------------------- ActiveThorns = "CarpetMask" CarpetMask::excluded_surface [0] = 1 CarpetMask::excluded_surface_factor[0] = 1.0 # ############################################################# # Time integration #############################################################
Cactus::terminate = "time" Cactus::cctk_final_time = 2000000000000 #Cactus::cctk_itlast = 1
Time::timestep_method = "given" Time::timestep = 0.10
MethodOfLines::ODE_method = "RK4" MethodOfLines::MoL_Intermediate_Steps = 4 MethodOfLines::MoL_Num_Scratch_Levels = 1 MethodOfLines::MoL_NaN_Check = no
################################################################################ # Hydro evolution ################################################################################
SpaceMask::use_mask = "yes"
#==================================== # GRHydro #==================================== ActiveThorns = "EOS_Omni" ActiveThorns = "GRHydro"
HydroBase::evolution_method = "GRHydro" HydroBase::Bvec_evolution_method = "GRHydro" # use c++ version of GRHydro GRHydro::use_cxx_code = "yes"
GRHydro::method_type = "RSA FV" GRHydro::riemann_solver = "HLLE" GRHydro::recon_method = "ppm" GRHydro::recon_vars = "primitive" GRHydro::GRHydro_stencil = 3 GRHydro::bound = "flat" #also tried none GRHydro::rho_abs_min = 1e-15 GRHydro::initial_rho_abs_min = 5e-14 GRHydro::GRHydro_atmo_tolerance = 0.01
# -- settings for 3-param (tabulated) EOS -- grhydro::grhydro_eos_type = "General" grhydro::grhydro_eos_table = "nuc_eos" EOS_Omni::nuceos_read_table = "yes" EOS_Omni::nuceos_table_name = "/home/d/dsiegel/lcombi/repository/EOS_tables/etk_bns_evolution/sfho/Hempel_SFHoEOS_rho222_temp180_ye60_version_1.1_20120817.h5" GRHydro::GRHydro_eos_rf_prec = 1e-10 hydrobase::Y_e_evolution_method = "GRHydro" hydrobase::temperature_evolution_method = "GRHydro"
GRHydro::reconstruct_temper = "yes" # more robust for reconstruction GRHydro::GRHydro_hot_atmo_temp = 0.012 #8.62e-06=1e5K GRHydro::GRHydro_hot_atmo_Y_e = 0.5 grhydro::grhydro_y_e_min = 0.012 grhydro::grhydro_y_e_max = 0.599
GRHydro::Grhydro_MaxNumConstrainedVars = 34 #33 GRHydro::GRHydro_MaxNumEvolvedVars = 10
# -- settings for con2prim -- GRHydro::GRHydro_c2pMHD_method = 1 GRHydro::GRHydro_c2pMHD_method_backup = 5 GRHydro::GRHydro_use_c2pMHD_method_backup = "yes" GRHydro::GRHydro_c2p_grace_radius = 7.5 grhydro::grhydro_c2p_warnlevel = 1 #1 grhydro::grhydro_c2p_warn_from_reflevel = 5 #5 # 6
# --- EOS Omni --- eos_omni::poly_gamma = 1.33333333 #eos_omni::poly_gamma_initial = 1.333333333 eos_omni::poly_k = 0.3605 #145.877704 #100 eos_omni::gl_gamma = 1.33333333 eos_omni::gl_k = 0.3605 #145.877704 #100
################################################################################ # Spacetime evolution ################################################################################
ADMBase::metric_type = "physical" ADMBase::evolution_method = "static" ADMBase::lapse_evolution_method = "static" ADMBase::shift_evolution_method = "static" ADMBase::dtlapse_evolution_method= "static" ADMBase::dtshift_evolution_method= "static"
ADMBase::metric_timelevels = 3 ADMBase::lapse_timelevels = 3 ADMBase::shift_timelevels = 3 ADMMacros::spatial_order = 4
#==================================== # Apparent horizon & excision #==================================== ActiveThorns = "hydro_initexcision" hydro_initexcision::hydro_initexcision = "yes" hydro_initexcision::hydro_initexcision_type = "sphere" hydro_initexcision::hydro_initexcision_coordinate_length = 4.2 # should be slightly larger than a_BH = spin_BH*mass_BH
# --- Settings for apparent horizon --- ActiveThorns = "AHFinderDirect SetMask_SphericalSurface" AHFinderDirect::N_horizons = 1 AHFinderDirect::find_after_individual_time[1] = 0 AHFinderDirect::find_every = 20000 AHFinderDirect::output_h_every = 20000 AHFinderDirect::initial_guess_method[1] = "Kerr/Kerr-Schild" AHFinderDirect::initial_guess__Kerr_KerrSchild__x_posn[1] = 0.0 AHFinderDirect::initial_guess__Kerr_KerrSchild__y_posn[1] = 0.0 AHFinderDirect::initial_guess__Kerr_KerrSchild__z_posn[1] = 0.0 AHFinderDirect::initial_guess__Kerr_KerrSchild__mass[1] = 3.0 # adjust to settings for TorusID AHFinderDirect::initial_guess__Kerr_KerrSchild__spin[1] = 0.8 # adjust to settings for TorusID AHFinderDirect::geometry_interpolator_name = "Lagrange polynomial interpolation" AHFinderDirect::geometry_interpolator_pars = "order=4" AHFinderDirect::surface_interpolator_name = "Lagrange polynomial interpolation" AHFinderDirect::surface_interpolator_pars = "order=4" AHFinderDirect::mask_buffer_thickness = 0 AHFinderDirect::mask_radius_multiplier = 0.8 AHFinderDirect::mask_radius_offset = 0 AHFinderDirect::max_allowable_horizon_radius[1] = 10 AHFinderDirect::max_allowable_Theta_growth_iterations = 10 AHFinderDirect::max_allowable_Theta_nonshrink_iterations = 10 AHFinderDirect::max_Newton_iterations__initial = 50 AHFinderDirect::max_Newton_iterations__subsequent = 50 AHFinderDirect::which_surface_to_store_info_by_name [1] = "apparent horizon"
################################################################################ # Initial data ################################################################################
InitBase::initial_data_setup_method = "init_some_levels" ADMBase::initial_data = "exact" #TorusID" ADMBase::initial_lapse = "exact" #TorusID" ADMBase::initial_shift = "exact" #TorusID" ADMBase::initial_dtlapse = "zero" ADMBase::initial_dtshift = "zero"
ActiveThorns = "Exact" Exact::exact_model = "Kerr/Kerr-Schild" Exact::Kerr_KerrSchild__mass = 3.0 Exact::Kerr_KerrSchild__parabolic = "yes"
*Dr. Luciano Combi* Postdoctoral Researcher Perimeter Institute for Theoretical Physics CITA National Fellow (U. of Guelph)