<div dir="ltr">Hi everyone,<div><br></div><div>I started testing the Thornburg04nc coordinates (spherical patches without Cartesian box) from Llama with (a modified version) of GRHydro. I started with just an atmosphere around a Kerr-Schild metric evolving only hydro. I&#39;m seeing instabilities near the origin, so I was wondering if someone already encountered this issue. In particular, I wanted to know if someone has a good idea of how to treat the inner boundary. Right now, I&#39;m putting the inner boundary inside the horizon, above the ring singularity, and applying hydro excision. I&#39;ve tried a combination of things playing with those parameters but without success. Has anyone tried a similar thing evolving hydro? Just want to double check if there&#39;s something obvious in the grid setup I&#39;m missing.</div><div><br></div><div>Below I attach the relevant part of the par file I&#39;m using. Main difference with the public version of GRHydro is that I&#39;m using Siegel+2017 c2p which works just fine with the same setup in Cartesian coordinates.</div><div><br></div><div>Thanks a lot!!</div><div><br></div><div><br></div><div><br></div><div><br></div><div><br></div><div>######<br><br>ActiveThorns = &quot;Time MoL&quot;<br>ActiveThorns = &quot;Coordbase CartGrid3d Boundary StaticConformal&quot;<br>ActiveThorns = &quot;SymBase ADMBase TmunuBase HydroBase InitBase ADMCoupling ADMMacros&quot;<br>ActiveThorns = &quot;IOUtil Formaline&quot;<br>ActiveThorns = &quot;SpaceMask CoordGauge Constants LocalReduce aeilocalinterp LoopControl&quot;<br>ActiveThorns = &quot;Carpet CarpetLib CarpetReduce CarpetRegrid2 &quot;<br>ActiveThorns = &quot;Coordinates GenericFD Interpolate2 CarpetInterp2 CarpetInterp&quot;<br>ActiveThorns = &quot;CarpetIOASCII CarpetIOScalar CarpetIOHDF5 CarpetIOBasic&quot;<br><br>################################################################################<br># Grid setup and general parameters<br>################################################################################<br><br># Grid<br><br>CartGrid3D::type               = &quot;multipatch&quot;<br>Coordinates::coordinate_system = &quot;Thornburg04nc&quot;<br>CartGrid3D::set_coordinate_ranges_on    = &quot;all maps&quot;<br><br>Coordinates::h_radial            =   0.35 <br>Coordinates::sphere_inner_radius =  3.0<br>Coordinates::sphere_outer_radius =  90.0<br>Coordinates::n_angular           =  25 <br><br>#Coordinates::radial_stretch             = yes<br>#Coordinates::stretch_rmin_1             = 10.0<br>#Coordinates::stretch_rmax_1             = 90.0<br><br>Coordinates::stagger_patch_boundaries = &quot;no&quot;<br>Coordinates::stagger_outer_boundaries = &quot;no&quot;<br><br>Interpolate::interpolator_order = 4<br>Interpolate2::shift_edges        = no<br><br>Coordinates::store_jacobian            = yes # I need to<br>Coordinates::store_inverse_jacobian    = yes # Check for runtime errors<br>#Coordinates::store_jacobian_derivative = yes<br>Coordinates::store_volume_form         = yes<br><br>#GenericFD::jacobian_group            = &quot;Coordinates::jacobian&quot;<br>#GenericFD::jacobian_derivative_group = &quot;Coordinates::jacobian2&quot;<br><br># General Carpet parameters:<br>Carpet::enable_all_storage       = &quot;no&quot;<br>Carpet::use_buffer_zones         = &quot;yes&quot;<br>Carpet::use_overlap_zones        = &quot;no&quot;<br>Carpet::schedule_barriers        = &quot;no&quot;<br>#Carpet::processor_topology       = &quot;recursive&quot; # recursive breaks multipatch<br><br>Carpet::check_for_poison         = &quot;no&quot;<br>Carpet::init_3_timelevels        = &quot;no&quot;<br>Carpet::init_fill_timelevels     = &quot;yes&quot;<br><br>CarpetLib::poison_new_memory         = &quot;yes&quot;<br>CarpetLib::poison_value              = 114<br>CarpetLib::check_bboxes              = &quot;no&quot;<br>CarpetLib::interleave_communications = &quot;yes&quot;<br>CarpetLib::combine_sends             = &quot;no&quot; # Erik says this is faster and does not use more memory than yes<br>CarpetLib::combine_recompose         = &quot;no&quot; # yes is the default, set to no if we run out of memory during recomposing<br>#CarpetLib::restriction_order_space   = 0<br><br><br>CarpetInterp::tree_search = &quot;yes&quot;<br>CarpetInterp::check_tree_search = &quot;no&quot;<br><br># OVERLAP<br>Coordinates::additional_overlap_size = 3<br><br># BOUNDARY ZONES (needs to be set to three) <br>Coordinates::patch_boundary_size = 3<br>Coordinates::outer_boundary_size = 3<br>#Carpet::prolongation_order_space = 5<br>#Carpet::prolongation_order_time  = 2<br>Carpet::refinement_centering     = &quot;vertex&quot;<br><br>#====================================<br># TmunuBase &amp; HydroBase<br>#====================================<br>TmunuBase::stress_energy_storage = yes<br>TmunuBase::stress_energy_at_RHS  = yes<br>TmunuBase::timelevels            =  1<br>TmunuBase::prolongation_type     = none<br>HydroBase::timelevels            = 3<br><br>#############################################################<br># Carpet<br>#############################################################<br><br>Carpet::domain_from_multipatch = yes<br>Carpet::ghost_size             = 3<br># not for FD<br>#Carpet::granularity            = 5   # (order+1)*m<br>#Carpet::granularity_boundary   = 1<br>#Carpet::poison_new_timelevels = &quot;yes&quot;   # does not work with persistent RHS<br><br>#--------CarpetMask--------------------------<br>ActiveThorns = &quot;CarpetMask&quot;<br>CarpetMask::excluded_surface       [0] = 1<br>CarpetMask::excluded_surface_factor[0] = 1.0<br>#<br>#############################################################<br># Time integration<br>#############################################################<br><br>Cactus::terminate       = &quot;time&quot;<br>Cactus::cctk_final_time = 2000000000000<br>#Cactus::cctk_itlast = 1<br><br>Time::timestep_method = &quot;given&quot;<br>Time::timestep        = 0.10<br><br>MethodOfLines::ODE_method             = &quot;RK4&quot;<br>MethodOfLines::MoL_Intermediate_Steps = 4<br>MethodOfLines::MoL_Num_Scratch_Levels = 1<br>MethodOfLines::MoL_NaN_Check = no<br><br><br>################################################################################<br># Hydro evolution<br>################################################################################<br><br>SpaceMask::use_mask      = &quot;yes&quot;<br><br>#====================================<br># GRHydro<br>#====================================<br>ActiveThorns = &quot;EOS_Omni&quot;<br>ActiveThorns = &quot;GRHydro&quot;<br><br>HydroBase::evolution_method      = &quot;GRHydro&quot;<br>HydroBase::Bvec_evolution_method = &quot;GRHydro&quot;<br># use c++ version of GRHydro<br>GRHydro::use_cxx_code = &quot;yes&quot;<br><br>GRHydro::method_type               = &quot;RSA FV&quot;<br>GRHydro::riemann_solver            = &quot;HLLE&quot;<br>GRHydro::recon_method              = &quot;ppm&quot;<br>GRHydro::recon_vars                = &quot;primitive&quot;<br>GRHydro::GRHydro_stencil           = 3<br>GRHydro::bound                     = &quot;flat&quot; #also tried none<br>GRHydro::rho_abs_min               = 1e-15 <br>GRHydro::initial_rho_abs_min       = 5e-14 <br>GRHydro::GRHydro_atmo_tolerance    = 0.01 <br><br></div><div><br># -- settings for 3-param (tabulated) EOS --<br>grhydro::grhydro_eos_type                =  &quot;General&quot;<br>grhydro::grhydro_eos_table               =  &quot;nuc_eos&quot;<br>EOS_Omni::nuceos_read_table = &quot;yes&quot;<br>EOS_Omni::nuceos_table_name = &quot;/home/d/dsiegel/lcombi/repository/EOS_tables/etk_bns_evolution/sfho/Hempel_SFHoEOS_rho222_temp180_ye60_version_1.1_20120817.h5&quot;<br>GRHydro::GRHydro_eos_rf_prec = 1e-10<br>hydrobase::Y_e_evolution_method          =  &quot;GRHydro&quot;<br>hydrobase::temperature_evolution_method  =  &quot;GRHydro&quot;<br><br>GRHydro::reconstruct_temper             =  &quot;yes&quot; # more robust for reconstruction<br>GRHydro::GRHydro_hot_atmo_temp          =  0.012 #8.62e-06=1e5K<br>GRHydro::GRHydro_hot_atmo_Y_e           =  0.5<br>grhydro::grhydro_y_e_min               =  0.012<br>grhydro::grhydro_y_e_max               =  0.599<br><br><br>GRHydro::Grhydro_MaxNumConstrainedVars = 34 #33 <br>GRHydro::GRHydro_MaxNumEvolvedVars     = 10<br><br># -- settings for con2prim --<br>GRHydro::GRHydro_c2pMHD_method = 1<br>GRHydro::GRHydro_c2pMHD_method_backup = 5<br>GRHydro::GRHydro_use_c2pMHD_method_backup = &quot;yes&quot;<br>GRHydro::GRHydro_c2p_grace_radius  = 7.5<br>grhydro::grhydro_c2p_warnlevel          = 1 #1<br>grhydro::grhydro_c2p_warn_from_reflevel = 5 #5 # 6<br><br># --- EOS Omni ---<br>eos_omni::poly_gamma                            = 1.33333333<br>#eos_omni::poly_gamma_initial                    = 1.333333333<br>eos_omni::poly_k                                = 0.3605 #145.877704 #100<br>eos_omni::gl_gamma                              = 1.33333333<br>eos_omni::gl_k                                  = 0.3605 #145.877704 #100<br><br><br><br>################################################################################<br># Spacetime evolution<br>################################################################################<br><br>ADMBase::metric_type     = &quot;physical&quot;<br>ADMBase::evolution_method        = &quot;static&quot;<br>ADMBase::lapse_evolution_method  = &quot;static&quot;<br>ADMBase::shift_evolution_method  = &quot;static&quot;<br>ADMBase::dtlapse_evolution_method= &quot;static&quot;<br>ADMBase::dtshift_evolution_method= &quot;static&quot;<br><br>ADMBase::metric_timelevels = 3<br>ADMBase::lapse_timelevels = 3<br>ADMBase::shift_timelevels = 3<br>ADMMacros::spatial_order = 4<br><br>#====================================<br># Apparent horizon &amp; excision<br>#====================================<br>ActiveThorns = &quot;hydro_initexcision&quot;<br>hydro_initexcision::hydro_initexcision  = &quot;yes&quot;<br>hydro_initexcision::hydro_initexcision_type  = &quot;sphere&quot;<br>hydro_initexcision::hydro_initexcision_coordinate_length  = 4.2 # should be slightly larger than a_BH = spin_BH*mass_BH <br><br># --- Settings for apparent horizon ---<br>ActiveThorns = &quot;AHFinderDirect SetMask_SphericalSurface&quot;<br>AHFinderDirect::N_horizons                               = 1<br>AHFinderDirect::find_after_individual_time[1]            = 0<br>AHFinderDirect::find_every                               = 20000<br>AHFinderDirect::output_h_every                           = 20000<br>AHFinderDirect::initial_guess_method[1]                   = &quot;Kerr/Kerr-Schild&quot;<br>AHFinderDirect::initial_guess__Kerr_KerrSchild__x_posn[1] = 0.0<br>AHFinderDirect::initial_guess__Kerr_KerrSchild__y_posn[1] = 0.0<br>AHFinderDirect::initial_guess__Kerr_KerrSchild__z_posn[1] = 0.0<br>AHFinderDirect::initial_guess__Kerr_KerrSchild__mass[1] = 3.0  # adjust to settings for TorusID<br>AHFinderDirect::initial_guess__Kerr_KerrSchild__spin[1] = 0.8  # adjust to settings for TorusID<br>AHFinderDirect::geometry_interpolator_name               = &quot;Lagrange polynomial interpolation&quot;<br>AHFinderDirect::geometry_interpolator_pars               = &quot;order=4&quot;<br>AHFinderDirect::surface_interpolator_name                = &quot;Lagrange polynomial interpolation&quot;<br>AHFinderDirect::surface_interpolator_pars                = &quot;order=4&quot;<br>AHFinderDirect::mask_buffer_thickness                    = 0<br>AHFinderDirect::mask_radius_multiplier                   = 0.8<br>AHFinderDirect::mask_radius_offset                       = 0<br>AHFinderDirect::max_allowable_horizon_radius[1]          = 10<br>AHFinderDirect::max_allowable_Theta_growth_iterations    = 10<br>AHFinderDirect::max_allowable_Theta_nonshrink_iterations = 10<br>AHFinderDirect::max_Newton_iterations__initial           = 50<br>AHFinderDirect::max_Newton_iterations__subsequent        = 50<br>AHFinderDirect::which_surface_to_store_info_by_name  [1] = &quot;apparent horizon&quot;<br><br>################################################################################<br># Initial data<br>################################################################################<br><br>InitBase::initial_data_setup_method = &quot;init_some_levels&quot;<br>ADMBase::initial_data            = &quot;exact&quot; #TorusID&quot;<br>ADMBase::initial_lapse           = &quot;exact&quot; #TorusID&quot;<br>ADMBase::initial_shift           = &quot;exact&quot; #TorusID&quot;<br>ADMBase::initial_dtlapse         = &quot;zero&quot;<br>ADMBase::initial_dtshift         = &quot;zero&quot;<br><br>ActiveThorns = &quot;Exact&quot;<br>Exact::exact_model                 = &quot;Kerr/Kerr-Schild&quot;<br>Exact::Kerr_KerrSchild__mass     = 3.0<br>Exact::Kerr_KerrSchild__parabolic = &quot;yes&quot;<br></div><div><br></div><div><br></div><div><br></div><div><br clear="all"><div><div dir="ltr" class="gmail_signature" data-smartmail="gmail_signature"><div dir="ltr"><div>---<br></div><div dir="ltr"><font face="trebuchet ms, sans-serif"><b>Dr. Luciano Combi</b></font></div><div dir="ltr"><div style="font-family:sans-serif"><div style="color:rgb(117,117,117)"><div>Postdoctoral Researcher</div><div>Perimeter Institute for Theoretical Physics<br></div><div>CITA National Fellow (U. of Guelph)</div><div><div style="color:rgb(32,33,36);font-family:Arial,Helvetica,sans-serif">---</div></div></div></div></div></div></div></div></div></div>