[Users] question about Thornburg04nc on Llama

Roland Haas rhaas at illinois.edu
Thu Mar 9 11:37:45 CST 2023


Hello Luciano,

We quickly discussed this in last week's ET call.

Unfortunately not one present had any experience using Llaman with
GRHydro and a non-trivial topology.

Oleg Korobkin did some similar work on accretion disks using a Cactus
based code with a central cutout (but not Llama based but using an
earlier version of Cactus/Carpet based multiblock): 

https://arxiv.org/abs/1011.3010

You may be a able to reach out to him (at LANL).

My own recollection is that hydro_excision is not used very frequently
in GRHydro, to it may or may not actually work at all.

Note when you say "origin" since this is the Llama grid with a hole in
the center, you mean "inner cutout" (ie not r=0 but instead r=3.0)?

Yours,
Roland

> Hi everyone,
> 
> I started testing the Thornburg04nc coordinates (spherical patches without
> Cartesian box) from Llama with (a modified version) of GRHydro. I started
> with just an atmosphere around a Kerr-Schild metric evolving only hydro.
> I'm seeing instabilities near the origin, so I was wondering if someone
> already encountered this issue. In particular, I wanted to know if someone
> has a good idea of how to treat the inner boundary. Right now, I'm putting
> the inner boundary inside the horizon, above the ring singularity, and
> applying hydro excision. I've tried a combination of things playing with
> those parameters but without success. Has anyone tried a similar thing
> evolving hydro? Just want to double check if there's something obvious in
> the grid setup I'm missing.
> 
> Below I attach the relevant part of the par file I'm using. Main difference
> with the public version of GRHydro is that I'm using Siegel+2017 c2p which
> works just fine with the same setup in Cartesian coordinates.
> 
> Thanks a lot!!
> 
> 
> 
> 
> 
> ######
> 
> ActiveThorns = "Time MoL"
> ActiveThorns = "Coordbase CartGrid3d Boundary StaticConformal"
> ActiveThorns = "SymBase ADMBase TmunuBase HydroBase InitBase ADMCoupling
> ADMMacros"
> ActiveThorns = "IOUtil Formaline"
> ActiveThorns = "SpaceMask CoordGauge Constants LocalReduce aeilocalinterp
> LoopControl"
> ActiveThorns = "Carpet CarpetLib CarpetReduce CarpetRegrid2 "
> ActiveThorns = "Coordinates GenericFD Interpolate2 CarpetInterp2
> CarpetInterp"
> ActiveThorns = "CarpetIOASCII CarpetIOScalar CarpetIOHDF5 CarpetIOBasic"
> 
> ################################################################################
> # Grid setup and general parameters
> ################################################################################
> 
> # Grid
> 
> CartGrid3D::type               = "multipatch"
> Coordinates::coordinate_system = "Thornburg04nc"
> CartGrid3D::set_coordinate_ranges_on    = "all maps"
> 
> Coordinates::h_radial            =   0.35
> Coordinates::sphere_inner_radius =  3.0
> Coordinates::sphere_outer_radius =  90.0
> Coordinates::n_angular           =  25
> 
> #Coordinates::radial_stretch             = yes
> #Coordinates::stretch_rmin_1             = 10.0
> #Coordinates::stretch_rmax_1             = 90.0
> 
> Coordinates::stagger_patch_boundaries = "no"
> Coordinates::stagger_outer_boundaries = "no"
> 
> Interpolate::interpolator_order = 4
> Interpolate2::shift_edges        = no
> 
> Coordinates::store_jacobian            = yes # I need to
> Coordinates::store_inverse_jacobian    = yes # Check for runtime errors
> #Coordinates::store_jacobian_derivative = yes
> Coordinates::store_volume_form         = yes
> 
> #GenericFD::jacobian_group            = "Coordinates::jacobian"
> #GenericFD::jacobian_derivative_group = "Coordinates::jacobian2"
> 
> # General Carpet parameters:
> Carpet::enable_all_storage       = "no"
> Carpet::use_buffer_zones         = "yes"
> Carpet::use_overlap_zones        = "no"
> Carpet::schedule_barriers        = "no"
> #Carpet::processor_topology       = "recursive" # recursive breaks
> multipatch
> 
> Carpet::check_for_poison         = "no"
> Carpet::init_3_timelevels        = "no"
> Carpet::init_fill_timelevels     = "yes"
> 
> CarpetLib::poison_new_memory         = "yes"
> CarpetLib::poison_value              = 114
> CarpetLib::check_bboxes              = "no"
> CarpetLib::interleave_communications = "yes"
> CarpetLib::combine_sends             = "no" # Erik says this is faster and
> does not use more memory than yes
> CarpetLib::combine_recompose         = "no" # yes is the default, set to no
> if we run out of memory during recomposing
> #CarpetLib::restriction_order_space   = 0
> 
> 
> CarpetInterp::tree_search = "yes"
> CarpetInterp::check_tree_search = "no"
> 
> # OVERLAP
> Coordinates::additional_overlap_size = 3
> 
> # BOUNDARY ZONES (needs to be set to three)
> Coordinates::patch_boundary_size = 3
> Coordinates::outer_boundary_size = 3
> #Carpet::prolongation_order_space = 5
> #Carpet::prolongation_order_time  = 2
> Carpet::refinement_centering     = "vertex"
> 
> #====================================
> # TmunuBase & HydroBase
> #====================================
> TmunuBase::stress_energy_storage = yes
> TmunuBase::stress_energy_at_RHS  = yes
> TmunuBase::timelevels            =  1
> TmunuBase::prolongation_type     = none
> HydroBase::timelevels            = 3
> 
> #############################################################
> # Carpet
> #############################################################
> 
> Carpet::domain_from_multipatch = yes
> Carpet::ghost_size             = 3
> # not for FD
> #Carpet::granularity            = 5   # (order+1)*m
> #Carpet::granularity_boundary   = 1
> #Carpet::poison_new_timelevels = "yes"   # does not work with persistent RHS
> 
> #--------CarpetMask--------------------------
> ActiveThorns = "CarpetMask"
> CarpetMask::excluded_surface       [0] = 1
> CarpetMask::excluded_surface_factor[0] = 1.0
> #
> #############################################################
> # Time integration
> #############################################################
> 
> Cactus::terminate       = "time"
> Cactus::cctk_final_time = 2000000000000
> #Cactus::cctk_itlast = 1
> 
> Time::timestep_method = "given"
> Time::timestep        = 0.10
> 
> MethodOfLines::ODE_method             = "RK4"
> MethodOfLines::MoL_Intermediate_Steps = 4
> MethodOfLines::MoL_Num_Scratch_Levels = 1
> MethodOfLines::MoL_NaN_Check = no
> 
> 
> ################################################################################
> # Hydro evolution
> ################################################################################
> 
> SpaceMask::use_mask      = "yes"
> 
> #====================================
> # GRHydro
> #====================================
> ActiveThorns = "EOS_Omni"
> ActiveThorns = "GRHydro"
> 
> HydroBase::evolution_method      = "GRHydro"
> HydroBase::Bvec_evolution_method = "GRHydro"
> # use c++ version of GRHydro
> GRHydro::use_cxx_code = "yes"
> 
> GRHydro::method_type               = "RSA FV"
> GRHydro::riemann_solver            = "HLLE"
> GRHydro::recon_method              = "ppm"
> GRHydro::recon_vars                = "primitive"
> GRHydro::GRHydro_stencil           = 3
> GRHydro::bound                     = "flat" #also tried none
> GRHydro::rho_abs_min               = 1e-15
> GRHydro::initial_rho_abs_min       = 5e-14
> GRHydro::GRHydro_atmo_tolerance    = 0.01
> 
> 
> # -- settings for 3-param (tabulated) EOS --
> grhydro::grhydro_eos_type                =  "General"
> grhydro::grhydro_eos_table               =  "nuc_eos"
> EOS_Omni::nuceos_read_table = "yes"
> EOS_Omni::nuceos_table_name =
> "/home/d/dsiegel/lcombi/repository/EOS_tables/etk_bns_evolution/sfho/Hempel_SFHoEOS_rho222_temp180_ye60_version_1.1_20120817.h5"
> GRHydro::GRHydro_eos_rf_prec = 1e-10
> hydrobase::Y_e_evolution_method          =  "GRHydro"
> hydrobase::temperature_evolution_method  =  "GRHydro"
> 
> GRHydro::reconstruct_temper             =  "yes" # more robust for
> reconstruction
> GRHydro::GRHydro_hot_atmo_temp          =  0.012 #8.62e-06=1e5K
> GRHydro::GRHydro_hot_atmo_Y_e           =  0.5
> grhydro::grhydro_y_e_min               =  0.012
> grhydro::grhydro_y_e_max               =  0.599
> 
> 
> GRHydro::Grhydro_MaxNumConstrainedVars = 34 #33
> GRHydro::GRHydro_MaxNumEvolvedVars     = 10
> 
> # -- settings for con2prim --
> GRHydro::GRHydro_c2pMHD_method = 1
> GRHydro::GRHydro_c2pMHD_method_backup = 5
> GRHydro::GRHydro_use_c2pMHD_method_backup = "yes"
> GRHydro::GRHydro_c2p_grace_radius  = 7.5
> grhydro::grhydro_c2p_warnlevel          = 1 #1
> grhydro::grhydro_c2p_warn_from_reflevel = 5 #5 # 6
> 
> # --- EOS Omni ---
> eos_omni::poly_gamma                            = 1.33333333
> #eos_omni::poly_gamma_initial                    = 1.333333333
> eos_omni::poly_k                                = 0.3605 #145.877704 #100
> eos_omni::gl_gamma                              = 1.33333333
> eos_omni::gl_k                                  = 0.3605 #145.877704 #100
> 
> 
> 
> ################################################################################
> # Spacetime evolution
> ################################################################################
> 
> ADMBase::metric_type     = "physical"
> ADMBase::evolution_method        = "static"
> ADMBase::lapse_evolution_method  = "static"
> ADMBase::shift_evolution_method  = "static"
> ADMBase::dtlapse_evolution_method= "static"
> ADMBase::dtshift_evolution_method= "static"
> 
> ADMBase::metric_timelevels = 3
> ADMBase::lapse_timelevels = 3
> ADMBase::shift_timelevels = 3
> ADMMacros::spatial_order = 4
> 
> #====================================
> # Apparent horizon & excision
> #====================================
> ActiveThorns = "hydro_initexcision"
> hydro_initexcision::hydro_initexcision  = "yes"
> hydro_initexcision::hydro_initexcision_type  = "sphere"
> hydro_initexcision::hydro_initexcision_coordinate_length  = 4.2 # should be
> slightly larger than a_BH = spin_BH*mass_BH
> 
> # --- Settings for apparent horizon ---
> ActiveThorns = "AHFinderDirect SetMask_SphericalSurface"
> AHFinderDirect::N_horizons                               = 1
> AHFinderDirect::find_after_individual_time[1]            = 0
> AHFinderDirect::find_every                               = 20000
> AHFinderDirect::output_h_every                           = 20000
> AHFinderDirect::initial_guess_method[1]                   =
> "Kerr/Kerr-Schild"
> AHFinderDirect::initial_guess__Kerr_KerrSchild__x_posn[1] = 0.0
> AHFinderDirect::initial_guess__Kerr_KerrSchild__y_posn[1] = 0.0
> AHFinderDirect::initial_guess__Kerr_KerrSchild__z_posn[1] = 0.0
> AHFinderDirect::initial_guess__Kerr_KerrSchild__mass[1] = 3.0  # adjust to
> settings for TorusID
> AHFinderDirect::initial_guess__Kerr_KerrSchild__spin[1] = 0.8  # adjust to
> settings for TorusID
> AHFinderDirect::geometry_interpolator_name               = "Lagrange
> polynomial interpolation"
> AHFinderDirect::geometry_interpolator_pars               = "order=4"
> AHFinderDirect::surface_interpolator_name                = "Lagrange
> polynomial interpolation"
> AHFinderDirect::surface_interpolator_pars                = "order=4"
> AHFinderDirect::mask_buffer_thickness                    = 0
> AHFinderDirect::mask_radius_multiplier                   = 0.8
> AHFinderDirect::mask_radius_offset                       = 0
> AHFinderDirect::max_allowable_horizon_radius[1]          = 10
> AHFinderDirect::max_allowable_Theta_growth_iterations    = 10
> AHFinderDirect::max_allowable_Theta_nonshrink_iterations = 10
> AHFinderDirect::max_Newton_iterations__initial           = 50
> AHFinderDirect::max_Newton_iterations__subsequent        = 50
> AHFinderDirect::which_surface_to_store_info_by_name  [1] = "apparent
> horizon"
> 
> ################################################################################
> # Initial data
> ################################################################################
> 
> InitBase::initial_data_setup_method = "init_some_levels"
> ADMBase::initial_data            = "exact" #TorusID"
> ADMBase::initial_lapse           = "exact" #TorusID"
> ADMBase::initial_shift           = "exact" #TorusID"
> ADMBase::initial_dtlapse         = "zero"
> ADMBase::initial_dtshift         = "zero"
> 
> ActiveThorns = "Exact"
> Exact::exact_model = "Kerr/Kerr-Schild"
> Exact::Kerr_KerrSchild__mass     = 3.0
> Exact::Kerr_KerrSchild__parabolic = "yes"
> 
> 
> 
> 
> ---
> *Dr. Luciano Combi*
> Postdoctoral Researcher
> Perimeter Institute for Theoretical Physics
> CITA National Fellow (U. of Guelph)
> ---

-- 
My email is as private as my paper mail. I therefore support encrypting
and signing email messages. Get my PGP key from http://pgp.mit.edu .
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 833 bytes
Desc: OpenPGP digital signature
Url : http://lists.einsteintoolkit.org/pipermail/users/attachments/20230309/a48c864c/attachment-0001.bin 


More information about the Users mailing list