#2861: Running qc0 with CarpetX

Reporter: Alejandra Gonzalez Status: open Milestone: Version: Type: bug Priority: major Component: EinsteinToolkit thorn

Comment (by Lucas Timotheo Sanches):

Hi Alejandra.

I was unable to make `Punctures` work by simply modifying parameters, which suggests that the thorn has a more fundamental issue. This will need to be further investigated at another opportune moment.

Regardless, as you noted `TwoPuncturesX` is the preferred option for initial data and should be used instead.

Yes, your output seems to indicate that your execution of the parameter file was successful. As you may have noticed, this example parameter file only runs for a single iteration, which is probably not very useful for your needs.

As for output formats, here’s an abbreviated guideline

- Whenever you intend to visualize 3D data, use the `Silo` format. The `Silo` format can be used natively with [LLNL’s VisIt](https://visit-dav.github.io/visit-website/index.html). This is probably the easiest way to visualize `CarpetX` data, as it allows you to simply read the files and create plots by using a GUI. The `OpenPMD` file is also an option for visualization, but you may have to write custom Python scripts for it. - Whenever you wish to store data, in Checkpoints, for instance, the `ADIOS2` and `OpenPMD` files are adequate. - `TSV` data, which is plain text, not recommended for storage, in general. It is more of a debug format, where you can look at grid function values in a human readable manner. `TSV` data is also adequate for storing the multipolar extractions of the Weyl scalar and the location of the punctures throughout the simulation.

I have compiled a more complex `qc0` parameter file, which uses `TwoPunctureX` for initial data, but also performs Apparent Horizon finding, puncture tracking and multipole extractions of the Psi4 Weyl scalar. This example should give you a good idea on what production run looks like and may be useful to your applications.

``` # Settings ################################# $itlast = 1000000000 $final_time = 200 $max_runtime = 23 * 60 # In minutes

$xmin = -120.0 $xmax = +120.0

$ncells = 120 $nlevels = 7

$time_stepper = "RK4" $dtfac = 0.25 $error_threshold = 0.9

$out_every = 32 $out_silo_every = 64 * 8 $wave_extract_every = 32 #################################

ActiveThorns = " ADMBaseX AEILocalInterp AHFinderDirect BoxInBox CarpetX CoordinatesX IOUtil Multipole NewRadX ODESolvers PunctureTracker SphericalSurface TimerReport TmunuBaseX TwoPuncturesX WeylScal4 Z4c "

# Driver Cactus::cctk_show_schedule = yes Cactus::presync_mode = "mixed-error" CarpetX::verbose = no CarpetX::poison_undefined_values = no

# Run Termination Cactus::terminate = "any" Cactus::cctk_itlast = $itlast Cactus::cctk_final_time = $final_time Cactus::max_runtime = $max_runtime

# Grid setup CarpetX::xmin = $xmin CarpetX::ymin = $xmin CarpetX::zmin = $xmin CarpetX::xmax = $xmax CarpetX::ymax = $xmax CarpetX::zmax = $xmax CarpetX::ncells_x = $ncells CarpetX::ncells_y = $ncells CarpetX::ncells_z = $ncells CarpetX::ghost_size = 3 CarpetX::interpolation_order = 3 CarpetX::prolongation_type = "ddf" CarpetX::prolongation_order = 5

# Boundary conditions CarpetX::boundary_x = "linear extrapolation" CarpetX::boundary_y = "linear extrapolation" CarpetX::boundary_z = "linear extrapolation" CarpetX::boundary_upper_x = "linear extrapolation" CarpetX::boundary_upper_y = "linear extrapolation" CarpetX::boundary_upper_z = "linear extrapolation"

# Mesh refinement CarpetX::max_num_levels = $nlevels CarpetX::regrid_every = 128 CarpetX::regrid_error_threshold = $error_threshold BoxInBox::num_regions = 2

# Region 1 BoxInBox::shape_1 = "cube" BoxInBox::num_levels_1 = $nlevels BoxInBox::position_x_1 = +1.168642873 BoxInBox::radius_1 = [-1., 64., 16., 8., 4., 2., 1.]

# Region 2 BoxInBox::shape_2 = "cube" BoxInBox::num_levels_2 = $nlevels BoxInBox::position_x_2 = -1.168642873 BoxInBox::radius_2 = [-1., 64., 16., 8., 4., 2., 1.]

# Initial data ADMBaseX::initial_data = "TwoPunctures" ADMBaseX::initial_lapse = "TwoPunctures-averaged" TwoPuncturesX::verbose = yes TwoPuncturesX::par_b = 1.168642873 TwoPuncturesX::par_m_plus = 0.453 TwoPuncturesX::par_m_minus = 0.453 TwoPuncturesX::par_P_plus [1] = +0.3331917498 TwoPuncturesX::par_P_minus[1] = -0.3331917498 TwoPuncturesX::grid_setup_method = "evaluation" TwoPuncturesX::TP_epsilon = 1.0e-2 TwoPuncturesX::TP_Tiny = 1.0e-2

# Time Integration CarpetX::dtfac = $dtfac ODESolvers::verbose = no ODESolvers::method = $time_stepper

# Spacetime evolution Z4c::calc_ADM_vars = yes #Required for computing the Weyl scalar Z4c::calc_ADMRHS_vars = yes #Required for computing the Weyl scalar Z4c::calc_constraints = yes Z4c::chi_floor = 1.0e-6 Z4c::alphaG_floor = 1.0e-8 Z4c::epsdiss = 0.32 Z4C::boundary_conditions = "NewRadX"

# Spherical surfaces SphericalSurface::nsurfaces = 5 SphericalSurface::maxntheta = 39 SphericalSurface::maxnphi = 76 SphericalSurface::verbose = no

# Surfaces 0 and 1 are used by PunctureTracker

# Tracker 1 SphericalSurface::ntheta [0] = 39 SphericalSurface::nphi [0] = 76 SphericalSurface::nghoststheta [0] = 2 SphericalSurface::nghostsphi [0] = 2

# Tracker 2 SphericalSurface::ntheta [1] = 39 SphericalSurface::nphi [1] = 76 SphericalSurface::nghoststheta [1] = 2 SphericalSurface::nghostsphi [1] = 2

# Surfaces 2, 3, 4 are used by AHFinderDirect and QuasiLocalMeasures # Horizon 1 SphericalSurface::ntheta [2] = 39 SphericalSurface::nphi [2] = 76 SphericalSurface::nghoststheta [2] = 2 SphericalSurface::nghostsphi [2] = 2

# Horizon 2 SphericalSurface::ntheta [3] = 39 SphericalSurface::nphi [3] = 76 SphericalSurface::nghoststheta [3] = 2 SphericalSurface::nghostsphi [3] = 2

# Horizon 3 SphericalSurface::ntheta [4] = 39 SphericalSurface::nphi [4] = 76 SphericalSurface::nghoststheta [4] = 2 SphericalSurface::nghostsphi [4] = 2

# Puncture tracking PunctureTracker::verbose = no PunctureTracker::track_boxes = yes PunctureTracker::interp_order = 3 PunctureTracker::track [0] = yes PunctureTracker::initial_x [0] = +1.168642873 PunctureTracker::track [1] = yes PunctureTracker::initial_x [1] = -1.168642873

# Apparent horizons AHFinderDirect::find_every = $wave_extract_every AHFinderDirect::verbose_level = "physics details" AHFinderDirect::output_BH_diagnostics = "true" AHFinderDirect::run_at_CCTK_POST_RECOVER_VARIABLES = no AHFinderDirect::output_h_every = 0 # Turn off horizon shape output

AHFinderDirect::max_Newton_iterations__initial = 50 AHFinderDirect::max_Newton_iterations__subsequent = 50 AHFinderDirect::max_allowable_Theta_growth_iterations = 10 AHFinderDirect::max_allowable_Theta_nonshrink_iterations = 10 AHFinderDirect::geometry_interpolator_name = "CarpetX" AHFinderDirect::geometry_interpolator_pars = "order=3 boundary_off_centering_tolerance={1.0e-10 1.0e-10 1.0e-10 1.0e-10 1.0e-10 1.0e-10} boundary_extrapolation_tolerance={0.0 0.0 0.0 0.0 0.0 0.0}" AHFinderDirect::surface_interpolator_name = "Lagrange polynomial interpolation" AHFinderDirect::surface_interpolator_pars = "order=3"

AHFinderDirect::move_origins = yes AHFinderDirect::reshape_while_moving = yes AHFinderDirect::predict_origin_movement = yes

AHFinderDirect::N_horizons = 3

AHFinderDirect::origin_x [1] = +1.168642873 AHFinderDirect::origin_y [1] = 0 AHFinderDirect::origin_z [1] = 0 AHFinderDirect::initial_guess_method [1] = "coordinate sphere" AHFinderDirect::initial_guess__coord_sphere__x_center[1] = +1.168642873 AHFinderDirect::initial_guess__coord_sphere__y_center[1] = 0 AHFinderDirect::initial_guess__coord_sphere__z_center[1] = 0 AHFinderDirect::initial_guess__coord_sphere__radius [1] = 0.25 AHFinderDirect::which_surface_to_store_info [1] = 2 AHFinderDirect::set_mask_for_individual_horizon [1] = no AHFinderDirect::reset_horizon_after_not_finding [1] = no AHFinderDirect::dont_find_after_individual_time [1] = 30.0

AHFinderDirect::origin_x [2] = -1.168642873 AHFinderDirect::origin_y [2] = 0 AHFinderDirect::origin_z [2] = 0 AHFinderDirect::initial_guess_method [2] = "coordinate sphere" AHFinderDirect::initial_guess__coord_sphere__x_center[2] = -1.168642873 AHFinderDirect::initial_guess__coord_sphere__y_center[2] = 0 AHFinderDirect::initial_guess__coord_sphere__z_center[2] = 0 AHFinderDirect::initial_guess__coord_sphere__radius [2] = 0.25 AHFinderDirect::which_surface_to_store_info [2] = 3 AHFinderDirect::reset_horizon_after_not_finding [2] = no AHFinderDirect::dont_find_after_individual_time [2] = 30.0

AHFinderDirect::origin_x [3] = 0.0 AHFinderDirect::origin_y [3] = 0 AHFinderDirect::origin_z [3] = 0 AHFinderDirect::initial_guess_method [3] = "coordinate sphere" AHFinderDirect::initial_guess__coord_sphere__x_center[3] = 0.0 AHFinderDirect::initial_guess__coord_sphere__y_center[3] = 0 AHFinderDirect::initial_guess__coord_sphere__z_center[3] = 0 AHFinderDirect::initial_guess__coord_sphere__radius [3] = 0.6 AHFinderDirect::which_surface_to_store_info [3] = 4 AHFinderDirect::find_after_individual_time [3] = 0.0

# Wave extraction WeylScal4::WeylScal4_psi4_calc_4th_calc_every = $wave_extract_every

Multipole::verbose = no Multipole::nradii = 8 Multipole::radius[0] = 15 Multipole::radius[1] = 30 Multipole::radius[2] = 40 Multipole::radius[3] = 50 Multipole::radius[4] = 60 Multipole::radius[5] = 70 Multipole::radius[6] = 80 Multipole::radius[7] = 90 Multipole::ntheta = 120 Multipole::nphi = 240 Multipole::variables = " WeylScal4::Psi4r{sw=-2 cmplx='WeylScal4::Psi4i' name='Psi4'} " Multipole::out_every = $wave_extract_every Multipole::l_max = 4 Multipole::interpolator_pars = "order=3"

# Checkpoint and recovery IO::out_dir = $parfile IO::out_every = $out_every IO::parfile_write = no IO::checkpoint_dir = "checkpoints_$parfile" IO::recover_dir = "checkpoints_$parfile" IO::checkpoint_ID = no IO::checkpoint_every_walltime_hours = 6 IO::checkpoint_on_terminate = yes IO::recover = "autoprobe" CarpetX::checkpoint_method = "openpmd" CarpetX::recover_method = "openpmd"

# Output TimerReport::out_every = $out_every TimerReport::out_filename = "TimerReport" TimerReport::output_schedule_timers = no TimerReport::n_top_timers = 100

# Norm and reduction output CarpetX::out_norm_vars = "all" CarpetX::out_norm_omit_unstable = yes

# 1D output CarpetX::out_tsv_vars = " PunctureTracker::pt_loc PunctureTracker::pt_vel "

# 3D output CarpetX::out_silo_every = $out_silo_every CarpetX::out_silo_vars = " ADMBaseX::alp " ```

‌

-- Ticket URL: https://bitbucket.org/einsteintoolkit/tickets/issues/2861/running-qc0-with-c...