[Users] McLachlan and Cartoon2D conflicting?
Federico Maria Guercilena
fguercilena at theorie.ikp.physik.tu-darmstadt.de
Fri Sep 18 05:55:24 CDT 2020
Hello everybody,
I investigated a bit more the problem I've been seeing in running
McLachlan (ML_CCZ4) and Cartoon2D. I've simplified my setup to a simple,
single puncture BH in a cartoon-adapted domain, I attach the parfile I
used, I think it is now standard ET (in the sense that it does not
contain code that doesn't ship with the official ET releases). I'm using
the ET_2020_05 release, with all the bugfixes backported to it, plus
Roland fix for NewRad's extrapolate-gammas routine from the master
branch. I also attach some plots of the simulation output.
The problem persists even in pure vacuum. With ML_CCZ4 + Cartoon2D, at
high resolution there are NaNs in the constraints at it=0 around the
z=axis, despite all the metric fields looking regular. Evolving this
data leads to the simulation crashing, in the sense that e.g. the lapse
starts oscillating and gets pushed down to the minimum value of 1e-8
everywhere, and almost immediately (in my setup, by iteration ~80). At
that point there is no BH anymore (or maybe it has expanded to encompass
the whole domain, as sometimes happens in the case of gauge pathologies,
the "collapse of the lapse"?).
Retrying the same setup but with Lean instead of ML_CCZ4 shows none of
the issues above. No NaNs, no evolution issues, the expected result.
This confirms that the hydro had nothing to do with it, and neither did
the extrapolate-gammas fix for Cartoon (which as said above, I'm using
anyway). As I said in previous email, I even tried to manually change
the ML_CCZ4 schedule.ccl (and a few of ML_CCZ4 source code files as
well) to make it as similar as possible to Lean's, reasoning that the
issue might be in some ordering conflict between Cartoon2D boundary
operations and ML_CCZ4 handling of boundary conditions, syncing, etc.,
but this also didn't solve the problem.
So once again I'm stuck. Has anyone experienced such an issue with Cartoon?
Thank you very much,
Federico
On 9/14/20 10:30 PM, Federico Maria Guercilena wrote:
> Hello Roland,
>
>
> thank you for the swift reply. Unfortunately my issue is not related to
> NewRad's ExtrapolateGammas function. I pulled the modifications that you
> made to it, rebuilt the ET and re-ran my test, with the exact same
> result. I had a feeling it would turn out this way, because both Lean
> and ML_CCZ4 call ExtrapolateGammas (and at essentially the same point in
> the scheduler), but only ML_CCZ4 fails, so it's unlikely that the origin
> of the problem is in NewRad.
>
> As for GCC10, maybe my short post scriptum was not clear enough. I have
> no more compilation issues, thanks to the recent fix to the master and
> release branch (I'm using the latter). I just wanted to express my
> thanks regarding that, no more.
>
>
> Thank you,
>
> Federico
>
>
> On 9/14/20 8:59 PM, Roland Haas wrote:
>> Hello Federico,
>>
>> not having quite read the whole email (in between meetings), but seeing
>> the 2D, McLachlan, and Cartoon2D keywords: you are are possibly seeing
>> an variant of the the issue that is described here:
>>
>> https://bitbucket.org/einsteintoolkit/tickets/issues/2449/newrad-cannot-be-used-with-cartoon2d-due
>>
>> If this is indeed the case, then this (should be) is fixed in the
>> master branch of the NewRad thorn as of today (couple of hours ago,
>> namely it needs to include commit 080c20b "NewRad: do not extrapolate
>> into symmetry boundaries").
>>
>> Yes, I believe that the gcc-10/gfortran-10 issues have been cleaned up
>> in master (couple of weeks ago) and also on the release branch so a
>> fresh checkout (or GetComponenonts --update --thornlist
>> Cactus/manifest/einteiintoolkit.th) should no longer fail to compile.
>>
>> If it is failing for you, would you mind sending the file sim.log
>> produced by:
>>
>> VERBOSE=yes simfactory/bin/sim build 2>&1 | tee sim.log
>>
>> as well as the file git.log produced by:
>>
>> for i in repos/*/.git ; do echo $i ; GIT_DIR=$i git log --pretty=oneline HEAD~1..HEAD ; done 2>&1 | tee git.log
>>
>> please?
>>
>> The first file will contain the error message and the second file exactly
>> which commit is checked out.
>>
>> Are you still seeing issues compiling using a fresh, fully up to date
>> ET checkout (master or release)?
>>
>> Yours,
>> Roland
>>
>>
>>> Hello everyone,
>>>
>>>
>>> I have found a very weird issue in trying to run 2D axysimmetric simulations with Cartoon2D and McLachlan, I'm really puzzled on what could be the cause and how to address it.
>>>
>>> In brief: in running a test TOV simulation with the above setup, the constraints have NaNs around the z-axis although all hydro and spacetime variables are fine, and the simulation crashes after a few iterations due to the NaNs propagating. However, this behavior is resolution-dependent (nothing strange at low resolution). Also Lean doesn't show any problems at any resolution, and the simulations run just fine in this case.
>>>
>>>
>>> More in detail, the setup that show this problem (the parfile is attached) is just a test TOV. It is a 2D axisymmetric run, using Cartoon2D. The spacetime is handled by ML_CCZ4, and I'm using David Radice's THC code for the hydro evolution (which strictly speaking is not part of the ET, but as you'll see the hydro is not at fault here). The initial data is handled by TOVSolver. When running at the resolution labelled as "mid" in the parfile, everything goes as planned, the ID is set up properly (in particular constraints violations are negligible), and the evolution proceeds nicely (the amplitude of the star oscillations seems a bit higher than it would be in a 3D simulation, but that might just be noise from the Cartoon2D boundary conditions).
>>>
>>> However when using the "high" or "fine" resolutions in the parfile (respectively 1.25 and 1.25^2=1.5625 times higher than mid), there are NaNs in the constraints, especially the Hamiltonian constraint, around the z-axis, in a sort of band (see the attached plot). The NaNs are on all reflevels, and the width of the band on the xz-plane varies, seemingly not just because the resolution varies between reflevels, but the NaNs band is sometimes a 1 or 2 points wider or narrower. Trying to evolve this data the NaNs propagate, the con2prim kills the hydro and progressively I end up with a "vacuum" simulation. ML_BSSN behaves similarly.
>>>
>>> All variables (primitives, conserved, spacetime, stress-energy tensor) are properly set as far as I can tell (no NaNs or weird values, and correct symmetry), even ADMMass::ADMMass_VolumeMass_GF. The fact that this last one is ok gave me the hunch that the problem is in the spacetime solver, and in fact switching to Lean did the trick: proper evolution at all resolutions. This also tells me the hydro code is not to blame, nor the initial data code (tried with a different, custom written one, no change).
>>>
>>> For various reasons (Lean is slower and doesn't implement CCZ4) I'd like to stick to ML_CCZ4. At first I thought that the way ML_CCZ4 handles symmetries and boundaries doesn't play well with Cartoon2D (some ordering problem), so I tried to modify its scheduler to make it as close to Lean as possible, but no change. It might be a problem with my grid, but then I don't see why Lean should work. I'm starting to think there's an issue with McLachlan internals (... the derivative stencils?), but than it should show up in 3D runs as well.
>>>
>>> In brief, I'm baffled.
>>>
>>> I'd be grateful for any insights, and of course I'll open a ticket if need be. But I'm still harboring a glimpse of hope that this is just a trivial mistake on my part...
>>>
>>>
>>> Thank you very much,
>>>
>>> Federico Guercilena
>>>
>>>
>>> PS Unrelated: in a couple of previous mails I wrote about the GCC10 issues, and how on my laptop I couldn't compile despite setting the necessary flags. That is still the case and I still have no idea why, but thanks to the latest efforts (mainly by Roland, as I understand?) the ET has been made "GCC10-proof" and that is not an issue anymore. I just wanted to say thanks.
>>>
>>>
--
Dr. Federico Maria Guercilena
Technische Universität Darmstadt
Institut für Kernphysik
Theoriezentrum
S2|11
Schlossgartenstraße 12
64289 Darmstadt
Room 302
+49 6151 16 21562
fguercilena at theorie.ikp.physik.tu-darmstadt.de
-------------- next part --------------
#!/usr/bin/env python2
from math import *
import sys
import re
from string import Template
from subprocess import call
import os
###########################
# Outer boundary
outer_boundary = 5120.
###########################
# Resolution
# The resolutions increase in steps of 1.25, a constant factor,
# which should make it easy to perform a convergence study.
# The resolution is set by setting directly dx on the coarsest level.
# The values [12.8, 10.24, 8.192] result in resolutions of
# [0.2, 0.16, 0.128] on level 6
try:
resolution = str("@RESOLUTION@")
except:
resolution = "mid"
if resolution == "@"+"RESOLUTION"+"@":
resolution = "mid"
if resolution=="mid":
dx = 12.8
elif resolution=="high":
dx = 10.24
elif resolution=="fine":
dx = 8.192
else:
print("Unknown RESOLUTION keyword")
sys.exit()
###########################
# CIDRFILE
try :
cidr_file = str("@CIDRF@")
except :
cidr_file = "nosuchthing"
if cidr_file == "@"+"CIDRF"+"@":
cidr_file = "nosuchthing"
###########################
# EVOLVE
try :
evolve = str("@EVOLVE@")
except :
evolve = "no"
if evolve == "@"+"EVOLVE"+"@":
evolve = "no"
if evolve == "no":
end = "iteration"
elif evolve == "yes":
end = "never"
###########################
# betaDriver
try:
betaDriver = float(str("@BDRIVER@"))
except:
betaDriver = 1.0
if betaDriver == "@"+"BDRIVER"+"@":
betaDriver = 1.0
lines = """
ActiveThorns = "
ADMBase
ADMCoupling
ADMMacros
AEILocalInterp
Boundary
Carpet
CarpetIOASCII
CarpetIOBasic
CarpetIOHDF5
CarpetIOScalar
CarpetInterp
CarpetLib
CarpetReduce
CarpetRegrid2
CarpetSlab
CartGrid3D
Cartoon2D
Constants
CoordBase
CoordGauge
Dissipation
Fortran
GenericFD
IOUtil
InitBase
LeanBSSNMoL
LoopControl
MoL
NaNChecker
NewRad
Nice
ReflectionSymmetry
Slab
SpaceMask
SphericalSurface
StaticConformal
SymBase
SystemStatistics
SystemTopology
TerminationTrigger
Time
TimerReport
TmunuBase
TwoPunctures
WatchDog
hwloc
"
# =============================================================================
# General
# =============================================================================
Cactus::cctk_run_title = "Cartoon BH Lean"
Cactus::cctk_show_banners = "no"
Cactus::recovery_mode = "strict"
Cactus::schedule_sort_mode = "ascending"
Cactus::terminate = "$end"
Cactus::cctk_itlast = 0
TerminationTrigger::max_walltime = @WALLTIME_HOURS@
TerminationTrigger::on_remaining_walltime = 10
TerminationTrigger::create_termination_file = "yes"
TerminationTrigger::termination_from_file = "yes"
TerminationTrigger::termination_file = "../TERMINATE"
TerminationTrigger::output_remtime_every_minutes = 120
TimerReport::out_every = 0
TimerReport::output_all_timers_readable = "yes"
Nice::Nice_nice = 0
# =============================================================================
# Grid
# =============================================================================
Grid::avoid_origin = "no"
Grid::domain = "full"
Grid::type = "coordbase"
ReflectionSymmetry::reflection_x = "no"
ReflectionSymmetry::reflection_y = "no"
ReflectionSymmetry::reflection_z = "yes"
ReflectionSymmetry::avoid_origin_x = "no"
ReflectionSymmetry::avoid_origin_y = "no"
ReflectionSymmetry::avoid_origin_z = "no"
CoordBase::xmin = 0.
CoordBase::xmax = +$outer_boundary
CoordBase::ymin = 0.
CoordBase::ymax = 0.
CoordBase::zmin = 0.
CoordBase::zmax = +$outer_boundary
CoordBase::spacing = "gridspacing"
CoordBase::dx = $dx
CoordBase::dy = $dx
CoordBase::dz = $dx
CoordBase::boundary_size_x_lower = 3
CoordBase::boundary_size_x_upper = 3
CoordBase::boundary_shiftout_x_lower = 1
CoordBase::boundary_shiftout_x_upper = 1
CoordBase::boundary_staggered_x_lower = "no"
CoordBase::boundary_staggered_x_upper = "no"
CoordBase::boundary_size_y_lower = 3
CoordBase::boundary_size_y_upper = 3
CoordBase::boundary_shiftout_y_lower = 1
CoordBase::boundary_shiftout_y_upper = 1
CoordBase::boundary_staggered_y_lower = "no"
CoordBase::boundary_staggered_y_upper = "no"
CoordBase::boundary_size_z_lower = 3
CoordBase::boundary_size_z_upper = 3
CoordBase::boundary_shiftout_z_lower = 1
CoordBase::boundary_shiftout_z_upper = 1
CoordBase::boundary_staggered_z_lower = "no"
CoordBase::boundary_staggered_z_upper = "no"
Driver::ghost_size = 3
Driver::ghost_size_x = 3
Driver::ghost_size_y = 3
Driver::ghost_size_z = 3
Cartoon2D::cartoon_active = "yes"
Cartoon2D::order = 3
Cartoon2D::eno_order = 3
Carpet::processor_topology = "automatic"
Carpet::no_split_direction = 1
Carpet::domain_from_coordbase = "yes"
InitBase::initial_data_setup_method = "init_some_levels"
LoopControl::settle_after_iteration = 0
Carpet::max_refinement_levels = 7
Carpet::prolongation_order_space = 3
Carpet::prolongation_order_time = 2
CarpetLib::use_higher_order_restriction = "no"
Carpet::use_buffer_zones = "no"
Carpet::enable_all_storage = "no"
Carpet::init_fill_timelevels = "yes"
Carpet::time_refinement_factors = "[1,1,2,4,8,16,32,64,128,256,512]"
Carpet::grid_coordinates_filename = "grid.carpet"
CarpetLib::poison_new_memory = "yes"
CarpetRegrid2::num_centres = 1
CarpetRegrid2::regrid_every = -1
CarpetRegrid2::ensure_proper_nesting = "yes"
CarpetRegrid2::snap_to_coarse = "yes"
CarpetRegrid2::freeze_unaligned_levels = "yes"
CarpetRegrid2::freeze_unaligned_parent_levels = "yes"
CarpetRegrid2::active_1 = "yes"
CarpetRegrid2::num_levels_1 = 7
CarpetRegrid2::position_x_1 = 0.0
CarpetRegrid2::radius_1[1] = 512.0
CarpetRegrid2::radius_1[2] = 256.0
CarpetRegrid2::radius_1[3] = 128.0
CarpetRegrid2::radius_1[4] = 64.0
CarpetRegrid2::radius_1[5] = 32.0
CarpetRegrid2::radius_1[6] = 16.0
CarpetRegrid2::symmetry_rotating180 = "no"
CarpetRegrid2::verbose = "no"
Carpet::print_grid_info = "no"
NaNChecker::check_every = 4096
NaNChecker::check_vars = "
ADMBase::alp
"
NaNChecker::action_if_found = "abort"
# =============================================================================
# Time integration
# =============================================================================
Carpet::num_integrator_substeps = 4
MoL::ode_method = "RK4"
MoL::MoL_Intermediate_Steps = 4
MoL::MoL_Num_Scratch_Levels = 1
MoL::verbose = "register"
MoL::disable_prolongation = "no"
Time::timestep_method = "courant_static"
Time::dtfac = 0.15
# =============================================================================
# Initial data
# =============================================================================
ADMBase::initial_data = "twopunctures"
ADMBase::initial_lapse = "twopunctures-averaged"
ADMBase::initial_shift = "zero"
ADMBase::initial_dtlapse = "zero"
ADMBase::initial_dtshift = "zero"
TmunuBase::prolongation_type = "none"
TmunuBase::stress_energy_storage = "no"
TmunuBase::stress_energy_at_RHS = "no"
TmunuBase::support_old_CalcTmunu_mechanism = "no"
# These will give you a BH of mass 1 at the center of the domain,
# not boosteed.
TwoPunctures::target_M_plus = 1.0
TwoPunctures::target_M_minus = 0.0
TwoPunctures::par_b = 1
TwoPunctures::center_offset[0] = -1
TwoPunctures::par_P_plus[0] = 0
TwoPunctures::par_P_plus[1] = 0
TwoPunctures::par_P_plus[2] = 0
TwoPunctures::par_P_minus[0] = 0
TwoPunctures::par_P_minus[1] = 0
TwoPunctures::par_P_minus[2] = 0
TwoPunctures::par_S_plus[0] = 0
TwoPunctures::par_S_plus[1] = 0
TwoPunctures::par_S_plus[2] = 0
TwoPunctures::par_S_minus[0] = 0
TwoPunctures::par_S_minus[1] = 0
TwoPunctures::par_S_minus[2] = 0
TwoPunctures::npoints_A = 30
TwoPunctures::npoints_B = 30
TwoPunctures::npoints_phi = 16
TwoPunctures::grid_setup_method = "evaluation"
TwoPunctures::give_bare_mass = "no"
TwoPunctures::TP_epsilon = 1e-6
# =============================================================================
# Spacetime evolution
# =============================================================================
ADMBase::evolution_method = "LeanBSSNMoL"
ADMBase::lapse_evolution_method = "LeanBSSNMoL"
ADMBase::shift_evolution_method = "LeanBSSNMoL"
ADMBase::dtlapse_evolution_method = "LeanBSSNMoL"
ADMBase::dtshift_evolution_method = "LeanBSSNMoL"
ADMBase::lapse_timelevels = 3
ADMBase::shift_timelevels = 3
ADMBase::metric_timelevels = 3
LeanBSSNMoL::impose_conf_fac_floor_at_initial = "yes"
LeanBSSNMoL::eta_beta = 1
LeanBSSNMoL::eta_beta_dynamic = "no"
LeanBSSNMoL::conf_fac_floor = 1e-4
LeanBSSNMoL::derivs_order = 4
LeanBSSNMoL::use_advection_stencils = "yes"
LeanBSSNMoL::calculate_constraints = "yes"
LeanBSSNMoL::make_aa_tracefree = "yes"
LeanBSSNMoL::reset_dethh = "yes"
LeanBSSNMoL::precollapsed_lapse = "no"
LeanBSSNMoL::rescale_shift_initial = "no"
LeanBSSNMoL::zeta_alpha = 1.
LeanBSSNMoL::use_newrad = "no"
Dissipation::order = 5
Dissipation::epsdis = 0.2
Dissipation::vars = "
ADMBase::lapse
ADMBase::shift
LeanBSSNMoL::conf_fac
LeanBSSNMoL::hmetric
LeanBSSNMoL::hcurv
LeanBSSNMoL::trk
LeanBSSNMoL::gammat
"
# =============================================================================
# Checkpoint
# =============================================================================
CarpetIOHDF5::checkpoint = "yes"
CarpetIOHDF5::use_reflevels_from_checkpoint = "yes"
IOUtil::abort_on_io_errors = "yes"
IOUtil::checkpoint_on_terminate = "yes"
IOUtil::checkpoint_every_walltime_hours = 6.
IOUtil::checkpoint_keep = 1
IOUtil::recover = "autoprobe"
IOUtil::checkpoint_dir = "../checkpoints"
IOUtil::recover_dir = "../checkpoints"
# =============================================================================
# Output
# =============================================================================
IOUtil::out_dir = "data"
CarpetIOHDF5::out1D_dir = "data/out_1D"
CarpetIOHDF5::out2D_dir = "data/out_2D"
CarpetIOHDF5::out3D_dir = "data/out_3D"
IOUtil::strict_io_parameter_check = "yes"
IOUtil::parfile_write = "copy"
CarpetIOScalar::one_file_per_group = "yes"
CarpetIOScalar::all_reductions_in_one_file = "yes"
CarpetIOASCII::one_file_per_group = "yes"
CarpetIOHDF5::one_file_per_group = "yes"
IO::out_single_precision = "yes"
CarpetIOHDF5::output_symmetry_points = "yes"
CarpetIOHDF5::output_ghost_points = "yes"
CarpetIOHDF5::output_boundary_points = "yes"
CarpetIOHDF5::output_buffer_points = "no"
CarpetIOHDF5::compression_level = 0
CarpetIOHDF5::minimum_size_for_compression = 1000000
CarpetIOBasic::outinfo_vars = "
Carpet::physical_time_per_hour
ADMBase::lapse
LeanBSSNMoL::ham
"
CarpetIOScalar::outscalar_reductions = "
minimum
maximum
norm_inf
norm1
norm2
"
CarpetIOScalar::outscalar_vars = "
ADMBase::lapse{ reductions = 'minimum' }
ADMBase::shift{ reductions = 'norm_inf' }
LeanBSSNMoL::ham{ reductions = 'norm_inf' }
LeanBSSNMoL::mom{ reductions = 'norm_inf' }
LeanBSSNMoL::conf_fac{ reductions = 'maximum minimum' }
"
CarpetIOASCII::out0D_vars = "
Carpet::timing
"
CarpetIOHDF5::out1d_d = "no"
CarpetIOHDF5::out1d_x = "yes"
CarpetIOHDF5::out1d_y = "yes"
CarpetIOHDF5::out1d_z = "yes"
CarpetIOHDF5::out1d_vars = "
ADMBase::lapse
ADMBase::shift
ADMBase::metric
ADMBase::curv
LeanBSSNMoL::ham
LeanBSSNMoL::mom
LeanBSSNMoL::conf_fac
CarpetReduce::weight
"
CarpetIOHDF5::out2d_xy = "no"
CarpetIOHDF5::out2d_xz = "yes"
CarpetIOHDF5::out2d_yz = "no"
CarpetIOHDF5::out2d_vars = "
ADMBase::lapse
ADMBase::shift
ADMBase::metric
ADMBase::curv
LeanBSSNMoL::ham
LeanBSSNMoL::mom
LeanBSSNMoL::conf_fac
CarpetReduce::weight
"
CarpetIOBasic::outinfo_every = 1024
CarpetIOScalar::outscalar_every = 256
CarpetIOASCII::out0d_every = 256
CarpetIOHDF5::out1d_every = 2048
CarpetIOHDF5::out2d_every = 1024
CarpetIOHDF5::out_every = -1
"""
open(re.sub(r'(.*)\.rpar$', r'\1.par', sys.argv[0]), 'w').write(re.sub(r'\n *',r'\n',Template(Template(lines).substitute(locals())).substitute(locals())))
-------------- next part --------------
#!/usr/bin/env python2
from math import *
import sys
import re
from string import Template
from subprocess import call
import os
###########################
# Outer boundary
outer_boundary = 5120.
###########################
# Resolution
# The resolutions increase in steps of 1.25, a constant factor,
# which should make it easy to perform a convergence study.
# The resolution is set by setting directly dx on the coarsest level.
# The values [12.8, 10.24, 8.192] result in resolutions of
# [0.2, 0.16, 0.128] on level 6
try:
resolution = str("@RESOLUTION@")
except:
resolution = "mid"
if resolution == "@"+"RESOLUTION"+"@":
resolution = "mid"
if resolution=="mid":
dx = 12.8
elif resolution=="high":
dx = 10.24
elif resolution=="fine":
dx = 8.192
else:
print("Unknown RESOLUTION keyword")
sys.exit()
###########################
# CIDRFILE
try :
cidr_file = str("@CIDRF@")
except :
cidr_file = "nosuchthing"
if cidr_file == "@"+"CIDRF"+"@":
cidr_file = "nosuchthing"
###########################
# EVOLVE
try :
evolve = str("@EVOLVE@")
except :
evolve = "no"
if evolve == "@"+"EVOLVE"+"@":
evolve = "no"
if evolve == "no":
end = "iteration"
elif evolve == "yes":
end = "never"
###########################
# betaDriver
try:
betaDriver = float(str("@BDRIVER@"))
except:
betaDriver = 1.0
if betaDriver == "@"+"BDRIVER"+"@":
betaDriver = 1.0
lines = """
ActiveThorns = "
ADMBase
ADMCoupling
ADMMacros
AEILocalInterp
Boundary
Carpet
CarpetIOASCII
CarpetIOBasic
CarpetIOHDF5
CarpetIOScalar
CarpetInterp
CarpetLib
CarpetReduce
CarpetRegrid2
CarpetSlab
CartGrid3D
Cartoon2D
Constants
CoordBase
CoordGauge
Fortran
GenericFD
IOUtil
InitBase
LoopControl
ML_CCZ4
ML_CCZ4_Helper
MoL
NaNChecker
NewRad
Nice
ReflectionSymmetry
Slab
SpaceMask
SphericalSurface
StaticConformal
SymBase
SystemStatistics
SystemTopology
TerminationTrigger
Time
TimerReport
TmunuBase
TwoPunctures
WatchDog
hwloc
"
# =============================================================================
# General
# =============================================================================
Cactus::cctk_run_title = "Cartoon BH McLachlan"
Cactus::cctk_show_banners = "no"
Cactus::recovery_mode = "strict"
Cactus::schedule_sort_mode = "ascending"
Cactus::terminate = "$end"
Cactus::cctk_itlast = 0
TerminationTrigger::max_walltime = @WALLTIME_HOURS@
TerminationTrigger::on_remaining_walltime = 10
TerminationTrigger::create_termination_file = "yes"
TerminationTrigger::termination_from_file = "yes"
TerminationTrigger::termination_file = "../TERMINATE"
TerminationTrigger::output_remtime_every_minutes = 120
TimerReport::out_every = 0
TimerReport::output_all_timers_readable = "yes"
Nice::Nice_nice = 0
# =============================================================================
# Grid
# =============================================================================
Grid::avoid_origin = "no"
Grid::domain = "full"
Grid::type = "coordbase"
ReflectionSymmetry::reflection_x = "no"
ReflectionSymmetry::reflection_y = "no"
ReflectionSymmetry::reflection_z = "yes"
ReflectionSymmetry::avoid_origin_x = "no"
ReflectionSymmetry::avoid_origin_y = "no"
ReflectionSymmetry::avoid_origin_z = "no"
CoordBase::xmin = 0.
CoordBase::xmax = +$outer_boundary
CoordBase::ymin = 0.
CoordBase::ymax = 0.
CoordBase::zmin = 0.
CoordBase::zmax = +$outer_boundary
CoordBase::spacing = "gridspacing"
CoordBase::dx = $dx
CoordBase::dy = $dx
CoordBase::dz = $dx
CoordBase::boundary_size_x_lower = 3
CoordBase::boundary_size_x_upper = 3
CoordBase::boundary_shiftout_x_lower = 1
CoordBase::boundary_shiftout_x_upper = 1
CoordBase::boundary_staggered_x_lower = "no"
CoordBase::boundary_staggered_x_upper = "no"
CoordBase::boundary_size_y_lower = 3
CoordBase::boundary_size_y_upper = 3
CoordBase::boundary_shiftout_y_lower = 1
CoordBase::boundary_shiftout_y_upper = 1
CoordBase::boundary_staggered_y_lower = "no"
CoordBase::boundary_staggered_y_upper = "no"
CoordBase::boundary_size_z_lower = 3
CoordBase::boundary_size_z_upper = 3
CoordBase::boundary_shiftout_z_lower = 1
CoordBase::boundary_shiftout_z_upper = 1
CoordBase::boundary_staggered_z_lower = "no"
CoordBase::boundary_staggered_z_upper = "no"
Driver::ghost_size = 3
Driver::ghost_size_x = 3
Driver::ghost_size_y = 3
Driver::ghost_size_z = 3
Cartoon2D::cartoon_active = "yes"
Cartoon2D::order = 3
Cartoon2D::eno_order = 3
Carpet::processor_topology = "automatic"
Carpet::no_split_direction = 1
Carpet::domain_from_coordbase = "yes"
InitBase::initial_data_setup_method = "init_some_levels"
LoopControl::settle_after_iteration = 0
Carpet::max_refinement_levels = 7
Carpet::prolongation_order_space = 3
Carpet::prolongation_order_time = 2
CarpetLib::use_higher_order_restriction = "no"
Carpet::use_buffer_zones = "no"
Carpet::enable_all_storage = "no"
Carpet::init_fill_timelevels = "yes"
Carpet::time_refinement_factors = "[1,1,2,4,8,16,32,64,128,256,512]"
Carpet::grid_coordinates_filename = "grid.carpet"
CarpetLib::poison_new_memory = "yes"
CarpetRegrid2::num_centres = 1
CarpetRegrid2::regrid_every = -1
CarpetRegrid2::ensure_proper_nesting = "yes"
CarpetRegrid2::snap_to_coarse = "yes"
CarpetRegrid2::freeze_unaligned_levels = "yes"
CarpetRegrid2::freeze_unaligned_parent_levels = "yes"
CarpetRegrid2::active_1 = "yes"
CarpetRegrid2::num_levels_1 = 7
CarpetRegrid2::position_x_1 = 0.0
CarpetRegrid2::radius_1[1] = 512.0
CarpetRegrid2::radius_1[2] = 256.0
CarpetRegrid2::radius_1[3] = 128.0
CarpetRegrid2::radius_1[4] = 64.0
CarpetRegrid2::radius_1[5] = 32.0
CarpetRegrid2::radius_1[6] = 16.0
CarpetRegrid2::symmetry_rotating180 = "no"
CarpetRegrid2::verbose = "no"
Carpet::print_grid_info = "no"
NaNChecker::check_every = 4096
NaNChecker::check_vars = "
ADMBase::alp
"
NaNChecker::action_if_found = "abort"
# =============================================================================
# Time integration
# =============================================================================
Carpet::num_integrator_substeps = 4
MoL::ode_method = "RK4"
MoL::MoL_Intermediate_Steps = 4
MoL::MoL_Num_Scratch_Levels = 1
MoL::verbose = "register"
MoL::disable_prolongation = "no"
Time::timestep_method = "courant_static"
Time::dtfac = 0.15
# =============================================================================
# Initial data
# =============================================================================
ADMBase::initial_data = "twopunctures"
ADMBase::initial_lapse = "twopunctures-averaged"
ADMBase::initial_shift = "zero"
ADMBase::initial_dtlapse = "zero"
ADMBase::initial_dtshift = "zero"
TmunuBase::prolongation_type = "none"
TmunuBase::stress_energy_storage = "no"
TmunuBase::stress_energy_at_RHS = "no"
TmunuBase::support_old_CalcTmunu_mechanism = "no"
# These will give you a BH of mass 1 at the center of the domain,
# not boosteed.
TwoPunctures::target_M_plus = 1.0
TwoPunctures::target_M_minus = 0.0
TwoPunctures::par_b = 1
TwoPunctures::center_offset[0] = -1
TwoPunctures::par_P_plus[0] = 0
TwoPunctures::par_P_plus[1] = 0
TwoPunctures::par_P_plus[2] = 0
TwoPunctures::par_P_minus[0] = 0
TwoPunctures::par_P_minus[1] = 0
TwoPunctures::par_P_minus[2] = 0
TwoPunctures::par_S_plus[0] = 0
TwoPunctures::par_S_plus[1] = 0
TwoPunctures::par_S_plus[2] = 0
TwoPunctures::par_S_minus[0] = 0
TwoPunctures::par_S_minus[1] = 0
TwoPunctures::par_S_minus[2] = 0
TwoPunctures::npoints_A = 30
TwoPunctures::npoints_B = 30
TwoPunctures::npoints_phi = 16
TwoPunctures::grid_setup_method = "evaluation"
TwoPunctures::give_bare_mass = "no"
TwoPunctures::TP_epsilon = 1e-6
# =============================================================================
# Spacetime evolution
# =============================================================================
ADMBase::evolution_method = "ML_CCZ4"
ADMBase::lapse_evolution_method = "ML_CCZ4"
ADMBase::shift_evolution_method = "ML_CCZ4"
ADMBase::dtlapse_evolution_method = "ML_CCZ4"
ADMBase::dtshift_evolution_method = "ML_CCZ4"
ML_CCZ4::timelevels = 3
ML_CCZ4::harmonicN = 1.0 # 1+log
ML_CCZ4::harmonicF = 2.0 # 1+log
ML_CCZ4::alphaDriver = 0.0 # not more than 0.5
ML_CCZ4::betaDriver = $betaDriver # ~1/M (eta)
ML_CCZ4::advectLapse = 1
ML_CCZ4::advectShift = 1
ML_CCZ4::evolveA = 0
ML_CCZ4::evolveB = 1
ML_CCZ4::shiftGammaCoeff = 0.75
ML_CCZ4::shiftFormulation = 0 # Gamma driver
ML_CCZ4::fixAdvectionTerms = 0
ML_CCZ4::dampk1 = 0.02
ML_CCZ4::dampk2 = 0.0
ML_CCZ4::GammaShift = 0.5
ML_CCZ4::MinimumLapse = 1.0e-8
ML_CCZ4::conformalMethod = 1 # 1 for W
ML_CCZ4::dt_lapse_shift_method = "noLapseShiftAdvection"
ML_CCZ4::initial_boundary_condition = "extrapolate-gammas"
ML_CCZ4::rhs_boundary_condition = "scalar"
Boundary::radpower = 2
ML_CCZ4::fdOrder = 4
ML_CCZ4::epsDiss = 0.2
ML_CCZ4::ML_CCZ4_EvolutionAnalysisInit_calc_every = 1
ML_CCZ4::ML_CCZ4_EvolutionAnalysisInit_calc_offset = 2
ML_CCZ4::ML_CCZ4_EvolutionAnalysisInterior_calc_every = 1
ML_CCZ4::ML_CCZ4_EvolutionAnalysisInterior_calc_offset = 2
ML_CCZ4::ML_CCZ4_ConstraintsEverywhere_calc_every = 1
ML_CCZ4::ML_CCZ4_ConstraintsEverywhere_calc_offset = 2
ML_CCZ4::ML_CCZ4_ConstraintsInterior_calc_every = 1
ML_CCZ4::ML_CCZ4_ConstraintsInterior_calc_offset = 0
ML_CCZ4::ML_log_confac_bound = "none"
ML_CCZ4::ML_log_confac_bound_speed = 1.0
ML_CCZ4::ML_log_confac_bound_limit = 0.0
ML_CCZ4::ML_log_confac_bound_scalar = 0.0
ML_CCZ4::ML_metric_bound = "none"
ML_CCZ4::ML_metric_bound_speed = 1.0
ML_CCZ4::ML_metric_bound_limit = 0.0
ML_CCZ4::ML_metric_bound_scalar = 0.0
ML_CCZ4::ML_Gamma_bound = "none"
ML_CCZ4::ML_Gamma_bound_speed = 1.0
ML_CCZ4::ML_Gamma_bound_limit = 0.0
ML_CCZ4::ML_Gamma_bound_scalar = 0.0
ML_CCZ4::ML_trace_curv_bound = "none"
ML_CCZ4::ML_trace_curv_bound_speed = 1.0
ML_CCZ4::ML_trace_curv_bound_limit = 0.0
ML_CCZ4::ML_trace_curv_bound_scalar = 0.0
ML_CCZ4::ML_curv_bound = "none"
ML_CCZ4::ML_curv_bound_speed = 1.0
ML_CCZ4::ML_curv_bound_limit = 0.0
ML_CCZ4::ML_curv_bound_scalar = 0.0
ML_CCZ4::ML_Theta_bound = "none"
ML_CCZ4::ML_Theta_bound_speed = 1.0
ML_CCZ4::ML_Theta_bound_limit = 0.0
ML_CCZ4::ML_Theta_bound_scalar = 0.0
ML_CCZ4::ML_lapse_bound = "none"
ML_CCZ4::ML_lapse_bound_speed = 1.0
ML_CCZ4::ML_lapse_bound_limit = 0.0
ML_CCZ4::ML_lapse_bound_scalar = 0.0
ML_CCZ4::ML_dtlapse_bound = "none"
ML_CCZ4::ML_dtlapse_bound_speed = 1.0
ML_CCZ4::ML_dtlapse_bound_limit = 0.0
ML_CCZ4::ML_dtlapse_bound_scalar = 0.0
ML_CCZ4::ML_shift_bound = "none"
ML_CCZ4::ML_shift_bound_speed = 1.0
ML_CCZ4::ML_shift_bound_limit = 0.0
ML_CCZ4::ML_shift_bound_scalar = 0.0
ML_CCZ4::ML_dtshift_bound = "none"
ML_CCZ4::ML_dtshift_bound_speed = 1.0
ML_CCZ4::ML_dtshift_bound_limit = 0.0
ML_CCZ4::ML_dtshift_bound_scalar = 0.0
ML_CCZ4::phi_bound = "skip"
ML_CCZ4::phi_bound_speed = 1.0
ML_CCZ4::phi_bound_limit = 0.0
ML_CCZ4::phi_bound_scalar = 0.0
ML_CCZ4::gt11_bound = "skip"
ML_CCZ4::gt11_bound_speed = 1.0
ML_CCZ4::gt11_bound_limit = 0.0
ML_CCZ4::gt11_bound_scalar = 0.0
ML_CCZ4::gt12_bound = "skip"
ML_CCZ4::gt12_bound_speed = 1.0
ML_CCZ4::gt12_bound_limit = 0.0
ML_CCZ4::gt12_bound_scalar = 0.0
ML_CCZ4::gt13_bound = "skip"
ML_CCZ4::gt13_bound_speed = 1.0
ML_CCZ4::gt13_bound_limit = 0.0
ML_CCZ4::gt13_bound_scalar = 0.0
ML_CCZ4::gt22_bound = "skip"
ML_CCZ4::gt22_bound_speed = 1.0
ML_CCZ4::gt22_bound_limit = 0.0
ML_CCZ4::gt22_bound_scalar = 0.0
ML_CCZ4::gt23_bound = "skip"
ML_CCZ4::gt23_bound_speed = 1.0
ML_CCZ4::gt23_bound_limit = 0.0
ML_CCZ4::gt23_bound_scalar = 0.0
ML_CCZ4::gt33_bound = "skip"
ML_CCZ4::gt33_bound_speed = 1.0
ML_CCZ4::gt33_bound_limit = 0.0
ML_CCZ4::gt33_bound_scalar = 0.0
ML_CCZ4::Xt1_bound = "skip"
ML_CCZ4::Xt1_bound_speed = 1.0
ML_CCZ4::Xt1_bound_limit = 0.0
ML_CCZ4::Xt1_bound_scalar = 0.0
ML_CCZ4::Xt2_bound = "skip"
ML_CCZ4::Xt2_bound_speed = 1.0
ML_CCZ4::Xt2_bound_limit = 0.0
ML_CCZ4::Xt2_bound_scalar = 0.0
ML_CCZ4::Xt3_bound = "skip"
ML_CCZ4::Xt3_bound_speed = 1.0
ML_CCZ4::Xt3_bound_limit = 0.0
ML_CCZ4::Xt3_bound_scalar = 0.0
ML_CCZ4::trK_bound = "skip"
ML_CCZ4::trK_bound_speed = 1.0
ML_CCZ4::trK_bound_limit = 0.0
ML_CCZ4::trK_bound_scalar = 0.0
ML_CCZ4::At11_bound = "skip"
ML_CCZ4::At11_bound_speed = 1.0
ML_CCZ4::At11_bound_limit = 0.0
ML_CCZ4::At11_bound_scalar = 0.0
ML_CCZ4::At12_bound = "skip"
ML_CCZ4::At12_bound_speed = 1.0
ML_CCZ4::At12_bound_limit = 0.0
ML_CCZ4::At12_bound_scalar = 0.0
ML_CCZ4::At13_bound = "skip"
ML_CCZ4::At13_bound_speed = 1.0
ML_CCZ4::At13_bound_limit = 0.0
ML_CCZ4::At13_bound_scalar = 0.0
ML_CCZ4::At22_bound = "skip"
ML_CCZ4::At22_bound_speed = 1.0
ML_CCZ4::At22_bound_limit = 0.0
ML_CCZ4::At22_bound_scalar = 0.0
ML_CCZ4::At23_bound = "skip"
ML_CCZ4::At23_bound_speed = 1.0
ML_CCZ4::At23_bound_limit = 0.0
ML_CCZ4::At23_bound_scalar = 0.0
ML_CCZ4::At33_bound = "skip"
ML_CCZ4::At33_bound_speed = 1.0
ML_CCZ4::At33_bound_limit = 0.0
ML_CCZ4::At33_bound_scalar = 0.0
ML_CCZ4::Theta_bound = "skip"
ML_CCZ4::Theta_bound_speed = 1.0
ML_CCZ4::Theta_bound_limit = 0.0
ML_CCZ4::Theta_bound_scalar = 0.0
ML_CCZ4::alpha_bound = "skip"
ML_CCZ4::alpha_bound_speed = 1.0
ML_CCZ4::alpha_bound_limit = 0.0
ML_CCZ4::alpha_bound_scalar = 0.0
ML_CCZ4::A_bound = "skip"
ML_CCZ4::A_bound_speed = 1.0
ML_CCZ4::A_bound_limit = 0.0
ML_CCZ4::A_bound_scalar = 0.0
ML_CCZ4::beta1_bound = "skip"
ML_CCZ4::beta1_bound_speed = 1.0
ML_CCZ4::beta1_bound_limit = 0.0
ML_CCZ4::beta1_bound_scalar = 0.0
ML_CCZ4::beta2_bound = "skip"
ML_CCZ4::beta2_bound_speed = 1.0
ML_CCZ4::beta2_bound_limit = 0.0
ML_CCZ4::beta2_bound_scalar = 0.0
ML_CCZ4::beta3_bound = "skip"
ML_CCZ4::beta3_bound_speed = 1.0
ML_CCZ4::beta3_bound_limit = 0.0
ML_CCZ4::beta3_bound_scalar = 0.0
ML_CCZ4::B1_bound = "skip"
ML_CCZ4::B1_bound_speed = 1.0
ML_CCZ4::B1_bound_limit = 0.0
ML_CCZ4::B1_bound_scalar = 0.0
ML_CCZ4::B2_bound = "skip"
ML_CCZ4::B2_bound_speed = 1.0
ML_CCZ4::B2_bound_limit = 0.0
ML_CCZ4::B2_bound_scalar = 0.0
ML_CCZ4::B3_bound = "skip"
ML_CCZ4::B3_bound_speed = 1.0
ML_CCZ4::B3_bound_limit = 0.0
ML_CCZ4::B3_bound_scalar = 0.0
# =============================================================================
# Checkpoint
# =============================================================================
CarpetIOHDF5::checkpoint = "yes"
CarpetIOHDF5::use_reflevels_from_checkpoint = "yes"
IOUtil::abort_on_io_errors = "yes"
IOUtil::checkpoint_on_terminate = "yes"
IOUtil::checkpoint_every_walltime_hours = 6.
IOUtil::checkpoint_keep = 1
IOUtil::recover = "autoprobe"
IOUtil::checkpoint_dir = "../checkpoints"
IOUtil::recover_dir = "../checkpoints"
# =============================================================================
# Output
# =============================================================================
IOUtil::out_dir = "data"
CarpetIOHDF5::out1D_dir = "data/out_1D"
CarpetIOHDF5::out2D_dir = "data/out_2D"
CarpetIOHDF5::out3D_dir = "data/out_3D"
IOUtil::strict_io_parameter_check = "yes"
IOUtil::parfile_write = "copy"
CarpetIOScalar::one_file_per_group = "yes"
CarpetIOScalar::all_reductions_in_one_file = "yes"
CarpetIOASCII::one_file_per_group = "yes"
CarpetIOHDF5::one_file_per_group = "yes"
IO::out_single_precision = "yes"
CarpetIOHDF5::output_symmetry_points = "yes"
CarpetIOHDF5::output_ghost_points = "yes"
CarpetIOHDF5::output_boundary_points = "yes"
CarpetIOHDF5::output_buffer_points = "no"
CarpetIOHDF5::compression_level = 0
CarpetIOHDF5::minimum_size_for_compression = 1000000
CarpetIOBasic::outinfo_vars = "
Carpet::physical_time_per_hour
ADMBase::lapse
ML_CCZ4::ML_Ham
"
CarpetIOScalar::outscalar_reductions = "
minimum
maximum
norm_inf
norm1
norm2
"
CarpetIOScalar::outscalar_vars = "
ADMBase::lapse{ reductions = 'minimum' }
ADMBase::shift{ reductions = 'norm_inf' }
ML_CCZ4::ML_Ham{ reductions = 'norm_inf' }
ML_CCZ4::ML_Mom{ reductions = 'norm_inf' }
ML_CCZ4::ML_log_confac{ reductions = 'maximum minimum' }
"
CarpetIOASCII::out0D_vars = "
Carpet::timing
"
CarpetIOHDF5::out1d_d = "no"
CarpetIOHDF5::out1d_x = "yes"
CarpetIOHDF5::out1d_y = "yes"
CarpetIOHDF5::out1d_z = "yes"
CarpetIOHDF5::out1d_vars = "
ADMBase::lapse
ADMBase::shift
ADMBase::metric
ADMBase::curv
ML_CCZ4::ML_Ham
ML_CCZ4::ML_Mom
ML_CCZ4::ML_log_confac
CarpetReduce::weight
"
CarpetIOHDF5::out2d_xy = "no"
CarpetIOHDF5::out2d_xz = "yes"
CarpetIOHDF5::out2d_yz = "no"
CarpetIOHDF5::out2d_vars = "
ADMBase::lapse
ADMBase::shift
ADMBase::metric
ADMBase::curv
ML_CCZ4::ML_Ham
ML_CCZ4::ML_Mom
ML_CCZ4::ML_log_confac
CarpetReduce::weight
"
CarpetIOBasic::outinfo_every = 1024
CarpetIOScalar::outscalar_every = 256
CarpetIOASCII::out0d_every = 256
CarpetIOHDF5::out1d_every = 2048
CarpetIOHDF5::out2d_every = 1024
CarpetIOHDF5::out_every = -1
"""
open(re.sub(r'(.*)\.rpar$', r'\1.par', sys.argv[0]), 'w').write(re.sub(r'\n *',r'\n',Template(Template(lines).substitute(locals())).substitute(locals())))
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.einsteintoolkit.org/pipermail/users/attachments/20200918/93c33dab/attachment-0001.html
More information about the Users
mailing list