[Users] Working ETK par file for BBH run?

Scott Hawley scott.hawley at belmont.edu
Fri Aug 5 17:12:43 CDT 2011


the ~/Cactus/par/static_tov.par  example works for me, but the two others don't, e.g. below.


% cd ~/Cactus
% ./exe/cactus_etk par/ks-mclachlan.par
--------------------------------------------------------------------------------

       10
  1   0101       ************************
  01  1010 10      The Cactus Code V4.0
 1010 1101 011      www.cactuscode.org
  1001 100101    ************************
    00010101
     100011     (c) Copyright The Authors
      0100      GNU Licensed. No Warranty
      0101

--------------------------------------------------------------------------------

Cactus version:    4.0.b17
Compile date:      Aug 05 2011 (16:33:22)
Run date:          Aug 05 2011 (17:07:14-0500)
Run host:          hedges.belmont.edu (pid=30592)
Working directory: /home/shawley/Cactus
Executable:        ./exe/cactus_etk
Parameter file:    par/ks-mclachlan.par
--------------------------------------------------------------------------------

Activating thorn Cactus...Success -> active implementation Cactus
Activation requested for
--->IOUtil<---
Activating thorn IOUtil...Success -> active implementation IO
Activation requested for
--->AEILocalInterp<---
Activating thorn AEILocalInterp...Success -> active implementation AEILocalInterp
Activation requested for
--->Fortran<---
Activating thorn Fortran...Success -> active implementation Fortran
Activation requested for
--->GSL<---
Activating thorn GSL...Success -> active implementation GSL
Activation requested for
--->GenericFD<---
Activating thorn GenericFD...Success -> active implementation GenericFD
Activation requested for
--->HDF5<---
Activating thorn HDF5...Success -> active implementation HDF5
Activation requested for
--->LocalInterp<---
Activating thorn LocalInterp...Success -> active implementation LocalInterp
Activation requested for
--->LoopControl<---
Activating thorn LoopControl...Success -> active implementation LoopControl
Activation requested for
--->Slab<---
Activating thorn Slab...Success -> active implementation Slab
Activation requested for
--->InitBase<---
Activating thorn InitBase...Success -> active implementation InitBase
Activation requested for
--->Carpet CarpetLib CarpetInterp CarpetReduce CarpetSlab<---
Activating thorn Carpet...Success -> active implementation Driver
Activating thorn CarpetInterp...Success -> active implementation interp
Activating thorn CarpetLib...Success -> active implementation CarpetLib
Activating thorn CarpetReduce...Success -> active implementation reduce
Activating thorn CarpetSlab...Success -> active implementation Hyperslab
Activation requested for
--->NaNChecker<---
Activating thorn NaNChecker...Success -> active implementation NaNChecker
Activation requested for
--->Boundary CartGrid3D CoordBase ReflectionSymmetry RotatingSymmetry90 SymBase<---
Activating thorn Boundary...Success -> active implementation boundary
Activating thorn CartGrid3D...Success -> active implementation grid
Activating thorn CoordBase...Success -> active implementation CoordBase
Activating thorn ReflectionSymmetry...Success -> active implementation ReflectionSymmetry
Activating thorn RotatingSymmetry90...Success -> active implementation RotatingSymmetry90
Activating thorn SymBase...Success -> active implementation SymBase
Activation requested for
--->SphericalSurface<---
Activating thorn SphericalSurface...Success -> active implementation SphericalSurface
Activation requested for
--->CarpetMask<---
Activating thorn CarpetMask...Success -> active implementation CarpetMask
Activation requested for
--->CarpetRegrid2 CarpetTracker<---
Activating thorn CarpetRegrid2...Success -> active implementation CarpetRegrid2
Activating thorn CarpetTracker...Success -> active implementation CarpetTracker
Activation requested for
--->MoL Time<---
Activating thorn MoL...Success -> active implementation MethodOfLines
Activating thorn Time...Success -> active implementation time
Activation requested for
--->ADMBase ADMCoupling ADMMacros CoordGauge SpaceMask StaticConformal TmunuBase<---
Activating thorn ADMBase...Success -> active implementation ADMBase
Activating thorn ADMCoupling...Success -> active implementation ADMCoupling
Activating thorn ADMMacros...Success -> active implementation ADMMacros
Activating thorn CoordGauge...Success -> active implementation CoordGauge
Activating thorn SpaceMask...Success -> active implementation SpaceMask
Activating thorn StaticConformal...Success -> active implementation StaticConformal
Activating thorn TmunuBase...Success -> active implementation TmunuBase
Activation requested for
--->Exact<---
Activating thorn Exact...Success -> active implementation exact
Activation requested for
--->ML_BSSN ML_BSSN_Helper NewRad<---
Activating thorn ML_BSSN...Success -> active implementation ML_BSSN
Activating thorn ML_BSSN_Helper...Success -> active implementation ML_BSSN_Helper
Activating thorn NewRad...Success -> active implementation NewRad
Activation requested for
--->Dissipation<---
Activating thorn Dissipation...Success -> active implementation Dissipation
Activation requested for
--->ML_ADMConstraints<---
Activating thorn ML_ADMConstraints...Success -> active implementation ML_ADMConstraints
Activation requested for
--->AHFinderDirect<---
Activating thorn AHFinderDirect...Success -> active implementation AHFinderDirect
Activation requested for
--->CarpetIOBasic<---
Activating thorn CarpetIOBasic...Success -> active implementation IOBasic
Activation requested for
--->CarpetIOScalar<---
Activating thorn CarpetIOScalar...Success -> active implementation IOScalar
Activation requested for
--->CarpetIOASCII<---
Activating thorn CarpetIOASCII...Success -> active implementation IOASCII
Activation requested for
--->CarpetIOHDF5<---
Activating thorn CarpetIOHDF5...Success -> active implementation IOHDF5
Activation requested for
--->Formaline<---
Activating thorn Formaline...Success -> active implementation Formaline
Activation requested for
--->TimerReport<---
Activating thorn TimerReport...Success -> active implementation timerreport
--------------------------------------------------------------------------------
  if (recover initial data)
    Recover parameters
  endif

  Startup routines
    [CCTK_STARTUP]
      Carpet::MultiModel_Startup: Multi-model Startup routine
      ML_BSSN_Helper::ML_BSSN_SetGroupTags: [meta] Set checkpointing and prolongation group tags
      Carpet::Driver_Startup: Startup routine
      IOUtil::IOUtil_Startup: Startup routine
      CarpetIOBasic::CarpetIOBasicStartup: [global] Startup routine
      CarpetIOHDF5::CarpetIOHDF5_Startup: Startup routine
      CarpetIOScalar::CarpetIOScalarStartup: [global] Startup routine
      CarpetInterp::CarpetInterpStartup: Startup routine
      CarpetReduce::CarpetReduceStartup: Startup routine
      CartGrid3D::SymmetryStartup: Register GH Extension for GridSymmetry
      CoordBase::CoordBase_Startup: Register a GH extension to store the coordinate system handles
      Formaline::Formaline_PrintIDs: [meta] Print the build and simulation ids
      CarpetIOASCII::CarpetIOASCIIStartup: [global] Startup routine
      LocalInterp::LocalInterp_Startup: register LocalInterp's interpolation operators
      ML_ADMConstraints::ML_ADMConstraints_Startup: [meta] create banner
      ML_BSSN::ML_BSSN_Startup: [meta] create banner
      ML_BSSN_Helper::ML_BSSN_RegisterSlicing: [meta] Register slicing
      AEILocalInterp::AEILocalInterp_U_Startup: register CCTK_InterpLocalUniform() interpolation operators
      MoL::MoL_Startup: Startup banner
      Slab::Slab_InitMPIDatatypes: Create MPI datatypes for complex variables in C
      SymBase::SymBase_Startup: Register GH Extension for SymBase

  Startup routines which need an existing grid hierarchy
    [CCTK_WRAGH]
      ADMBase::Einstein_InitSymBound: [global] Set up GF symmetries
      Boundary::Boundary_RegisterBCs: [global] Register boundary conditions that this thorn provides
      CarpetLib::CarpetLib_setmemlimit: [global] Set operating system memory limit
      CarpetRegrid2::CarpetRegrid2_Initialise: [global] Initialise locations of refined regions
      CartGrid3D::RegisterCartGrid3DCoords: [meta] Register coordinates for the Cartesian grid
      CoordGauge::Einstein_ActivateSlicing: Initialize slicing, setup priorities for mixed slicings
      CoordGauge::Einstein_SetNextSlicing: Identify the slicing for the next iteration
      Formaline::Formaline_OutputSource: Output Cactus source tree
      Formaline::Formaline_RegisterWarnings: [meta] Register to receive warnings and info messages from the flesh
      Formaline::Formaline_AnnounceInitial: [global] Put some meta information about the current run into permanent storage
      MoL::MoL_SetupIndexArrays: Set up the MoL bookkeeping index arrays
      MoL::MoL_SetScheduleStatus: [global] Set the flag so it is ok to register with MoL
      GROUP MoL_Register: The group where physics thorns register variables with MoL
        ML_ADMConstraints::ML_ADMConstraints_RegisterVars: [meta] Register Variables for MoL
        ML_BSSN::ML_BSSN_RegisterVars: [meta] Register Variables for MoL
        ML_BSSN_Helper::ML_BSSN_RegisterConstrained: [meta] Register ADMBase variables as constrained
      MoL::MoL_ReportNumberVariables: [meta] Report how many of each type of variable there are
      Slab::Slab_InitTimers: Initialise timers
      GROUP SymBase_Wrapper: Wrapper group for SymBase
        GROUP SymmetryRegister: Register your symmetries here
          CartGrid3D::RegisterSymmetryBoundaries: [meta] Register symmetry boundaries
          ML_ADMConstraints::ML_ADMConstraints_RegisterSymmetries: [meta] register symmetries
          ML_BSSN::ML_BSSN_RegisterSymmetries: [meta] register symmetries
          ReflectionSymmetry::ReflectionSymmetry_Register: Register reflection symmetry boundaries
          RotatingSymmetry90::Rot90_RegisterSymmetry: Register symmetry boundaries
        SymBase::SymBase_Statistics: Print symmetry boundary face descriptions
  Parameter checking routines
    [CCTK_PARAMCHECK]
      ADMBase::ADMBase_ParamCheck: [global] Check consistency of parameters
      Boundary::Boundary_Check: Check dimension of grid variables
      Carpet::CarpetParamCheck: Parameter checking routine
      CarpetRegrid2::CarpetRegrid2_ParamCheck: Check parameters
      CartGrid3D::ParamCheck_CartGrid3D: Check coordinates for CartGrid3D
      Dissipation::dissipation_paramcheck: Check dissipation parameters for consistency
      Exact::Exact_ParamCheck: [global] do consistency checks on our parameters
      Fortran::CheckFortranParameters: Test whether Fortran parameters work correctly
      MoL::MoL_ParamCheck: Basic parameter checking
      RotatingSymmetry90::Rot90_CheckTensorTypes: Check tensor type definitions for consistency

  Initialisation
    if (NOT (recover initial data AND recovery_mode is 'strict'))
      [CCTK_PREREGRIDINITIAL]
      Set up grid hierarchy
      [CCTK_POSTREGRIDINITIAL]
        AHFinderDirect::AHFinderDirect_maybe_do_masks: [global] [loop-local] set mask(s) based on apparent horizon position(s)
        CartGrid3D::SpatialCoordinates: Set Coordinates after regridding
        GROUP MaskBase_SetupMask: Set up the weight function
          CarpetReduce::MaskBase_InitMask: [global] [loop-local] Initialise the weight function
          GROUP SetupMask: Set up the weight function (schedule other routines in here)
            CarpetMask::CarpetExcludedSetup: [global] [loop-local] Set up the weight function for the excluded regions
            CarpetMask::CarpetSurfaceSetup: [global] [loop-local] Set up the weight function for the excluded spherical surfaces
            CarpetReduce::CoordBase_SetupMask: [global] [loop-local] Set up the outer boundaries of the weight function
            CarpetReduce::CarpetMaskSetup: [global] [loop-singlemap] Set up the weight function for the restriction regions
        Dissipation::setup_epsdis: Setup spatially varying dissipation
        GROUP MoL_PseudoEvolution: Calculate pseudo-evolved quantities
          GROUP ADMBase_SetADMVars: Set the ADM variables before this group, and use them afterwards
          GROUP ML_ADMConstraints_group: ML_ADMConstraints
            ML_ADMConstraints::ML_ADMConstraints: ML_ADMConstraints
            GROUP ML_ADMConstraints_bc_group: ML_ADMConstraints
              ML_ADMConstraints::ML_ADMConstraints_SelectBCs: [level] ML_ADMConstraints_SelectBCs
              GROUP ML_ADMConstraints_ApplyBCs: Apply BCs for groups set in ML_ADMConstraints
                GROUP BoundaryConditions: Execute all boundary conditions
                  Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions
                  CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions
                  ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries
                  RotatingSymmetry90::Rot90_ApplyBC: Apply 90 degree rotational symmetry boundary condition
                Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions
          ML_ADMConstraints::ML_ADMConstraints_SelectBoundConds: [level] select boundary conditions
          GROUP ML_ADMConstraints_ApplyBCs: Apply boundary conditions controlled by thorn Boundary
            GROUP BoundaryConditions: Execute all boundary conditions
              Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions
              CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions
              ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries
              RotatingSymmetry90::Rot90_ApplyBC: Apply 90 degree rotational symmetry boundary condition
            Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions
          GROUP ML_BSSN_constraints1_group: ML_BSSN_constraints1
            ML_BSSN::ML_BSSN_constraints1: ML_BSSN_constraints1
            GROUP ML_BSSN_constraints1_bc_group: ML_BSSN_constraints1
              ML_BSSN::ML_BSSN_constraints1_SelectBCs: [level] ML_BSSN_constraints1_SelectBCs
              GROUP ML_BSSN_constraints1_ApplyBCs: Apply BCs for groups set in ML_BSSN_constraints1
                GROUP BoundaryConditions: Execute all boundary conditions
                  Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions
                  CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions
                  ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries
                  RotatingSymmetry90::Rot90_ApplyBC: Apply 90 degree rotational symmetry boundary condition
                Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions
          GROUP ML_BSSN_constraints2_group: ML_BSSN_constraints2
            ML_BSSN::ML_BSSN_constraints2: ML_BSSN_constraints2
            GROUP ML_BSSN_constraints2_bc_group: ML_BSSN_constraints2
              ML_BSSN::ML_BSSN_constraints2_SelectBCs: [level] ML_BSSN_constraints2_SelectBCs
              GROUP ML_BSSN_constraints2_ApplyBCs: Apply BCs for groups set in ML_BSSN_constraints2
                GROUP BoundaryConditions: Execute all boundary conditions
                  Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions
                  CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions
                  ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries
                  RotatingSymmetry90::Rot90_ApplyBC: Apply 90 degree rotational symmetry boundary condition
                Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions
      [CCTK_BASEGRID]
        ADMBase::ADMBase_SetShiftStateOn: Set the shift_state variable to 1
        ADMBase::ADMBase_SetDtLapseStateOn: Set the dtlapse_state variable to 1
        ADMBase::ADMBase_SetDtShiftStateOn: Set the dtshift_state variable to 1
        ADMMacros::ADMMacros_SetLocalSpatialOrder: Initialize the local_spatial_order
        CartGrid3D::CartGrid3D_SetRanges: Set up ranges for spatial 3D Cartesian coordinates (on all grids)
        CarpetIOASCII::CarpetIOASCIIInit: [global] Initialisation routine
        CarpetIOBasic::CarpetIOBasicInit: [global] Initialisation routine
        CarpetIOHDF5::CarpetIOHDF5_Init: [global] Initialisation routine
        CarpetIOScalar::CarpetIOScalarInit: [global] Initialisation routine
        CartGrid3D::SpatialCoordinates: Set up spatial 3D Cartesian coordinates on the GH
        SphericalSurface::SphericalSurface_SetupRes: [global] [loop-local] Set surface resolution automatically
        AHFinderDirect::AHFinderDirect_setup: [global] setup data structures
        Dissipation::dissipation_basegrid: Ensure that there are enough ghost zones
        ML_ADMConstraints::ML_ADMConstraints_CheckBoundaries: [meta] check boundaries treatment
        ML_BSSN::ML_BSSN_CheckBoundaries: [meta] check boundaries treatment
        SphericalSurface::SphericalSurface_Setup: [global] Calculate surface coordinate descriptors
        GROUP MaskBase_SetupMask: Set up the weight function
          CarpetReduce::MaskBase_InitMask: [global] [loop-local] Initialise the weight function
          GROUP SetupMask: Set up the weight function (schedule other routines in here)
            CarpetMask::CarpetExcludedSetup: [global] [loop-local] Set up the weight function for the excluded regions
            CarpetMask::CarpetSurfaceSetup: [global] [loop-local] Set up the weight function for the excluded spherical surfaces
            CarpetReduce::CoordBase_SetupMask: [global] [loop-local] Set up the outer boundaries of the weight function
            CarpetReduce::CarpetMaskSetup: [global] [loop-singlemap] Set up the weight function for the restriction regions
        SphericalSurface::SphericalSurface_Set: [global] Set surface radii to be used for initial setup in other thorns
        GROUP SphericalSurface_HasBeenSet: Set the spherical surfaces before this group, and use it afterwards
          SphericalSurface::SphericalSurface_CheckState: [global] Test the state of the spherical surfaces
        SymBase::SymBase_Check: Check whether the driver set up the grid consistently
        Time::Time_Initialise: [global] Initialise Time variables
        Time::Time_Simple: [singlemap] Set timestep based on Courant condition (courant_static)
        TmunuBase::TmunuBase_SetStressEnergyState: [global] Set the stress_energy_state variable
      [CCTK_INITIAL]
        StaticConformal::StaticConformal_InitialiseState: Set the conformal_state variable to 0
        GROUP ADMBase_InitialData: Schedule group for calculating ADM initial data
          Exact::Exact__decode_pars: decode/copy thorn Exact parameters into grid scalars
          Exact::Exact__initialize: Set initial data from exact solution on a trivial slice
        GROUP ADMBase_InitialGauge: Schedule group for the ADM initial gauge condition
          Exact::Exact__gauge: Set initial lapse and/or shift from exact solution on a trivial slice
        GROUP ADMBase_PostInitial: Schedule group for modifying the ADM initial data, such as e.g. adding noise
        ML_BSSN::ML_BSSN_convertFromADMBase: ML_BSSN_convertFromADMBase
        ML_BSSN::ML_BSSN_convertFromADMBaseGamma: ML_BSSN_convertFromADMBaseGamma
        MoL::MoL_StartLoop: [level] Initialise the step size control
        MoL::MoL_SetInitialTime: [level] Ensure the correct time and timestep are used - initial
        NaNChecker::NaNChecker_ResetCounter: [global] Reset the NaNChecker::NaNsFound counter
        ML_BSSN_Helper::ML_BSSN_ExtrapolateGammas: Extrapolate Gammas and time derivatives of lapse and shift
      [CCTK_POSTINITIAL]
        CarpetIOHDF5::CarpetIOHDF5_CloseFiles: [global] Close all filereader input files
        GROUP MoL_PostStep: Ensure that everything is correct after the initial data have been set up
          ML_BSSN::ML_BSSN_enforce: ML_BSSN_enforce
          ML_BSSN::ML_BSSN_SelectBoundConds: [level] select boundary conditions
          GROUP ML_BSSN_ApplyBCs: Apply boundary conditions controlled by thorn Boundary
            GROUP BoundaryConditions: Execute all boundary conditions
              Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions
              CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions
              ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries
              RotatingSymmetry90::Rot90_ApplyBC: Apply 90 degree rotational symmetry boundary condition
            Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions
          GROUP ML_BSSN_convertToADMBaseGroupWrapper: Calculate ADM variables
            GROUP ML_BSSN_convertToADMBaseGroup: Calculate ADM variables
              ML_BSSN::ML_BSSN_convertToADMBase: ML_BSSN_convertToADMBase
              ML_BSSN::ML_BSSN_convertToADMBaseFakeDtLapseShift: ML_BSSN_convertToADMBaseFakeDtLapseShift
            ML_BSSN_Helper::ML_BSSN_SelectBCsADMBase: [level] Select boundary conditions for ADMBase variables
            GROUP ML_BSSN_ApplyBCsADMBase: Apply boundary conditions to ADMBase variables
              GROUP BoundaryConditions: Execute all boundary conditions
                Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions
                CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions
                ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries
                RotatingSymmetry90::Rot90_ApplyBC: Apply 90 degree rotational symmetry boundary condition
              Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions
          GROUP ADMBase_SetADMVars: Set the ADM variables before this group, and use them afterwards
          GROUP MoL_OldBdry_Wrap: Wrapper group, do not schedule directly into this group
            MoL::MoL_OldBdry_SetDt: [level] Store and change dt
            GROUP MoL_OldStyleBoundaries: Place old style boundary routines here
            MoL::MoL_OldBdry_ResetDt: [level] Reset dt
        GROUP MoL_PseudoEvolution: Calculate pseudo-evolved quantities
          GROUP ADMBase_SetADMVars: Set the ADM variables before this group, and use them afterwards
          GROUP ML_ADMConstraints_group: ML_ADMConstraints
            ML_ADMConstraints::ML_ADMConstraints: ML_ADMConstraints
            GROUP ML_ADMConstraints_bc_group: ML_ADMConstraints
              ML_ADMConstraints::ML_ADMConstraints_SelectBCs: [level] ML_ADMConstraints_SelectBCs
              GROUP ML_ADMConstraints_ApplyBCs: Apply BCs for groups set in ML_ADMConstraints
                GROUP BoundaryConditions: Execute all boundary conditions
                  Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions
                  CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions
                  ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries
                  RotatingSymmetry90::Rot90_ApplyBC: Apply 90 degree rotational symmetry boundary condition
                Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions
          ML_ADMConstraints::ML_ADMConstraints_SelectBoundConds: [level] select boundary conditions
          GROUP ML_ADMConstraints_ApplyBCs: Apply boundary conditions controlled by thorn Boundary
            GROUP BoundaryConditions: Execute all boundary conditions
              Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions
              CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions
              ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries
              RotatingSymmetry90::Rot90_ApplyBC: Apply 90 degree rotational symmetry boundary condition
            Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions
          GROUP ML_BSSN_constraints1_group: ML_BSSN_constraints1
            ML_BSSN::ML_BSSN_constraints1: ML_BSSN_constraints1
            GROUP ML_BSSN_constraints1_bc_group: ML_BSSN_constraints1
              ML_BSSN::ML_BSSN_constraints1_SelectBCs: [level] ML_BSSN_constraints1_SelectBCs
              GROUP ML_BSSN_constraints1_ApplyBCs: Apply BCs for groups set in ML_BSSN_constraints1
                GROUP BoundaryConditions: Execute all boundary conditions
                  Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions
                  CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions
                  ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries
                  RotatingSymmetry90::Rot90_ApplyBC: Apply 90 degree rotational symmetry boundary condition
                Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions
          GROUP ML_BSSN_constraints2_group: ML_BSSN_constraints2
            ML_BSSN::ML_BSSN_constraints2: ML_BSSN_constraints2
            GROUP ML_BSSN_constraints2_bc_group: ML_BSSN_constraints2
              ML_BSSN::ML_BSSN_constraints2_SelectBCs: [level] ML_BSSN_constraints2_SelectBCs
              GROUP ML_BSSN_constraints2_ApplyBCs: Apply BCs for groups set in ML_BSSN_constraints2
                GROUP BoundaryConditions: Execute all boundary conditions
                  Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions
                  CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions
                  ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries
                  RotatingSymmetry90::Rot90_ApplyBC: Apply 90 degree rotational symmetry boundary condition
                Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions
      Initialise finer grids recursively
      Restrict from finer grids
      [CCTK_POSTRESTRICTINITIAL]
        GROUP ML_ADMConstraints_bc_group: ML_ADMConstraints
          ML_ADMConstraints::ML_ADMConstraints_SelectBCs: [level] ML_ADMConstraints_SelectBCs
          GROUP ML_ADMConstraints_ApplyBCs: Apply BCs for groups set in ML_ADMConstraints
            GROUP BoundaryConditions: Execute all boundary conditions
              Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions
              CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions
              ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries
              RotatingSymmetry90::Rot90_ApplyBC: Apply 90 degree rotational symmetry boundary condition
            Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions
        GROUP ML_BSSN_constraints1_bc_group: ML_BSSN_constraints1
          ML_BSSN::ML_BSSN_constraints1_SelectBCs: [level] ML_BSSN_constraints1_SelectBCs
          GROUP ML_BSSN_constraints1_ApplyBCs: Apply BCs for groups set in ML_BSSN_constraints1
            GROUP BoundaryConditions: Execute all boundary conditions
              Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions
              CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions
              ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries
              RotatingSymmetry90::Rot90_ApplyBC: Apply 90 degree rotational symmetry boundary condition
            Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions
        GROUP ML_BSSN_constraints2_bc_group: ML_BSSN_constraints2
          ML_BSSN::ML_BSSN_constraints2_SelectBCs: [level] ML_BSSN_constraints2_SelectBCs
          GROUP ML_BSSN_constraints2_ApplyBCs: Apply BCs for groups set in ML_BSSN_constraints2
            GROUP BoundaryConditions: Execute all boundary conditions
              Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions
              CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions
              ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries
              RotatingSymmetry90::Rot90_ApplyBC: Apply 90 degree rotational symmetry boundary condition
            Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions
        GROUP MoL_PostStep: Ensure that everything is correct after restriction
          ML_BSSN::ML_BSSN_enforce: ML_BSSN_enforce
          ML_BSSN::ML_BSSN_SelectBoundConds: [level] select boundary conditions
          GROUP ML_BSSN_ApplyBCs: Apply boundary conditions controlled by thorn Boundary
            GROUP BoundaryConditions: Execute all boundary conditions
              Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions
              CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions
              ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries
              RotatingSymmetry90::Rot90_ApplyBC: Apply 90 degree rotational symmetry boundary condition
            Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions
          GROUP ML_BSSN_convertToADMBaseGroupWrapper: Calculate ADM variables
            GROUP ML_BSSN_convertToADMBaseGroup: Calculate ADM variables
              ML_BSSN::ML_BSSN_convertToADMBase: ML_BSSN_convertToADMBase
              ML_BSSN::ML_BSSN_convertToADMBaseFakeDtLapseShift: ML_BSSN_convertToADMBaseFakeDtLapseShift
            ML_BSSN_Helper::ML_BSSN_SelectBCsADMBase: [level] Select boundary conditions for ADMBase variables
            GROUP ML_BSSN_ApplyBCsADMBase: Apply boundary conditions to ADMBase variables
              GROUP BoundaryConditions: Execute all boundary conditions
                Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions
                CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions
                ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries
                RotatingSymmetry90::Rot90_ApplyBC: Apply 90 degree rotational symmetry boundary condition
              Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions
          GROUP ADMBase_SetADMVars: Set the ADM variables before this group, and use them afterwards
          GROUP MoL_OldBdry_Wrap: Wrapper group, do not schedule directly into this group
            MoL::MoL_OldBdry_SetDt: [level] Store and change dt
            GROUP MoL_OldStyleBoundaries: Place old style boundary routines here
            MoL::MoL_OldBdry_ResetDt: [level] Reset dt
      [CCTK_POSTPOSTINITIAL]
      [CCTK_POSTSTEP]
        AHFinderDirect::AHFinderDirect_import_mask: [global] [loop-local] import the excision mask
        AHFinderDirect::AHFinderDirect_find_horizons: [global] find apparent horizon(s) after this time step
        AHFinderDirect::AHFinderDirect_store: [global] store apparent horizon(s) into spherical surface(s)
        AHFinderDirect::AHFinderDirect_save: [global] save apparent horizon(s) into Cactus variables
        AHFinderDirect::AHFinderDirect_maybe_do_masks: [global] [loop-local] set mask(s) based on apparent horizon position(s)
        SphericalSurface::SphericalSurface_Set: [global] Set surface radii
        GROUP SphericalSurface_HasBeenSet: Set the spherical surfaces before this group, and use it afterwards
          SphericalSurface::SphericalSurface_CheckState: [global] Test the state of the spherical surfaces
        NaNChecker::NaNChecker_NaNCheck: [local] Check for NaNs and count them in NaNChecker::NaNsFound
        NaNChecker::NaNChecker_TakeAction: [global] [loop-level] Output NaNChecker::NaNmask and take action according to NaNChecker::action_if_found
        Dissipation::setup_epsdis: Setup spatially varying dissipation
        GROUP MaskBase_SetupMask: Set up the weight function
          CarpetReduce::MaskBase_InitMask: [global] [loop-local] Initialise the weight function
          GROUP SetupMask: Set up the weight function (schedule other routines in here)
            CarpetMask::CarpetExcludedSetup: [global] [loop-local] Set up the weight function for the excluded regions
            CarpetMask::CarpetSurfaceSetup: [global] [loop-local] Set up the weight function for the excluded spherical surfaces
            CarpetReduce::CoordBase_SetupMask: [global] [loop-local] Set up the outer boundaries of the weight function
            CarpetReduce::CarpetMaskSetup: [global] [loop-singlemap] Set up the weight function for the restriction regions
    endif
    if (recover initial data)
      [CCTK_BASEGRID]
        ADMBase::ADMBase_SetShiftStateOn: Set the shift_state variable to 1
        ADMBase::ADMBase_SetDtLapseStateOn: Set the dtlapse_state variable to 1
        ADMBase::ADMBase_SetDtShiftStateOn: Set the dtshift_state variable to 1
        ADMMacros::ADMMacros_SetLocalSpatialOrder: Initialize the local_spatial_order
        CartGrid3D::CartGrid3D_SetRanges: Set up ranges for spatial 3D Cartesian coordinates (on all grids)
        CarpetIOASCII::CarpetIOASCIIInit: [global] Initialisation routine
        CarpetIOBasic::CarpetIOBasicInit: [global] Initialisation routine
        CarpetIOHDF5::CarpetIOHDF5_Init: [global] Initialisation routine
        CarpetIOScalar::CarpetIOScalarInit: [global] Initialisation routine
        CartGrid3D::SpatialCoordinates: Set up spatial 3D Cartesian coordinates on the GH
        SphericalSurface::SphericalSurface_SetupRes: [global] [loop-local] Set surface resolution automatically
        AHFinderDirect::AHFinderDirect_setup: [global] setup data structures
        Dissipation::dissipation_basegrid: Ensure that there are enough ghost zones
        ML_ADMConstraints::ML_ADMConstraints_CheckBoundaries: [meta] check boundaries treatment
        ML_BSSN::ML_BSSN_CheckBoundaries: [meta] check boundaries treatment
        SphericalSurface::SphericalSurface_Setup: [global] Calculate surface coordinate descriptors
        GROUP MaskBase_SetupMask: Set up the weight function
          CarpetReduce::MaskBase_InitMask: [global] [loop-local] Initialise the weight function
          GROUP SetupMask: Set up the weight function (schedule other routines in here)
            CarpetMask::CarpetExcludedSetup: [global] [loop-local] Set up the weight function for the excluded regions
            CarpetMask::CarpetSurfaceSetup: [global] [loop-local] Set up the weight function for the excluded spherical surfaces
            CarpetReduce::CoordBase_SetupMask: [global] [loop-local] Set up the outer boundaries of the weight function
            CarpetReduce::CarpetMaskSetup: [global] [loop-singlemap] Set up the weight function for the restriction regions
        SphericalSurface::SphericalSurface_Set: [global] Set surface radii to be used for initial setup in other thorns
        GROUP SphericalSurface_HasBeenSet: Set the spherical surfaces before this group, and use it afterwards
          SphericalSurface::SphericalSurface_CheckState: [global] Test the state of the spherical surfaces
        SymBase::SymBase_Check: Check whether the driver set up the grid consistently
        Time::Time_Initialise: [global] Initialise Time variables
        Time::Time_Simple: [singlemap] Set timestep based on Courant condition (courant_static)
        TmunuBase::TmunuBase_SetStressEnergyState: [global] Set the stress_energy_state variable
      [CCTK_RECOVER_VARIABLES]
      [CCTK_POST_RECOVER_VARIABLES]
        AHFinderDirect::AHFinderDirect_recover: [global] import horizon data from Cactus variables
        CarpetIOHDF5::CarpetIOHDF5_InitCheckpointingIntervals: [global] Initialisation of checkpointing intervals after recovery
        GROUP MaskBase_SetupMask: Set up the weight function
          CarpetReduce::MaskBase_InitMask: [global] [loop-local] Initialise the weight function
          GROUP SetupMask: Set up the weight function (schedule other routines in here)
            CarpetMask::CarpetExcludedSetup: [global] [loop-local] Set up the weight function for the excluded regions
            CarpetMask::CarpetSurfaceSetup: [global] [loop-local] Set up the weight function for the excluded spherical surfaces
            CarpetReduce::CoordBase_SetupMask: [global] [loop-local] Set up the outer boundaries of the weight function
            CarpetReduce::CarpetMaskSetup: [global] [loop-singlemap] Set up the weight function for the restriction regions
        GROUP MoL_PostStep: Ensure that everything is correct after recovery
          ML_BSSN::ML_BSSN_enforce: ML_BSSN_enforce
          ML_BSSN::ML_BSSN_SelectBoundConds: [level] select boundary conditions
          GROUP ML_BSSN_ApplyBCs: Apply boundary conditions controlled by thorn Boundary
            GROUP BoundaryConditions: Execute all boundary conditions
              Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions
              CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions
              ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries
              RotatingSymmetry90::Rot90_ApplyBC: Apply 90 degree rotational symmetry boundary condition
            Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions
          GROUP ML_BSSN_convertToADMBaseGroupWrapper: Calculate ADM variables
            GROUP ML_BSSN_convertToADMBaseGroup: Calculate ADM variables
              ML_BSSN::ML_BSSN_convertToADMBase: ML_BSSN_convertToADMBase
              ML_BSSN::ML_BSSN_convertToADMBaseFakeDtLapseShift: ML_BSSN_convertToADMBaseFakeDtLapseShift
            ML_BSSN_Helper::ML_BSSN_SelectBCsADMBase: [level] Select boundary conditions for ADMBase variables
            GROUP ML_BSSN_ApplyBCsADMBase: Apply boundary conditions to ADMBase variables
              GROUP BoundaryConditions: Execute all boundary conditions
                Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions
                CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions
                ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries
                RotatingSymmetry90::Rot90_ApplyBC: Apply 90 degree rotational symmetry boundary condition
              Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions
          GROUP ADMBase_SetADMVars: Set the ADM variables before this group, and use them afterwards
          GROUP MoL_OldBdry_Wrap: Wrapper group, do not schedule directly into this group
            MoL::MoL_OldBdry_SetDt: [level] Store and change dt
            GROUP MoL_OldStyleBoundaries: Place old style boundary routines here
            MoL::MoL_OldBdry_ResetDt: [level] Reset dt
        GROUP MoL_PseudoEvolution: Calculate pseudo-evolved quantities
          GROUP ADMBase_SetADMVars: Set the ADM variables before this group, and use them afterwards
          GROUP ML_ADMConstraints_group: ML_ADMConstraints
            ML_ADMConstraints::ML_ADMConstraints: ML_ADMConstraints
            GROUP ML_ADMConstraints_bc_group: ML_ADMConstraints
              ML_ADMConstraints::ML_ADMConstraints_SelectBCs: [level] ML_ADMConstraints_SelectBCs
              GROUP ML_ADMConstraints_ApplyBCs: Apply BCs for groups set in ML_ADMConstraints
                GROUP BoundaryConditions: Execute all boundary conditions
                  Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions
                  CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions
                  ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries
                  RotatingSymmetry90::Rot90_ApplyBC: Apply 90 degree rotational symmetry boundary condition
                Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions
          ML_ADMConstraints::ML_ADMConstraints_SelectBoundConds: [level] select boundary conditions
          GROUP ML_ADMConstraints_ApplyBCs: Apply boundary conditions controlled by thorn Boundary
            GROUP BoundaryConditions: Execute all boundary conditions
              Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions
              CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions
              ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries
              RotatingSymmetry90::Rot90_ApplyBC: Apply 90 degree rotational symmetry boundary condition
            Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions
          GROUP ML_BSSN_constraints1_group: ML_BSSN_constraints1
            ML_BSSN::ML_BSSN_constraints1: ML_BSSN_constraints1
            GROUP ML_BSSN_constraints1_bc_group: ML_BSSN_constraints1
              ML_BSSN::ML_BSSN_constraints1_SelectBCs: [level] ML_BSSN_constraints1_SelectBCs
              GROUP ML_BSSN_constraints1_ApplyBCs: Apply BCs for groups set in ML_BSSN_constraints1
                GROUP BoundaryConditions: Execute all boundary conditions
                  Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions
                  CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions
                  ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries
                  RotatingSymmetry90::Rot90_ApplyBC: Apply 90 degree rotational symmetry boundary condition
                Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions
          GROUP ML_BSSN_constraints2_group: ML_BSSN_constraints2
            ML_BSSN::ML_BSSN_constraints2: ML_BSSN_constraints2
            GROUP ML_BSSN_constraints2_bc_group: ML_BSSN_constraints2
              ML_BSSN::ML_BSSN_constraints2_SelectBCs: [level] ML_BSSN_constraints2_SelectBCs
              GROUP ML_BSSN_constraints2_ApplyBCs: Apply BCs for groups set in ML_BSSN_constraints2
                GROUP BoundaryConditions: Execute all boundary conditions
                  Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions
                  CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions
                  ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries
                  RotatingSymmetry90::Rot90_ApplyBC: Apply 90 degree rotational symmetry boundary condition
                Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions
    endif
    if (checkpoint initial data)
      [CCTK_CPINITIAL]
        CarpetIOHDF5::CarpetIOHDF5_InitialDataCheckpoint: [meta] Initial data checkpoint routine
    endif
    if (analysis)
      [CCTK_ANALYSIS]
        CarpetLib::CarpetLib_printtimestats: [global] Print timing statistics if desired
        CarpetLib::CarpetLib_printmemstats: [global] Print memory statistics if desired
        Formaline::Formaline_AnnounceUpdate: [global] Put some meta information about the current run into permanent storage
        GROUP ML_BSSN_evolCalcGroup: Calculate BSSN RHS
          ML_BSSN::ML_BSSN_RHS1: ML_BSSN_RHS1
          ML_BSSN::ML_BSSN_RHS2: ML_BSSN_RHS2
          ML_BSSN_Helper::ML_BSSN_NewRad: Apply NewRad boundary conditions to RHS
        TimerReport::zzz_TimerReport_Output: [global] Print the timer report
  endif
  Output grid variables

  do loop over timesteps
    [CCTK_PREREGRID]
      CarpetTracker::CarpetTracker_SetPositions: [global] Set positions of refined regions
    Change grid hierarchy
    [CCTK_POSTREGRID]
      AHFinderDirect::AHFinderDirect_maybe_do_masks: [global] [loop-local] set mask(s) based on apparent horizon position(s)
      CartGrid3D::SpatialCoordinates: Set Coordinates after regridding
      GROUP MaskBase_SetupMask: Set up the weight function
        CarpetReduce::MaskBase_InitMask: [global] [loop-local] Initialise the weight function
        GROUP SetupMask: Set up the weight function (schedule other routines in here)
          CarpetMask::CarpetExcludedSetup: [global] [loop-local] Set up the weight function for the excluded regions
          CarpetMask::CarpetSurfaceSetup: [global] [loop-local] Set up the weight function for the excluded spherical surfaces
          CarpetReduce::CoordBase_SetupMask: [global] [loop-local] Set up the outer boundaries of the weight function
          CarpetReduce::CarpetMaskSetup: [global] [loop-singlemap] Set up the weight function for the restriction regions
      Dissipation::setup_epsdis: Setup spatially varying dissipation
      GROUP MoL_PostStep: Ensure that everything is correct after regridding
        ML_BSSN::ML_BSSN_enforce: ML_BSSN_enforce
        ML_BSSN::ML_BSSN_SelectBoundConds: [level] select boundary conditions
        GROUP ML_BSSN_ApplyBCs: Apply boundary conditions controlled by thorn Boundary
          GROUP BoundaryConditions: Execute all boundary conditions
            Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions
            CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions
            ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries
            RotatingSymmetry90::Rot90_ApplyBC: Apply 90 degree rotational symmetry boundary condition
          Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions
        GROUP ML_BSSN_convertToADMBaseGroupWrapper: Calculate ADM variables
          GROUP ML_BSSN_convertToADMBaseGroup: Calculate ADM variables
            ML_BSSN::ML_BSSN_convertToADMBase: ML_BSSN_convertToADMBase
            ML_BSSN::ML_BSSN_convertToADMBaseFakeDtLapseShift: ML_BSSN_convertToADMBaseFakeDtLapseShift
          ML_BSSN_Helper::ML_BSSN_SelectBCsADMBase: [level] Select boundary conditions for ADMBase variables
          GROUP ML_BSSN_ApplyBCsADMBase: Apply boundary conditions to ADMBase variables
            GROUP BoundaryConditions: Execute all boundary conditions
              Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions
              CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions
              ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries
              RotatingSymmetry90::Rot90_ApplyBC: Apply 90 degree rotational symmetry boundary condition
            Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions
        GROUP ADMBase_SetADMVars: Set the ADM variables before this group, and use them afterwards
        GROUP MoL_OldBdry_Wrap: Wrapper group, do not schedule directly into this group
          MoL::MoL_OldBdry_SetDt: [level] Store and change dt
          GROUP MoL_OldStyleBoundaries: Place old style boundary routines here
          MoL::MoL_OldBdry_ResetDt: [level] Reset dt
      GROUP MoL_PseudoEvolution: Calculate pseudo-evolved quantities
        GROUP ADMBase_SetADMVars: Set the ADM variables before this group, and use them afterwards
        GROUP ML_ADMConstraints_group: ML_ADMConstraints
          ML_ADMConstraints::ML_ADMConstraints: ML_ADMConstraints
          GROUP ML_ADMConstraints_bc_group: ML_ADMConstraints
            ML_ADMConstraints::ML_ADMConstraints_SelectBCs: [level] ML_ADMConstraints_SelectBCs
            GROUP ML_ADMConstraints_ApplyBCs: Apply BCs for groups set in ML_ADMConstraints
              GROUP BoundaryConditions: Execute all boundary conditions
                Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions
                CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions
                ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries
                RotatingSymmetry90::Rot90_ApplyBC: Apply 90 degree rotational symmetry boundary condition
              Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions
        ML_ADMConstraints::ML_ADMConstraints_SelectBoundConds: [level] select boundary conditions
        GROUP ML_ADMConstraints_ApplyBCs: Apply boundary conditions controlled by thorn Boundary
          GROUP BoundaryConditions: Execute all boundary conditions
            Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions
            CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions
            ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries
            RotatingSymmetry90::Rot90_ApplyBC: Apply 90 degree rotational symmetry boundary condition
          Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions
        GROUP ML_BSSN_constraints1_group: ML_BSSN_constraints1
          ML_BSSN::ML_BSSN_constraints1: ML_BSSN_constraints1
          GROUP ML_BSSN_constraints1_bc_group: ML_BSSN_constraints1
            ML_BSSN::ML_BSSN_constraints1_SelectBCs: [level] ML_BSSN_constraints1_SelectBCs
            GROUP ML_BSSN_constraints1_ApplyBCs: Apply BCs for groups set in ML_BSSN_constraints1
              GROUP BoundaryConditions: Execute all boundary conditions
                Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions
                CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions
                ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries
                RotatingSymmetry90::Rot90_ApplyBC: Apply 90 degree rotational symmetry boundary condition
              Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions
        GROUP ML_BSSN_constraints2_group: ML_BSSN_constraints2
          ML_BSSN::ML_BSSN_constraints2: ML_BSSN_constraints2
          GROUP ML_BSSN_constraints2_bc_group: ML_BSSN_constraints2
            ML_BSSN::ML_BSSN_constraints2_SelectBCs: [level] ML_BSSN_constraints2_SelectBCs
            GROUP ML_BSSN_constraints2_ApplyBCs: Apply BCs for groups set in ML_BSSN_constraints2
              GROUP BoundaryConditions: Execute all boundary conditions
                Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions
                CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions
                ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries
                RotatingSymmetry90::Rot90_ApplyBC: Apply 90 degree rotational symmetry boundary condition
              Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions
    Rotate timelevels
    iteration = iteration+1
    t = t+dt
    [CCTK_PRESTEP]
      CoordGauge::Einstein_SetNextSlicing: Identify the slicing for the next iteration
      NaNChecker::NaNChecker_ResetCounter: [global] Reset the NaNChecker::NaNsFound counter
    [CCTK_EVOL]
      MoL::MoL_StartLoop: [level] Initialise the step size control
      while (MoL::MoL_Stepsize_Bad)
        GROUP MoL_Evolution: A single Cactus evolution step using MoL
          GROUP MoL_StartStep: MoL internal setup for the evolution step
            MoL::MoL_SetCounter: [level] Set the counter for the ODE method to loop over
            MoL::MoL_SetTime: [level] Ensure the correct time and timestep are used
          GROUP MoL_PreStep: Physics thorns can schedule preloop setup routines in here
          MoL::MoL_InitialCopy: Ensure the data is in the correct timelevel
          while (MoL::MoL_Intermediate_Step)
            GROUP MoL_Step: The loop over the intermediate steps for the ODE integrator
              MoL::MoL_InitRHS: Initialise the RHS functions
              GROUP MoL_CalcRHS: Physics thorns schedule the calculation of the discrete spatial operator in here
                GROUP ML_BSSN_evolCalcGroup: Calculate BSSN RHS
                  ML_BSSN::ML_BSSN_RHS1: ML_BSSN_RHS1
                  ML_BSSN::ML_BSSN_RHS2: ML_BSSN_RHS2
                  ML_BSSN_Helper::ML_BSSN_NewRad: Apply NewRad boundary conditions to RHS
              GROUP MoL_PostRHS: Modify RHS functions
                Dissipation::dissipation_add: Add Kreiss-Oliger dissipation to the right hand sides
              GROUP MoL_RHSBoundaries: Any 'final' modifications to the RHS functions (boundaries etc.)
              MoL::MoL_Add: Updates calculated with the efficient Runge-Kutta 4 method
              MoL::MoL_DecrementCounter: [level] Alter the counter number
              MoL::MoL_ResetTime: [level] If necessary, change the time
              GROUP MoL_PostStep: The group for physics thorns to schedule boundary calls etc.
                ML_BSSN::ML_BSSN_enforce: ML_BSSN_enforce
                ML_BSSN::ML_BSSN_SelectBoundConds: [level] select boundary conditions
                GROUP ML_BSSN_ApplyBCs: Apply boundary conditions controlled by thorn Boundary
                  GROUP BoundaryConditions: Execute all boundary conditions
                    Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions
                    CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions
                    ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries
                    RotatingSymmetry90::Rot90_ApplyBC: Apply 90 degree rotational symmetry boundary condition
                  Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions
                GROUP ML_BSSN_convertToADMBaseGroupWrapper: Calculate ADM variables
                  GROUP ML_BSSN_convertToADMBaseGroup: Calculate ADM variables
                    ML_BSSN::ML_BSSN_convertToADMBase: ML_BSSN_convertToADMBase
                    ML_BSSN::ML_BSSN_convertToADMBaseFakeDtLapseShift: ML_BSSN_convertToADMBaseFakeDtLapseShift
                  ML_BSSN_Helper::ML_BSSN_SelectBCsADMBase: [level] Select boundary conditions for ADMBase variables
                  GROUP ML_BSSN_ApplyBCsADMBase: Apply boundary conditions to ADMBase variables
                    GROUP BoundaryConditions: Execute all boundary conditions
                      Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions
                      CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions
                      ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries
                      RotatingSymmetry90::Rot90_ApplyBC: Apply 90 degree rotational symmetry boundary condition
                    Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions
                GROUP ADMBase_SetADMVars: Set the ADM variables before this group, and use them afterwards
                GROUP MoL_OldBdry_Wrap: Wrapper group, do not schedule directly into this group
                  MoL::MoL_OldBdry_SetDt: [level] Store and change dt
                  GROUP MoL_OldStyleBoundaries: Place old style boundary routines here
                  MoL::MoL_OldBdry_ResetDt: [level] Reset dt
              MoL::MoL_ResetDeltaTime: [level] If necessary, change the timestep
          end while
          MoL::MoL_FinishLoop: [level] Control the step size
          MoL::MoL_RestoreSandR: Restoring the Save and Restore variables to the original state
      end while
      GROUP MoL_PseudoEvolution: Calculate pseudo-evolved quantities
        GROUP ADMBase_SetADMVars: Set the ADM variables before this group, and use them afterwards
        GROUP ML_ADMConstraints_group: ML_ADMConstraints
          ML_ADMConstraints::ML_ADMConstraints: ML_ADMConstraints
          GROUP ML_ADMConstraints_bc_group: ML_ADMConstraints
            ML_ADMConstraints::ML_ADMConstraints_SelectBCs: [level] ML_ADMConstraints_SelectBCs
            GROUP ML_ADMConstraints_ApplyBCs: Apply BCs for groups set in ML_ADMConstraints
              GROUP BoundaryConditions: Execute all boundary conditions
                Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions
                CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions
                ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries
                RotatingSymmetry90::Rot90_ApplyBC: Apply 90 degree rotational symmetry boundary condition
              Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions
        ML_ADMConstraints::ML_ADMConstraints_SelectBoundConds: [level] select boundary conditions
        GROUP ML_ADMConstraints_ApplyBCs: Apply boundary conditions controlled by thorn Boundary
          GROUP BoundaryConditions: Execute all boundary conditions
            Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions
            CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions
            ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries
            RotatingSymmetry90::Rot90_ApplyBC: Apply 90 degree rotational symmetry boundary condition
          Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions
        GROUP ML_BSSN_constraints1_group: ML_BSSN_constraints1
          ML_BSSN::ML_BSSN_constraints1: ML_BSSN_constraints1
          GROUP ML_BSSN_constraints1_bc_group: ML_BSSN_constraints1
            ML_BSSN::ML_BSSN_constraints1_SelectBCs: [level] ML_BSSN_constraints1_SelectBCs
            GROUP ML_BSSN_constraints1_ApplyBCs: Apply BCs for groups set in ML_BSSN_constraints1
              GROUP BoundaryConditions: Execute all boundary conditions
                Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions
                CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions
                ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries
                RotatingSymmetry90::Rot90_ApplyBC: Apply 90 degree rotational symmetry boundary condition
              Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions
        GROUP ML_BSSN_constraints2_group: ML_BSSN_constraints2
          ML_BSSN::ML_BSSN_constraints2: ML_BSSN_constraints2
          GROUP ML_BSSN_constraints2_bc_group: ML_BSSN_constraints2
            ML_BSSN::ML_BSSN_constraints2_SelectBCs: [level] ML_BSSN_constraints2_SelectBCs
            GROUP ML_BSSN_constraints2_ApplyBCs: Apply BCs for groups set in ML_BSSN_constraints2
              GROUP BoundaryConditions: Execute all boundary conditions
                Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions
                CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions
                ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries
                RotatingSymmetry90::Rot90_ApplyBC: Apply 90 degree rotational symmetry boundary condition
              Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions
    Evolve finer grids recursively
    Restrict from finer grids
    [CCTK_POSTRESTRICT]
      GROUP ML_ADMConstraints_bc_group: ML_ADMConstraints
        ML_ADMConstraints::ML_ADMConstraints_SelectBCs: [level] ML_ADMConstraints_SelectBCs
        GROUP ML_ADMConstraints_ApplyBCs: Apply BCs for groups set in ML_ADMConstraints
          GROUP BoundaryConditions: Execute all boundary conditions
            Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions
            CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions
            ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries
            RotatingSymmetry90::Rot90_ApplyBC: Apply 90 degree rotational symmetry boundary condition
          Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions
      GROUP ML_BSSN_constraints1_bc_group: ML_BSSN_constraints1
        ML_BSSN::ML_BSSN_constraints1_SelectBCs: [level] ML_BSSN_constraints1_SelectBCs
        GROUP ML_BSSN_constraints1_ApplyBCs: Apply BCs for groups set in ML_BSSN_constraints1
          GROUP BoundaryConditions: Execute all boundary conditions
            Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions
            CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions
            ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries
            RotatingSymmetry90::Rot90_ApplyBC: Apply 90 degree rotational symmetry boundary condition
          Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions
      GROUP ML_BSSN_constraints2_bc_group: ML_BSSN_constraints2
        ML_BSSN::ML_BSSN_constraints2_SelectBCs: [level] ML_BSSN_constraints2_SelectBCs
        GROUP ML_BSSN_constraints2_ApplyBCs: Apply BCs for groups set in ML_BSSN_constraints2
          GROUP BoundaryConditions: Execute all boundary conditions
            Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions
            CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions
            ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries
            RotatingSymmetry90::Rot90_ApplyBC: Apply 90 degree rotational symmetry boundary condition
          Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions
      GROUP MoL_PostStep: Ensure that everything is correct after restriction
        ML_BSSN::ML_BSSN_enforce: ML_BSSN_enforce
        ML_BSSN::ML_BSSN_SelectBoundConds: [level] select boundary conditions
        GROUP ML_BSSN_ApplyBCs: Apply boundary conditions controlled by thorn Boundary
          GROUP BoundaryConditions: Execute all boundary conditions
            Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions
            CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions
            ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries
            RotatingSymmetry90::Rot90_ApplyBC: Apply 90 degree rotational symmetry boundary condition
          Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions
        GROUP ML_BSSN_convertToADMBaseGroupWrapper: Calculate ADM variables
          GROUP ML_BSSN_convertToADMBaseGroup: Calculate ADM variables
            ML_BSSN::ML_BSSN_convertToADMBase: ML_BSSN_convertToADMBase
            ML_BSSN::ML_BSSN_convertToADMBaseFakeDtLapseShift: ML_BSSN_convertToADMBaseFakeDtLapseShift
          ML_BSSN_Helper::ML_BSSN_SelectBCsADMBase: [level] Select boundary conditions for ADMBase variables
          GROUP ML_BSSN_ApplyBCsADMBase: Apply boundary conditions to ADMBase variables
            GROUP BoundaryConditions: Execute all boundary conditions
              Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions
              CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions
              ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries
              RotatingSymmetry90::Rot90_ApplyBC: Apply 90 degree rotational symmetry boundary condition
            Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions
        GROUP ADMBase_SetADMVars: Set the ADM variables before this group, and use them afterwards
        GROUP MoL_OldBdry_Wrap: Wrapper group, do not schedule directly into this group
          MoL::MoL_OldBdry_SetDt: [level] Store and change dt
          GROUP MoL_OldStyleBoundaries: Place old style boundary routines here
          MoL::MoL_OldBdry_ResetDt: [level] Reset dt
    [CCTK_POSTSTEP]
      AHFinderDirect::AHFinderDirect_import_mask: [global] [loop-local] import the excision mask
      AHFinderDirect::AHFinderDirect_find_horizons: [global] find apparent horizon(s) after this time step
      AHFinderDirect::AHFinderDirect_store: [global] store apparent horizon(s) into spherical surface(s)
      AHFinderDirect::AHFinderDirect_save: [global] save apparent horizon(s) into Cactus variables
      AHFinderDirect::AHFinderDirect_maybe_do_masks: [global] [loop-local] set mask(s) based on apparent horizon position(s)
      SphericalSurface::SphericalSurface_Set: [global] Set surface radii
      GROUP SphericalSurface_HasBeenSet: Set the spherical surfaces before this group, and use it afterwards
        SphericalSurface::SphericalSurface_CheckState: [global] Test the state of the spherical surfaces
      NaNChecker::NaNChecker_NaNCheck: [local] Check for NaNs and count them in NaNChecker::NaNsFound
      NaNChecker::NaNChecker_TakeAction: [global] [loop-level] Output NaNChecker::NaNmask and take action according to NaNChecker::action_if_found
      Dissipation::setup_epsdis: Setup spatially varying dissipation
      GROUP MaskBase_SetupMask: Set up the weight function
        CarpetReduce::MaskBase_InitMask: [global] [loop-local] Initialise the weight function
        GROUP SetupMask: Set up the weight function (schedule other routines in here)
          CarpetMask::CarpetExcludedSetup: [global] [loop-local] Set up the weight function for the excluded regions
          CarpetMask::CarpetSurfaceSetup: [global] [loop-local] Set up the weight function for the excluded spherical surfaces
          CarpetReduce::CoordBase_SetupMask: [global] [loop-local] Set up the outer boundaries of the weight function
          CarpetReduce::CarpetMaskSetup: [global] [loop-singlemap] Set up the weight function for the restriction regions
  if (checkpoint)
    [CCTK_CHECKPOINT]
      CarpetIOHDF5::CarpetIOHDF5_EvolutionCheckpoint: [meta] Evolution checkpoint routine
      TimerReport::zzz_TimerReport_Checkpoint: [global] Print the timer report
  endif
  if (analysis)
    [CCTK_ANALYSIS]
      CarpetLib::CarpetLib_printtimestats: [global] Print timing statistics if desired
      CarpetLib::CarpetLib_printmemstats: [global] Print memory statistics if desired
      Formaline::Formaline_AnnounceUpdate: [global] Put some meta information about the current run into permanent storage
      GROUP ML_BSSN_evolCalcGroup: Calculate BSSN RHS
        ML_BSSN::ML_BSSN_RHS1: ML_BSSN_RHS1
        ML_BSSN::ML_BSSN_RHS2: ML_BSSN_RHS2
        ML_BSSN_Helper::ML_BSSN_NewRad: Apply NewRad boundary conditions to RHS
      TimerReport::zzz_TimerReport_Output: [global] Print the timer report
  endif
  Output grid variables
  enddo

  Termination routines
    [CCTK_TERMINATE]
      CarpetIOHDF5::CarpetIOHDF5_TerminationCheckpoint: [meta] Termination checkpoint routine
      Formaline::Formaline_AnnounceFinal: [global] Put some meta information about the current run into permanent storage
      MoL::MoL_FreeIndexArrays: Free the MoL bookkeeping index arrays
      TimerReport::zzz_TimerReport_Output: [global] Print the timer report

  Shutdown routines
    [CCTK_SHUTDOWN]

  Routines run after changing the grid hierarchy:
    [CCTK_POSTREGRID]
      AHFinderDirect::AHFinderDirect_maybe_do_masks: [global] [loop-local] set mask(s) based on apparent horizon position(s)
      CartGrid3D::SpatialCoordinates: Set Coordinates after regridding
      GROUP MaskBase_SetupMask: Set up the weight function
        CarpetReduce::MaskBase_InitMask: [global] [loop-local] Initialise the weight function
        GROUP SetupMask: Set up the weight function (schedule other routines in here)
          CarpetMask::CarpetExcludedSetup: [global] [loop-local] Set up the weight function for the excluded regions
          CarpetMask::CarpetSurfaceSetup: [global] [loop-local] Set up the weight function for the excluded spherical surfaces
          CarpetReduce::CoordBase_SetupMask: [global] [loop-local] Set up the outer boundaries of the weight function
          CarpetReduce::CarpetMaskSetup: [global] [loop-singlemap] Set up the weight function for the restriction regions
      Dissipation::setup_epsdis: Setup spatially varying dissipation
      GROUP MoL_PostStep: Ensure that everything is correct after regridding
        ML_BSSN::ML_BSSN_enforce: ML_BSSN_enforce
        ML_BSSN::ML_BSSN_SelectBoundConds: [level] select boundary conditions
        GROUP ML_BSSN_ApplyBCs: Apply boundary conditions controlled by thorn Boundary
          GROUP BoundaryConditions: Execute all boundary conditions
            Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions
            CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions
            ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries
            RotatingSymmetry90::Rot90_ApplyBC: Apply 90 degree rotational symmetry boundary condition
          Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions
        GROUP ML_BSSN_convertToADMBaseGroupWrapper: Calculate ADM variables
          GROUP ML_BSSN_convertToADMBaseGroup: Calculate ADM variables
            ML_BSSN::ML_BSSN_convertToADMBase: ML_BSSN_convertToADMBase
            ML_BSSN::ML_BSSN_convertToADMBaseFakeDtLapseShift: ML_BSSN_convertToADMBaseFakeDtLapseShift
          ML_BSSN_Helper::ML_BSSN_SelectBCsADMBase: [level] Select boundary conditions for ADMBase variables
          GROUP ML_BSSN_ApplyBCsADMBase: Apply boundary conditions to ADMBase variables
            GROUP BoundaryConditions: Execute all boundary conditions
              Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions
              CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions
              ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries
              RotatingSymmetry90::Rot90_ApplyBC: Apply 90 degree rotational symmetry boundary condition
            Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions
        GROUP ADMBase_SetADMVars: Set the ADM variables before this group, and use them afterwards
        GROUP MoL_OldBdry_Wrap: Wrapper group, do not schedule directly into this group
          MoL::MoL_OldBdry_SetDt: [level] Store and change dt
          GROUP MoL_OldStyleBoundaries: Place old style boundary routines here
          MoL::MoL_OldBdry_ResetDt: [level] Reset dt
      GROUP MoL_PseudoEvolution: Calculate pseudo-evolved quantities
        GROUP ADMBase_SetADMVars: Set the ADM variables before this group, and use them afterwards
        GROUP ML_ADMConstraints_group: ML_ADMConstraints
          ML_ADMConstraints::ML_ADMConstraints: ML_ADMConstraints
          GROUP ML_ADMConstraints_bc_group: ML_ADMConstraints
            ML_ADMConstraints::ML_ADMConstraints_SelectBCs: [level] ML_ADMConstraints_SelectBCs
            GROUP ML_ADMConstraints_ApplyBCs: Apply BCs for groups set in ML_ADMConstraints
              GROUP BoundaryConditions: Execute all boundary conditions
                Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions
                CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions
                ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries
                RotatingSymmetry90::Rot90_ApplyBC: Apply 90 degree rotational symmetry boundary condition
              Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions
        ML_ADMConstraints::ML_ADMConstraints_SelectBoundConds: [level] select boundary conditions
        GROUP ML_ADMConstraints_ApplyBCs: Apply boundary conditions controlled by thorn Boundary
          GROUP BoundaryConditions: Execute all boundary conditions
            Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions
            CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions
            ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries
            RotatingSymmetry90::Rot90_ApplyBC: Apply 90 degree rotational symmetry boundary condition
          Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions
        GROUP ML_BSSN_constraints1_group: ML_BSSN_constraints1
          ML_BSSN::ML_BSSN_constraints1: ML_BSSN_constraints1
          GROUP ML_BSSN_constraints1_bc_group: ML_BSSN_constraints1
            ML_BSSN::ML_BSSN_constraints1_SelectBCs: [level] ML_BSSN_constraints1_SelectBCs
            GROUP ML_BSSN_constraints1_ApplyBCs: Apply BCs for groups set in ML_BSSN_constraints1
              GROUP BoundaryConditions: Execute all boundary conditions
                Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions
                CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions
                ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries
                RotatingSymmetry90::Rot90_ApplyBC: Apply 90 degree rotational symmetry boundary condition
              Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions
        GROUP ML_BSSN_constraints2_group: ML_BSSN_constraints2
          ML_BSSN::ML_BSSN_constraints2: ML_BSSN_constraints2
          GROUP ML_BSSN_constraints2_bc_group: ML_BSSN_constraints2
            ML_BSSN::ML_BSSN_constraints2_SelectBCs: [level] ML_BSSN_constraints2_SelectBCs
            GROUP ML_BSSN_constraints2_ApplyBCs: Apply BCs for groups set in ML_BSSN_constraints2
              GROUP BoundaryConditions: Execute all boundary conditions
                Boundary::Boundary_ApplyPhysicalBCs: Apply all requested local physical boundary conditions
                CartGrid3D::CartGrid3D_ApplyBC: Apply symmetry boundary conditions
                ReflectionSymmetry::ReflectionSymmetry_Apply: Apply reflection symmetries
                RotatingSymmetry90::Rot90_ApplyBC: Apply 90 degree rotational symmetry boundary condition
              Boundary::Boundary_ClearSelection: [level] Unselect all grid variables for boundary conditions
--------------------------------------------------------------------------------
INFO (Carpet): Multi-Model listing:
   model 0: "world"
INFO (Carpet): Multi-Model processor distribution:
   processor 0: model 0 "world"
INFO (Carpet): Multi-Model: This is processor 0, model 0 "world"
INFO (Formaline): Configuration id: config-etk-hedges.belmont.edu-home-shawley-Cactus
INFO (Formaline): Build id: build-etk-hedges.belmont.edu-shawley-2011.08.05-21.33.22-29655
INFO (Formaline): Simulation id: run-ks-mclachlan-hedges.belmont.edu-shawley-2011.08.05-22.07.14-30592
INFO (Formaline): Run id: run-ks-mclachlan-hedges.belmont.edu-shawley-2011.08.05-22.07.14-30592
--------------------------------------------------------------------------------
AMR driver provided by Carpet
--------------------------------------------------------------------------------
AMR info I/O provided by CarpetIOBasic
--------------------------------------------------------------------------------
AMR HDF5 I/O provided by CarpetIOHDF5
--------------------------------------------------------------------------------
AMR 0D HDF5 I/O provided by CarpetIOHDF5
--------------------------------------------------------------------------------
AMR 1D HDF5 I/O provided by CarpetIOHDF5
--------------------------------------------------------------------------------
AMR 2D HDF5 I/O provided by CarpetIOHDF5
--------------------------------------------------------------------------------
AMR scalar I/O provided by CarpetIOScalar
--------------------------------------------------------------------------------
AMR 0D ASCII I/O provided by CarpetIOASCII
--------------------------------------------------------------------------------
AMR 1D ASCII I/O provided by CarpetIOASCII
--------------------------------------------------------------------------------
AMR 2D ASCII I/O provided by CarpetIOASCII
--------------------------------------------------------------------------------
AMR 3D ASCII I/O provided by CarpetIOASCII
--------------------------------------------------------------------------------
ML_ADMConstraints
--------------------------------------------------------------------------------
ML_BSSN
--------------------------------------------------------------------------------
MoL: Generalized time integration.
--------------------------------------------------------------------------------

Current core file size limit: hard=[unlimited], soft=0 MB
Current memory size limit: hard=[unlimited], soft=[unlimited]
INFO (Carpet): Carpet is running on 1 processes
INFO (Carpet): This is process 0
INFO (Carpet): This process contains 1 threads
INFO (Carpet): There are 1 threads in total
INFO (Carpet): Boundary specification for map 0:
   nboundaryzones: [[3,3,3],[3,3,3]]
   is_internal   : [[0,0,0],[0,0,0]]
   is_staggered  : [[0,0,0],[0,0,0]]
   shiftout      : [[1,1,1],[0,0,0]]
INFO (Carpet): CoordBase domain specification for map 0:
   physical extent: [0,0,0] : [120,120,120]   ([120,120,120])
   interior extent: [0,0,0] : [118,118,118]   ([118,118,118])
   exterior extent: [-6,-6,-6] : [124,124,124]   ([130,130,130])
   base_spacing   : [2,2,2]
INFO (Carpet): Adapted domain specification for map 0:
   convergence factor: 2
   convergence level : 0
   physical extent   : [0,0,0] : [120,120,120]   ([120,120,120])
   interior extent   : [0,0,0] : [118,118,118]   ([118,118,118])
   exterior extent   : [-6,-6,-6] : [124,124,124]   ([130,130,130])
   spacing           : [2,2,2]
INFO (Carpet): Base grid specification for map 0:
   number of grid points : [66,66,66]
   number of ghost points: [[3,3,3],[3,3,3]]
INFO (Carpet): Grid structure (grid points):
   [0][0][0][0]   exterior: proc 0   [0,0,0] : [65,65,65]   ([66,66,66]) 287496
INFO (Carpet): Group and variable statistics:
INFO (Carpet):    There are 355 grid functions in 53 groups
INFO (Carpet):    There are 246 grid scalars in 53 groups
INFO (Carpet):    There are 23 1-dimensional grid arrays in 4 groups
INFO (Carpet):    There are 2 2-dimensional grid arrays in 2 groups
INFO (Carpet):    There are 1 3-dimensional grid arrays in 1 groups
INFO (Carpet):    (The number of variables counts all time levels)
INFO (CarpetIOHDF5): I/O Method 'IOHDF5' registered: AMR output of grid variables to HDF5 files
INFO (CarpetIOHDF5): Periodic HDF5 output requested for 'ML_ADMCONSTRAINTS::M3', 'ML_ADMCONSTRAINTS::M2', 'ML_ADMCONSTRAINTS::M1', 'ML_ADMCONSTRAINTS::H', 'CARPETREDUCE::weight', 'ADMBASE::dtbetaz', 'ADMBASE::dtbetay', 'ADMBASE::dtbetax', 'ADMBASE::dtalp', 'ADMBASE::betaz', 'ADMBASE::betay', 'ADMBASE::betax', 'ADMBASE::alp', 'ADMBASE::kzz', 'ADMBASE::kyz', 'ADMBASE::kyy', 'ADMBASE::kxz', 'ADMBASE::kxy', 'ADMBASE::kxx', 'ADMBASE::gzz', 'ADMBASE::gyz', 'ADMBASE::gyy', 'ADMBASE::gxz', 'ADMBASE::gxy', 'ADMBASE::gxx'
INFO (CarpetIOHDF5): I/O Method 'IOHDF5_0D' registered: 0D AMR output of grid variables to HDF5 files
INFO (CarpetIOHDF5): I/O Method 'IOHDF5_1D' registered: 1D AMR output of grid variables to HDF5 files
INFO (CarpetIOHDF5): I/O Method 'IOHDF5_2D' registered: 2D AMR output of grid variables to HDF5 files
INFO (CarpetIOScalar): Periodic scalar output requested for 'SPHERICALSURFACE::sf_radius[0]', 'ML_ADMCONSTRAINTS::M3', 'ML_ADMCONSTRAINTS::M2', 'ML_ADMCONSTRAINTS::M1', 'ML_ADMCONSTRAINTS::H', 'CARPETREDUCE::weight', 'ADMBASE::dtbetaz', 'ADMBASE::dtbetay', 'ADMBASE::dtbetax', 'ADMBASE::dtalp', 'ADMBASE::betaz', 'ADMBASE::betay', 'ADMBASE::betax', 'ADMBASE::alp', 'ADMBASE::kzz', 'ADMBASE::kyz', 'ADMBASE::kyy', 'ADMBASE::kxz', 'ADMBASE::kxy', 'ADMBASE::kxx', 'ADMBASE::gzz', 'ADMBASE::gyz', 'ADMBASE::gyy', 'ADMBASE::gxz', 'ADMBASE::gxy', 'ADMBASE::gxx'
INFO (CarpetIOASCII): I/O Method 'IOASCII_0D' registered: 0D AMR output of grid variables to ASCII files
INFO (CarpetIOASCII): Periodic 0D AMR output requested for:
   ADMBASE::gxx
   ADMBASE::gxy
   ADMBASE::gxz
   ADMBASE::gyy
   ADMBASE::gyz
   ADMBASE::gzz
   ADMBASE::kxx
   ADMBASE::kxy
   ADMBASE::kxz
   ADMBASE::kyy
   ADMBASE::kyz
   ADMBASE::kzz
   ADMBASE::alp
   ADMBASE::betax
   ADMBASE::betay
   ADMBASE::betaz
   ADMBASE::dtalp
   ADMBASE::dtbetax
   ADMBASE::dtbetay
   ADMBASE::dtbetaz
   CARPET::physical_time_per_hour
   CARPET::time_total
   CARPET::time_computing
   CARPET::time_communicating
   CARPET::time_io
   CARPET::local_grid_points_per_second
   CARPET::total_grid_points_per_second
   CARPET::local_grid_point_updates_count
   CARPET::total_grid_point_updates_count
   CARPET::io_per_second
   CARPET::io_bytes_per_second
   CARPET::io_bytes_ascii_per_second
   CARPET::io_bytes_binary_per_second
   CARPET::io_count
   CARPET::io_bytes_count
   CARPET::io_bytes_ascii_count
   CARPET::io_bytes_binary_count
   CARPET::comm_per_second
   CARPET::comm_bytes_per_second
   CARPET::comm_count
   CARPET::comm_bytes_count
   CARPETREDUCE::weight
   ML_ADMCONSTRAINTS::H
   ML_ADMCONSTRAINTS::M1
   ML_ADMCONSTRAINTS::M2
   ML_ADMCONSTRAINTS::M3
   SPHERICALSURFACE::sf_active[0]
   SPHERICALSURFACE::sf_valid[0]
   SPHERICALSURFACE::sf_area[0]
   SPHERICALSURFACE::sf_mean_radius[0]
   SPHERICALSURFACE::sf_centroid_x[0]
   SPHERICALSURFACE::sf_centroid_y[0]
   SPHERICALSURFACE::sf_centroid_z[0]
   SPHERICALSURFACE::sf_quadrupole_xx[0]
   SPHERICALSURFACE::sf_quadrupole_xy[0]
   SPHERICALSURFACE::sf_quadrupole_xz[0]
   SPHERICALSURFACE::sf_quadrupole_yy[0]
   SPHERICALSURFACE::sf_quadrupole_yz[0]
   SPHERICALSURFACE::sf_quadrupole_zz[0]
   SPHERICALSURFACE::sf_min_radius[0]
   SPHERICALSURFACE::sf_max_radius[0]
   SPHERICALSURFACE::sf_min_x[0]
   SPHERICALSURFACE::sf_min_y[0]
   SPHERICALSURFACE::sf_min_z[0]
   SPHERICALSURFACE::sf_max_x[0]
   SPHERICALSURFACE::sf_max_y[0]
   SPHERICALSURFACE::sf_max_z[0]
   SPHERICALSURFACE::sf_radius[0]
   SPHERICALSURFACE::sf_origin_x[0]
   SPHERICALSURFACE::sf_origin_y[0]
   SPHERICALSURFACE::sf_origin_z[0]
   SPHERICALSURFACE::sf_origin_theta[0]
   SPHERICALSURFACE::sf_origin_phi[0]
   SPHERICALSURFACE::sf_delta_theta[0]
   SPHERICALSURFACE::sf_delta_phi[0]
INFO (CarpetIOASCII): I/O Method 'IOASCII_1D' registered: 1D AMR output of grid variables to ASCII files
INFO (CarpetIOASCII): Periodic 1D AMR output requested for:
   ADMBASE::gxx
   ADMBASE::gxy
   ADMBASE::gxz
   ADMBASE::gyy
   ADMBASE::gyz
   ADMBASE::gzz
   ADMBASE::kxx
   ADMBASE::kxy
   ADMBASE::kxz
   ADMBASE::kyy
   ADMBASE::kyz
   ADMBASE::kzz
   ADMBASE::alp
   ADMBASE::betax
   ADMBASE::betay
   ADMBASE::betaz
   ADMBASE::dtalp
   ADMBASE::dtbetax
   ADMBASE::dtbetay
   ADMBASE::dtbetaz
   CARPETREDUCE::weight
   ML_ADMCONSTRAINTS::H
   ML_ADMCONSTRAINTS::M1
   ML_ADMCONSTRAINTS::M2
   ML_ADMCONSTRAINTS::M3
   SPHERICALSURFACE::sf_radius[0]
INFO (CarpetIOASCII): I/O Method 'IOASCII_2D' registered: 2D AMR output of grid variables to ASCII files
INFO (CarpetIOASCII): Periodic 2D AMR output requested for:
   SPHERICALSURFACE::sf_radius[0]
INFO (CarpetIOASCII): I/O Method 'IOASCII_3D' registered: 3D AMR output of grid variables to ASCII files
INFO (CarpetLib): Old address space size limit: hard=-1, soft=-1
INFO (CarpetLib): New address space size limit: hard=-1, soft=-1
INFO (CarpetLib): (Unlimited address space size indicated by -1)
INFO (CarpetRegrid2): Initialising position of centre 0 to [0,0,0]
--------------------------------------------------------------------------------
INFO (Formaline): Writing tarballs with the Cactus sources into the directory "ks-mclachlan/cactus-source"
INFO (MoL): Using Runge-Kutta 4 as the time integrator.
INFO (MoL): The maximum number of evolved variables is 25. 25 are registered.
INFO (MoL): The maximum number of constrained variables is 20. 20 are registered.
INFO (MoL): The maximum number of SandR variables is 0. 0 are registered.
INFO (MoL): The maximum number of evolved complex variables is 0. 0 are registered.
INFO (MoL): The maximum number of constrained complex variables is 0. 0 are registered.
INFO (MoL): The maximum number of SandR complex variables is 0. 0 are registered.
INFO (MoL): The maximum number of evolved array variables is 0. 0 are registered.
INFO (MoL): The maximum number of constrained array variables is 0. 0 are registered.
INFO (MoL): The maximum number of SandR array variables is 0. 0 are registered.
INFO (MoL): The maximum number of evolved complex array variables is 0. 0 are registered.
INFO (MoL): The maximum number of constrained complex array variables is 0. 0 are registered.
INFO (MoL): The maximum number of SandR complex array variables is 0. 0 are registered.
INFO (MoL): The maximum size of any array variables is 0.
INFO (SymBase): Symmetry on lower x-face: rotating_symmetry_90
INFO (SymBase): Symmetry on lower y-face: rotating_symmetry_90
INFO (SymBase): Symmetry on lower z-face: reflection_symmetry
INFO (CarpetRegrid2): Centre 1 is at position [0,0,0]
INFO (CarpetRegrid2): Regridding
INFO (CarpetRegrid2): Regridding levels 1 and up
INFO (Carpet): Grid structure statistics:
INFO (Carpet): GF: rhs: 227k active, 341k owned (+50%), 437k total (+28%), 63 steps/time
INFO (Carpet): GF: vars: 71, pts: 96M active, 137M owned (+43%), 176M total (+29%), 1.0 comp/proc
INFO (Carpet): GA: vars: 267, pts: 0M active, 0M total (+0%)
INFO (ADMMacros): Spatial finite differencing order: 2
INFO (CartGrid3D): Grid Spacings:
INFO (CartGrid3D): dx=>2.0000000e+00  dy=>2.0000000e+00  dz=>2.0000000e+00
INFO (CartGrid3D): Computational Coordinates:
INFO (CartGrid3D): x=>[-6.000,124.000]  y=>[-6.000,124.000]  z=>[-6.000,124.000]
INFO (CartGrid3D): Indices of Physical Coordinates:
INFO (CartGrid3D): x=>[0,65]  y=>[0,65]  z=>[0,65]
INFO (AHFinderDirect): setting up AHFinderDirect data structures
INFO (AHFinderDirect):            to search for 1 horizon on 1 processor
INFO (AHFinderDirect):       constructing full sphere patch system
INFO (AHFinderDirect):                    at origin (0,0,0)
INFO (AHFinderDirect):                    with 18 angular zones per right angle
INFO (AHFinderDirect):       ==> 2166 nominal, 3174 ghosted angular grid points
INFO (Time): Timestep set to 0.8 (courant_static)
INFO (Exact): setting exact data on slice
[hedges:30592] *** Process received signal ***
[hedges:30592] Signal: Segmentation fault (11)
[hedges:30592] Signal code: Address not mapped (1)
[hedges:30592] Failing at address: 0xffffffffe7de3a00
[hedges:30592] [ 0] /lib64/libpthread.so.0(+0xef60) [0x7f68b462cf60]
[hedges:30592] [ 1] /lib64/libc.so.6(+0x1064e6) [0x7f68b43c34e6]
[hedges:30592] [ 2] ./exe/cactus_etk() [0x4155a4]
[hedges:30592] [ 3] ./exe/cactus_etk() [0x41b62b]
[hedges:30592] [ 4] ./exe/cactus_etk() [0x41b4cd]
[hedges:30592] *** End of error message ***
Segmentation fault
hedges.belmont.edu


More information about the Users mailing list