[Users] Carpet hg vs git

[Luca Baiotti]

Hello,

I am trying to switch to carpet mercurial. I compiled two executables, 
identical except for the Carpet thorns, one with thorns from the git 
repository and the other with thorns from the hg repository.

The BBH simulations runs with the git version, but (with the same 
parfile) it fails with the hg version:

...
INFO (CarpetReduce): Simulation domain volume: 512000
INFO (CarpetReduce): Reduction weight sum:     511999.999154925
WARNING[L1,P0] (CarpetReduce): Simulation domain volume and reduction 
weight sum differ
Memory statistics from CarpetLib:
    Current number of objects: 2611
    Current allocated memory:  649 MB
    Maximum number of objects: 2.63e+03
    Maximum allocated memory:  652 MB
    Current administrative memory: 2.79 MB
    Total allocated used system memory: 665 MB
    Total allocated free system memory: 17.6 MB
    gh::allmemory:    708688
    dh::allmemory:    1301208
    th::allmemory:    26608
    ggf::allmemory:   576000
    gdata::allmemory: 182224
INFO (IsolatedHorizon): Low Spin < 1e-3 ! Skipping 
coordinate-transformation!
[node154:6197] *** An error occurred in MPI_Allreduce: the reduction 
operation MPI_SUM is not defined for non-intrinsic datat
ypes
[node154:6197] *** on communicator MPI COMMUNICATOR 3 SPLIT FROM 0
[node154:6197] *** MPI_ERR_OP: invalid reduce operation
[node154:6197] *** MPI_ERRORS_ARE_FATAL (your MPI job will now abort)


I am running on damiana, with the same configuration options and qsub 
scripts for the two executables. When switching from git to hg, should I 
change, add, or remove some parameter?


Here is a list of the carpet parameters (and a few others) that I set in 
my parfiles:

ActiveThorns"
   Carpet
   CarpetIOScalar
   CarpetIOASCII
   CarpetIOBasic
   CarpetIOHDF5
   CarpetInterp
   CarpetLib
   CarpetMask
   CarpetReduce
   CarpetRegrid2

Parameters:

Carpet::domain_from_coordbase           = yes
Carpet::max_refinement_levels           = 9
CarpetRegrid2::num_centres              = 2
CarpetRegrid2::num_levels_1             = 9
CarpetRegrid2::position_x_1             = 4
CarpetRegrid2::radius_1             [1] = 100.0

etc.

Carpet::time_refinement_factors         = "[1,1,1,2,4,8,16,32,64]"

Carpet::use_buffer_zones                = yes
Carpet::prolongation_order_space        = 5
Carpet::prolongation_order_time         = 2
Carpet::regrid_in_level_mode            = yes
Carpet::regrid_during_initialisation    = no
CarpetRegrid2::regrid_every             = 256
Carpet::init_fill_timelevels            = yes
InitBase::initial_data_setup_method     = "init_all_levels"
CarpetTracker::surface                      [0] = 0
CarpetTracker::surface                      [1] = 1
MoL::ODE_Method                         = "rk4"
MoL::MoL_Intermediate_Steps             = 4
Carpet::num_integrator_substeps         = 4
MoL::MoL_Num_Scratch_Levels             = 1
CarpetMask::excluded_surface        [0] = 2
CarpetMask::excluded_surface_factor [0] = 1.
CarpetMask::excluded_surface        [1] = 3
CarpetMask::excluded_surface        [2] = 4
Carpet::poison_new_timelevels           = yes
Carpet::check_for_poison                = no
CarpetLib::poison_new_memory            = yes
CarpetLib::poison_value                 = 200
Carpet::output_timers_every             = 768
CarpetLib::print_timestats_every        = 768
CarpetLib::print_memstats_every         = 768


Cheers,
Luca