While trying to understand why the results from a BBH run differed when using the 2011_05 and 2011_10 version of ET I noticed that the two versions gave different results on every other point of the buffer region (in a 2 level run) on the 2nd time level.
I reran the test using McLachlan and I found a similar behavior, but for fewer points (here, only 3, rather than 8) and only the innermost points.
diff -w -u ~/ET/Old/Cactus/test/mc/gxx.x.asc ~/ET/Develop/Cactus/test/mc | grep "^[+-]2" -2 0 0 0 0 26 8 8 0.1 1.8 0 0 5.86163790879802 +2 0 0 0 0 26 8 8 0.1 1.8 0 0 5.86163790879801 -2 0 1 0 0 26 8 8 0.1 1.8 0 0 5.86163790879802 +2 0 1 0 0 26 8 8 0.1 1.8 0 0 5.86163790879801 -2 0 1 0 0 59 8 8 0.1 5.1 0 0 2.06954055131151 +2 0 1 0 0 59 8 8 0.1 5.1 0 0 2.06940807802512 -2 0 1 0 0 61 8 8 0.1 5.3 0 0 2.01672019300839 +2 0 1 0 0 61 8 8 0.1 5.3 0 0 2.01674238678588 -2 0 1 0 0 63 8 8 0.1 5.5 0 0 1.96875965038995 +2 0 1 0 0 63 8 8 0.1 5.5 0 0 1.96875664800624
One of the major differences between these two versions of ET is the switch from the git to hg version of carpet.
Here is the full parfile that I used.
#==============================================================================
ActiveThorns = "admanalysis admbase admcoupling admmacros coordgauge spacemask StaticConformal boundary time cartgrid3d ioutil iobasic nanchecker CoordBase aeilocalinterp Slab SphericalSurface LocalReduce MoL Carpet CarpetInterp CarpetIOASCII CarpetLib CarpetReduce CarpetSlab CarpetRegrid2 SphericalSurface CarpetIOHDF5 CarpetIOScalar InitBase SymBase LoopControl GSL ReflectionSymmetry"
ActiveThorns = "TimerReport"
TimerReport::out_every = 512 TimerReport::out_filename = "TimerReport" Carpet::output_timers_every = 512 #------------------------------------------------------------------------------
# Run parameters #------------------------------------------------------------------------------
# Grid
mol::ode_method = "RK4" mol::mol_intermediate_steps = 4 mol::mol_num_scratch_levels = 1
CoordBase::domainsize = "minmax" CoordBase::xmin = 0 CoordBase::ymin = 0 CoordBase::zmin = 0 CoordBase::xmax = 10 CoordBase::ymax = 10 CoordBase::zmax = 10 CoordBase::spacing = "numcells" CoordBase::ncells_x = 50 CoordBase::ncells_y = 50 CoordBase::ncells_z = 50 CartGrid3D::type = "coordbase" CartGrid3D::domain = "full" CartGrid3D::bitant_plane = "xy" CartGrid3D::avoid_originx = no CartGrid3D::avoid_originy = no CartGrid3D::avoid_originz = no
CoordBase::boundary_size_x_lower = 4 CoordBase::boundary_size_y_lower = 4 CoordBase::boundary_size_z_lower = 4 CoordBase::boundary_shiftout_x_lower = 1 CoordBase::boundary_shiftout_y_lower = 1 CoordBase::boundary_shiftout_z_lower = 1
CoordBase::boundary_size_x_upper = 4 CoordBase::boundary_size_y_upper = 4 CoordBase::boundary_size_z_upper = 4
ReflectionSymmetry::reflection_x = yes ReflectionSymmetry::reflection_y = yes ReflectionSymmetry::reflection_z = yes ReflectionSymmetry::avoid_origin_x = no ReflectionSymmetry::avoid_origin_y = no ReflectionSymmetry::avoid_origin_z = no
driver::ghost_size = 4
#---------------------------------------------------------------------------
cactus::cctk_itlast = 16
time::dtfac = .5 Carpet::domain_from_coordbase = yes Carpet::prolongation_order_space = 5 Carpet::prolongation_order_time = 2 Carpet::max_refinement_levels = 2 Carpet::use_buffer_zones = yes Carpet::verbose = no Carpet::schedule_barriers = no Carpet::veryverbose = no
CarpetLib::output_bboxes = no CarpetLib::check_bboxes = no CarpetLib::interleave_communications = yes CarpetLib::combine_sends = yes
#Carpet::time_refinement_factors = "[1,1,1,1,2,2,4,8,16,32,64]" Carpet::init_3_timelevels = "no" Carpet::init_each_timelevel = "no" Carpet::init_fill_timelevels = "yes" Carpet::enable_all_storage = no Carpet::regrid_during_recovery = no
ActiveThorns="CarpetRegrid2" CarpetRegrid2::regrid_every = 16 CarpetRegrid2::num_centres = 1 CarpetRegrid2::num_levels_1 = 2 CarpetRegrid2::radius_1[ 1] = 5
# MODEL ActiveThorns="IDAnalyticBH" ADMBase::initial_data="bl_bh" IDAnalyticBH::bl_nbh = 2 IDAnalyticBH::bl_x0_1 = 1.01001 IDAnalyticBH::bl_y0_1 = 1.01001 IDAnalyticBH::bl_x0_2 = -1.01001 IDAnalyticBH::bl_y0_2 = -1.01001 initbase::initial_data_setup_method = init_some_levels
admbase::metric_type = "Physical" #staticconformal::conformal_storage = "factor+derivs+2nd derivs"
#--- numerics
ActiveThorns = "ML_BSSN ML_BSSN_Helper TmunuBase GenericFD"
ADMBase::evolution_method = "ML_BSSN" ADMBase::lapse_evolution_method = "ML_BSSN" ADMBase::shift_evolution_method = "ML_BSSN"
ML_BSSN::timelevels = 3
ML_BSSN::harmonicN = 1 # 1+log ML_BSSN::harmonicF = 2.0 # 1+log ML_BSSN::ShiftGammaCoeff = 0.75 ML_BSSN::BetaDriver = 1.0 ML_BSSN::LapseAdvectionCoeff = 1 ML_BSSN::ShiftAdvectionCoeff = 1
ML_BSSN::my_boundary_condition = "Minkowski"
ML_BSSN::ML_log_confac_bound = "none" ML_BSSN::ML_metric_bound = "none" ML_BSSN::ML_Gamma_bound = "none" ML_BSSN::ML_trace_curv_bound = "none" ML_BSSN::ML_curv_bound = "none" ML_BSSN::ML_lapse_bound = "none" ML_BSSN::ML_dtlapse_bound = "none" ML_BSSN::ML_shift_bound = "none" ML_BSSN::ML_dtshift_bound = "none"
admbase::initial_lapse = "one"
ADMBase::initial_shift = "zero" ADMBase::initial_dtlapse = "zero" ADMBase::initial_dtshift = "zero"
#------------------------------------------------------------------------------ # Output #------------------------------------------------------------------------------ IO::out_dir = $parfile IO::out_fileinfo = "none"
CarpetIOASCII::out1D_every = 1 CarpetIOASCII::out1D_vars = "admbase::alp admbase::metric admbase::curv admbase::shift " CarpetIOASCII::out1D_d=no
Hi Yosef,
I believe the code was reorganised in carpet and this needed the algorithm to change in a way that meant numerical results would change. I can't check right now, but this should have been mentioned in the announcement of the new version of carpet as well as the ET release notes.
On 07/27/2012 04:21 AM, Ian Hinder wrote:
Hi Yosef,
I believe the code was reorganised in carpet and this needed the algorithm to change in a way that meant numerical results would change. I can't check right now, but this should have been mentioned in the announcement of the new version of carpet as well as the ET release notes.
Thanks. I was surprised that effect of this change was measurable in the phase at merger (it was of order the differences in phase when I increased the resolution by a factor of 1.2).
-- Ian Hinder ian.hinder@aei.mpg.de mailto:ian.hinder@aei.mpg.de
----- Original message -----
While trying to understand why the results from a BBH run differed when using the 2011_05 and 2011_10 version of ET I noticed that the two versions gave different results on every other point of the buffer region (in a 2 level run) on the 2nd time level.
I reran the test using McLachlan and I found a similar behavior, but for fewer points (here, only 3, rather than 8) and only the innermost points.
diff -w -u ~/ET/Old/Cactus/test/mc/gxx.x.asc ~/ET/Develop/Cactus/test/mc | grep "^[+-]2" -2 0 0 0 0 26 8 8 0.1 1.8 0 0 5.86163790879802 +2 0 0 0 0 26 8 8 0.1 1.8 0 0 5.86163790879801 -2 0 1 0 0 26 8 8 0.1 1.8 0 0 5.86163790879802 +2 0 1 0 0 26 8 8 0.1 1.8 0 0 5.86163790879801 -2 0 1 0 0 59 8 8 0.1 5.1 0 0 2.06954055131151 +2 0 1 0 0 59 8 8 0.1 5.1 0 0 2.06940807802512 -2 0 1 0 0 61 8 8 0.1 5.3 0 0 2.01672019300839 +2 0 1 0 0 61 8 8 0.1 5.3 0 0 2.01674238678588 -2 0 1 0 0 63 8 8 0.1 5.5 0 0 1.96875965038995 +2 0 1 0 0 63 8 8 0.1 5.5 0 0 1.96875664800624
Yosef
As Ian mentioned, there are two changes that may affect this:
(1) The way in which the shape of the buffer region and the restricted region are determined may be different now, even if a grid has the same number of grid points. The old way was inconsistent under certain circumstances. If this is the case, then certain grid points that used to be defined via interpolation are now defined via time interpolation or vice versa. You can see such differences if you output the grid structure, e.g. via setting Carpet::verbose or CarpetLib::output_bboxes. (The resulting output may be difficult to read.)
(2) Carpet's prolongation operators are implemented differently and may be more efficient. If so, you should be seeing differences at the round-off level. If you see differences only at every other point, then this is likely the case – every second point is copied from the coarse grid, the points in between are interpolated. Carpet also contains self-tests to ensure that the operators have the desired order of accuracy.
-erik
On Friday, July 27, 2012, Ian Hinder wrote:
**
Hi Yosef,
I believe the code was reorganised in carpet and this needed the algorithm to change in a way that meant numerical results would change. I can't check right now, but this should have been mentioned in the announcement of the new version of carpet as well as the ET release notes.
-- Ian Hinder ian.hinder@aei.mpg.de <javascript:_e({}, 'cvml', 'ian.hinder@aei.mpg.de');>
----- Original message -----
While trying to understand why the results from a BBH run differed when using the 2011_05 and 2011_10 version of ET I noticed that the two versions gave different results on every other point of the buffer region (in a 2 level run) on the 2nd time level.
I reran the test using McLachlan and I found a similar behavior, but for fewer points (here, only 3, rather than 8) and only the innermost points.
diff -w -u ~/ET/Old/Cactus/test/mc/gxx.x.asc ~/ET/Develop/Cactus/test/mc | grep "^[+-]2" -2 0 0 0 0 26 8 8 0.1 1.8 0 0 5.86163790879802 +2 0 0 0 0 26 8 8 0.1 1.8 0 0
5.86163790879801
-2 0 1 0 0 26 8 8 0.1 1.8 0 0 5.86163790879802 +2 0 1 0 0 26 8 8 0.1 1.8 0 0
5.86163790879801
-2 0 1 0 0 59 8 8 0.1 5.1 0 0 2.06954055131151 +2 0 1 0 0 59 8 8 0.1 5.1 0 0
2.06940807802512
-2 0 1 0 0 61 8 8 0.1 5.3 0 0 2.01672019300839 +2 0 1 0 0 61 8 8 0.1 5.3 0 0
2.01674238678588
-2 0 1 0 0 63 8 8 0.1 5.5 0 0 1.96875965038995 +2 0 1 0 0 63 8 8 0.1 5.5 0 0
1.96875664800624
One of the major differences between these two versions of ET is the switch from the git to hg version of carpet.
Here is the full parfile that I used.
#==============================================================================
ActiveThorns = "admanalysis admbase admcoupling admmacros coordgauge spacemask StaticConformal boundary time cartgrid3d ioutil iobasic nanchecker CoordBase aeilocalinterp Slab SphericalSurface LocalReduce MoL Carpet CarpetInterp CarpetIOASCII CarpetLib CarpetReduce CarpetSlab CarpetRegrid2 SphericalSurface CarpetIOHDF5 CarpetIOScalar InitBase SymBase LoopControl GSL ReflectionSymmetry"
ActiveThorns = "TimerReport"
TimerReport::out_every = 512 TimerReport::out_filename = "TimerReport" Carpet::output_timers_every = 512
#------------------------------------------------------------------------------
# Run parameters
#------------------------------------------------------------------------------
# Grid
mol::ode_method = "RK4" mol::mol_intermediate_steps = 4 mol::mol_num_scratch_levels = 1
CoordBase::domainsize = "minmax" CoordBase::xmin = 0 CoordBase::ymin = 0 CoordBase::zmin = 0 CoordBase::xmax = 10 CoordBase::ymax = 10 CoordBase::zmax = 10 CoordBase::spacing = "numcells" CoordBase::ncells_x = 50 CoordBase::ncells_y = 50 CoordBase::ncells_z = 50 CartGrid3D::type = "coordbase" CartGrid3D::domain = "full" CartGrid3D::bitant_plane = "xy" CartGrid3D::avoid_originx = no CartGrid3D::avoid_originy = no CartGrid3D::avoid_originz = no
CoordBase::boundary_size_x_lower = 4 CoordBase::boundary_size_y_lower = 4 CoordBase::boundary_size_z_lower = 4 CoordBase::boundary_shiftout_x_lower = 1 CoordBase::boundary_shiftout_y_lower = 1 CoordBase::boundary_shiftout_z_lower = 1
CoordBase::boundary_size_x_upper = 4 CoordBase::boundary_size_y_upper = 4 CoordBase::boundary_size_z_upper = 4
ReflectionSymmetry::reflection_x = yes ReflectionSymmetry::reflection_y = yes ReflectionSymmetry::reflection_z = yes ReflectionSymmetry::avoid_origin_x = no ReflectionSymmetry::avoid_origin_y = no ReflectionSymmetry::avoid_origin_z = no
driver::ghost_size = 4
#-----------------------------------------------------------
users@lists.einsteintoolkit.org
-
Erik Schnetter -
Ian Hinder -
Yosef Zlochower