[Users] McLachlan help
Erik Schnetter
schnetter at cct.lsu.edu
Mon Jul 27 21:21:20 CDT 2015
Jim
By default, McLachlan is configured to be generic, i.e. many choices can be
made at run time. For example, there are 4 additional gauge variables
evolved (A and B^i). If you are willing to restrict your run-time choices,
for example by fixing derivative order, disabling multi-block systems,
choosing a particular gauge, choice of conformal factor, etc., then this
will lead to a more efficient code. You can easily do so for McLachlan by
generating a new version in Kranc. If you look in the beginning of
McLachlan_BSSN.m in the section "Choose code to generate", then you'll see
the parameters you can choose. We have made a particular set of choices for
a configuration "BSSN_bench", but these are apparently different from
LazEv. For example, we use 4th order finite differencing, and evolve B^i in
time as well.
Can you specify all the details of your formulation? I notice that you
didn't respond to my request to point me to the source code, and there are
quite a few details that you also didn't describe yet (and are also not
described in your paper), such as your choice of conformal factor,
whether/how your code is integrated with ADMBase, whether the constraints
are calculated from the BSSN or the ADM variables, how advection
derivatives are handled, etc. For a true comparison, this all needs to be
known. Unless we both know exactly what both the codes do, we'll never know
where the physics differences are.
-erik
On Mon, Jul 27, 2015 at 5:39 PM, James Healy <jchsma at rit.edu> wrote:
> Hi Erik,
>
> We evolve alpha and beta for the gauge variables as given in equations 7a
> and 7b of http://arxiv.org/pdf/1506.06153.pdf. The number I reported for
> LazEv was using 5th order dissipation applied to all 21 evolved variables
> (alp, beta^i, At_{ij}, gt_{ij}, Gammat^{i}, trK, and xi).
>
> LazEv is calculating the Hamiltonian and Momentum constraints as well as
> the constraints on Gammat^i using 4th order stencils.
>
> I also forgot to mention that I did these tests on 16 nodes, and the
> memory usage was relatively low ( ~2 GB/MPI process ).
>
> I will give the internal dissipation for McLachlan a try.
>
> Thanks,
> Jim
>
>
> On 07/27/2015 05:56 PM, Erik Schnetter wrote:
>
> Jim
>
> Thanks for posting the details.
>
> Can you give us more details about the LazEv scheme? In particular there
> may be differences in the gauge. What (gauge) variables do you evolve? What
> gauge conditions do you use? And what kind of dissipation do you apply? Can
> you point us to the source code?
>
> For the new McLachlan, you would probably use the built-in dissipation
> instead of thorn Dissipation, which should lead to a small speed-up.
>
> -erik
>
> On Mon, Jul 27, 2015 at 4:43 PM, James Healy <jchsma at rit.edu> wrote:
>
>> Hello,
>>
>> I have been running some tests on Stampede comparing the run speed of
>> McLachlan to RIT's evolution thorn LazEv. I started with the
>> qc0-mclachlan.par parameter file included with the Einstein Toolkit, added
>> a few refinement levels, increased the resolution and changed McLachlan to
>> be 8th order (and increased the number of ghost zones to 5). I also
>> increased the initial separation so the finest grids aren't already
>> overlapping. To compare with LazEv, I removed the McLachlan and
>> Dissipation thorns and replaced them with LazEv. Everything else in the
>> parameter file is exactly the same. I tried using both the McLachlan
>> master and rewrite branches.
>>
>> The grid setup is 10 levels of refinement, dx=4M on the coarsest with
>> outer boundary at 400M, M/128 on the finest with r=0.6M, CFL is 0.25. Both
>> use 8th order spatial differencing with ghost_size=5 and 5th order
>> dissipation.
>>
>> Below is a summary of the results as reported at iteration 256 from
>> Carpet::physical_time_per_hour:
>>
>> McLachlan - rewrite branch: 3.0596110 M/hr
>> McLachlan - master branch: 3.8033607 M/hr
>> LazEv - 4.1941544 M/hr
>>
>> I am using the stampede-impi.cfg configuration file in simfactory.
>> "module list" returns:
>>
>> 1) TACC-paths 3) cluster-paths 5) xalt/0.4.6 7) TACC
>> 2) Linux 4) intel/13.0.2.146 6) cluster 8) impi/4.1.0.030
>>
>> Attached is my parameter file. I pasted the McLachlan parameters below.
>> Are there any optimizations that I can use for McLachlan? Are the
>> parameters I am using for it what would be used for production runs?
>>
>> ML_BSSN::harmonicN = 1 # 1+log
>> ML_BSSN::harmonicF = 2.0 # 1+log
>> ML_BSSN::ShiftGammaCoeff = 0.75
>> ML_BSSN::BetaDriver = 1.0
>> ML_BSSN::LapseAdvectionCoeff = 1.0
>> ML_BSSN::ShiftAdvectionCoeff = 1.0
>>
>> ML_BSSN::MinimumLapse = 1.0e-8
>>
>> ML_BSSN::my_initial_boundary_condition = "extrapolate-gammas"
>> ML_BSSN::my_rhs_boundary_condition = "NewRad"
>> Boundary::radpower = 2
>>
>> ML_BSSN::ML_log_confac_bound = "none"
>> ML_BSSN::ML_metric_bound = "none"
>> ML_BSSN::ML_Gamma_bound = "none"
>> ML_BSSN::ML_trace_curv_bound = "none"
>> ML_BSSN::ML_curv_bound = "none"
>> ML_BSSN::ML_lapse_bound = "none"
>> ML_BSSN::ML_dtlapse_bound = "none"
>> ML_BSSN::ML_shift_bound = "none"
>> ML_BSSN::ML_dtshift_bound = "none"
>>
>> ML_BSSN::fdOrder = 8
>>
>> ActiveThorns = "Dissipation"
>>
>> Dissipation::order = 5
>> Dissipation::vars = "
>> ML_BSSN::ML_metric
>> ML_BSSN::ML_trace_curv
>> ML_BSSN::ML_curv
>> ML_BSSN::ML_Gamma
>> ML_BSSN::ML_lapse
>> ML_BSSN::ML_shift
>> ML_BSSN::ML_dtlapse
>> ML_BSSN::ML_dtshift
>> "
>>
>> ActiveThorns = "ML_ADMConstraints"
>>
>>
>> Thanks,
>> Jim Healy
>>
>> _______________________________________________
>> Users mailing list
>> Users at einsteintoolkit.org
>> http://lists.einsteintoolkit.org/mailman/listinfo/users
>>
>>
>
>
> --
> Erik Schnetter <schnetter at cct.lsu.edu>
> http://www.perimeterinstitute.ca/personal/eschnetter/
>
>
>
--
Erik Schnetter <schnetter at cct.lsu.edu>
http://www.perimeterinstitute.ca/personal/eschnetter/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.einsteintoolkit.org/pipermail/users/attachments/20150727/d8a68163/attachment-0001.html
More information about the Users
mailing list