Hello Karima,

I am not a black hole evolution expert, but I can give this a try.

Looking at the AHFinderDirect output in your *.out file (and the grid structure output):

--8<-- INFO (AHFinderDirect): AH 1/3: r=0.25 at (3.000000,-0.000000,0.000000) INFO (AHFinderDirect): AH 1/3: area=9.555893546e+36 m_irreducible=4.360142907e+17 INFO (AHFinderDirect): AH 2/3: r=0.25 at (-3.000000,-0.000000,-0.000000) INFO (AHFinderDirect): AH 2/3: area=9.555893546e+36 m_irreducible=4.360142907e+17 --8<--

your black holes are located exactly on grid points. Since there is a singularity that goes as 1/r at the center of the black holes this means that your data contains very large numbers which can develop into NaNs and PunctureTracker picks this up. In particular this can be compiler and optimization level specific (in particular the Intel and GNU compilers may produce different results).

To fix I would suggest moving your black holes way from the exact grid point. By slightly increasing or decreasing their separation or pick a grid spacing that avoids putting a grind point there.

Your parfile seems to use a smaller domain (only to 60) and coarser resolution (3 instead of 2) on the coarsest grid which can increase the likelyhood of this to happen. Eg using a coarsest grid spacing of 3.03 (and a grid coordinate size of 20*3.03 = 60.6) may help since it (should) avoid bringing any grid point very close to the center of the black hole.

Yours, Roland

Thank you, it removed the error, but I again encounter another error in the PunctureTracker. I am unable to adjust it, maybe a lack of some knowledge. Your support would be greatly appreciated in getting around it.

Thank you

Best regards, Karima S

On Wed, Dec 23, 2020 at 3:50 AM Roland Haas rhaas@illinois.edu wrote:

...

Hello Karima,

looking at the error message and the parfile (BBHLowRes.par) that you included in your email then there are these lines in the parfile:

--8<-- Contents successfully written to /home/karima/Cactus/repos/simfactory2/etc/defs.local.ini %%bash ./simfactory/bin/sim build -j2 --thornlist ../einsteintoolkit.th %%bash ./simfactory/bin/sim build -j2 --thornlist ../einsteintoolkit.th Using configuration: sim Updated thorn list for configuration sim Building sim Cactus - version: 4.9.0 Building configuration sim Reconfiguring thorns Reading ThornList... Parsing configuration files... ADMAnalysis

--8<--

which clearly should not be there. They looks to me like some part of screen output produced when copying and pasting the cell content from the tutorial (which in itself is not wrong).

You will have to remove those lines and check that your parfile actually agrees with the one from Cactus/par/arXiv-1111.3344 that you would like to use.

Yours, Roland

...

Hello there, I am running a simple binary black hole simulation from the parameter

file

...

kept in the Cactus/par/arXiv-1111.3344 where I simulated high resolution. Furthermore, I assume simulation for lower resolution is less costly than higher, but I wonder it keeps giving me errors in the low-resolution run. (kernel dies for the MedResolution). I tried almost all the ways of adjusting the par file according to my local machine strength, but I failed. The error is below along with the par file, *.err, and *.out filesgenerators attached. Note: There is no storage issue at all.

Any help would be highly appreciated.

Warning: Too many threads per process specified: specified num-threads=2 (ppn-used is 2) Warning: Total number of threads and number of threads per process are inconsistent: procs=1, num-threads=2 (procs*num-smt must be an integer multiple of num-threads) Warning: Total number of threads and number of cores per node are inconsistent: procs=1, ppn-used=2 (procs must be an integer multiple of ppn-used)

• set -e
• cd /home/karima/simulations/bbhL/output-0000-active
• echo Checking:
• pwd
• hostname
• date
• echo Environment:
• export CACTUS_NUM_PROCS=1
• export CACTUS_NUM_THREADS=2
• export GMON_OUT_PREFIX=gmon.out
• export OMP_NUM_THREADS=2
• env
• sort
• echo Starting:
• date +%s
• export CACTUS_STARTTIME=1608653732
• [ 1 = 1 ]
• [ 0 -eq 0 ]
• /home/karima/simulations/bbhL/SIMFACTORY/exe/cactus_sim -L 3

/home/karima/simulations/bbhL/output-0000/BBHLowRes.par WARNING level 0 from host karima-Latitude-E5470 process 0 in thorn cactus, file BBHLowRes.par:1: -> ERROR IN PARAMETER FILE:Parse Error Expected one of the following characters: ':' CarpetMask::excluded_surface_factor[1] = 1.0 CarpetMask::excluded_surface [2] = 2 CarpetMask::excluded_surface_factor[2] = 1.0 Contents successfully written to /home/karima/Cactus/repos/simfactory2/etc/defs.local.ini ^

---

...

MPI_ABORT was invoked on rank 0 in communicator MPI_COMM_WORLD with errorcode 1.

NOTE: invoking MPI_ABORT causes Open MPI to kill all MPI processes. You may or may not see output from other processes, depending on exactly when Open MPI kills them.

Thanks in advance

best,

Karima S

-- My email is as private as my paper mail. I therefore support encrypting and signing email messages. Get my PGP key from https://urldefense.com/v3/__http://pgp.mit.edu__;!!DZ3fjg!vK_laSF6UHgmUOfxg3... .

Hi,

I don't know why the mailing list formatted my email in that way! It has added line breaks where there should not be any. Here is another attempt:

I use the parameter

TwoPunctures::TP_epsilon = 1e-6

to ensure that the NaN never appears, no matter where I place the BHs. This replaces a denominator

sqrt (pow (r2_plus, 2))

with

sqrt (pow (r2_plus, 2) + pow (TP_epsilon, 4))

See

https://bitbucket.org/einsteintoolkit/einsteininitialdata/src/master/TwoPunc...

so adds a fixed source of error to your evolution, but the effect of this is negligible.

-- Ian Hinder Research Software Engineer University of Manchester, UK