Aha, it looks like you have found a bug.  Congratulations!  A similar problem was discussed in https://trac.einsteintoolkit.org/ticket/1936#comment:4 for the QuasiLocalMeasures thorn.  The solution is to merge the multiple tags entries.  It's called "development version" for a reason :)  There is also a problem with the ADMConstraints thorn.  I have fixed both of these. 


This is great, thanks a lot Ian. I tested it and the GW150914 now compiles on Stampede-KNL. I will run a test job and see if there are any further issues.


You will need to be more specific about what you mean by "analyse the performance".  What do you want to know?


I simply want  to compute the wall time taken per iteration, and per coordinate time or roughly the speed of the code. So all I need is to output iteration, coordinate time and walltime at that point of simulation. Is there some thorn for that?


> Lastly, I am still stuck with the BBox error on the GT code. Could you please  suggest on how can I resolve this?

I didn't understand the email that you sent about this.  We said that the bug was in the Intel 17 compiler, and that you should be able to fix the problem by using the development version of the toolkit, since a workaround has been added there.  It looks like you are now using the development version, so the problem shouldn't be there any more.  Does the development version still not work?

The code of Georgia Tech also uses Cactus interface and I am trying to compile it on Stampede-KNL. I tried using the same machine files and optionlist but ran into the same BBox error (for the Georgia Tech code) which I mentioned on the call . Since the ETK development version does not have this issue properly, I tried updating the Cactus infrastructure of GT code using this version but again run into some errors. I will first try to test if I can get ETK to work properly on Stampede-KNL and then probably send a more detailed update with the errors I get for this case.


Thank you so much Ian.

.............................

Bhavesh Khamesra

Graduate Student

Centre of Relativistic Astrophysics

Georgia Institute of Technology

From: Ian Hinder <ian.hinder@aei.mpg.de>
Sent: Thursday, April 13, 2017 6:07:52 AM
To: Khamesra, Bhavesh
Cc: Einstein Toolkit Users
Subject: Re: [Users] GW150914 failing compilation
 

On 13 Apr 2017, at 06:07, Khamesra, Bhavesh <bhaveshkhamesra@gatech.edu> wrote:

> Hey everyone,
>
> Continuing the test of the ETK on Stampede-KNL, I ran into some issues. After some modification to optionlist, I was able to compile the einsteintoolkit.ththornlist using the development version. Then, I was trying to compile the GW150914 thornlist but ended up this error -
>
> CST Error:
> Repeated element 'tag' in metric_tmp
> Line: 190
> File: /work/04082/tg833814/Cactus_ETK_dev/arrangements/Llama/WaveExtractCPM/interface.ccl
> Repeated item: tags in metric_tmp at /work/04082/tg833814/Cactus_ETK_dev/lib/sbin/interface_parser.pl line 881.
>     main::parse_interface_ccl('Llama', 'WaveExtractCPM', 'ARRAY(0x237fd68)', 'Group=HASH(0x28d98a8)', 'HASH(0x23759e0)') called at /work/04082/tg833814/Cactus_ETK_dev/lib/sbin/interface_parser.pl line 103
>     main::create_interface_database(3, 'BINDINGS_DIR', '/work/04082/tg833814/Cactus_ETK_dev/configs/GW150914_KNL_test...', 'CONFIG_DIR', '/work/04082/tg833814/Cactus_ETK_dev/configs/GW150914_KNL_test...', 'CCTK_HOME', '/work/04082/tg833814/Cactus_ETK_dev', 'TwoPunctures', '/work/04082/tg833814/Cactus_ETK_dev/arrangements/EinsteinInit...', ...) called at /work/04082/tg833814/Cactus_ETK_dev/lib/sbin/CST line 135
>
> I get this error even on my local machine (Red Hat 7) I am not sure how to resolve this one? Could someone please help?

Aha, it looks like you have found a bug.  Congratulations!  A similar problem was discussed in https://trac.einsteintoolkit.org/ticket/1936#comment:4 for the QuasiLocalMeasures thorn.  The solution is to merge the multiple tags entries.  It's called "development version" for a reason :)  There is also a problem with the ADMConstraints thorn.  I have fixed both of these.  You should be able to update using

cd Cactus_ETK_dev
repos/CRL/GetComponents --update --root .

(I don't use GetComponents, so I'm guessing that this is the recommended way to update)

The reason this is not caught by the automated tests is that the WaveExtractCPM thorn is not part of the toolkit thornlist, so is not tested.  It is used in the GW150914 parameter file and thornlist, because I thought it was a useful thing to have, though it is by no means essential.  The Cactus parser has been rewritten since this was last tested, and the new version is much stricter.  This is good, because the old behaviour was to ignore all but the last of repeated tags entries, which is definitely not what was intended here!

I don't think that this bug would affect results; the code will currently ignore the first tags entry specifying the tensor type, but that would only be used for symmetry boundary conditions, which shouldn't be relevant for the metric_tmp gridfunction, since it is a temporary variable which is probably only accessed point-wise.

> Also, is there any way to analyse the performance of the code for various simulations? Is there any thorn for this purpose?

You will need to be more specific about what you mean by "analyse the performance".  What do you want to know?

> Lastly, I am still stuck with the BBox error on the GT code. Could you please  suggest on how can I resolve this?

I didn't understand the email that you sent about this.  We said that the bug was in the Intel 17 compiler, and that you should be able to fix the problem by using the development version of the toolkit, since a workaround has been added there.  It looks like you are now using the development version, so the problem shouldn't be there any more.  Does the development version still not work?

--
Ian Hinder
http://members.aei.mpg.de/ianhin