[Users] Meeting Minutes

[Roland Haas]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hello all,

> ET release: * most tests pass on all machines, exceptions are
> GRHydro's weno and slow_sector tests. Roland will look into them
> (since he put them into the repo). Suspicions are that they were
> generated using gcc but are now tested using intel (thing
> --fast-math)
I looked into this and so far it is not clear to me where the
differences come from. The points that differ are in the outer
boundary in the "---" corner of the domain. (in 2D) They form a square
of size of the ghost zones. This is idependent of using -O2 or -O0.
Also compiling the WENO routines with gfortran (and everything else
with ifort) makes no difference.

I also looked into the issue that many tests fail on bethe when
oversubscribing the machine using 2 MPI processes and 8 threads each
(there are only 8 physical cores). This turns out to be related to
vectorization/OpenMP in the Kranc generated thorns, in particular
McLachlan. The errors go away if I either run without openmp, or fewer
openmp threads, eg. 1x16 fails but 1x8 is fine. They also go away if I
disable vectorization. Changing various Vectors options for ALIGNED
storage and read/write seems to make no difference.

This actually seems serious if really there is a bug in the
interaction between vectorization and OpenMP (note that since the
error goes away and comes back if I disable OpenMP only in eg. the
ML_BSSN_enforce.cc file it does not seem to be related to reductions
being not bit identical when using OpenMP.).

Yours,
Roland

- -- 
My email is as private as my paper mail. I therefore support encrypting
and signing email messages. Get my PGP key from http://keys.gnupg.net.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.12 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAlGOlpAACgkQTiFSTN7SboU4KgCdGarZ+QERdh2wYNFS+Hn+CFRQ
dNMAoIWStwoFmegaj+1WmlVu3Ql/eG6a
=ci8n
-----END PGP SIGNATURE-----