[Users] Setting up ETK on an AMD Rome cluster

[Roland Haas]

Hello Vaishak,

Sorry for the delay, and thank you for including the various log files.

I have been running on a new AMD based system (NCSA Delta, Milan, not
Rome) during the last week (with Vectors active), though it is a
slightly older ET code (no changes to Vectors though). I also ran on
SDSC Expanse (Rome, Epyc 7742) for the ET testsuite for the 2022_11
release (http://einsteintoolkit.org/testsuite_results/index.php)
without SEGFAULT failures.
 
This unfortunately makes debugging the issue that you are facing harder.

One (possible) issue could be related to using  -march=native in you
compilation flags. Since this instructs GCC to compile for the CPU
architecture it finds itself running on, I would double check that
indeed the login nodes on sonic use the same CPU as the compute nodes.

Yours,
Roland

> Dear All,
> 
> Greetings from India. I am trying to get the ETK working on an AMD Rome
> powered supercomputer at ICTS, India. I am working with gcc (11.1.0,
> 12.2.0) and openmpi. The compilation is successful but every one of the
> tests and runs fails due to seg faults at the vectorization stage. On
> recompiling the toolkit without vectorization, the tests run  (except for
> one test of ML_BSSN which fails due to a relative error ~ 1e-14). I am
> attaching the backtrace (from the gallery BBH run), make log (with
> vectorization) and the optionlist herewith.
> 
> Requesting help!
> 
> 
> With regards
> 



-- 
My email is as private as my paper mail. I therefore support encrypting
and signing email messages. Get my PGP key from http://keys.gnupg.net.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 833 bytes
Desc: OpenPGP digital signature
Url : http://lists.einsteintoolkit.org/pipermail/users/attachments/20230302/8cf10e3c/attachment-0001.bin