[Users] question about ET on Stampede (TACC)

[Erik Schnetter]

Bruno

I'm currently testing the Einstein Toolkit on Stampede. Even without the
MICs, this is a very nice and fast system, almost ideal for the ET. The
scripts in Simfactory should almost work; I will update them in the coming
days. (The system was unavailable in the past days, so I couldn't test them
recently.)

The MICs (Xeon Phis) can be used in two ways. One can view them as an
accelerator (like a GPU), and then offload certain calculations there.
Unfortunately neither CUDA nor OpenCL are available there, hence one has to
use PGI's OpenACC and deal with PGI's compiler that has serious problems
with OpenMP.

The other option is to view the Xeon Phis as regular processors. I believe
that I/O will be quite slow from them, and MPI does not (yet?) fully work
there, but apart from this, one can run the ET there directly. The small
caches and in-order execution will make for some challenges; in particular,
multi-threading e.g. via OpenMP will be a must.

In the end, I assume that one would want to use them either as accelerator
(if not via OpenCL then via something equivalent, just different), or as
regular compute nodes where Carpet will have to be enhanced to handle the
different CPU speeds between Xeons and Xeon Phis.

-erik



On Tue, Jan 8, 2013 at 2:35 PM, Bruno Giacomazzo <
bruno.giacomazzo at jila.colorado.edu> wrote:

> Hi,
> I'd like to compile and run ET on the new TACC cluster Stampede. Did
> anybody run ET on stampede?
> I see that there are already simfactory scripts for stampede in the ET
> development version, but I'm not sure if they have been already fully
> tested (I see that there are comments about job submission not having been
> tested yet).
>
> I'm also curious to know if you think that the hybrid approach can be of
> any use for ET/GRHydro (or ET/Whisky) on that machine. I'm not very
> familiar with the Intel Phi coprocessor and MIC programming (but it seems
> similar to GPU, except for being much easier to use).
>
> Thanks,
> Bruno
>
> Dr. Bruno Giacomazzo
> JILA - University of Colorado
> 440 UCB
> Boulder, CO 80309
> USA
>
> Tel.  : +1 303-492-5170
> Fax  : +1 303-492-5235
> email : bruno.giacomazzo at jila.colorado.edu
> web: http://www.brunogiacomazzo.org
>
> ----------------------------------------------------------------------
> There are only 10 types of people in the world:
> Those who understand binary, and those who don't
> ----------------------------------------------------------------------
>
>
> _______________________________________________
> Users mailing list
> Users at einsteintoolkit.org
> http://lists.einsteintoolkit.org/mailman/listinfo/users
>
>


-- 
Erik Schnetter <schnetter at cct.lsu.edu>
http://www.perimeterinstitute.ca/personal/eschnetter/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.einsteintoolkit.org/pipermail/users/attachments/20130108/1be064b2/attachment.html