[Users] GPU support for the Einstein Toolkit

[Erik Schnetter]

Bruno

We are looking into GPU support in the context of the CarpetX project.
CarpetX <https://bitbucket.org/eschnett/cactusamrex/src/master/> is a
new driver, currently being developed, that uses AMReX
<https://amrex-codes.github.io/amrex/> as back-end to provide the
lower-level AMR capabilities. AMReX supports GPUs, and we intend to
ensure that CarpetX (and thus the Einstein Toolkit) supports GPUs from
the beginning. I am (personally) currently not using the larger EU HPC
systems, but we have Summit <https://www.olcf.ornl.gov/summit/> in the
North American region, and Summit requires highly efficient GPU codes
as well.

Having said this, CarpetX is still in the very early stages, so I can
only present a plan and not yet a solution. We know that AMReX
supports GPUs, and AMReX is funded and actively developed, so we have
a very well engineered foundation, but exactly how to expose this to
Cactus and the Einstein Toolkit isn't quite clear yet. My hope
(dream?) is that it will be possible to write somewhat generic C++
code that will run unchanged on both CPUs and GPUs, and that making
this work efficiently will work without having to maintain two
different codes. Whether this is possible will remain to be seen.
Maybe we need to go towards automated code generation (generating both
CPU and GPU codes from the same script) to make this happen.

I don't see AMReX on the list of supported software for Marconi. Do
you want to investigate? I wonder whether this is due to lack of
interest from users, or whether they looked into AMReX and there was a
roadblock. (I would not assume so, but you never know.)

-erik

On Fri, Jul 10, 2020 at 11:11 AM Bruno Giacomazzo
<bruno.giacomazzo at unimib.it> wrote:
>
> Dear All,
>   I would like to know if there is any plan to port the Einstein Toolkit to GPUs. This is something that we may also discuss at the next Einstein Toolkit workshop if there is time.
>
>   The reason I'm asking is that in Italy, CINECA (part of the EU HPC consortium) just bought its new HPC cluster and it is completely GPU based (Marconi100, https://www.hpc.cineca.it/hardware/marconi100, currently ranked in the top 10 of the fastest supercomputers in the world).
>
>    Speaking with people at CINECA, I was informed that HPC-Europe is moving to GPU and, while cpu codes will be still supported for the next 5 years (with some of the new clusters still having a limited number of nodes for cpu codes), the plan is to move to 100% GPU clusters at the EU level. I was indeed told that the latest 3 HPC clusters bought by HPC-Europe are GPU clusters. The next Italian HPC machine will also be a GPU cluster, with a limited number of cpu nodes (they estimate ~3500 nodes with GPUs and ~1000 nodes with cpus).
>
> Thanks,
> Bruno
>
> --
>
> Prof. Bruno Giacomazzo
> Department of Physics
> University of Milano-Bicocca
> Piazza della Scienza 3
> 20126 Milano
> Italy
>
> email: bruno.giacomazzo at unimib.it
> phone: (+39) 02 6448 2321
> web: http://www.brunogiacomazzo.org
>
> ----------------------------------------------------------------------
> There are only 10 types of people in the world:
> Those who understand binary, and those who don't
> ----------------------------------------------------------------------
>
> _______________________________________________
> Users mailing list
> Users at einsteintoolkit.org
> http://lists.einsteintoolkit.org/mailman/listinfo/users



-- 
Erik Schnetter <schnetter at cct.lsu.edu>
http://www.perimeterinstitute.ca/personal/eschnetter/