Title: CarpetX developer call
When: Weekly from 10am to 11am on Wednesday Eastern Time
Join Zoom Meeting
https://pitp.zoom.us/j/93861697232
Join by phone
(CA) +1 647-558-0588
Join using SIP
93861697232(a)zoomcrc.com
Title: CarpetX developer call
When: Weekly from 10am to 11am on Wednesday Eastern Time
Join Zoom Meeting
https://pitp.zoom.us/j/93861697232
Join by phone
(CA) +1 647-558-0588
Join using SIP
93861697232(a)zoomcrc.com
Title: CarpetX developer call
When: Weekly from 10am to 11am on Wednesday Eastern Time
Join Zoom Meeting
https://pitp.zoom.us/j/93861697232
Join by phone
(CA) +1 647-558-0588
Join using SIP
93861697232(a)zoomcrc.com
Title: CarpetX developer call
When: Weekly from 10am to 11am on Wednesday Eastern Time
Join Zoom Meeting
https://pitp.zoom.us/j/93861697232
Join by phone
(CA) +1 647-558-0588
Join using SIP
93861697232(a)zoomcrc.com
Title: CarpetX developer call
When: Weekly from 10am to 11am on Tuesday Eastern Time
Join Zoom Meeting
https://pitp.zoom.us/j/93861697232
Join by phone
(CA) +1 647-558-0588
Join using SIP
93861697232(a)zoomcrc.com
Hello all,
since this was originally shared only with a subset of the devs, here
it is once more. These are the original (tested) instructions on how to
compile CarpetX for CUDA.
The ones on
https://bitbucket.org/eschnett/cactusamrex/wiki/Getting%20Started#markdown-… are a derivative and also use slightly different code branches (in an attempt to support building ExternalLibraries).
Yours,
Roland
Begin forwarded message:
Date: Wed, 4 Aug 2021 11:32:11 -0400
From: Erik Schnetter <eschnetter(a)perimeterinstitute.ca>
To: Bruno Giacomazzo <bruno.giacomazzo(a)unimib.it>
Cc: Lorenzo Ennoggi <lorenzo.ennoggi(a)gmail.com>, Lucas Timotheo
Sanches <lucas.t.s.carneiro(a)gmail.com>, Jay Vijay Kalinani
<jayvijay.kalinani(a)phd.unipd.it>, Annamalai P S
<annamalaips.97(a)gmail.com>, Roland Haas <rhaas(a)illinois.edu>,
Soham Mukherjee <smukherjee(a)perimeterinstitute.ca> Subject: Re:
Building CarpetX on a laptop
To run on a GPU, you need to install AMReX differently, probably into
a different directory: You need to configure it with additional cmake
options such as
-DAMReX_GPU_BACKEND=CUDA
-DAMReX_CUDA_ERROR_CAPTURE_THIS=ON
-DAMReX_CUDA_ERROR_CROSS_EXECUTION_SPACE_CALL=ON
-DAMReX_CUDA_ARCH=75
where "75" stands for the CUDA compute capability 7.5. Choose other
values there, depending on the hardware on which you will run.
Then continue as before. Create a different option list (which points
to the new AMReX library) and, which uses nvcc as C++ compiler. For
me, the respective modified or new options are:
CXX = /home/eschnetter/Cactus/view-cuda/bin/nvcc --compiler-bindir
/home/eschnetter/Cactus/view-cuda-compilers/bin/g++ -x cu
CXXFLAGS = -pipe -g --compiler-options -march=native -std=c++17
--compiler-options -std=gnu++17 --expt-relaxed-constexpr
--extended-lambda --gpu-architecture sm_75
--forward-unknown-to-host-compiler --Werror cross-execution-space-call
--Werror ext-lambda-captures-this --relocatable-device-code=true
--objdir-as-tempdir
DISABLE_INT16 = yes
DISABLE_REAL16 = yes
VECTORISE = no
CUDA_DIR = /home/eschnetter/Cactus/view-cuda
CUDA_INC_DIRS =
/home/eschnetter/Cactus/view-cuda/targets/x86_64-linux/include
CUDA_LIB_DIRS =
/home/eschnetter/Cactus/view-cuda/targets/x86_64-linux/lib
Then create a new Cactus configuration (executable). This executable
will only run when you have a GPU available; AMReX will check this at
startup.
Depending on how you write your loops, they will either still run on a
CPU, or will now run on a GPU (if you use "loop_device", as in
HydroToyGPU).
I left out the steps where you get lots of errors from the compiler,
and where you sprinkle the code with macros such as CCTK_DEVICE,
CCTK_HOST, or CCTK_ATTRIBUTE_ALWAYS_INLINE, and where you change C++
lambda expressions to use [=] to ensure the captured variables are
copied to the GPU instead of leading to compiler errors (good, because
they come with line numbers) or run-time errors (bad, because they
just say "something went wrong").
And once you're done, you would start looking at performance, trying
to beat our well-optimized CPU gode.
-erik
--
My email is as private as my paper mail. I therefore support encrypting
and signing email messages. Get my PGP key from http://pgp.mit.edu .