Carpetx-developers August 2021

carpetx-developers@lists.einsteintoolkit.org

1 participants
1 discussions

Fwd: Building CarpetX on a laptop
by Roland Haas 31 Aug '21

31 Aug '21

Hello all, since this was originally shared only with a subset of the devs, here it is once more. These are the original (tested) instructions on how to compile CarpetX for CUDA. The ones on https://bitbucket.org/eschnett/cactusamrex/wiki/Getting%20Started#markdown-… are a derivative and also use slightly different code branches (in an attempt to support building ExternalLibraries). Yours, Roland Begin forwarded message: Date: Wed, 4 Aug 2021 11:32:11 -0400 From: Erik Schnetter <eschnetter(a)perimeterinstitute.ca> To: Bruno Giacomazzo <bruno.giacomazzo(a)unimib.it> Cc: Lorenzo Ennoggi <lorenzo.ennoggi(a)gmail.com>, Lucas Timotheo Sanches <lucas.t.s.carneiro(a)gmail.com>, Jay Vijay Kalinani <jayvijay.kalinani(a)phd.unipd.it>, Annamalai P S <annamalaips.97(a)gmail.com>, Roland Haas <rhaas(a)illinois.edu>, Soham Mukherjee <smukherjee(a)perimeterinstitute.ca> Subject: Re: Building CarpetX on a laptop To run on a GPU, you need to install AMReX differently, probably into a different directory: You need to configure it with additional cmake options such as -DAMReX_GPU_BACKEND=CUDA -DAMReX_CUDA_ERROR_CAPTURE_THIS=ON -DAMReX_CUDA_ERROR_CROSS_EXECUTION_SPACE_CALL=ON -DAMReX_CUDA_ARCH=75 where "75" stands for the CUDA compute capability 7.5. Choose other values there, depending on the hardware on which you will run. Then continue as before. Create a different option list (which points to the new AMReX library) and, which uses nvcc as C++ compiler. For me, the respective modified or new options are: CXX = /home/eschnetter/Cactus/view-cuda/bin/nvcc --compiler-bindir /home/eschnetter/Cactus/view-cuda-compilers/bin/g++ -x cu CXXFLAGS = -pipe -g --compiler-options -march=native -std=c++17 --compiler-options -std=gnu++17 --expt-relaxed-constexpr --extended-lambda --gpu-architecture sm_75 --forward-unknown-to-host-compiler --Werror cross-execution-space-call --Werror ext-lambda-captures-this --relocatable-device-code=true --objdir-as-tempdir DISABLE_INT16 = yes DISABLE_REAL16 = yes VECTORISE = no CUDA_DIR = /home/eschnetter/Cactus/view-cuda CUDA_INC_DIRS = /home/eschnetter/Cactus/view-cuda/targets/x86_64-linux/include CUDA_LIB_DIRS = /home/eschnetter/Cactus/view-cuda/targets/x86_64-linux/lib Then create a new Cactus configuration (executable). This executable will only run when you have a GPU available; AMReX will check this at startup. Depending on how you write your loops, they will either still run on a CPU, or will now run on a GPU (if you use "loop_device", as in HydroToyGPU). I left out the steps where you get lots of errors from the compiler, and where you sprinkle the code with macros such as CCTK_DEVICE, CCTK_HOST, or CCTK_ATTRIBUTE_ALWAYS_INLINE, and where you change C++ lambda expressions to use [=] to ensure the captured variables are copied to the GPU instead of leading to compiler errors (good, because they come with line numbers) or run-time errors (bad, because they just say "something went wrong"). And once you're done, you would start looking at performance, trying to beat our well-optimized CPU gode. -erik -- My email is as private as my paper mail. I therefore support encrypting and signing email messages. Get my PGP key from http://pgp.mit.edu .

1 0

2026

2025

2024

2023

2022

2021

Carpetx-developers August 2021