#2553: Cactus' link command uses CPPFLAGS and CXXFLAGS
Reporter: Roland Haas
Status: open
Milestone:
Version: development version
Type: bug
Priority: minor
Component: Cactus
Comment (by Erik Schnetter):
You would need to point Spack to the compiler and MPI libraries that it should use \(and maybe a few more\), and then build the rest. Building your own MPI library is easy; using it is next to impossible since it’s very difficult to configure it to use the existing hardware \(and it’s probably actually impossible on Crays\).
--
Ticket URL: https://bitbucket.org/einsteintoolkit/tickets/issues/2553/cactus-link-comma…
#2407: Cactus math work-arounds break CarpetX reduction operators
Reporter: Erik Schnetter
Status: open
Milestone:
Version:
Type: bug
Priority: major
Component: Cactus
Comment (by Roland Haas):
The workarounds \(indirectly\) also cause issues on new M1 macOS machines since Apple seems to have removed the `finite` function call that is used in some places instead of `std::isfininite` \(and just `isfinite` does not work\).
--
Ticket URL: https://bitbucket.org/einsteintoolkit/tickets/issues/2407/cactus-math-work-…
#2553: Cactus' link command uses CPPFLAGS and CXXFLAGS
Reporter: Roland Haas
Status: open
Milestone:
Version: development version
Type: bug
Priority: minor
Component: Cactus
Comment (by Roland Haas):
My biggest worry with Spac is that, unless the cluster’s module system is also based on Spac, it is often hard to integrate Spac generated packages with the cluster ones. I realize that one can often just build everything using Spac incl. MPI etc. but am not sure how confident I am in this always providing a good solution. I expect it will do somewhat poorly on clusters with proprietary hardware, eg Crays of any shape and size or clusters where MPI required a lot of hand tuning.
--
Ticket URL: https://bitbucket.org/einsteintoolkit/tickets/issues/2553/cactus-link-comma…
#2553: Cactus' link command uses CPPFLAGS and CXXFLAGS
Reporter: Roland Haas
Status: open
Milestone:
Version: development version
Type: bug
Priority: minor
Component: Cactus
Comment (by Erik Schnetter):
I’m working on a Spack recipe for the Einstein Toolkit. It’s working \(with many external libraries, of course\), but it currently uses plain `make` instead of Simfactory. Once that’s done I’ll submit it as official package.
--
Ticket URL: https://bitbucket.org/einsteintoolkit/tickets/issues/2553/cactus-link-comma…
#2553: Cactus' link command uses CPPFLAGS and CXXFLAGS
Reporter: Roland Haas
Status: open
Milestone:
Version: development version
Type: bug
Priority: minor
Component: Cactus
Comment (by Erik Schnetter):
Yes, external libraries really want traditional compiler settings. I always build external libraries externally; I think it was a mistake to use Cactus’s build system for that \(downloads are large, build times are large, external libraries are not shared across configurations, etc.\) I’m using Spack for all external libraries these days.
We could introduce `EXTERNAL_CC` etc. which we would use in external libraries.
--
Ticket URL: https://bitbucket.org/einsteintoolkit/tickets/issues/2553/cactus-link-comma…
#2553: Cactus' link command uses CPPFLAGS and CXXFLAGS
Reporter: Roland Haas
Status: open
Milestone:
Version: development version
Type: bug
Priority: minor
Component: Cactus
Comment (by Roland Haas):
That one also worked for me, and is what I used at first. However it fails to compile the ExternalLibraries. I am not 100% sure why but two issues seem to be that CMake really does not like $CXX to be more than one word and then that “-x cu” seems to also interpreset \*.o files as source files ocompile, which of course leas to failures.
I also have to add `-DSIMD_CPU` to avoid issues in Arith where I otherwise get an error that the return value of a constexpr function must be of integral type \(this is with gcc 10.1 as the host complier so sufficiently modern\).
This work is part of an effort is to avoid having to do any of this and be able to use a more regular Cactus build setup where one can use the ExternalLibraries to automatically build the required libraries which should help reduce the amount of build failures when one starts with CarpetX.
I will attach a thornlist once I have tested things once or twice on my latpop \(with and without GPU\) and on some cluster.
--
Ticket URL: https://bitbucket.org/einsteintoolkit/tickets/issues/2553/cactus-link-comma…
#2553: Cactus' link command uses CPPFLAGS and CXXFLAGS
Reporter: Roland Haas
Status: open
Milestone:
Version: development version
Type: bug
Priority: minor
Component: Cactus
Comment (by Erik Schnetter):
I didn’t realize you had trouble compiling these thorns.
For me, these settings work:
```shell
CXX = /home/eschnetter/Cactus/view-cuda/bin/nvcc --compiler-bindir /home/eschnetter/Cactus/view-cuda-compilers/bin/g++ -x cu
CXXFLAGS = -pipe -g --compiler-options -march=native -std=c++17 --compiler-options -std=gnu++17 --expt-relaxed-constexpr --extended-lambda --gpu-architecture sm_75 --forward-unknown-to-host-compiler --Werror cross-execution-space-call --Werror ext-lambda-captures-this --relocatable-device-code=true --objdir-as-tempdir
```
with standard Cactus. This uses `nvcc` for all C\+\+ code as well as for linking.
--
Ticket URL: https://bitbucket.org/einsteintoolkit/tickets/issues/2553/cactus-link-comma…
#2553: Cactus' link command uses CPPFLAGS and CXXFLAGS
Reporter: Roland Haas
Status: open
Milestone:
Version: development version
Type: bug
Priority: minor
Component: Cactus
Comment (by Roland Haas):
And version 8aa7a36 lets me compile CarpetX, Z4c, Weyl, AMReX.
--
Ticket URL: https://bitbucket.org/einsteintoolkit/tickets/issues/2553/cactus-link-comma…
#2553: Cactus' link command uses CPPFLAGS and CXXFLAGS
Reporter: Roland Haas
Status: open
Milestone:
Version: development version
Type: bug
Priority: minor
Component: Cactus
Comment (by Roland Haas):
Well making LD be nvcc gives me extra trouble when compiling ExternalLibraries \(same sort of issues that come from having Cactus LIBS mean something different then autoconf’s LIBS\).
Instead I peeked at Formaline and wrote a short ExternallLibraries/CUDA thorn that adapts Cactus' link step based on the documentation by NVIDIA. The thorn is here: [https://github.com/rhaas80/ExternalLibraries-CUDA.git](https://github.com/r… and the trick is to collect all CUDA code in a new library using nvcc \(just as NVIDIA shows\):
```
CACTUSLIBLINKLINE += -l$(CCTK_LIBNAME_PREFIX)CUDA-gpucode
CUDA-LIB = $(CCTK_LIBDIR)/$(LIBNAME_PREFIX)$(CCTK_LIBNAME_PREFIX)CUDA-gpucode$(LIBNAME_SUFFIX)
$(EXEDIR)$(DIRSEP)$(EXE): $(CUDA-LIB)
# TODO: make this depend on only the thorns that REQUIRE CUDA
# TODO: check if depending on LINKLIST would be enough
$(CUDA-LIB): $(CONFIG)/make.thornlist $(CONFIG)/cctki_version.h $(patsubst %,$(CCTK_LIBDIR)/$(LIBNAME_PREFIX)$(CCTK_LIBNAME_PREFIX)%$(LIBNAME_SUFFIX),$(notdir $(THORNS) $(CACTUSLIBS))) $(CCTK_LIBDIR)/LINKLIST
$(CUCC) $(patsubst %,$(CCTK_LIBDIR)/$(LIBNAME_PREFIX)%$(LIBNAME_SUFFIX),$(ALLCACTUSLIBS)) -dlink -o $@
if test "x$(USE_RANLIB)" = "xyes"; then $(RANLIB) $(RANLIBFLAGS) $@; fi
@echo $(DIVIDER)
```
which lets me compile this thornlist:
```
ExternalLibraries/CUDA
```
and run this parfile:
```
ActiveThorns = CUDA
CUDA::test = yes
```
I think this should work, though I have not tested it on a system where `-filelist` is actually supported \(given that Linux does not, I assume nothing does anymore\), in which case maybe there are no libthornFOO.a files but only the raw object files.
--
Ticket URL: https://bitbucket.org/einsteintoolkit/tickets/issues/2553/cactus-link-comma…