Hi All,
Thanks so much for making the Einstein Toolkit software available!
I'm using the CactusTutorial.ipynb that I got from your site. I'm trying to build ET on my 2015 Macbook Pro running OS 10.15.7.
I'm stuck at the cell (after) *Building the Einstein Toolkit* . I modified that cell to get past issues with zlib . The cell now looks like:
%%bash export ZLIB_DIR=/usr/local/opt/zlib/lib export LDFLAGS="-L/usr/local/opt/zlib/lib" export CPPFLAGS="-I/usr/local/opt/zlib/include" export PKG_CONFIG_PATH="/usr/local/opt/zlib/lib/pkgconfig" ./simfactory/bin/sim build -j2 --thornlist ../einsteintoolkit.th
Running that cell eventually stops with the errors
COMPILING CactusBase/CoordBase/src/Domain.c COMPILING configs/sim/bindings/build/ADMCoupling/cctk_ThornBindings.c clang: error: unsupported option '-fopenmp' make[3]: *** [Domain.c.o] Error 1 make[2]: *** [make.checked] Error 2 make[1]: *** [/Users/murray/Cactus/configs/sim/lib/libthorn_CoordBase.a] Error 2 make[1]: *** Waiting for unfinished jobs.... clang: error: unsupported option '-fopenmp' make[3]: *** [cctk_ThornBindings.c.o] Error 1 make[2]: *** [cctk_Bindings/make.checked] Error 2 make[1]: *** [/Users/murray/Cactus/configs/sim/lib/libthorn_ADMCoupling.a] Error 2 make: *** [sim] Error 2
Can you help? I'm confused by the error message coming from clang, since the ET seems to use gcc, but my attempt to force gcc, by including the exports below, made no difference:
export CC=gcc-10 export CXX=gcc-10
I really appreciate any help you are able to provide!
Thank you.
Murray (Wolinsky) ______________________________________________________
Additional (likely irrelevant) notes:
I used brew to install the prerequisites. It seemed to have gone more-or-less ok.
Brew did issue a somewhat concerning message regarding hdf5:
==> Pouring hdf5-1.12.0_1.catalina.bottle.1.tar.gz
Warning: hdf5 dependency gcc was built with a different C++ standard
library (libstdc++ from clang). This may cause problems at runtime.
But it didn't provide any guidance into fixing the potential problem. Itried
brew reinstall hdf5
and brew no longer complains.
I also had some issues with the subversion installation. But I don't think they're responsible for my current problem.
Also, when I did the step *Configuring SimFactory for your machine*, I got the output:
Here we will define some necessary Simulation Factory defaults.
Determining local machine name: mw-3.local
--------------------SUMMARY--------------------:
[default] user = murray email = murray allocation = NO_ALLOCATION
------------------END SUMMARY------------------:
Contents successfully written to /Users/murray/Cactus/repos/simfactory2/etc/defs.local.ini
which looks OK, but the NO_ALLOCATION message seems like it might be of concern?
Hello Murray,
clang (in particular Apple's version delivered in OSX) is not fully supported by the ET (due to, at least historically, no OpenMP support, no paired Fortran compiler).
Please see the initial section of:
https://github.com/nds-org/jupyter-et/blob/master/CactusTutorial.ipynb
for how to use the ET on OSX using homebrew or macports.
If you have a Fortran compiler installed you can try disabling openmp by setting OPENMP=no in your option list. I strongly advise consulting the link I provided above and following the (tested and verified) instructions there in particular if this is your first time using the toolkit.
Yours, Roland
Hi All,
Thanks so much for making the Einstein Toolkit software available!
I'm using the CactusTutorial.ipynb that I got from your site. I'm trying to build ET on my 2015 Macbook Pro running OS 10.15.7.
I'm stuck at the cell (after) *Building the Einstein Toolkit* . I modified that cell to get past issues with zlib . The cell now looks like:
%%bash export ZLIB_DIR=/usr/local/opt/zlib/lib export LDFLAGS="-L/usr/local/opt/zlib/lib" export CPPFLAGS="-I/usr/local/opt/zlib/include" export PKG_CONFIG_PATH="/usr/local/opt/zlib/lib/pkgconfig" ./simfactory/bin/sim build -j2 --thornlist ../einsteintoolkit.th
Running that cell eventually stops with the errors
COMPILING CactusBase/CoordBase/src/Domain.c COMPILING configs/sim/bindings/build/ADMCoupling/cctk_ThornBindings.c clang: error: unsupported option '-fopenmp' make[3]: *** [Domain.c.o] Error 1 make[2]: *** [make.checked] Error 2 make[1]: *** [/Users/murray/Cactus/configs/sim/lib/libthorn_CoordBase.a] Error 2 make[1]: *** Waiting for unfinished jobs.... clang: error: unsupported option '-fopenmp' make[3]: *** [cctk_ThornBindings.c.o] Error 1 make[2]: *** [cctk_Bindings/make.checked] Error 2 make[1]: *** [/Users/murray/Cactus/configs/sim/lib/libthorn_ADMCoupling.a] Error 2 make: *** [sim] Error 2
Can you help? I'm confused by the error message coming from clang, since the ET seems to use gcc, but my attempt to force gcc, by including the exports below, made no difference:
export CC=gcc-10 export CXX=gcc-10
I really appreciate any help you are able to provide!
Thank you.
Murray (Wolinsky) ______________________________________________________
Additional (likely irrelevant) notes:
I used brew to install the prerequisites. It seemed to have gone more-or-less ok.
Brew did issue a somewhat concerning message regarding hdf5:
==> Pouring hdf5-1.12.0_1.catalina.bottle.1.tar.gz
Warning: hdf5 dependency gcc was built with a different C++ standard
library (libstdc++ from clang). This may cause problems at runtime.
But it didn't provide any guidance into fixing the potential problem. Itried
brew reinstall hdf5
and brew no longer complains.
I also had some issues with the subversion installation. But I don't think they're responsible for my current problem.
Also, when I did the step *Configuring SimFactory for your machine*, I got the output:
Here we will define some necessary Simulation Factory defaults.
Determining local machine name: mw-3.local
--------------------SUMMARY--------------------:
[default] user = murray email = murray allocation = NO_ALLOCATION
------------------END SUMMARY------------------:
Contents successfully written to /Users/murray/Cactus/repos/simfactory2/etc/defs.local.ini
which looks OK, but the NO_ALLOCATION message seems like it might be of concern?
Hello Murray,
thinking about this a bit more.
The notebook currently (as of last week) lets me compile and run the ET using Homebrew (and worked on Catalina in the past).
One thing to keep in mind is that the various CC/CXX and other exported env variables are only used the *first* time you attempt to build and not afterwards.
You may have more luck having the options be respected by using:
./simfactory/bin/sim build --clean --reconfig
which will recompile from scratch and re-run the configuration stage or even by manually removing the "configs/sim" directory completely.
Did you run "simfactory/bin/sim setup-silent" before trying to compile?
Yours, Roland
Hello Murray,
clang (in particular Apple's version delivered in OSX) is not fully supported by the ET (due to, at least historically, no OpenMP support, no paired Fortran compiler).
Please see the initial section of:
https://github.com/nds-org/jupyter-et/blob/master/CactusTutorial.ipynb
for how to use the ET on OSX using homebrew or macports.
If you have a Fortran compiler installed you can try disabling openmp by setting OPENMP=no in your option list. I strongly advise consulting the link I provided above and following the (tested and verified) instructions there in particular if this is your first time using the toolkit.
Yours, Roland
Hi All,
Thanks so much for making the Einstein Toolkit software available!
I'm using the CactusTutorial.ipynb that I got from your site. I'm trying to build ET on my 2015 Macbook Pro running OS 10.15.7.
I'm stuck at the cell (after) *Building the Einstein Toolkit* . I modified that cell to get past issues with zlib . The cell now looks like:
%%bash export ZLIB_DIR=/usr/local/opt/zlib/lib export LDFLAGS="-L/usr/local/opt/zlib/lib" export CPPFLAGS="-I/usr/local/opt/zlib/include" export PKG_CONFIG_PATH="/usr/local/opt/zlib/lib/pkgconfig" ./simfactory/bin/sim build -j2 --thornlist ../einsteintoolkit.th
Running that cell eventually stops with the errors
COMPILING CactusBase/CoordBase/src/Domain.c COMPILING configs/sim/bindings/build/ADMCoupling/cctk_ThornBindings.c clang: error: unsupported option '-fopenmp' make[3]: *** [Domain.c.o] Error 1 make[2]: *** [make.checked] Error 2 make[1]: *** [/Users/murray/Cactus/configs/sim/lib/libthorn_CoordBase.a] Error 2 make[1]: *** Waiting for unfinished jobs.... clang: error: unsupported option '-fopenmp' make[3]: *** [cctk_ThornBindings.c.o] Error 1 make[2]: *** [cctk_Bindings/make.checked] Error 2 make[1]: *** [/Users/murray/Cactus/configs/sim/lib/libthorn_ADMCoupling.a] Error 2 make: *** [sim] Error 2
Can you help? I'm confused by the error message coming from clang, since the ET seems to use gcc, but my attempt to force gcc, by including the exports below, made no difference:
export CC=gcc-10 export CXX=gcc-10
I really appreciate any help you are able to provide!
Thank you.
Murray (Wolinsky) ______________________________________________________
Additional (likely irrelevant) notes:
I used brew to install the prerequisites. It seemed to have gone more-or-less ok.
Brew did issue a somewhat concerning message regarding hdf5:
==> Pouring hdf5-1.12.0_1.catalina.bottle.1.tar.gz
Warning: hdf5 dependency gcc was built with a different C++ standard
library (libstdc++ from clang). This may cause problems at runtime.
But it didn't provide any guidance into fixing the potential problem. Itried
brew reinstall hdf5
and brew no longer complains.
I also had some issues with the subversion installation. But I don't think they're responsible for my current problem.
Also, when I did the step *Configuring SimFactory for your machine*, I got the output:
Here we will define some necessary Simulation Factory defaults.
Determining local machine name: mw-3.local
--------------------SUMMARY--------------------:
[default] user = murray email = murray allocation = NO_ALLOCATION
------------------END SUMMARY------------------:
Contents successfully written to /Users/murray/Cactus/repos/simfactory2/etc/defs.local.ini
which looks OK, but the NO_ALLOCATION message seems like it might be of concern?
Hi Roland,
Thanks so much for your response! I've been struggling with the -fopenmp issue for a couple of days now. I'll try your suggestions and let you know if it works. It's encouraging to know that the ET works with Catalina: that gives me hope I'll get it to work eventually.
I'm using the notebook CactusTutorial.ipynb, and I certainly don't want to use clang. But the Mac makes using gcc instead of clang tricky. On my machine, gcc gets clang; you need to enter gcc-10 to get gcc.
I verified that gcc (gcc-10) accepts the -fopenmp flag properly. So both gcc and OpenMP seem to work properly on my machine. My problem is convincing ET to use gcc instead of clang! BTW, I'm using brew. It's possible that MacPorts would work better, but I think brew and MacPorts are incompatible.
Again, thank you and I'll let you know my progress later today.
Yours,
Murray
On Tue, Apr 13, 2021 at 9:55 AM Roland Haas rhaas@illinois.edu wrote:
Hello Murray,
thinking about this a bit more.
The notebook currently (as of last week) lets me compile and run the ET using Homebrew (and worked on Catalina in the past).
One thing to keep in mind is that the various CC/CXX and other exported env variables are only used the *first* time you attempt to build and not afterwards.
You may have more luck having the options be respected by using:
./simfactory/bin/sim build --clean --reconfig
which will recompile from scratch and re-run the configuration stage or even by manually removing the "configs/sim" directory completely.
Did you run "simfactory/bin/sim setup-silent" before trying to compile?
Yours, Roland
Hello Murray,
clang (in particular Apple's version delivered in OSX) is not fully supported by the ET (due to, at least historically, no OpenMP support, no paired Fortran compiler).
Please see the initial section of:
https://github.com/nds-org/jupyter-et/blob/master/CactusTutorial.ipynb
for how to use the ET on OSX using homebrew or macports.
If you have a Fortran compiler installed you can try disabling openmp by setting OPENMP=no in your option list. I strongly advise consulting the link I provided above and following the (tested and verified) instructions there in particular if this is your first time using the toolkit.
Yours, Roland
Hi All,
Thanks so much for making the Einstein Toolkit software available!
I'm using the CactusTutorial.ipynb that I got from your site. I'm
trying to
build ET on my 2015 Macbook Pro running OS 10.15.7.
I'm stuck at the cell (after) *Building the Einstein Toolkit* . I modified that cell to get past issues with zlib . The cell now looks
like:
%%bash export ZLIB_DIR=/usr/local/opt/zlib/lib export LDFLAGS="-L/usr/local/opt/zlib/lib" export CPPFLAGS="-I/usr/local/opt/zlib/include" export PKG_CONFIG_PATH="/usr/local/opt/zlib/lib/pkgconfig" ./simfactory/bin/sim build -j2 --thornlist ../einsteintoolkit.th
Running that cell eventually stops with the errors
COMPILING CactusBase/CoordBase/src/Domain.c COMPILING configs/sim/bindings/build/ADMCoupling/cctk_ThornBindings.c clang: error: unsupported option '-fopenmp' make[3]: *** [Domain.c.o] Error 1 make[2]: *** [make.checked] Error 2 make[1]: ***
[/Users/murray/Cactus/configs/sim/lib/libthorn_CoordBase.a] Error 2
make[1]: *** Waiting for unfinished jobs.... clang: error: unsupported option '-fopenmp' make[3]: *** [cctk_ThornBindings.c.o] Error 1 make[2]: *** [cctk_Bindings/make.checked] Error 2 make[1]: ***
[/Users/murray/Cactus/configs/sim/lib/libthorn_ADMCoupling.a]
Error 2 make: *** [sim] Error 2
Can you help? I'm confused by the error message coming from clang,
since
the ET seems to use gcc, but my attempt to force gcc, by including the exports below, made no difference:
export CC=gcc-10 export CXX=gcc-10
I really appreciate any help you are able to provide!
Thank you.
Murray (Wolinsky) ______________________________________________________
Additional (likely irrelevant) notes:
I used brew to install the prerequisites. It seemed to have gone more-or-less ok.
Brew did issue a somewhat concerning message regarding hdf5:
==> Pouring hdf5-1.12.0_1.catalina.bottle.1.tar.gz
Warning: hdf5 dependency gcc was built with a different C++ standard
library (libstdc++ from clang). This may cause problems at runtime.
But it didn't provide any guidance into fixing the potential problem.
Itried
brew reinstall hdf5
and brew no longer complains.
I also had some issues with the subversion installation. But I don't
think
they're responsible for my current problem.
Also, when I did the step *Configuring SimFactory for your machine*, I
got
the output:
Here we will define some necessary Simulation Factory defaults.
Determining local machine name: mw-3.local
--------------------SUMMARY--------------------:
[default] user = murray email = murray allocation = NO_ALLOCATION
------------------END SUMMARY------------------:
Contents successfully written to /Users/murray/Cactus/repos/simfactory2/etc/defs.local.ini
which looks OK, but the NO_ALLOCATION message seems like it might be
of concern?
-- My email is as private as my paper mail. I therefore support encrypting and signing email messages. Get my PGP key from http://pgp.mit.edu .
Hello Murray,
ok. Good to hear that.
My initial email was a send a bit off the cuff since I had not quite realized that you were already using the notebook.
OSX is a bit tricky for the ET to support for the reasons that you outlined.
When you run "./simfactory/bin/sim setup-silent" simulation factory will find the gcc-10 compiler and creates a set of of make options for your laptop (there's explicit code in simfactory to recongnize OSX). So running this command is essential.
You should not have to adjust anything (though given that the last test with Catalina rather than BigSur was about a year ago that may not be true anymore if HomeBrew changed too much).
This set of options is tied to your laptop's hostname as reported by "hostname -f". Unfortunately this hostname is not "stable" in the sense that it may change with time since it can be derived from your laptop's IP address obtained from the wireless network. Eg on my test laptop it currently reads:
wirelessprvnat-172-16-210-128.near.illinois.edu
So you may have to check that
"./simfactory/bin/sim whoami"
returns the expected machine name and run "./simfactory/bin/sim setup-silent" again if it reports an unknown machine.
You can also create a file $HOME/.hostname and add a name that you would like simfactory to use for your laptop eg:
echo >$HOME/.hostname "murrays-laptop"
(no spaces allowed).
Yours, Roland
Hi Roland,
Thanks so much for your response! I've been struggling with the -fopenmp issue for a couple of days now. I'll try your suggestions and let you know if it works. It's encouraging to know that the ET works with Catalina: that gives me hope I'll get it to work eventually.
I'm using the notebook CactusTutorial.ipynb, and I certainly don't want to use clang. But the Mac makes using gcc instead of clang tricky. On my machine, gcc gets clang; you need to enter gcc-10 to get gcc.
I verified that gcc (gcc-10) accepts the -fopenmp flag properly. So both gcc and OpenMP seem to work properly on my machine. My problem is convincing ET to use gcc instead of clang! BTW, I'm using brew. It's possible that MacPorts would work better, but I think brew and MacPorts are incompatible.
Again, thank you and I'll let you know my progress later today.
Yours,
Murray
On Tue, Apr 13, 2021 at 9:55 AM Roland Haas rhaas@illinois.edu wrote:
Hello Murray,
thinking about this a bit more.
The notebook currently (as of last week) lets me compile and run the ET using Homebrew (and worked on Catalina in the past).
One thing to keep in mind is that the various CC/CXX and other exported env variables are only used the *first* time you attempt to build and not afterwards.
You may have more luck having the options be respected by using:
./simfactory/bin/sim build --clean --reconfig
which will recompile from scratch and re-run the configuration stage or even by manually removing the "configs/sim" directory completely.
Did you run "simfactory/bin/sim setup-silent" before trying to compile?
Yours, Roland
Hello Murray,
clang (in particular Apple's version delivered in OSX) is not fully supported by the ET (due to, at least historically, no OpenMP support, no paired Fortran compiler).
Please see the initial section of:
https://urldefense.com/v3/__https://github.com/nds-org/jupyter-et/blob/maste...
for how to use the ET on OSX using homebrew or macports.
If you have a Fortran compiler installed you can try disabling openmp by setting OPENMP=no in your option list. I strongly advise consulting the link I provided above and following the (tested and verified) instructions there in particular if this is your first time using the toolkit.
Yours, Roland
Hi All,
Thanks so much for making the Einstein Toolkit software available!
I'm using the CactusTutorial.ipynb that I got from your site. I'm
trying to
build ET on my 2015 Macbook Pro running OS 10.15.7.
I'm stuck at the cell (after) *Building the Einstein Toolkit* . I modified that cell to get past issues with zlib . The cell now looks
like:
%%bash export ZLIB_DIR=/usr/local/opt/zlib/lib export LDFLAGS="-L/usr/local/opt/zlib/lib" export CPPFLAGS="-I/usr/local/opt/zlib/include" export PKG_CONFIG_PATH="/usr/local/opt/zlib/lib/pkgconfig" ./simfactory/bin/sim build -j2 --thornlist ../einsteintoolkit.th
Running that cell eventually stops with the errors
COMPILING CactusBase/CoordBase/src/Domain.c COMPILING configs/sim/bindings/build/ADMCoupling/cctk_ThornBindings.c clang: error: unsupported option '-fopenmp' make[3]: *** [Domain.c.o] Error 1 make[2]: *** [make.checked] Error 2 make[1]: ***
[/Users/murray/Cactus/configs/sim/lib/libthorn_CoordBase.a] Error 2
make[1]: *** Waiting for unfinished jobs.... clang: error: unsupported option '-fopenmp' make[3]: *** [cctk_ThornBindings.c.o] Error 1 make[2]: *** [cctk_Bindings/make.checked] Error 2 make[1]: ***
[/Users/murray/Cactus/configs/sim/lib/libthorn_ADMCoupling.a]
Error 2 make: *** [sim] Error 2
Can you help? I'm confused by the error message coming from clang,
since
the ET seems to use gcc, but my attempt to force gcc, by including the exports below, made no difference:
export CC=gcc-10 export CXX=gcc-10
I really appreciate any help you are able to provide!
Thank you.
Murray (Wolinsky) ______________________________________________________
Additional (likely irrelevant) notes:
I used brew to install the prerequisites. It seemed to have gone more-or-less ok.
Brew did issue a somewhat concerning message regarding hdf5:
==> Pouring hdf5-1.12.0_1.catalina.bottle.1.tar.gz
Warning: hdf5 dependency gcc was built with a different C++ standard
library (libstdc++ from clang). This may cause problems at runtime.
But it didn't provide any guidance into fixing the potential problem.
Itried
brew reinstall hdf5
and brew no longer complains.
I also had some issues with the subversion installation. But I don't
think
they're responsible for my current problem.
Also, when I did the step *Configuring SimFactory for your machine*, I
got
the output:
Here we will define some necessary Simulation Factory defaults.
Determining local machine name: mw-3.local
--------------------SUMMARY--------------------:
[default] user = murray email = murray allocation = NO_ALLOCATION
------------------END SUMMARY------------------:
Contents successfully written to /Users/murray/Cactus/repos/simfactory2/etc/defs.local.ini
which looks OK, but the NO_ALLOCATION message seems like it might be
of concern?
-- My email is as private as my paper mail. I therefore support encrypting and signing email messages. Get my PGP key from https://urldefense.com/v3/__http://pgp.mit.edu__;!!DZ3fjg!vBkInm5NFkd2yeuitJ... .
Hi Roland,
The toolkit is still building, but it looks like things are working properly and I want to thank you again for your help!
As you suggested, I added a .hostname file and ran ./simfactory/bin/sim build --clean --reconfig . I'm past the -fopenmp issue which plagued me for days! Thank you.
While I was trying to get around the -fopenmp problem, I edited the file /Users/murray/Cactus/configs/sim/OptionList . I'm not sure if that file is input or output. It's input, but it might be both. I changed the version name (added a '_' at the end of the version string); set the compiler and preprocessor directives to use gcc-10, etc.; and turned the OpenMP options off.
When I did the reconfig, the options I set were retained. And it looks like the toolkit is building properly!
I'm going to see if I can build again while turning OpenMP back on, but I'll wait until this build is complete. It's taking a lot longer to build than I expected (my Macbook Pro is old -- early 2015), so I won't know until tomorrow.
Again, thanks so much!
Yours,
Murray
On Tue, Apr 13, 2021 at 11:50 AM Roland Haas rhaas@illinois.edu wrote:
Hello Murray,
ok. Good to hear that.
My initial email was a send a bit off the cuff since I had not quite realized that you were already using the notebook.
OSX is a bit tricky for the ET to support for the reasons that you outlined.
When you run "./simfactory/bin/sim setup-silent" simulation factory will find the gcc-10 compiler and creates a set of of make options for your laptop (there's explicit code in simfactory to recongnize OSX). So running this command is essential.
You should not have to adjust anything (though given that the last test with Catalina rather than BigSur was about a year ago that may not be true anymore if HomeBrew changed too much).
This set of options is tied to your laptop's hostname as reported by "hostname -f". Unfortunately this hostname is not "stable" in the sense that it may change with time since it can be derived from your laptop's IP address obtained from the wireless network. Eg on my test laptop it currently reads:
wirelessprvnat-172-16-210-128.near.illinois.edu
So you may have to check that
"./simfactory/bin/sim whoami"
returns the expected machine name and run "./simfactory/bin/sim setup-silent" again if it reports an unknown machine.
You can also create a file $HOME/.hostname and add a name that you would like simfactory to use for your laptop eg:
echo >$HOME/.hostname "murrays-laptop"
(no spaces allowed).
Yours, Roland
Hi Roland,
Thanks so much for your response! I've been struggling with the -fopenmp issue for a couple of days now. I'll try your suggestions and let you
know
if it works. It's encouraging to know that the ET works with Catalina:
that
gives me hope I'll get it to work eventually.
I'm using the notebook CactusTutorial.ipynb, and I certainly don't want
to
use clang. But the Mac makes using gcc instead of clang tricky. On my machine, gcc gets clang; you need to enter gcc-10 to get gcc.
I verified that gcc (gcc-10) accepts the -fopenmp flag properly. So both gcc and OpenMP seem to work properly on my machine. My problem is convincing ET to use gcc instead of clang! BTW, I'm using brew. It's possible that MacPorts would work better, but I think brew and MacPorts
are
incompatible.
Again, thank you and I'll let you know my progress later today.
Yours,
Murray
On Tue, Apr 13, 2021 at 9:55 AM Roland Haas rhaas@illinois.edu wrote:
Hello Murray,
thinking about this a bit more.
The notebook currently (as of last week) lets me compile and run the ET using Homebrew (and worked on Catalina in the past).
One thing to keep in mind is that the various CC/CXX and other exported env variables are only used the *first* time you attempt to build and not afterwards.
You may have more luck having the options be respected by using:
./simfactory/bin/sim build --clean --reconfig
which will recompile from scratch and re-run the configuration stage or even by manually removing the "configs/sim" directory completely.
Did you run "simfactory/bin/sim setup-silent" before trying to compile?
Yours, Roland
Hello Murray,
clang (in particular Apple's version delivered in OSX) is not fully supported by the ET (due to, at least historically, no OpenMP
support,
no paired Fortran compiler).
Please see the initial section of:
https://urldefense.com/v3/__https://github.com/nds-org/jupyter-et/blob/maste...
for how to use the ET on OSX using homebrew or macports.
If you have a Fortran compiler installed you can try disabling openmp by setting OPENMP=no in your option list. I strongly advise
consulting
the link I provided above and following the (tested and verified) instructions there in particular if this is your first time using the toolkit.
Yours, Roland
Hi All,
Thanks so much for making the Einstein Toolkit software available!
I'm using the CactusTutorial.ipynb that I got from your site. I'm
trying to
build ET on my 2015 Macbook Pro running OS 10.15.7.
I'm stuck at the cell (after) *Building the Einstein Toolkit* . I modified that cell to get past issues with zlib . The cell now
looks
like:
%%bash export ZLIB_DIR=/usr/local/opt/zlib/lib export LDFLAGS="-L/usr/local/opt/zlib/lib" export CPPFLAGS="-I/usr/local/opt/zlib/include" export PKG_CONFIG_PATH="/usr/local/opt/zlib/lib/pkgconfig" ./simfactory/bin/sim build -j2 --thornlist ../einsteintoolkit.th
Running that cell eventually stops with the errors
COMPILING CactusBase/CoordBase/src/Domain.c COMPILING
configs/sim/bindings/build/ADMCoupling/cctk_ThornBindings.c
clang: error: unsupported option '-fopenmp' make[3]: *** [Domain.c.o] Error 1 make[2]: *** [make.checked] Error 2 make[1]: ***
[/Users/murray/Cactus/configs/sim/lib/libthorn_CoordBase.a] Error 2
make[1]: *** Waiting for unfinished jobs.... clang: error: unsupported option '-fopenmp' make[3]: *** [cctk_ThornBindings.c.o] Error 1 make[2]: *** [cctk_Bindings/make.checked] Error 2 make[1]: ***
[/Users/murray/Cactus/configs/sim/lib/libthorn_ADMCoupling.a]
Error 2 make: *** [sim] Error 2
Can you help? I'm confused by the error message coming from
clang,
since
the ET seems to use gcc, but my attempt to force gcc, by including
the
exports below, made no difference:
export CC=gcc-10 export CXX=gcc-10
I really appreciate any help you are able to provide!
Thank you.
Murray (Wolinsky) ______________________________________________________
Additional (likely irrelevant) notes:
I used brew to install the prerequisites. It seemed to have gone more-or-less ok.
Brew did issue a somewhat concerning message regarding hdf5:
==> Pouring hdf5-1.12.0_1.catalina.bottle.1.tar.gz
Warning: hdf5 dependency gcc was built with a different C++
standard
library (libstdc++ from clang). This may cause problems at runtime.
But it didn't provide any guidance into fixing the potential
problem.
Itried
brew reinstall hdf5
and brew no longer complains.
I also had some issues with the subversion installation. But I
don't
think
they're responsible for my current problem.
Also, when I did the step *Configuring SimFactory for your
machine*, I
got
the output:
Here we will define some necessary Simulation Factory defaults.
Determining local machine name: mw-3.local
--------------------SUMMARY--------------------:
[default] user = murray email = murray allocation = NO_ALLOCATION
------------------END SUMMARY------------------:
Contents successfully written to /Users/murray/Cactus/repos/simfactory2/etc/defs.local.ini
which looks OK, but the NO_ALLOCATION message seems like it might
be
of concern?
-- My email is as private as my paper mail. I therefore support encrypting and signing email messages. Get my PGP key from
https://urldefense.com/v3/__http://pgp.mit.edu__;!!DZ3fjg!vBkInm5NFkd2yeuitJ... .
-- My email is as private as my paper mail. I therefore support encrypting and signing email messages. Get my PGP key from http://pgp.mit.edu .
Hello Murray,
As you suggested, I added a .hostname file and ran ./simfactory/bin/sim build --clean --reconfig . I'm past the -fopenmp issue which plagued me for days! Thank you.
Glad to have been able to help.
While I was trying to get around the -fopenmp problem, I edited the file /Users/murray/Cactus/configs/sim/OptionList . I'm not sure if that file is input or output. It's input, but it might be both. I changed the version name (added a '_' at the end of the version string); set the compiler and preprocessor directives to use gcc-10, etc.; and turned the OpenMP options off.
The file is both. When you ran ./simfactory/bin/sim setup-silent then simfactory created a custom option list named generic-10.cfg or so in simfactory/mdb/optionlists/ and a custom machine ini file pointing to that option list in simfactory/mdb/machines/. When you then ran simfactory/bin/sim build it copied that option list to OptionList and used it with Cactus to compile the code. Cactus then generates an almost copy of that file in configs/sim/config-info and eventually a makefile fragment based on that information in configs/sim/config-data/make.config.defn.
Yours, Roland
Hi Roland,
It almost worked -- but not quite. The final link failed because, I believe, of a single missing entry point: "__gfortran_os_error_at" .
The error message is:
Undefined symbols for architecture x86_64: "__gfortran_os_error_at", referenced from: _ahfinder_ in AHFinder.F.o _ahfinder_dis_ in AHFinder_dis.F.o _ahfinder_int_ in AHFinder_int.F.o _linmin_ in AHFinder_pow.F.o _ahfinder_initguess_ in AHFinder_initguess.F.o _ahfinder_gau_ in AHFinder_gau.F.o ___carpet_boxtypes_MOD_splitregionsrecursive in carpet_boxtypes.F90.o ... ld: symbol(s) not found for architecture x86_64 collect2: error: ld returned 1 exit status make[1]: *** [/Users/murray/Cactus/exe/cactus_sim] Error 1 make: *** [sim] Error 2
The compilations completed without errors, but with a fair number of warnings.
I'm googling ("mac gfortran_os_error_at") to see what I can learn. I've found some possibilities, but nothing that seems terribly promising. I brew reinstalled gfortran and I re-downloaded the CactusTutorial.ipynb notebook and I'm trying again.
The file OptionList I used before sets FPP = cpp-10 F90 = gfortran
That file also specifies: FPPFLAGS = -traditional F90FLAGS = -g -fcray-pointer -ffixed-line-length-none F90_OPTIMISE_FLAGS = -O2 F90_WARN_FLAGS = -Wall LDFLAGS = -rdynamic F90_OPENMP_FLAGS = F90_PROFILE_FLAGS = -pg
I get
*which gfortran*
/usr/local/bin/gfortran ❯ *gfortran --version* GNU Fortran (Homebrew GCC 10.2.0_4) 10.2.0 Copyright (C) 2020 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
*which gfortran-10*
/usr/local/bin/gfortran-10 ❯ *gfortran-10 --version* GNU Fortran (Homebrew GCC 10.2.0_4) 10.2.0 Copyright (C) 2020 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
Possibly I should add a link library somewhere, but I don't know what, where or how. (Is it possible I inadvertently deleted a link library specification in the OptionList file or elsewhere? I thought I saved the original file, but I overwrote it, which is part of the reason I'm trying again from the start. The only difference I found from the OptionsList file I used is that the correct file sets F90 = gfortran-10. But gfortran is symlinked to gfortran-10, so I don't expect that'll make a difference.)
Any thoughts on fixing the missing reference?
Again, thanks!
Murray
__________________________________________________________________
The build process takes slightly over 3 hours on my Mac. The final output of the sim build, clean, reconfig run is:
--------------------------------------------- Creating cactus_sim in /Users/murray/Cactus/exe from EinsteinAnalysis/ADMAnalysis EinsteinBase/ADMBase EinsteinBase/ADMCoupling Llama/ADMDerivatives EinsteinBase/ADMMacros EinsteinAnalysis/ADMMass Numerical/AEILocalInterp EinsteinAnalysis/AHFinder EinsteinAnalysis/AHFinderDirect ExternalLibraries/BLAS WVUThorns/Baikal WVUThorns/BaikalVacuum CactusBase/Boundary CTThorns/CT_Analytic CTThorns/CT_MultiLevel EinsteinAnalysis/CalcK Carpet/Carpet Carpet/CarpetEvolutionMask Carpet/CarpetIOASCII Carpet/CarpetIOBasic Carpet/CarpetIOHDF5 Carpet/CarpetIOScalar Carpet/CarpetIntegrateTest Carpet/CarpetInterp Carpet/CarpetInterp2 Carpet/CarpetLib Carpet/CarpetMask Carpet/CarpetProlongateTest Carpet/CarpetReduce Carpet/CarpetRegrid Carpet/CarpetRegrid2 Carpet/CarpetRegridTest Carpet/CarpetSlab Carpet/CarpetTracker CactusBase/CartGrid3D CactusNumerical/Cartoon2D EinsteinBase/Constants WVUThorns/Convert_to_HydroBase CactusBase/CoordBase EinsteinBase/CoordGauge Llama/Coordinates Llama/CoordinatesSymmetry Carpet/CycleClock CactusExamples/DemoInterp CactusNumerical/Dissipation EinsteinInitialData/DistortedBHIVP EinsteinAnalysis/EHFinder EinsteinBase/EOS_Base EinsteinEOS/EOS_Hybrid EinsteinEOS/EOS_IdealFluid EinsteinEOS/EOS_Omni EinsteinEOS/EOS_Polytrope EinsteinExact/EinsteinExact_Test CactusElliptic/EllBase CactusElliptic/EllSOR EinsteinInitialData/Exact EinsteinAnalysis/Extract ExternalLibraries/FFTW3 WVUThorns/FishboneMoncriefID CactusExamples/FleshInfo CactusUtils/Formaline CactusBase/Fortran EinsteinEvolve/GRHydro EinsteinEvolve/GRHydro_InitData ExternalLibraries/GSL EinsteinExact/GaugeWave KrancNumericalTools/GenericFD WVUThorns/GiRaFFE WVUThorns/GiRaFFE_to_HydroBase WVUThorns/GiRaFFEfood Llama/GlobalDerivative ExternalLibraries/HDF5 CactusConnect/HTTPD CactusConnect/HTTPDExtra CactusExamples/HelloWorld Carpet/HighOrderWaveTest EinsteinBase/HydroBase EinsteinAnalysis/Hydro_Analysis EinsteinInitialData/Hydro_InitExcision EinsteinInitialData/Hydro_RNSID EinsteinInitialData/IDAnalyticBH EinsteinInitialData/IDAxiBrillBH EinsteinInitialData/IDAxiOddBrillBH EinsteinInitialData/IDBrillData EinsteinInitialData/IDConstraintViolate EinsteinInitialData/IDFileADM EinsteinInitialData/IDLinearWaves CactusWave/IDScalarWave CactusWave/IDScalarWaveC CactusWave/IDScalarWaveCXX CactusWave/IDScalarWaveElliptic CactusExamples/IDWaveMoL WVUThorns/ID_converter_GiRaFFE WVUThorns/ID_converter_ILGRMHD CactusBase/IOASCII CactusBase/IOBasic CactusPUGHIO/IOHDF5 CactusPUGHIO/IOHDF5Util CactusIO/IOJpeg CactusBase/IOUtil WVUThorns/IllinoisGRMHD CactusBase/InitBase CactusNumerical/InterpToArray Llama/Interpolate2 EinsteinExact/KerrSchild ExternalLibraries/LAPACK ExternalLibraries/LORENE Lean/LeanBSSNMoL Llama/LlamaWaveToy CactusNumerical/LocalInterp CactusNumerical/LocalInterp2 CactusNumerical/LocalReduce Carpet/LoopControl McLachlan/ML_ADMConstraints McLachlan/ML_ADMQuantities McLachlan/ML_BSSN McLachlan/ML_BSSN_Helper McLachlan/ML_BSSN_Test McLachlan/ML_CCZ4 McLachlan/ML_CCZ4_Helper McLachlan/ML_CCZ4_Test McLachlan/ML_WaveToy McLachlan/ML_WaveToy_Test ExternalLibraries/MPI CactusUtils/MemSpeed EinsteinInitialData/Meudon_Bin_BH EinsteinInitialData/Meudon_Bin_NS EinsteinInitialData/Meudon_Mag_NS EinsteinExact/Minkowski CactusNumerical/MoL EinsteinExact/ModifiedSchwarzschildBL EinsteinAnalysis/Multipole Lean/NPScalars Proca/NPScalars_Proca CactusUtils/NaNCatcher CactusUtils/NaNChecker EinsteinEvolve/NewRad CactusUtils/Nice EinsteinInitialData/NoExcision CactusUtils/NoMPI CactusNumerical/Noise CactusNumerical/Norms PITTNullCode/NullConstr PITTNullCode/NullDecomp PITTNullCode/NullEvolve PITTNullCode/NullExact PITTNullCode/NullGrid PITTNullCode/NullInterp PITTNullCode/NullNews PITTNullCode/NullPsiInt PITTNullCode/NullSHRExtract PITTNullCode/NullVars ExternalLibraries/OpenSSL EinsteinAnalysis/Outflow ExternalLibraries/PAPI CactusPUGH/PUGH CactusPUGH/PUGHInterp CactusPUGH/PUGHReduce CactusPUGH/PUGHSlab CactusNumerical/Periodic Carpet/PeriodicCarpet CactusExamples/Poisson Proca/ProcaBase Proca/ProcaEvolve Proca/Proca_simpleID EinsteinAnalysis/PunctureTracker EinsteinAnalysis/QuasiLocalMeasures EinsteinInitialData/ReadInterpolate Carpet/ReductionTest Carpet/ReductionTest2 Carpet/ReductionTest3 CactusNumerical/ReflectionSymmetry Carpet/RegridSyncTest EinsteinInitialData/RotatingDBHIVP CactusNumerical/RotatingSymmetry180 CactusNumerical/RotatingSymmetry90 CactusExamples/SampleBoundary CactusExamples/SampleIO WVUThorns/Seed_Magnetic_Fields WVUThorns_Diagnostics/Seed_Magnetic_Fields_BNS EinsteinUtils/SetMask_SphericalSurface EinsteinExact/ShiftedGaugeWave WVUThorns/ShiftedKerrSchild CactusNumerical/Slab CactusNumerical/SlabTest CactusConnect/Socket CactusNumerical/SpaceMask PITTNullCode/SphericalHarmonicDecomp PITTNullCode/SphericalHarmonicRecon PITTNullCode/SphericalHarmonicReconGen CactusNumerical/SphericalSurface EinsteinBase/StaticConformal CactusNumerical/SummationByParts CactusBase/SymBase CactusUtils/SystemStatistics CactusUtils/SystemTopology CactusElliptic/TATelliptic EinsteinUtils/TGRtensor EinsteinInitialData/TOVSolver CactusNumerical/TensorTypes CactusUtils/TerminationTrigger CactusTest/TestArrays Carpet/TestCarpetGridInfo CactusTest/TestComplex CactusTest/TestCoordinates CactusTest/TestFortranCrayPointers CactusTest/TestFortranDependencies1 CactusTest/TestFortranDependencies2 CactusTest/TestFpointerNULL CactusTest/TestFreeF90 CactusTest/TestGlobalReduce CactusTest/TestInclude1 CactusTest/TestInclude2 CactusNumerical/TestLocalInterp2 CactusNumerical/TestLocalReduce CactusTest/TestLoop Carpet/TestLoopControl CactusTest/TestMath CactusTest/TestMoL CactusTest/TestPar CactusTest/TestReadWrite CactusTest/TestReduce CactusTest/TestSchedule CactusTest/TestStrings CactusTest/TestTable CactusTest/TestTimers CactusTest/TestTypes CactusBase/Time CactusExamples/TimerInfo CactusUtils/TimerReport Carpet/Timers EinsteinBase/TmunuBase CactusUtils/Trigger EinsteinInitialData/TwoPunctures Proca/TwoPunctures_KerrProca EinsteinExact/Vaidya2 CactusUtils/Vectors WVUThorns_Diagnostics/VolumeIntegrals_GRMHD WVUThorns_Diagnostics/VolumeIntegrals_vacuum CactusUtils/WatchDog CactusWave/WaveBinarySource Llama/WaveExtractL CactusExamples/WaveMoL CactusExamples/WaveToy1DF77 CactusExamples/WaveToy2DF77 CactusWave/WaveToyC CactusWave/WaveToyCXX CactusWave/WaveToyExtra CactusWave/WaveToyF77 CactusWave/WaveToyF90 CactusWave/WaveToyFreeF90 EinsteinAnalysis/WeylScal4 ExternalLibraries/hwloc ExternalLibraries/libjpeg WVUThorns_Diagnostics/particle_tracerET ExternalLibraries/pthreads WVUThorns_Diagnostics/smallbPoynET ExternalLibraries/zlib Undefined symbols for architecture x86_64: "__gfortran_os_error_at", referenced from: _ahfinder_ in AHFinder.F.o _ahfinder_dis_ in AHFinder_dis.F.o _ahfinder_int_ in AHFinder_int.F.o _linmin_ in AHFinder_pow.F.o _ahfinder_initguess_ in AHFinder_initguess.F.o _ahfinder_gau_ in AHFinder_gau.F.o ___carpet_boxtypes_MOD_splitregionsrecursive in carpet_boxtypes.F90.o ... ld: symbol(s) not found for architecture x86_64 collect2: error: ld returned 1 exit status make[1]: *** [/Users/murray/Cactus/exe/cactus_sim] Error 1 make: *** [sim] Error 2
============================================================ On Tue, Apr 13, 2021 at 11:50 AM Roland Haas rhaas@illinois.edu wrote:
Hello Murray,
ok. Good to hear that.
My initial email was a send a bit off the cuff since I had not quite realized that you were already using the notebook.
OSX is a bit tricky for the ET to support for the reasons that you outlined.
When you run "./simfactory/bin/sim setup-silent" simulation factory will find the gcc-10 compiler and creates a set of of make options for your laptop (there's explicit code in simfactory to recongnize OSX). So running this command is essential.
You should not have to adjust anything (though given that the last test with Catalina rather than BigSur was about a year ago that may not be true anymore if HomeBrew changed too much).
This set of options is tied to your laptop's hostname as reported by "hostname -f". Unfortunately this hostname is not "stable" in the sense that it may change with time since it can be derived from your laptop's IP address obtained from the wireless network. Eg on my test laptop it currently reads:
wirelessprvnat-172-16-210-128.near.illinois.edu
So you may have to check that
"./simfactory/bin/sim whoami"
returns the expected machine name and run "./simfactory/bin/sim setup-silent" again if it reports an unknown machine.
You can also create a file $HOME/.hostname and add a name that you would like simfactory to use for your laptop eg:
echo >$HOME/.hostname "murrays-laptop"
(no spaces allowed).
Yours, Roland
Hi Roland,
Thanks so much for your response! I've been struggling with the -fopenmp issue for a couple of days now. I'll try your suggestions and let you
know
if it works. It's encouraging to know that the ET works with Catalina:
that
gives me hope I'll get it to work eventually.
I'm using the notebook CactusTutorial.ipynb, and I certainly don't want
to
use clang. But the Mac makes using gcc instead of clang tricky. On my machine, gcc gets clang; you need to enter gcc-10 to get gcc.
I verified that gcc (gcc-10) accepts the -fopenmp flag properly. So both gcc and OpenMP seem to work properly on my machine. My problem is convincing ET to use gcc instead of clang! BTW, I'm using brew. It's possible that MacPorts would work better, but I think brew and MacPorts
are
incompatible.
Again, thank you and I'll let you know my progress later today.
Yours,
Murray
On Tue, Apr 13, 2021 at 9:55 AM Roland Haas rhaas@illinois.edu wrote:
Hello Murray,
thinking about this a bit more.
The notebook currently (as of last week) lets me compile and run the ET using Homebrew (and worked on Catalina in the past).
One thing to keep in mind is that the various CC/CXX and other exported env variables are only used the *first* time you attempt to build and not afterwards.
You may have more luck having the options be respected by using:
./simfactory/bin/sim build --clean --reconfig
which will recompile from scratch and re-run the configuration stage or even by manually removing the "configs/sim" directory completely.
Did you run "simfactory/bin/sim setup-silent" before trying to compile?
Yours, Roland
Hello Murray,
clang (in particular Apple's version delivered in OSX) is not fully supported by the ET (due to, at least historically, no OpenMP
support,
no paired Fortran compiler).
Please see the initial section of:
https://urldefense.com/v3/__https://github.com/nds-org/jupyter-et/blob/maste...
for how to use the ET on OSX using homebrew or macports.
If you have a Fortran compiler installed you can try disabling openmp by setting OPENMP=no in your option list. I strongly advise
consulting
the link I provided above and following the (tested and verified) instructions there in particular if this is your first time using the toolkit.
Yours, Roland
Hi All,
Thanks so much for making the Einstein Toolkit software available!
I'm using the CactusTutorial.ipynb that I got from your site. I'm
trying to
build ET on my 2015 Macbook Pro running OS 10.15.7.
I'm stuck at the cell (after) *Building the Einstein Toolkit* . I modified that cell to get past issues with zlib . The cell now
looks
like:
%%bash export ZLIB_DIR=/usr/local/opt/zlib/lib export LDFLAGS="-L/usr/local/opt/zlib/lib" export CPPFLAGS="-I/usr/local/opt/zlib/include" export PKG_CONFIG_PATH="/usr/local/opt/zlib/lib/pkgconfig" ./simfactory/bin/sim build -j2 --thornlist ../einsteintoolkit.th
Running that cell eventually stops with the errors
COMPILING CactusBase/CoordBase/src/Domain.c COMPILING
configs/sim/bindings/build/ADMCoupling/cctk_ThornBindings.c
clang: error: unsupported option '-fopenmp' make[3]: *** [Domain.c.o] Error 1 make[2]: *** [make.checked] Error 2 make[1]: ***
[/Users/murray/Cactus/configs/sim/lib/libthorn_CoordBase.a] Error 2
make[1]: *** Waiting for unfinished jobs.... clang: error: unsupported option '-fopenmp' make[3]: *** [cctk_ThornBindings.c.o] Error 1 make[2]: *** [cctk_Bindings/make.checked] Error 2 make[1]: ***
[/Users/murray/Cactus/configs/sim/lib/libthorn_ADMCoupling.a]
Error 2 make: *** [sim] Error 2
Can you help? I'm confused by the error message coming from
clang,
since
the ET seems to use gcc, but my attempt to force gcc, by including
the
exports below, made no difference:
export CC=gcc-10 export CXX=gcc-10
I really appreciate any help you are able to provide!
Thank you.
Murray (Wolinsky) ______________________________________________________
Additional (likely irrelevant) notes:
I used brew to install the prerequisites. It seemed to have gone more-or-less ok.
Brew did issue a somewhat concerning message regarding hdf5:
==> Pouring hdf5-1.12.0_1.catalina.bottle.1.tar.gz
Warning: hdf5 dependency gcc was built with a different C++
standard
library (libstdc++ from clang). This may cause problems at runtime.
But it didn't provide any guidance into fixing the potential
problem.
Itried
brew reinstall hdf5
and brew no longer complains.
I also had some issues with the subversion installation. But I
don't
think
they're responsible for my current problem.
Also, when I did the step *Configuring SimFactory for your
machine*, I
got
the output:
Here we will define some necessary Simulation Factory defaults.
Determining local machine name: mw-3.local
--------------------SUMMARY--------------------:
[default] user = murray email = murray allocation = NO_ALLOCATION
------------------END SUMMARY------------------:
Contents successfully written to /Users/murray/Cactus/repos/simfactory2/etc/defs.local.ini
which looks OK, but the NO_ALLOCATION message seems like it might
be
of concern?
-- My email is as private as my paper mail. I therefore support encrypting and signing email messages. Get my PGP key from
https://urldefense.com/v3/__http://pgp.mit.edu__;!!DZ3fjg!vBkInm5NFkd2yeuitJ... .
-- My email is as private as my paper mail. I therefore support encrypting and signing email messages. Get my PGP key from http://pgp.mit.edu .
Hello Murray,
this may be caused by having set LDFLAGS or LIBS or similar. Since Cactus combine C, C++ and Fortran code and links using the g++ wrapper one must manually list the Fotran runtime library "gfortran" as a library to link.
It should figure that out on its own when it detects the compiler in use at configure time (unless you got something like "Cactus has never been build on this architecture" in which case somehow it failed to recognize the OS / compiler).
You can probably fix this by adding:
LIBS=gfortran
to your OptionList file in configs/sim/ the run
./simfactory/bin/sim build --reconfig
(--clean should not be necessary).
I am not quite sure how ended up with this error if you used the files provided by simfactory (can you include your OptionList file and the file configs/sim/config-data/make.config.defn as an email attachment?), though.
Do you happen to have a environment variable "LIBS" set, such a variable would most likely confuse things?
Yours, Roland
Hi Roland,
It almost worked -- but not quite. The final link failed because, I believe, of a single missing entry point: "__gfortran_os_error_at" .
The error message is:
Undefined symbols for architecture x86_64: "__gfortran_os_error_at", referenced from: _ahfinder_ in AHFinder.F.o _ahfinder_dis_ in AHFinder_dis.F.o _ahfinder_int_ in AHFinder_int.F.o _linmin_ in AHFinder_pow.F.o _ahfinder_initguess_ in AHFinder_initguess.F.o _ahfinder_gau_ in AHFinder_gau.F.o ___carpet_boxtypes_MOD_splitregionsrecursive in carpet_boxtypes.F90.o ... ld: symbol(s) not found for architecture x86_64 collect2: error: ld returned 1 exit status make[1]: *** [/Users/murray/Cactus/exe/cactus_sim] Error 1 make: *** [sim] Error 2
The compilations completed without errors, but with a fair number of warnings.
I'm googling ("mac gfortran_os_error_at") to see what I can learn. I've found some possibilities, but nothing that seems terribly promising. I brew reinstalled gfortran and I re-downloaded the CactusTutorial.ipynb notebook and I'm trying again.
The file OptionList I used before sets FPP = cpp-10 F90 = gfortran
That file also specifies: FPPFLAGS = -traditional F90FLAGS = -g -fcray-pointer -ffixed-line-length-none F90_OPTIMISE_FLAGS = -O2 F90_WARN_FLAGS = -Wall LDFLAGS = -rdynamic F90_OPENMP_FLAGS = F90_PROFILE_FLAGS = -pg
I get
*which gfortran*
/usr/local/bin/gfortran ❯ *gfortran --version* GNU Fortran (Homebrew GCC 10.2.0_4) 10.2.0 Copyright (C) 2020 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
*which gfortran-10*
/usr/local/bin/gfortran-10 ❯ *gfortran-10 --version* GNU Fortran (Homebrew GCC 10.2.0_4) 10.2.0 Copyright (C) 2020 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
Possibly I should add a link library somewhere, but I don't know what, where or how. (Is it possible I inadvertently deleted a link library specification in the OptionList file or elsewhere? I thought I saved the original file, but I overwrote it, which is part of the reason I'm trying again from the start. The only difference I found from the OptionsList file I used is that the correct file sets F90 = gfortran-10. But gfortran is symlinked to gfortran-10, so I don't expect that'll make a difference.)
Any thoughts on fixing the missing reference?
Again, thanks!
Murray
The build process takes slightly over 3 hours on my Mac. The final output of the sim build, clean, reconfig run is:
Creating cactus_sim in /Users/murray/Cactus/exe from EinsteinAnalysis/ADMAnalysis EinsteinBase/ADMBase EinsteinBase/ADMCoupling Llama/ADMDerivatives EinsteinBase/ADMMacros EinsteinAnalysis/ADMMass Numerical/AEILocalInterp EinsteinAnalysis/AHFinder EinsteinAnalysis/AHFinderDirect ExternalLibraries/BLAS WVUThorns/Baikal WVUThorns/BaikalVacuum CactusBase/Boundary CTThorns/CT_Analytic CTThorns/CT_MultiLevel EinsteinAnalysis/CalcK Carpet/Carpet Carpet/CarpetEvolutionMask Carpet/CarpetIOASCII Carpet/CarpetIOBasic Carpet/CarpetIOHDF5 Carpet/CarpetIOScalar Carpet/CarpetIntegrateTest Carpet/CarpetInterp Carpet/CarpetInterp2 Carpet/CarpetLib Carpet/CarpetMask Carpet/CarpetProlongateTest Carpet/CarpetReduce Carpet/CarpetRegrid Carpet/CarpetRegrid2 Carpet/CarpetRegridTest Carpet/CarpetSlab Carpet/CarpetTracker CactusBase/CartGrid3D CactusNumerical/Cartoon2D EinsteinBase/Constants WVUThorns/Convert_to_HydroBase CactusBase/CoordBase EinsteinBase/CoordGauge Llama/Coordinates Llama/CoordinatesSymmetry Carpet/CycleClock CactusExamples/DemoInterp CactusNumerical/Dissipation EinsteinInitialData/DistortedBHIVP EinsteinAnalysis/EHFinder EinsteinBase/EOS_Base EinsteinEOS/EOS_Hybrid EinsteinEOS/EOS_IdealFluid EinsteinEOS/EOS_Omni EinsteinEOS/EOS_Polytrope EinsteinExact/EinsteinExact_Test CactusElliptic/EllBase CactusElliptic/EllSOR EinsteinInitialData/Exact EinsteinAnalysis/Extract ExternalLibraries/FFTW3 WVUThorns/FishboneMoncriefID CactusExamples/FleshInfo CactusUtils/Formaline CactusBase/Fortran EinsteinEvolve/GRHydro EinsteinEvolve/GRHydro_InitData ExternalLibraries/GSL EinsteinExact/GaugeWave KrancNumericalTools/GenericFD WVUThorns/GiRaFFE WVUThorns/GiRaFFE_to_HydroBase WVUThorns/GiRaFFEfood Llama/GlobalDerivative ExternalLibraries/HDF5 CactusConnect/HTTPD CactusConnect/HTTPDExtra CactusExamples/HelloWorld Carpet/HighOrderWaveTest EinsteinBase/HydroBase EinsteinAnalysis/Hydro_Analysis EinsteinInitialData/Hydro_InitExcision EinsteinInitialData/Hydro_RNSID EinsteinInitialData/IDAnalyticBH EinsteinInitialData/IDAxiBrillBH EinsteinInitialData/IDAxiOddBrillBH EinsteinInitialData/IDBrillData EinsteinInitialData/IDConstraintViolate EinsteinInitialData/IDFileADM EinsteinInitialData/IDLinearWaves CactusWave/IDScalarWave CactusWave/IDScalarWaveC CactusWave/IDScalarWaveCXX CactusWave/IDScalarWaveElliptic CactusExamples/IDWaveMoL WVUThorns/ID_converter_GiRaFFE WVUThorns/ID_converter_ILGRMHD CactusBase/IOASCII CactusBase/IOBasic CactusPUGHIO/IOHDF5 CactusPUGHIO/IOHDF5Util CactusIO/IOJpeg CactusBase/IOUtil WVUThorns/IllinoisGRMHD CactusBase/InitBase CactusNumerical/InterpToArray Llama/Interpolate2 EinsteinExact/KerrSchild ExternalLibraries/LAPACK ExternalLibraries/LORENE Lean/LeanBSSNMoL Llama/LlamaWaveToy CactusNumerical/LocalInterp CactusNumerical/LocalInterp2 CactusNumerical/LocalReduce Carpet/LoopControl McLachlan/ML_ADMConstraints McLachlan/ML_ADMQuantities McLachlan/ML_BSSN McLachlan/ML_BSSN_Helper McLachlan/ML_BSSN_Test McLachlan/ML_CCZ4 McLachlan/ML_CCZ4_Helper McLachlan/ML_CCZ4_Test McLachlan/ML_WaveToy McLachlan/ML_WaveToy_Test ExternalLibraries/MPI CactusUtils/MemSpeed EinsteinInitialData/Meudon_Bin_BH EinsteinInitialData/Meudon_Bin_NS EinsteinInitialData/Meudon_Mag_NS EinsteinExact/Minkowski CactusNumerical/MoL EinsteinExact/ModifiedSchwarzschildBL EinsteinAnalysis/Multipole Lean/NPScalars Proca/NPScalars_Proca CactusUtils/NaNCatcher CactusUtils/NaNChecker EinsteinEvolve/NewRad CactusUtils/Nice EinsteinInitialData/NoExcision CactusUtils/NoMPI CactusNumerical/Noise CactusNumerical/Norms PITTNullCode/NullConstr PITTNullCode/NullDecomp PITTNullCode/NullEvolve PITTNullCode/NullExact PITTNullCode/NullGrid PITTNullCode/NullInterp PITTNullCode/NullNews PITTNullCode/NullPsiInt PITTNullCode/NullSHRExtract PITTNullCode/NullVars ExternalLibraries/OpenSSL EinsteinAnalysis/Outflow ExternalLibraries/PAPI CactusPUGH/PUGH CactusPUGH/PUGHInterp CactusPUGH/PUGHReduce CactusPUGH/PUGHSlab CactusNumerical/Periodic Carpet/PeriodicCarpet CactusExamples/Poisson Proca/ProcaBase Proca/ProcaEvolve Proca/Proca_simpleID EinsteinAnalysis/PunctureTracker EinsteinAnalysis/QuasiLocalMeasures EinsteinInitialData/ReadInterpolate Carpet/ReductionTest Carpet/ReductionTest2 Carpet/ReductionTest3 CactusNumerical/ReflectionSymmetry Carpet/RegridSyncTest EinsteinInitialData/RotatingDBHIVP CactusNumerical/RotatingSymmetry180 CactusNumerical/RotatingSymmetry90 CactusExamples/SampleBoundary CactusExamples/SampleIO WVUThorns/Seed_Magnetic_Fields WVUThorns_Diagnostics/Seed_Magnetic_Fields_BNS EinsteinUtils/SetMask_SphericalSurface EinsteinExact/ShiftedGaugeWave WVUThorns/ShiftedKerrSchild CactusNumerical/Slab CactusNumerical/SlabTest CactusConnect/Socket CactusNumerical/SpaceMask PITTNullCode/SphericalHarmonicDecomp PITTNullCode/SphericalHarmonicRecon PITTNullCode/SphericalHarmonicReconGen CactusNumerical/SphericalSurface EinsteinBase/StaticConformal CactusNumerical/SummationByParts CactusBase/SymBase CactusUtils/SystemStatistics CactusUtils/SystemTopology CactusElliptic/TATelliptic EinsteinUtils/TGRtensor EinsteinInitialData/TOVSolver CactusNumerical/TensorTypes CactusUtils/TerminationTrigger CactusTest/TestArrays Carpet/TestCarpetGridInfo CactusTest/TestComplex CactusTest/TestCoordinates CactusTest/TestFortranCrayPointers CactusTest/TestFortranDependencies1 CactusTest/TestFortranDependencies2 CactusTest/TestFpointerNULL CactusTest/TestFreeF90 CactusTest/TestGlobalReduce CactusTest/TestInclude1 CactusTest/TestInclude2 CactusNumerical/TestLocalInterp2 CactusNumerical/TestLocalReduce CactusTest/TestLoop Carpet/TestLoopControl CactusTest/TestMath CactusTest/TestMoL CactusTest/TestPar CactusTest/TestReadWrite CactusTest/TestReduce CactusTest/TestSchedule CactusTest/TestStrings CactusTest/TestTable CactusTest/TestTimers CactusTest/TestTypes CactusBase/Time CactusExamples/TimerInfo CactusUtils/TimerReport Carpet/Timers EinsteinBase/TmunuBase CactusUtils/Trigger EinsteinInitialData/TwoPunctures Proca/TwoPunctures_KerrProca EinsteinExact/Vaidya2 CactusUtils/Vectors WVUThorns_Diagnostics/VolumeIntegrals_GRMHD WVUThorns_Diagnostics/VolumeIntegrals_vacuum CactusUtils/WatchDog CactusWave/WaveBinarySource Llama/WaveExtractL CactusExamples/WaveMoL CactusExamples/WaveToy1DF77 CactusExamples/WaveToy2DF77 CactusWave/WaveToyC CactusWave/WaveToyCXX CactusWave/WaveToyExtra CactusWave/WaveToyF77 CactusWave/WaveToyF90 CactusWave/WaveToyFreeF90 EinsteinAnalysis/WeylScal4 ExternalLibraries/hwloc ExternalLibraries/libjpeg WVUThorns_Diagnostics/particle_tracerET ExternalLibraries/pthreads WVUThorns_Diagnostics/smallbPoynET ExternalLibraries/zlib Undefined symbols for architecture x86_64: "__gfortran_os_error_at", referenced from: _ahfinder_ in AHFinder.F.o _ahfinder_dis_ in AHFinder_dis.F.o _ahfinder_int_ in AHFinder_int.F.o _linmin_ in AHFinder_pow.F.o _ahfinder_initguess_ in AHFinder_initguess.F.o _ahfinder_gau_ in AHFinder_gau.F.o ___carpet_boxtypes_MOD_splitregionsrecursive in carpet_boxtypes.F90.o ... ld: symbol(s) not found for architecture x86_64 collect2: error: ld returned 1 exit status make[1]: *** [/Users/murray/Cactus/exe/cactus_sim] Error 1 make: *** [sim] Error 2
============================================================ On Tue, Apr 13, 2021 at 11:50 AM Roland Haas rhaas@illinois.edu wrote:
Hello Murray,
ok. Good to hear that.
My initial email was a send a bit off the cuff since I had not quite realized that you were already using the notebook.
OSX is a bit tricky for the ET to support for the reasons that you outlined.
When you run "./simfactory/bin/sim setup-silent" simulation factory will find the gcc-10 compiler and creates a set of of make options for your laptop (there's explicit code in simfactory to recongnize OSX). So running this command is essential.
You should not have to adjust anything (though given that the last test with Catalina rather than BigSur was about a year ago that may not be true anymore if HomeBrew changed too much).
This set of options is tied to your laptop's hostname as reported by "hostname -f". Unfortunately this hostname is not "stable" in the sense that it may change with time since it can be derived from your laptop's IP address obtained from the wireless network. Eg on my test laptop it currently reads:
wirelessprvnat-172-16-210-128.near.illinois.edu
So you may have to check that
"./simfactory/bin/sim whoami"
returns the expected machine name and run "./simfactory/bin/sim setup-silent" again if it reports an unknown machine.
You can also create a file $HOME/.hostname and add a name that you would like simfactory to use for your laptop eg:
echo >$HOME/.hostname "murrays-laptop"
(no spaces allowed).
Yours, Roland
Hi Roland,
Thanks so much for your response! I've been struggling with the -fopenmp issue for a couple of days now. I'll try your suggestions and let you
know
if it works. It's encouraging to know that the ET works with Catalina:
that
gives me hope I'll get it to work eventually.
I'm using the notebook CactusTutorial.ipynb, and I certainly don't want
to
use clang. But the Mac makes using gcc instead of clang tricky. On my machine, gcc gets clang; you need to enter gcc-10 to get gcc.
I verified that gcc (gcc-10) accepts the -fopenmp flag properly. So both gcc and OpenMP seem to work properly on my machine. My problem is convincing ET to use gcc instead of clang! BTW, I'm using brew. It's possible that MacPorts would work better, but I think brew and MacPorts
are
incompatible.
Again, thank you and I'll let you know my progress later today.
Yours,
Murray
On Tue, Apr 13, 2021 at 9:55 AM Roland Haas rhaas@illinois.edu wrote:
Hello Murray,
thinking about this a bit more.
The notebook currently (as of last week) lets me compile and run the ET using Homebrew (and worked on Catalina in the past).
One thing to keep in mind is that the various CC/CXX and other exported env variables are only used the *first* time you attempt to build and not afterwards.
You may have more luck having the options be respected by using:
./simfactory/bin/sim build --clean --reconfig
which will recompile from scratch and re-run the configuration stage or even by manually removing the "configs/sim" directory completely.
Did you run "simfactory/bin/sim setup-silent" before trying to compile?
Yours, Roland
Hello Murray,
clang (in particular Apple's version delivered in OSX) is not fully supported by the ET (due to, at least historically, no OpenMP
support,
no paired Fortran compiler).
Please see the initial section of:
https://urldefense.com/v3/__https://github.com/nds-org/jupyter-et/blob/maste...
for how to use the ET on OSX using homebrew or macports.
If you have a Fortran compiler installed you can try disabling openmp by setting OPENMP=no in your option list. I strongly advise
consulting
the link I provided above and following the (tested and verified) instructions there in particular if this is your first time using the toolkit.
Yours, Roland
Hi All,
Thanks so much for making the Einstein Toolkit software available!
I'm using the CactusTutorial.ipynb that I got from your site. I'm
trying to
build ET on my 2015 Macbook Pro running OS 10.15.7.
I'm stuck at the cell (after) *Building the Einstein Toolkit* . I modified that cell to get past issues with zlib . The cell now
looks
like:
%%bash export ZLIB_DIR=/usr/local/opt/zlib/lib export LDFLAGS="-L/usr/local/opt/zlib/lib" export CPPFLAGS="-I/usr/local/opt/zlib/include" export PKG_CONFIG_PATH="/usr/local/opt/zlib/lib/pkgconfig" ./simfactory/bin/sim build -j2 --thornlist ../einsteintoolkit.th
Running that cell eventually stops with the errors
COMPILING CactusBase/CoordBase/src/Domain.c COMPILING
configs/sim/bindings/build/ADMCoupling/cctk_ThornBindings.c
clang: error: unsupported option '-fopenmp' make[3]: *** [Domain.c.o] Error 1 make[2]: *** [make.checked] Error 2 make[1]: ***
[/Users/murray/Cactus/configs/sim/lib/libthorn_CoordBase.a] Error 2
make[1]: *** Waiting for unfinished jobs.... clang: error: unsupported option '-fopenmp' make[3]: *** [cctk_ThornBindings.c.o] Error 1 make[2]: *** [cctk_Bindings/make.checked] Error 2 make[1]: ***
[/Users/murray/Cactus/configs/sim/lib/libthorn_ADMCoupling.a]
Error 2 make: *** [sim] Error 2
Can you help? I'm confused by the error message coming from
clang,
since
the ET seems to use gcc, but my attempt to force gcc, by including
the
exports below, made no difference:
export CC=gcc-10 export CXX=gcc-10
I really appreciate any help you are able to provide!
Thank you.
Murray (Wolinsky) ______________________________________________________
Additional (likely irrelevant) notes:
I used brew to install the prerequisites. It seemed to have gone more-or-less ok.
Brew did issue a somewhat concerning message regarding hdf5:
==> Pouring hdf5-1.12.0_1.catalina.bottle.1.tar.gz
Warning: hdf5 dependency gcc was built with a different C++
standard
library (libstdc++ from clang). This may cause problems at runtime.
But it didn't provide any guidance into fixing the potential
problem.
Itried
brew reinstall hdf5
and brew no longer complains.
I also had some issues with the subversion installation. But I
don't
think
they're responsible for my current problem.
Also, when I did the step *Configuring SimFactory for your
machine*, I
got
the output:
Here we will define some necessary Simulation Factory defaults.
Determining local machine name: mw-3.local
--------------------SUMMARY--------------------:
[default] user = murray email = murray allocation = NO_ALLOCATION
------------------END SUMMARY------------------:
Contents successfully written to /Users/murray/Cactus/repos/simfactory2/etc/defs.local.ini
which looks OK, but the NO_ALLOCATION message seems like it might
be
of concern?
-- My email is as private as my paper mail. I therefore support encrypting and signing email messages. Get my PGP key from
https://urldefense.com/v3/__http://pgp.mit.edu__;!!DZ3fjg!vBkInm5NFkd2yeuitJ... .
-- My email is as private as my paper mail. I therefore support encrypting and signing email messages. Get my PGP key from https://urldefense.com/v3/__http://pgp.mit.edu__;!!DZ3fjg!rWmlL2OXUBrkT9rQ9m... .
Hi Roland,
Again, thank you!
I added LIBS=gfortran to the OptionList file and re-ran. It didn't work. Same error -- but much, much faster!
I checked (echo $LIBS) in my console and the variable is *not* set there.
(I deleted the remainder of our email thread -- way too long -- and we both have all the text in previous emails.)
I attached the two files you requested. The OptionsList file has the LIBS=gfortran statement added, and it appears in the listing when I re-ran.
I believe the variable LIBS (and perhaps LIBDIRS) is supposed to be set in the file
./repos/flesh/lib/make/known-architectures/darwin
That file is very old (2004) and, admittedly I'm not a shell programming guy, but it contains lines which may no longer work. Specifically, if you look at
FPP='/usr/bin/cpp' FPP_VERSION="`$FPP --version | awk '{print $3; exit}'`"
then FPP_VERSION will not pick up the version string for the HomeBrew install. Because when I enter ❯ */usr/local/bin/cpp-10 --version*
I get
cpp-10 (Homebrew GCC 10.2.0_4) 10.2.0 Copyright (C) 2020 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
so the version string is no longer the 3rd field in the line as the awk command seems to require.
Of course this is probably irrelevant -- the file may no longer be used. I played around with awk -- which I know little about -- and came up with a replacement for the FPP_VERSION line above which seems to work. It clearly isn't enough by itself to update the file darwin (and was likely a waste of time -- although somewhat educational). Specifically,
echo -e "`$FPP --version | awk 'match ($0, /[0-9]+.[0-9]+.[0-9]+/){print substr($0,RSTART,RLENGTH); exit}'`"
seems to work if FPP = /usr/local/bin/cpp-10
Be all that as it may, I wonder if I should add a line like LIBDIRS = /usr/local/lib/libgfortran.a to the OptionList file?
(Because when I enter *❯ locate libgfortran.a* I get /usr/local/Cellar/gcc/10.2.0_4.reinstall/lib/gcc/10/libgfortran.a /usr/local/lib/i386/libgfortran.a /usr/local/lib/libgfortran.a )
What do you suggest?
I really appreciate all your help! Thank you!
Murray
On Tue, Apr 13, 2021 at 6:41 PM Roland Haas rhaas@illinois.edu wrote:
Hello Murray,
this may be caused by having set LDFLAGS or LIBS or similar. Since Cactus combine C, C++ and Fortran code and links using the g++ wrapper one must manually list the Fotran runtime library "gfortran" as a library to link.
It should figure that out on its own when it detects the compiler in use at configure time (unless you got something like "Cactus has never been build on this architecture" in which case somehow it failed to recognize the OS / compiler).
You can probably fix this by adding:
LIBS=gfortran
to your OptionList file in configs/sim/ the run
./simfactory/bin/sim build --reconfig
(--clean should not be necessary).
I am not quite sure how ended up with this error if you used the files provided by simfactory (can you include your OptionList file and the file configs/sim/config-data/make.config.defn as an email attachment?), though.
Do you happen to have a environment variable "LIBS" set, such a variable would most likely confuse things?
Yours, Roland
users@lists.einsteintoolkit.org
-
Murray Wolinsky -
Roland Haas