Dear Roland,
thank you for gathering these links, they are indeed useful to point me in the right direction.
I also found this seminar talk by Erik Schnetter on using Spack to install the ET dependencies. I also copy it in this thread, so that it is grouped with the other resources you gathered:
https://einsteintoolkit.org/seminars/2022_05_19/index.html
I will work my way through this material and let you know if I encounter other problems.
Best, Panagiotis
________________________________ From: Roland Haas Sent: Friday, February 7, 2025 6:41 PM To: IOSIF PANAGIOTIS Cc: users@einsteintoolkit.org Subject: Re: [Users] installation of ET on a cluster
Hello Panagiotis,
Thank you for your inquiry.
This may help:
http://einsteintoolkit.org/seminars/2022_02_24/index.html
https://youtu.be/8dnOfQIk32c?feature=shared
whose slides are:
https://github.com/rhaas80/2024_LSU/blob/main/CactusBuildSytemTour.ipynb
There's also some advanced simulation factory docs here:
https://docs.einsteintoolkit.org/et-docs/Configuring_a_new_machine
(linked on https://docs.einsteintoolkit.org/et-docs/Main_Page#Documentation)
The CactusTutorial jupyter notebook for new users does not tackle this scenario in detail. Understandably, it would be unrealistic to provide generic instructions that cover all cluster configurations. How about guidelines that cover some common scenarios though? (e.g. a cluster that organizes libraries and packages as modules and also uses the Spack manager)
See above. Spack is tricky and often does not work out of the box unless one really builds everything from scratch. Some of the ET users have experience and they may be able to provide help.
Is there some documentation that gives basic guidelines as to how setup ET on a cluster aimed at a user profile that has (obviously) no previous experience with ET, simfactory etc and (quite possibly) also limited experience on how HPC clusters are setup, how packages are installed etc?
The recording of the seminar (and the slides therein) tries to do that. I would give it a try.
This request stems from the fact that I have been discovering little-by-little and through trial-and-error that e.g. running `./simfactory/bin/sim setup-silent` as outlined in the CactusTutorial, makes no sense for the case of a cluster. In my opinion, and if I may offer a suggestion, it would be a good idea to underline this distinction in a future revision of the tutorial.
Unfortunately each cluster is different and often one needs specific detailed workarounds. A lot of that unfortuntely comes down to experience.
Furthermore, another crucial step to setup ET on a cluster seems to be that the user must create "manually" their respective "<my_cluster>.cfg" and "<my_cluster>.ini" files and possibly the corresponding run and submit scripts. Could you confirm that this is indeed the case? What is the recommended course of action to do this? Maybe one should start by copying a sample ".cfg" file and amend it with information found from their cluster's technical documentation?
I'd start with an existing cluster that uses the same compiler / queuing system. Queueing system these days should always be SLURM and one can then try and see if the cluster supports the GNU compilers which should make things easy.
The only libraries that one really needs from the cluster are MPI. Everything else can be built by the EinsteinToolkit if needed (by setting the various FOO_DIR variable to the string BUILD).
I can follow up with more specific questions about problems I encountered during my so far unsuccessful attempts to install ET on a cluster, but to avoid people having to repeat explanations already given in the past, I would like first to make sure that I am not missing some source of information/documentation already available.
There is no real good docs unfortunately. Even the ones that I had linked above are (were, they are now) not collected in a single place.
Also, in the long shot that someone has already installed ET on one of the CINECA clusters in Italy (I am trying to install ET on the Leonardo cluster), I would be grateful if you could share your insights!
Bruno Giacommazzo installed the ET on some of the CINECA clusters in the past. I would try and reach out to him and his group at Milan. The newest machine still in simfactory is marconiA3 which however is already quite old. He also is subscribed to this mailing list.
Yours, Roland
-- My email is as private as my paper mail. I therefore support encrypting and signing email messages. Get my PGP key from http://pgp.mit.edu .