<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
<style type="text/css" style="display:none;"> P {margin-top:0;margin-bottom:0;} </style>
</head>
<body dir="ltr">
<div class="elementToProof">Dear Roland,<br>
<br>
thank you for gathering these links, they are indeed useful to point me in the right direction.<br>
<br>
I also found this seminar talk by Erik Schnetter on using Spack to install the ET dependencies.<br>
I also copy it in this thread, so that it is grouped with the other resources you gathered:<br>
<br>
</div>
<div class="elementToProof" style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<a href="https://einsteintoolkit.org/seminars/2022_05_19/index.html" id="LPlnk149802">https://einsteintoolkit.org/seminars/2022_05_19/index.html</a><br>
<br>
I will work my way through this material and let you know if I encounter other problems.<br>
<br>
Best,<br>
Panagiotis</div>
<div id="appendonsend"></div>
<div><br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<hr style="display: inline-block; width: 98%;">
<span style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);"><b>From:</b> Roland Haas<br>
<b>Sent:</b> Friday, February 7, 2025 6:41 PM<br>
<b>To:</b> IOSIF PANAGIOTIS<br>
<b>Cc:</b> users@einsteintoolkit.org<br>
<b>Subject:</b> Re: [Users] installation of ET on a cluster </span>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div class="elementToProof" style="font-size: 11pt;">Hello Panagiotis,<br>
<br>
Thank you for your inquiry.<br>
<br>
This may help:<br>
<br>
<a href="http://einsteintoolkit.org/seminars/2022_02_24/index.html" target="_blank" id="OWA1972fde8-b78f-0863-ef2d-4ab42ac46ce4" class="OWAAutoLink" rel="noopener noreferrer" data-auth="NotApplicable">http://einsteintoolkit.org/seminars/2022_02_24/index.html</a><br>
<br>
<a href="https://youtu.be/8dnOfQIk32c?feature=shared" target="_blank" id="OWA3c8eafd4-0b3b-bb3d-7b72-a759da9c3bac" class="OWAAutoLink" rel="noopener noreferrer" data-auth="NotApplicable">https://youtu.be/8dnOfQIk32c?feature=shared</a><br>
<br>
whose slides are:<br>
<br>
<a href="https://github.com/rhaas80/2024_LSU/blob/main/CactusBuildSytemTour.ipynb" target="_blank" id="OWAe1eaf94f-43bf-f545-9abf-f19d3a99d485" class="OWAAutoLink" rel="noopener noreferrer" data-auth="NotApplicable">https://github.com/rhaas80/2024_LSU/blob/main/CactusBuildSytemTour.ipynb</a><br>
<br>
There's also some advanced simulation factory docs here:<br>
<br>
<a href="https://docs.einsteintoolkit.org/et-docs/Configuring_a_new_machine" target="_blank" id="OWA20ecb1d9-c5d8-fea5-35be-183b83bcbafe" class="OWAAutoLink" rel="noopener noreferrer" data-auth="NotApplicable">https://docs.einsteintoolkit.org/et-docs/Configuring_a_new_machine</a><br>
<br>
(linked on<br>
<a href="https://docs.einsteintoolkit.org/et-docs/Main_Page#Documentation)" target="_blank" id="OWAdf978919-8496-b432-1f28-19e0f6635fc6" class="OWAAutoLink" rel="noopener noreferrer" data-auth="NotApplicable">https://docs.einsteintoolkit.org/et-docs/Main_Page#Documentation)</a><br>
<br>
> The CactusTutorial jupyter notebook for new users does not tackle<br>
> this scenario in detail. Understandably, it would be unrealistic to<br>
> provide generic instructions that cover all cluster configurations.<br>
> How about guidelines that cover some common scenarios though? (e.g. a<br>
> cluster that organizes libraries and packages as modules and also<br>
> uses the Spack manager)<br>
<br>
See above. Spack is tricky and often does not work out of the box<br>
unless one really builds everything from scratch. Some of the ET users<br>
have experience and they may be able to provide help.<br>
<br>
> Is there some documentation that gives basic guidelines as to how<br>
> setup ET on a cluster aimed at a user profile that has (obviously) no<br>
> previous experience with ET, simfactory etc and (quite possibly) also<br>
> limited experience on how HPC clusters are setup, how packages are<br>
> installed etc?<br>
<br>
The recording of the seminar (and the slides therein) tries to do that.<br>
I would give it a try.<br>
<br>
> This request stems from the fact that I have been discovering<br>
> little-by-little and through trial-and-error that e.g. running<br>
> `./simfactory/bin/sim setup-silent` as outlined in the<br>
> CactusTutorial, makes no sense for the case of a cluster. In my<br>
> opinion, and if I may offer a suggestion, it would be a good idea to<br>
> underline this distinction in a future revision of the tutorial.<br>
<br>
Unfortunately each cluster is different and often one needs specific<br>
detailed workarounds. A lot of that unfortuntely comes down to<br>
experience.<br>
<br>
><br>
> Furthermore, another crucial step to setup ET on a cluster seems to<br>
> be that the user must create "manually" their respective<br>
> "<my_cluster>.cfg" and "<my_cluster>.ini" files and possibly the<br>
> corresponding run and submit scripts. Could you confirm that this is<br>
> indeed the case? What is the recommended course of action to do this?<br>
> Maybe one should start by copying a sample ".cfg" file and amend it<br>
> with information found from their cluster's technical documentation?<br>
<br>
I'd start with an existing cluster that uses the same compiler / queuing<br>
system. Queueing system these days should always be SLURM and one can<br>
then try and see if the cluster supports the GNU compilers which should<br>
make things easy.<br>
<br>
The only libraries that one really needs from the cluster are MPI.<br>
Everything else can be built by the EinsteinToolkit if needed (by<br>
setting the various FOO_DIR variable to the string BUILD).<br>
<br>
> I can follow up with more specific questions about problems I<br>
> encountered during my so far unsuccessful attempts to install ET on a<br>
> cluster, but to avoid people having to repeat explanations already<br>
> given in the past, I would like first to make sure that I am not<br>
> missing some source of information/documentation already available.<br>
<br>
There is no real good docs unfortunately. Even the ones that I had<br>
linked above are (were, they are now) not collected in a single place.<br>
<br>
> Also, in the long shot that someone has already installed ET on one<br>
> of the CINECA clusters in Italy (I am trying to install ET on the<br>
> Leonardo cluster), I would be grateful if you could share your<br>
> insights!<br>
<br>
Bruno Giacommazzo installed the ET on some of the CINECA clusters in the<br>
past. I would try and reach out to him and his group at Milan. The<br>
newest machine still in simfactory is marconiA3 which however is<br>
already quite old. He also is subscribed to this mailing list.<br>
<br>
Yours,<br>
Roland<br>
<br>
--<br>
My email is as private as my paper mail. I therefore support encrypting<br>
and signing email messages. Get my PGP key from <a href="http://pgp.mit.edu" target="_blank" id="OWA0dc1ccbf-5b54-111a-583f-324909f66689" class="OWAAutoLink" rel="noopener noreferrer" data-auth="NotApplicable">
http://pgp.mit.edu</a> .<br>
</div>
</body>
</html>