Users September 2025

users@lists.einsteintoolkit.org

18 participants
21 discussions

Inconsistency warnings: cores/threads mismatch [Leonardo cluster]
by IOSIF PANAGIOTIS 24 Oct '25

24 Oct '25

Hi all, After having installed the toolkit on Leonardo, CINECA, I wanted to try the TOV examples. However, I am getting the following warnings: Warning: Too many threads per process specified: specified num-threads=56 (ppn-used is 112) Warning: Total number of threads and number of threads per process are inconsistent: procs=16, num-threads=56 (procs*num-smt must be an integer multiple of num-threads) Warning: Total number of threads and number of cores per node are inconsistent: procs=16, ppn-used=112 (procs must be an integer multiple of ppn-used) >From those, it seems that I am doing something wrong either in my machine.ini file or in the resources request when submitting the job. Here is the command I use to run the TOV example: ./simfactory/bin/sim submit tov_ET --parfile par/tov_ET.par --cores 16 --walltime 00:20:00 And here is the ini file I am using (the "Cluster characteristics" section is identical to Bruno Giacomazzo's file<https://bitbucket.org/simfactory/simfactory2/commits/0d4011bf554ee695f9543e…>: https://github.com/piosif/einstein-toolkit-config-leonardo/blob/main/leonar… The job ran and completed fine, but from what I understand, it ran on 56 cores instead of the 16 that I requested. >From what I understand, the problem is that I am requesting just 16 cores for this simple test, but due to the cluster specs in the ini file, SimFactory expects me to request a multiple of 56. Is that so? So, my question is the following: * I want to do some tests, running a single star e.g. on a single node using 16, 32, 64 cores (to understand the timescale required). Do I need to change my ini file? * Or is there some option to specify when submitting the job that will override the cluster specs? I read the simfactory's terminology explanation on nodes, threads etc<https://simfactory.bitbucket.io/simfactory2/userguide/processterminology.ht…>, but it is not clear to me how to request the cores I need. Best, Panayotis

4 11

Questions about old ET releases and bundled libraries on clusters
by Omar Elías Velasco Castillo 23 Oct '25

23 Oct '25

Dear Einstein Toolkit team, I hope this message finds you well. I am a beginner with the Einstein Toolkit. On personal workstations I have been able to compile and run tutorial simulations at low resolution, but I am facing problems on two different clusters. I would like to ask two questions: 1. *Are there ET versions prior to 2022_05 (e.g. 2019–2020 releases) that can still be downloaded and compiled successfully?* When I try to fetch them from the website using ./GetComponents, the process fails (CactusSourceJar.git is not created and some components do not download). Since some of the nodes I use have older GCC versions (8 or 10) and limited modules, a stable older release might be more practical. 2. During compilation, I notice that thorns (such as GSL and HDF5, for example) fall back to using the bundled versions because system modules are not found. The build completes successfully, but jobs fail immediately after submission to PBS or SLURM queues. *What is the role of the bundled versions in this case*?* If the build uses bundled GSL/HDF5, do I still need to load corresponding, compatible modules in the runscript?* Could this mismatch explain why jobs die right after submission? Any advice on handling these issues would be very helpful. Thank you very much for your time and support. Greetings, O.V.

2 5

Agenda for Thursday's Meeting
by users＠einsteintoolkit.org 29 Sep '25

29 Sep '25

Please update the Wiki with agenda items for Thursday's meeting. Thanks! https://docs.einsteintoolkit.org/et-docs/meeting_agenda --The Maintainers

1 0

Re: [Users] Inconsistency warnings: cores/threads mismatch [Leonardo cluster]
by Wei Sun 29 Sep '25

29 Sep '25

Hi Panayotis, Can you try specifying --procs, --num-threads, and --ppn-used at the same time? It looks like your machine defaulted to --ppn-used=112, which is not what you want, so you may need to overwrite it. You can check your allocation details in /output-000x/SIMFACTORY/properties.ini. A useful strategy is to verify whether the allocation matches these relations: ppn-used = numthreads * nodeprocsprocs = numthreads * numprocs = nodes * ppn-used Below is my understanding (I personally use numsmt=1, so threads = cores in my case). Please correct me if I’m wrong on them: nodes = number of nodes procsrequested = total number of cores requested (may not all be used, but you still occupy them on the HPC) ppn = number of cores per node requested (may not all be used) numprocs = total number of MPI ranks actually used nodeprocs = number of MPI ranks per node actually used procs = total number of cores actually used numthreads = number of OpenMP threads per MPI rank actually used ppnused = number of cores per node actually used numsmt = number of threads per physical core Best, Wei

2 1

Meeting Minutes for 2025-09-18
by Lucas Timotheo Sanches 25 Sep '25

25 Sep '25

Present: Lucas(minutes), Peter Diener, Leo Rosa Werneck (chair), Roland Hass, Keith Dow, Beyhan # Agenda items - BBH gallery example: Peter says ran the example with the executable from back then (the Annie Jump Cannon release binary) and it works. He still needs to visualize it. - Move repos to GitHub: Currently, there's a user named CactusCodewe which we could potentially use. We would need to reach out to this person and ask them if they would be interested in releasing the name. Potential issues on the migration includes some loss of functionality like the ability to download large files. Moving issues is also an issue, but we are probably sticking to BitBucket for that. It was decided to vote on this issue again once Steve is back. # Unanswered questions on the mailing list. - "Issues Running BNS and GW150914 Example Simulations + storing BSSN variables": Roland is in contact with the person and trying to help them. - "Questions about old ET releases and bundled libraries on clusters": You can't download old releases. Too bad. No sure if old cold will compile with modern compilers. # Tickets - 2706: Zach says there are updates, but things are moving slowly. Needs more time. - 2886: Roland says this is more of a TODO than a ticket. - 2866: Just comment it out. Still waiting on a response from the reporter. - 2858: Lucas still didn't have time to look into these files. - 2882: Alejandra is still working on testing if there are performance improvements. - 2664: No updates. - 963: Improve accuracy: Peters says testings are running, to early to tell.

1 0

Einstein Toolkit Meeting Reminder
by users＠einsteintoolkit.org 24 Sep '25

24 Sep '25

Hello, Please consider joining the weekly Einstein Toolkit phone call at 9:00 am US central time on Thursdays. For details on how to connect and what agenda items are to be discussed, use the link below. https://docs.einsteintoolkit.org/et-docs/Main_Page#Weekly_Users_Call --The Maintainers

1 0

Agenda for Thursday's Meeting
by users＠einsteintoolkit.org 22 Sep '25

22 Sep '25

Please update the Wiki with agenda items for Thursday's meeting. Thanks! https://docs.einsteintoolkit.org/et-docs/meeting_agenda --The Maintainers

1 0

Assistant Professor - Computational Astrophysics, The University of Tennessee, Knoxville
by Mewes, Vassilios 22 Sep '25

22 Sep '25

Hi all, See the attached pdf document for an Assistant Professor in Computational Astrophysics opening at the nearby University of Tennessee in Knoxville. Please share this with any interested candidates. Best wishes, Vassili

1 0

clarification on folders output-xxxx
by IOSIF PANAGIOTIS 18 Sep '25

18 Sep '25

Hi all, I am not sure I understand what the output-xxxx folders do under each individual simulation folder. Suppose we "create-submit" a new simulation named test. The results will appear in the output-0000 folder under the folder "home/username/simulations/test/". OK, great. Suppose now we want to re-run that simulation. And suppose we want to re-run it from scratch and not continue from a previous checkpoint (e.g. maybe we forgot to save some output we needed). The skeleton is already created, so we can just use the command submit this time. The results will appear in the output-0001 folder. Is the above understanding correct? Questions: * When we submit again the simulation for the re-run, do we have to specify the same computational resources? * I.e. is this info something already encoded in the simulation's skeleton and therefore we should keep it fixed for consistency? * Does the skeleton of the simulation also fix what quantities are saved and therefore if we forgot to save some quantity we have to create a new simulation skeleton? Additionally, from what I understand, the output-xxxx folders mainly handle different checkpoints of a larger simulation that could take weeks or months. * Does the user have to do anything to combine all the information from the checkpoints, or is this handled automatically by ETK / SimFactory? * Is there any advice on best practices regarding this scenario? Best, Panayotis ------------------- Panagiotis Iosif postdoctoral researcher Department of Physics, University of Trieste Via Alfonso Valerio 2, Trieste 34127 Italy -------------------

2 1

Einstein Toolkit Meeting Reminder
by users＠einsteintoolkit.org 17 Sep '25

17 Sep '25

1 0

2026

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

Users September 2025