Hello all,
This is also addressed in the chat:
https://matrix.to/#/%21mYSgqyShRCoThqmUmY%3Agitter.im/%24GoGyO7PMmaXQPBip5Jj...
where the most recent issue is
--8<-- /home/jorge/simulations/ProcaStar6/SIMFACTORY/exe/cactus_sim -L 3 /home/jorge/simulations/ProcaStar6/output-0000/ProcaStar.par cactus_sim: /home/jorge/Einstein-Toolkit/Cactus/arrangements/Carpet/CarpetLib/src/th.hh:70: CCTK_REAL8 CarpetLib::th::get_time(int, int, int) const: Assertion `tl >= 0 and tl < timelevels' failed. Rank 0 with PID 57937 received signal 6 --8<--
Yours, Roland
[CAUTION: Non-UBC Email]
Dear Einstein Toolkit users,
I am currently working with the Einstein Toolkit and trying to use checkpoint/restart functionality to recover simulations.
I understand the parameters that must be included in the parameter file, but I am not sure how I should correctly restart a simulation from existing checkpoint data.
So far, I have been launching simulations with commands such as
./simfactory/bin/sim run ProcaStar -workstation-parfile=par/ProcaStar.par
and the simulation runs correctly and generates checkpoint files. However, when I try to restart from the checkpoint data, I am unsure what the correct procedure should be. I do not know whether I should use a different SimFactory command, modify the submission procedure, or provide additional options.
The checkpoint-related parameters in my parameter file are:
CarpetIOHDF5::checkpoint = yes IOUtil::checkpoint_every_walltime_hours = 0.1 IOUtil::checkpoint_dir = "checkpoints_prueba" IOUtil::checkpoint_keep = 1 IOUtil::checkpoint_ID = no IOUtil::checkpoint_on_terminate = yes IOUtil::recover = "autoprobe" IOUtil::recover_dir = "checkpoints_prueba" IOUtil::abort_on_io_errors = yes CarpetIOHDF5::open_one_input_file_at_a_time = yes
Could someone please clarify the recommended procedure for restarting a simulation from existing checkpoint files? In particular:
Which SimFactory command should be used to restart a simulation? 2.
Is it necessary to create a new run or continue the existing one? 3.
Are there any additional parameters or directory requirements for recovery to work correctly?
My environment is: Einstein Toolkit version: ET_2025_05 Operating system: Linux Ubuntu 22.04.5 LTS Cluster/machine: Local
Thank you very much for your help.
Best regards, Yahir Mio