Also, I have just realized I missed the last email from Roland. I'll also have a look at branch rhaas/mpiio of Carpet and try to setup a test on few nodes to see if/how much using the parameter CarpetIOHDF5::user_MPIIO reduces the peak in the amount of data written to disk at the same time (which was the actual issue).
Thank you very much Roland, Lorenzo
On Tue, Nov 1, 2022, 23:59 Lorenzo Ennoggi lorenzo.ennoggi@gmail.com wrote:
Hi Erik, thank you very much for getting back to me. I'll have a look a that and try to test it.
Thanks again for your help, Lorenzo
On Tue, Nov 1, 2022, 13:49 Erik Schnetter schnetter@gmail.com wrote:
Lorenzo
The thorn `Carpet/CarpetSimulationIO` can write N output files on M processes, and uses an efficient mechanism to map in between. That is, it uses the high-speed interconnect instead of disk I/O to exchange data.
This thorn is, to my knowledge, not used in production, so you'd want to test it before you use it. It needs the `SimulationIO` external library.
-erik
On Thu, Oct 6, 2022 at 12:28 PM Lorenzo Ennoggi lorenzo.ennoggi@gmail.com wrote:
Hello, In order to avoid stressing the filesystem on the cluster I'm running
on, I was suggested to avoid writing one output/checkpoint file per MPI process and instead collecting data from multiple processes before outputting/checkpointing happens. I found the combination of parameters
IO::out_mode = "np" IO::out_proc_every = 8
does the job for output files, but I still have one checkpoint file per
process. Is there a similar parameter, or combination of parameters, which can be used for checkpoint files?
Thank you very much, Lorenzo Ennoggi _______________________________________________ Users mailing list Users@einsteintoolkit.org http://lists.einsteintoolkit.org/mailman/listinfo/users
-- Erik Schnetter schnetter@gmail.com http://www.perimeterinstitute.ca/personal/eschnetter/