Also, I have just realized I missed the last email from Roland. I'll also have a look at branch rhaas/mpiio of Carpet and try to setup a test on few nodes to see if/how much using the parameter CarpetIOHDF5::user_MPIIO  reduces the peak in the amount of data written to disk at the same time (which was the actual issue).

Thank you very much Roland,
Lorenzo

On Tue, Nov 1, 2022, 23:59 Lorenzo Ennoggi <lorenzo.ennoggi@gmail.com> wrote:
Hi Erik,
thank you very much for getting back to me. I'll have a look a that and try to test it.

Thanks again for your help,
Lorenzo

On Tue, Nov 1, 2022, 13:49 Erik Schnetter <schnetter@gmail.com> wrote:
Lorenzo

The thorn `Carpet/CarpetSimulationIO` can write N output files on M
processes, and uses an efficient mechanism to map in between. That is,
it uses the high-speed interconnect instead of disk I/O to exchange
data.

This thorn is, to my knowledge, not used in production, so you'd want
to test it before you use it. It needs the `SimulationIO` external
library.

-erik

On Thu, Oct 6, 2022 at 12:28 PM Lorenzo Ennoggi
<lorenzo.ennoggi@gmail.com> wrote:
>
> Hello,
> In order to avoid stressing the filesystem on the cluster I'm running on, I was suggested to avoid writing one output/checkpoint file per MPI process and instead collecting data from multiple processes before outputting/checkpointing happens. I found the combination of parameters
>
> IO::out_mode       = "np"
> IO::out_proc_every = 8
>
> does the job for output files, but I still have one checkpoint file per process. Is there a similar parameter, or combination of parameters, which can be used for checkpoint files?
>
> Thank you very much,
> Lorenzo Ennoggi
> _______________________________________________
> Users mailing list
> Users@einsteintoolkit.org
> http://lists.einsteintoolkit.org/mailman/listinfo/users



--
Erik Schnetter <schnetter@gmail.com>
http://www.perimeterinstitute.ca/personal/eschnetter/