Hello Wei,
I encountered an error with CarpetIOHDF5 when reaching the checkpoint iteration, which stops my job if I use too many nodes for parallel processing. I am wondering if this issue is related to the HDF5 library I used during building and if there is a way to resolve it. I utilized the bundled HDF5 from the toolkit itself.
These are only warnings and will not make the code abort. The warning lets you know that an array TERMINATIONTRIGGER::watchminutes that exists on each process and is claimed to be identical on each, does in fact have differing values.
Only the data by process 0 will be saved, and the other data discarded.
The array in question TERMINATIONTRIGGER::watchminutes is used by the TeminationTrigger thorn only to keep reminding you every so often of how much time is left. Given the way to code is written it will always differ, since only rank 0 actually updates it. This is a bug, but a harmless one and you can ignore this particular warning.
Yours, Roland
The warnings I received are as follows:
*WARNING Level 1* from host d24cepyc083.crc.nd.edu, process 0, while executing schedule bin CCTK_CHECKPOINT, routine CarpetIOHDF5::CarpetIOHDF5_EvolutionCheckpoint in thorn CarpetIOHDF5, file /Cactus/configs/sim/build/CarpetIOHDF5/Output.cc:442: "Values for DISTRIB=CONSTANT grid variable 'TERMINATIONTRIGGER::watchminutes' (time level 0) differ between processors 0 and 95; only the array from processor 0 will be stored."
*WARNING Level 1* from host d24cepyc083.crc.nd.edu, process 0, while executing schedule bin CCTK_CHECKPOINT, routine CarpetIOHDF5::CarpetIOHDF5_EvolutionCheckpoint in thorn CarpetIOHDF5, file /Cactus/configs/sim/build/CarpetIOHDF5/Output.cc:442: "Values for DISTRIB=CONSTANT grid variable 'QUASILOCALMEASURES::qlm_z_p_p[2]' (time level 0) differ between processors 0 and 2; only the array from processor 0 will be stored."
Best regards,
Wei