Hi Roland,
In the release notes for the current release, it is written
http://einsteintoolkit.org/about/releases/ET_2017_06_announcement.html:
-(#) On Comet, using more than one MPI process might produce corrupt output files. This is a known file system bug on this machine.
https://docs.einsteintoolkit.org/et-docs/Release_coordination:
• comet • failures on multiple MPI ranks are almost certainly due to the file system bug that corrupts ASCII output files. Nothing we can do about it at this point (there is a ticket by Milton Ruiz with SDSC) short of adding various strange workarounds to every output routine (and it may still fail for checkpoint and recovery as is happening to the SpEC code right now)
Do you have any more details? I had the feeling that it was you who was fighting this. Would it be possible to add an autoconf or runtime test, so that users are at least made aware of the problem? Do you know of other systems that are affected? Do we have an ET ticket for this?