Hi,
I am wondering if the default behavior of Cactus output has been changed recently. When starting a simulation from scratch (i.e. no recovery), already existing output files used to get overwritten. Instead now, Cactus appends data to the end of the files. How can I change this behavior? Is there a parameter for this?
Thanks, Christian
This behaviour is not supposed to have changed. The basic mechanism is implemented in CactusBase/IOUtil, where the aliased function IO_TruncateOutputFiles decides whether to truncate (overwrite) or not. Of course, each thorn is supposed to contain logic on top of this to ensure that this function is only called once, during the beginning, so that things are not overwritten at every time step...
This function basically determines whether this run started from scratch or whether it is a restart.
Which thorn is misbehaving?
-erik
On Mon, Mar 12, 2012 at 3:25 PM, Christian Reisswig reisswig@tapir.caltech.edu wrote:
Hi,
I am wondering if the default behavior of Cactus output has been changed recently. When starting a simulation from scratch (i.e. no recovery), already existing output files used to get overwritten. Instead now, Cactus appends data to the end of the files. How can I change this behavior? Is there a parameter for this?
Thanks, Christian _______________________________________________ Users mailing list Users@einsteintoolkit.org http://lists.einsteintoolkit.org/mailman/listinfo/users
On 12 Mar 2012, at 20:25, Christian Reisswig wrote:
Hi,
I am wondering if the default behavior of Cactus output has been changed recently. When starting a simulation from scratch (i.e. no recovery), already existing output files used to get overwritten. Instead now, Cactus appends data to the end of the files. How can I change this behavior? Is there a parameter for this?
Hi Christian,
From IOUtil/param.ccl:
BOOLEAN truncate_files "Truncate existing output files from previous runs (except when recovering) ?" STEERABLE = ALWAYS { } "yes"
I don't think this has been changed. If it is not truncating the files, then either you have this parameter set accidentally, or there is something very wrong!
On the other hand, many people who might have been involved in such a change are probably using simfactory, which ensures that you have a new empty directory for each Cactus run, so a problem might not have been noticed.
Hi Ian, Erik,
thanks for the reply! I played around with IO::truncate_files = yes/no but without success. In particular, CarpetIOScalar does not overwrite files (I am looking at the L2 norm of some grid function) anymore.
cheers, Christian
On 12 Mar 2012, at 20:25, Christian Reisswig wrote:
Hi,
I am wondering if the default behavior of Cactus output has been changed recently. When starting a simulation from scratch (i.e. no recovery), already existing output files used to get overwritten. Instead now, Cactus appends data to the end of the files. How can I change this behavior? Is there a parameter for this?
Hi Christian,
From IOUtil/param.ccl:
BOOLEAN truncate_files "Truncate existing output files from previous runs (except when recovering) ?" STEERABLE = ALWAYS { } "yes"
I don't think this has been changed. If it is not truncating the files, then either you have this parameter set accidentally, or there is something very wrong!
On the other hand, many people who might have been involved in such a change are probably using simfactory, which ensures that you have a new empty directory for each Cactus run, so a problem might not have been noticed.
On 12 Mar 2012, at 22:06, Christian Reisswig wrote:
Hi Ian, Erik,
thanks for the reply! I played around with IO::truncate_files = yes/no but without success. In particular, CarpetIOScalar does not overwrite files (I am looking at the L2 norm of some grid function) anymore.
What machine/OS/filesystem are you using? Could it be that in some new version of Linux the file semantics have changed?
cheers, Christian
On 12 Mar 2012, at 20:25, Christian Reisswig wrote:
Hi,
I am wondering if the default behavior of Cactus output has been changed recently. When starting a simulation from scratch (i.e. no recovery), already existing output files used to get overwritten. Instead now, Cactus appends data to the end of the files. How can I change this behavior? Is there a parameter for this?
Hi Christian,
From IOUtil/param.ccl:
BOOLEAN truncate_files "Truncate existing output files from previous runs (except when recovering) ?" STEERABLE = ALWAYS { } "yes"
I don't think this has been changed. If it is not truncating the files, then either you have this parameter set accidentally, or there is something very wrong!
On the other hand, many people who might have been involved in such a change are probably using simfactory, which ensures that you have a new empty directory for each Cactus run, so a problem might not have been noticed.
CarpetIOScalar has a new parameter all_reductions_in_one_file. This parameter is used near the checks for truncating. Can you check the behaviour from before this parameter existed?
-erik
On Mon, Mar 12, 2012 at 5:12 PM, Ian Hinder ian.hinder@aei.mpg.de wrote:
On 12 Mar 2012, at 22:06, Christian Reisswig wrote:
Hi Ian, Erik,
thanks for the reply! I played around with IO::truncate_files = yes/no but without success. In particular, CarpetIOScalar does not overwrite files (I am looking at the L2 norm of some grid function) anymore.
What machine/OS/filesystem are you using? Could it be that in some new version of Linux the file semantics have changed?
cheers, Christian
On 12 Mar 2012, at 20:25, Christian Reisswig wrote:
Hi,
I am wondering if the default behavior of Cactus output has been changed recently. When starting a simulation from scratch (i.e. no recovery), already existing output files used to get overwritten. Instead now, Cactus appends data to the end of the files. How can I change this behavior? Is there a parameter for this?
Hi Christian,
From IOUtil/param.ccl:
BOOLEAN truncate_files "Truncate existing output files from previous runs (except when recovering) ?" STEERABLE = ALWAYS { } "yes"
I don't think this has been changed. If it is not truncating the files, then either you have this parameter set accidentally, or there is something very wrong!
On the other hand, many people who might have been involved in such a change are probably using simfactory, which ensures that you have a new empty directory for each Cactus run, so a problem might not have been noticed.
-- Ian Hinder http://numrel.aei.mpg.de/people/hinder
Users mailing list Users@einsteintoolkit.org http://lists.einsteintoolkit.org/mailman/listinfo/users
Hello Christian, Erik, Ian, all,
CarpetIOScalar has a new parameter all_reductions_in_one_file. This parameter is used near the checks for truncating. Can you check the behaviour from before this parameter existed?
It should work again now. I tested all four combinations of io::truncate_files and carpetioscalar::all_reductions_in_one_file and they seem to reproduce the old behaviour. A unfortunate side effect of this, is that no header is written when the set of reductions is changed in the course of a checkpoint-and-recovery operation. The only way would seem to checkpoint the per-variable settings (unless it is possible to ask the flesh for parameter values pre and post checkpoint recovery).
The underlying error was not properly realising that do_truncate[] is a per-Cactus-variable vector, rather than a per-output-file vector. Truncation worked for the very first reduction (average in my case), which likely contributed to me to not noticing earlier.
Yours, Roland
Roland
You could output the current header after recovery unconditionally.
-erik
On Tue, Mar 13, 2012 at 9:54 PM, Roland Haas roland.haas@physics.gatech.edu wrote:
Hello Christian, Erik, Ian, all,
CarpetIOScalar has a new parameter all_reductions_in_one_file. This parameter is used near the checks for truncating. Can you check the behaviour from before this parameter existed?
It should work again now. I tested all four combinations of io::truncate_files and carpetioscalar::all_reductions_in_one_file and they seem to reproduce the old behaviour. A unfortunate side effect of this, is that no header is written when the set of reductions is changed in the course of a checkpoint-and-recovery operation. The only way would seem to checkpoint the per-variable settings (unless it is possible to ask the flesh for parameter values pre and post checkpoint recovery).
The underlying error was not properly realising that do_truncate[] is a per-Cactus-variable vector, rather than a per-output-file vector. Truncation worked for the very first reduction (average in my case), which likely contributed to me to not noticing earlier.
Yours, Roland
-- My email is as private as my paper mail. I therefore support encrypting and signing email messages. Get my PGP key from http://keys.gnupg.net.
Users mailing list Users@einsteintoolkit.org http://lists.einsteintoolkit.org/mailman/listinfo/users
Hello Erik,
You could output the current header after recovery unconditionally.
Yes, that would be a solution (and less code, right now I had put in an explicit check to _not_ write out a header upon recovery unless a new file is created). CarpetIOScalar should only output the new header if all_reductions_in_one_file==yes since (a) the old method never changes headers and (b) some users might rely on there being only a single block of comments in the files (eg. to open the file in MatLab which is evil that way and expects that one tells it how many header lines there are at the top).
I might be able to use Cactus CCTK_ParameterQueryTimesSet (the way that Carpet does for PUGH::periodic_x) to only output a header if either outScalar_vars or outScalar_reductions changed (and output a new header for all variables, not just the ones that changed). I'll give at one point (unless this is considered high priority, the bug certainly was).
Yours, Roland
Hi Roland,
thanks for the fix! It works again for me!
cheers, Christian
Hello Christian, Erik, Ian, all,
CarpetIOScalar has a new parameter all_reductions_in_one_file. This parameter is used near the checks for truncating. Can you check the behaviour from before this parameter existed?
It should work again now. I tested all four combinations of io::truncate_files and carpetioscalar::all_reductions_in_one_file and they seem to reproduce the old behaviour. A unfortunate side effect of this, is that no header is written when the set of reductions is changed in the course of a checkpoint-and-recovery operation. The only way would seem to checkpoint the per-variable settings (unless it is possible to ask the flesh for parameter values pre and post checkpoint recovery).
The underlying error was not properly realising that do_truncate[] is a per-Cactus-variable vector, rather than a per-output-file vector. Truncation worked for the very first reduction (average in my case), which likely contributed to me to not noticing earlier.
Yours, Roland
users@lists.einsteintoolkit.org
-
Christian Reisswig -
Erik Schnetter -
Ian Hinder -
Roland Haas