Hi everyone,
I am currently debugging some code and have been printing out (using either CCTK_INFO and/or printf) debugging information at particular grid points (LOCAL mode).
From what I can tell, Cactus only allows output from one process, say rank 0, to stop flooding of the output by printing everything once per process. However if I am wanting data from a particular grid point, and this grid point is not in the printing process, nothing will output.
I have been getting around this by turning off MPI (fine for one iteration or so) but now I am doing longer runs I need to be able to print out this information with MPI.
Thus my question:
How can you force Cactus to print out a statement from any process?
Cheers,
Chris
On 17 Jun 2018, at 10:52, Chris Stevens c.stevens@ru.ac.za wrote:
Hi everyone,
I am currently debugging some code and have been printing out (using either CCTK_INFO and/or printf) debugging information at particular grid points (LOCAL mode).
From what I can tell, Cactus only allows output from one process, say rank 0, to stop flooding of the output by printing everything once per process. However if I am wanting data from a particular grid point, and this grid point is not in the printing process, nothing will output.
I have been getting around this by turning off MPI (fine for one iteration or so) but now I am doing longer runs I need to be able to print out this information with MPI.
Thus my question:
How can you force Cactus to print out a statement from any process?
Add -roe to the Cactus command line and you will get separate output and error files from each process.
Hi Ian,
thanks for that, much appreciated!
Chris
On 06/17/2018 11:21 PM, ian.hinder@aei.mpg.de wrote:
On 17 Jun 2018, at 10:52, Chris Stevens <c.stevens@ru.ac.za mailto:c.stevens@ru.ac.za> wrote:
Hi everyone,
I am currently debugging some code and have been printing out (using either CCTK_INFO and/or printf) debugging information at particular grid points (LOCAL mode).
From what I can tell, Cactus only allows output from one process, say rank 0, to stop flooding of the output by printing everything once per process. However if I am wanting data from a particular grid point, and this grid point is not in the printing process, nothing will output.
I have been getting around this by turning off MPI (fine for one iteration or so) but now I am doing longer runs I need to be able to print out this information with MPI.
Thus my question:
How can you force Cactus to print out a statement from any process?
Add -roe to the Cactus command line and you will get separate output and error files from each process.
-- Ian Hinder https://ianhinder.net
Hi Ian,
I have just gone to implement your advice but am confused as to what you mean by Cactus command line. Is this additions to the .sub or .run file? I am using simfactory to submit my jobs.
Cheers,
Chris
On 06/17/2018 11:21 PM, ian.hinder@aei.mpg.de wrote:
On 17 Jun 2018, at 10:52, Chris Stevens <c.stevens@ru.ac.za mailto:c.stevens@ru.ac.za> wrote:
Hi everyone,
I am currently debugging some code and have been printing out (using either CCTK_INFO and/or printf) debugging information at particular grid points (LOCAL mode).
From what I can tell, Cactus only allows output from one process, say rank 0, to stop flooding of the output by printing everything once per process. However if I am wanting data from a particular grid point, and this grid point is not in the printing process, nothing will output.
I have been getting around this by turning off MPI (fine for one iteration or so) but now I am doing longer runs I need to be able to print out this information with MPI.
Thus my question:
How can you force Cactus to print out a statement from any process?
Add -roe to the Cactus command line and you will get separate output and error files from each process.
-- Ian Hinder https://ianhinder.net
On 21 Jun 2018, at 09:41, Chris Stevens c.stevens@ru.ac.za wrote:
Hi Ian,
I have just gone to implement your advice but am confused as to what you mean by Cactus command line. Is this additions to the .sub or .run file? I am using simfactory to submit my jobs.
Hi,
The .run file is the script that actually runs Cactus, and that's where the change needs to be made. For example, if you are using generic.run, you would change the line from
mpirun -np @NUM_PROCS@ @EXECUTABLE@ -L 3 @PARFILE@
to
mpirun -np @NUM_PROCS@ @EXECUTABLE@ -roe -L 3 @PARFILE@
Unfortunately it's all a little convoluted because of the layers of abstractions that simfactory provides. And the interface is not complete, which means do you have to delve into it from time to time. i.e. there is no way to pass through additional options to Cactus from the sim submit command line, as far as I know.
Note: in order to make SimFactory use the new runscript, you must rerun "sim build" and give it the runscript again, i.e.
sim build [<config>] --runscript generic.run
(replacing generic.run with whatever runscript you are using). It does not automatically detect the change and use the new runscript.
users@lists.einsteintoolkit.org
-
Chris Stevens -
ian.hinder＠aei.mpg.de