[Einstein Toolkit] #291: new python simfactory does not overtake the "-roe" option of Cactus is used with submit
#291: new python simfactory does not overtake the "-roe" option of Cactus is used with submit ----------------------------------------------+----------------------------- Reporter: alexander.beck-ratzka@… | Owner: mthomas Type: defect | Status: new Priority: minor | Milestone: ET_2010_11 Component: SimFactory | Version: ET_2010_11 Keywords: | ----------------------------------------------+----------------------------- I my template for RunScript, I have the line:
[snip] time ${MPIDIR}/bin/mpirun -v --mca btl openib,self --mca mpi_leave_pinned 0 -np @NUM_PROCS@ -npernode @(@PPN_USED@/@NUM_THREADS@)@ @EXECUTABLE@ -roe -L 3 @PARFILE@ [snip]
The option "-roe" will force Cactus to create a log for all processors, and not only for PE0.
The Runscript created by this template only contains the line
time ${MPIDIR}/bin/mpirun -v --mca btl openib,self --mca mpi_leave_pinned 0 -np 8 -npernode 4 /lustre/AEI/alibeck/s imulations/cactus_test-luca-no-openmp/SIMFACTORY/exe/cactus_test-luca-no- openmp -L 3 /lustre/AEI/alibeck/simulation s/cactus_test-luca-no- openmp/output-0003/TOVMHD_CarpetRegrid2_PPM_HLLE_BSSNMoL.par
so the option -roe is missing.
I have used simfactory submit (not create-submit) to start the job. If I use "create-submit" the -roe flag is respected.
#291: new python simfactory does not overtake the "-roe" option of Cactus is used with submit -----------------------------------------------+---------------------------- Reporter: alexander.beck-ratzka@… | Owner: mthomas Type: defect | Status: new Priority: minor | Milestone: ET_2010_11 Component: SimFactory | Version: ET_2010_11 Resolution: | Keywords: -----------------------------------------------+----------------------------
Comment (by eschnett):
The expected behaviour of Simfactory is the following:
- When a Cactus configuration is built and the --runscript option is given (or if the configuration does not have a run script), then Simfactory's runscript template is copied into the directory containing the Cactus configuration.
- When a simulation is created, then the run script (and the submit script) accompanying the Cactus configuration is copied into the simulation, and is partly expanded
- When a restart is submitted, then the simulation's run script is copied into the restart directory, and is fully expanded
The rationale behind the behaviour is that both Cactus configurations and simulations should be self-consistent if other things on the system change. The submit script and run script very likely depend on the configuration options that were used to build the Cactus configuration, and should therefore remain associated with the executable.
#291: new python simfactory does not overtake the "-roe" option of Cactus is used with submit -----------------------------------------------+---------------------------- Reporter: alexander.beck-ratzka@… | Owner: mthomas Type: defect | Status: new Priority: minor | Milestone: ET_2010_11 Component: SimFactory | Version: ET_2010_11 Resolution: | Keywords: -----------------------------------------------+----------------------------
Comment (by anonymous):
I principle these ideas are correct. However, I can imagine simulations where one would like to see the content of other PEs at a later stage of the simulations, e.g. if problems occur which do not lead to a error exit pof the slmulation, and where the raeson cannot be identified from the logging of PE0. In such a case it would be really helpful, if one could add ther -roe option. The -roe option is no change in a fundamental part of a simulation, it is only an extension of the logging
#291: new python simfactory does not overtake the "-roe" option of Cactus is used with submit -----------------------------------------------+---------------------------- Reporter: alexander.beck-ratzka@… | Owner: mthomas Type: defect | Status: new Priority: minor | Milestone: ET_2010_11 Component: SimFactory | Version: ET_2010_11 Resolution: | Keywords: -----------------------------------------------+----------------------------
Comment (by eschnett):
You can add the -roe option to the corresponding run script, and Simfactory will honour this. There is currently no command to automate this, though.
If you create a simulation, you need to modify the run script in the Cactus configuration directory. If the simulation already exists, then you need to modify the run script in the simulation.
#291: new python simfactory does not overtake the "-roe" option of Cactus is used with submit -----------------------------------------------+---------------------------- Reporter: alexander.beck-ratzka@… | Owner: mthomas Type: defect | Status: new Priority: minor | Milestone: ET_2010_11 Component: SimFactory | Version: ET_2010_11 Resolution: | Keywords: -----------------------------------------------+----------------------------
Comment (by anonymous):
Okay, I think this should help. Are you shure that simfactory is looking in the corresponing old output-xxxx directory befor it is starting to run?
#291: new python simfactory does not overtake the "-roe" option of Cactus is used with submit -----------------------------------------------+---------------------------- Reporter: alexander.beck-ratzka@… | Owner: mthomas Type: defect | Status: new Priority: minor | Milestone: ET_2010_11 Component: SimFactory | Version: ET_2010_11 Resolution: | Keywords: -----------------------------------------------+----------------------------
Comment (by eschnett):
Internally, Simfactory distinguishes between creating a simulation and submitting a restart. When creating a simulation look for the run script in the Cactus configuration directory, submitting a restart looks in the simulation directory (in the SIMFACTORY subdirectory there). Neither look in the output-NNNN directory; however, this is where the final run script is placed when the restart is submitted.
You can use the command find -iname "*runscript*" to find where runscripts are stored.
#291: new python simfactory does not overtake the "-roe" option of Cactus is used with submit -----------------------------------------------+---------------------------- Reporter: alexander.beck-ratzka@… | Owner: mthomas Type: defect | Status: new Priority: minor | Milestone: ET_2010_11 Component: SimFactory | Version: ET_2010_11 Resolution: | Keywords: -----------------------------------------------+----------------------------
Comment (by hinder):
Alexander, is this solution sufficient? Can we close the ticket?
#291: new python simfactory does not overtake the "-roe" option of Cactus is used with submit -----------------------------------------------+---------------------------- Reporter: alexander.beck-ratzka@… | Owner: mthomas Type: defect | Status: new Priority: minor | Milestone: ET_2010_11 Component: SimFactory | Version: ET_2010_11 Resolution: | Keywords: -----------------------------------------------+----------------------------
Comment (by alexander.beck-ratzka@…):
The ticket can be closed.
Alexander
#291: new python simfactory does not overtake the "-roe" option of Cactus is used with submit -----------------------------------------------+---------------------------- Reporter: alexander.beck-ratzka@… | Owner: mthomas Type: defect | Status: closed Priority: minor | Milestone: ET_2010_11 Component: SimFactory | Version: ET_2010_11 Resolution: invalid | Keywords: -----------------------------------------------+---------------------------- Changes (by hinder):
* status: new => closed * resolution: => invalid
trac@lists.einsteintoolkit.org
-
Einstein Toolkit