[ET Trac] [Einstein Toolkit] #73: Decouple submit and run scripts from configurations

[Einstein Toolkit]

#73: Decouple submit and run scripts from configurations
--------------------------+-------------------------------------------------
  Reporter:  hinder       |       Owner:  mthomas
      Type:  enhancement  |      Status:  new    
  Priority:  minor        |   Milestone:         
 Component:  SimFactory   |     Version:         
Resolution:               |    Keywords:         
--------------------------+-------------------------------------------------

Comment (by eschnett):

 1. The run script is logically part of the Cactus configuration, since one
 has to use the same MPI version to build and to run an executable. The
 submit script is admittedly different -- at the moment, one can either
 submit an executable to a queuing system, or can run it interactively. It
 makes thus sense to submit different queuing systems, or to choose a
 queuing system only at the time of submission. However, the run script
 needs to be closely tied to the configuration; it is an omission in Cactus
 that this is not so. If you have an executable on a particular system,
 then there is only one correct way of running it.

 2. When a run script is modified, the executable is rebuilt because it
 contains (via Formaline) a copy of the run script. It is unfortunate that
 make takes so long to rebuild -- it shouldn't if there are no changes.
 There is no way to find out whether anything changed except to look at all
 files, which is just what make does. I'm afraid there is no way to speed
 up things, except in an unsafe manner (by not checking source files). You
 can, of course, just manually copy the new run script into the Cactus
 configuration directory.

 3. It is a feature that editing a run script does not lead to it being
 used. This prevents problems when you have different configurations with
 different run scripts or different MPI versions. Each configuration is
 self-contained once it has been built, so that one can
 change/update/modify anything one wants, and the configuration remains
 usable, even after weeks or months. This is important in production
 situations. It may be necessary to highlight this in the documentation.

 4. No, the Cactus test suite mechanisms needs to be updated (or SimFactory
 needs to be able to translate) such that it uses the information in the
 mdb and in the Cactus submit/run scripts, depending on whether the test
 cases are checked interactively or in a submitted job.

 Moving the --submitscript to the submit command may make sense, or at
 least letting the user override the submit script.

 What kind of changes do you find yourself making to the run script and
 submit script? Are these corrections for debugging? Or would there be a
 more elegant way to make these changes via SimFactory, so that these
 changes are also logged?

 Could SimFactory make these changes automatically? This is the key point
 of SimFactory: to automate things to relieve people of low-level details.
 For example, running simulations on Damiana or on attached workstations is
 very similar to using a different queuing system, and SimFactory should
 totally support this, either by looking at the machine from where you
 run/submit, or via a --remote or --queue option.

-- 
Ticket URL: <https://trac.einsteintoolkit.org/ticket/73#comment:1>
Einstein Toolkit <http://einsteintoolkit.org>
The Einstein Toolkit