[Users] Issues configuring SimFactory for PBS cluster (Hubble) - Jobs failing to submit correctly

[MAKINO Taiki]

Dear Beyhan Karakaş,

Thank you for your response, and I apologize for the delayed response.

I have prepared hubble.ini, .cfg, .sub, .run files and edited some lines in hubble.ini and hubble.sub based on your advice.

But some problems appear.

.out files exist under the Cactus directory and they are named ‘SubmitScript.o123456’ (where 123456 is the PBS job_id).


Also, when I checked my_simulation/SIMFACTORY/run/SubmitScript file inside the simulation output, I found that the its content was same as generic.sub.


Why is SimFactory ignoring my change to the submitscript option and still applying the generic script? Is there a command I need to run to force SimFactory to reload the updated .ini configuration?

Additionally, is there a documentation webpage that explains the variables enclosed in @ (like @JOB_ID@ or @SCRIPTFILE@)? I would like to know the full list of available placeholders.


I have attached the ‘SubmitScript' file and ‘generic.sub' file for your reference.

Thank you for your help.

Best regards,


Taiki Makino

Graduate School of Science, Nagoya University

makino.taiki.g6 at s.mail.nagoya-u.ac.jp<mailto:makino.taiki.g6 at s.mail.nagoya-u.ac.jp>


取得 Outlook for Mac<https://aka.ms/GetOutlookForMac>
差出人: Beyhan Karakaş <beyhannkarakas at gmail.com>
日付: 土曜日, 2026年7月18日 3:53
宛先: MAKINO Taiki <makino.taiki.g6 at s.mail.nagoya-u.ac.jp>
CC: Einstein Toolkit Users <users at einsteintoolkit.org>
件名: Issues configuring SimFactory for PBS cluster (Hubble) - Jobs failing to submit correctly


Hello Taiki,

You are modifying a generic machine file and using it with pbs-normal submission file. Submission and running the simulation are done by different files and currently you use generic config and runscript (generic.cfg, generic.run). I would suggest you to use consistent simfactory files, lets say modify the files for crux (crux.ini, .cfg, .sub, .run) to work with your cluster. After having consistent files add "--machine hubble" option when you submit the job (./simfactory/bin/sim submit ... --machine hubble).  Also have a look at the all supported clusters that use PBS on simfactory/mdb/submitscripts.

Best regards,

Beyhan.

I am trying to configure SimFactory to submit jobs using PBS on a machine named "Hubble".

Initially, the automatically generated hubble.ini was configured for a local login node submission. I attempted to modify the submission commands to support our PBS environment, but the jobs are not running properly.

Here are the modifications I made to the .ini configuration:

  *   submit: Changed from: exec nohup @SCRIPTFILE@ < /dev/null > @RUNDIR@/@SIMULATION_NAME at .out 2> @RUNDIR@/@SIMULATION_NAME at .err & echo $! To: qsub @SCRIPTFILE@

  *   stop: Changed from: pkill -g $(ps -o pgid= -p @JOB_ID@) To: qdel @JOB_ID@

  *   getstatus: Changed from: ps @JOB_ID@ To: qstat -f @JOB_ID@

Symptoms:

  1.  When running simfactory/bin/sim submit, no .out or .err files are generated at all.

  2.  When running simfactory/bin/sim create-submit, the .out and .err files are generated, but the .out file only contains the PBS job ID, and the .err file is completely empty.

  3.  No simulation output files or directories are created inside the designated simulation directory.

As you can see from the symptoms,  create-submit successfully captures the job ID, but the simulation does not run and no output is generated.

Could anyone guide me on the correct way to write the submit option or the template script for PBS in SimFactory? Any working examples of a PBS configuration (.ini and .sub files) would be greatly appreciated.

For your reference, I have attached my .ini file and .sub file that I am currently using.


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.einsteintoolkit.org/pipermail/users/attachments/20260721/1dc8b5f7/attachment-0001.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: SubmitScript
Type: application/octet-stream
Size: 464 bytes
Desc: SubmitScript
URL: <http://lists.einsteintoolkit.org/pipermail/users/attachments/20260721/1dc8b5f7/attachment-0002.obj>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: generic.sub
Type: application/octet-stream
Size: 453 bytes
Desc: generic.sub
URL: <http://lists.einsteintoolkit.org/pipermail/users/attachments/20260721/1dc8b5f7/attachment-0003.obj>