Dear Peter,

Thank you for the clarifications regarding the Simfactory macros and the best practice advice.

best, Panayotis

________________________________ From: Peter Diener diener@cct.lsu.edu Sent: Thursday, August 7, 2025 4:26 PM To: IOSIF PANAGIOTIS PANAGIOTIS.IOSIF@units.it Cc: Leonardo Werneck wernecklr@gmail.com; users@einsteintoolkit.org users@einsteintoolkit.org Subject: Re: [Users] Users Digest, Vol 184, Issue 14

Dear Iosif,

Yes, @ALLOCATION@ will be read from the corresponding * .ini file if not specified with the --allocation=YOUR_ALLOCATION command line option. And yes the command line will override the information in the .ini file.

However, I would rather place the allocation information in simfactory/etc/defs.local.ini than in the <machine>.ini file. Multiple users with different allocations can use the same machine, so that information belongs in the user specific .ini file.

There is no list of macros documented anywhere, unfortunately.

Cheers,

Peter

Assistant Professor Research Center for Computation & Technology and Department of Physics & Astronomy Louisiana State University

On Sat, 2 Aug 2025, IOSIF PANAGIOTIS wrote:

Dear Leo,

Thank you for your answer and for sharing your experience. No, I hadn't received a reply to my question yet.

So it seems that macros such as @ALLOCATION@ (or any other SimFactory placeholder) can be changed via :

• either editing the <machine>.ini file
• or specifying the relevant command line option (i.e., among the ones that come up when doing "./simfactory/bin/sim --help").

And if I understand correctly, it seems that whatever is specified as a command-line option for sim takes priority over what is specified in the configuration files. Is this correct?

If yes, then these placeholders could be simply left alone when editing the submitscript and runscript, and set via the respective sim command-line option when submitting a simulation. Right?

And to reiterate part of my initial question:

• If these @FOO@ macros for SimFactory are documented somewhere, I would be grateful if you could point me to the relevant page.
• If such documentation does not exist, then it is also ok, and this empirical understanding will have to suffice for now. I was just asking to make sure I was not missing some obvious documentation source.

Best, Panayotis

---

From: Users users-bounces@einsteintoolkit.org on behalf of Leonardo Werneck wernecklr@gmail.com Sent: Thursday, July 31, 2025 7:26 PM To: users@einsteintoolkit.org users@einsteintoolkit.org Subject: Re: [Users] Users Digest, Vol 184, Issue 14 Dear Iosif,

I hope this message finds you well.

I'm not sure you already got a reply, but in case you haven't, my understanding is that placeholders like @FOO@ are there to indicate things you need to change yourself, manually.

So when you see something like:

#SBATCH -A @ALLOCATION@

then it means you should replace @ALLOCATION@ with the name of your allocation in the specific cluster.

Someone else can correct me if this is not correct, but that's how I usually handle these.

Cheers, Leo

On Fri, Jul 25, 2025 at 8:06 AM users-request@einsteintoolkit.org wrote: Send Users mailing list submissions to users@einsteintoolkit.org

  To subscribe or unsubscribe via the World Wide Web, visit
          http://lists.einsteintoolkit.org/mailman/listinfo/users
  or, via email, send a message with subject or body 'help' to
          users-request@einsteintoolkit.org

  You can reach the person managing the list at
          users-owner@einsteintoolkit.org

  When replying, please edit your Subject line so it is more
  specific
  than "Re: Contents of Users digest..."


  Today's Topics:

     1. SimFactory placeholder macros (IOSIF PANAGIOTIS)
     2. Issue with ReadInterpolate (Luciano Combi)


  ----------------------------------------------------------------------

  Message: 1
  Date: Fri, 25 Jul 2025 10:43:36 +0000
  From: IOSIF PANAGIOTIS <PANAGIOTIS.IOSIF@units.it>
  To: Einstein Toolkit Users <users@einsteintoolkit.org>
  Subject: [Users] SimFactory placeholder macros
  Message-ID:
         <AS8P250MB08637A69A316C8763E533B5FF959A@AS8P250MB0863.EURP250.PROD.OUTLOOK.
  COM>

  Content-Type: text/plain; charset="iso-8859-1"

  Hi,

  I have been following this document,
  https://einsteintoolkit.org/seminars/2022_02_24/index.html , to
  set up ET on my cluster, working also on some example
  configuration files.

  While preparing the submit and run scripts, I wanted to check
  the description of the @FOO@ macros that SimFactory uses.

  I checked these places but found no information.

    *
  https://simfactory.bitbucket.io/simfactory2/userguide/ (no
  mention of macros)
    *

https://docs.einsteintoolkit.org/et-docs/Simulation_Factory_Advanced_Tutori al (1 mention of the @USER@ macro and a pointer to a section#Macros<https://docs.einsteintoolkit.org/et-docs/Simulation_Factory_Advance d_Tutorial#Macros> that seems missing) * https://docs.einsteintoolkit.org/et-docs/Configuring_a_new_machine (there is no information on the submitscript and runscript)

  So I would like to ask:

  If e.g. I have this line in my submitscript:
  #SBATCH -A @ALLOCATION@

  Where is @ALLOCATION@ "read" from?

  My guess is that this is specified in the .ini file in this
  line:
  allocation = <project_account>

  Could you confirm if this is the case?

  Is there another source of documentation (that I might have
  missed) describing what these @FOO@ placeholders do?

  Best regards,
  Panagiotis


  -------------------
  Panagiotis Iosif
  postdoctoral researcher
  Department of Physics, University of Trieste
  Via Alfonso Valerio 2, Trieste 34127
  Italy
  -------------------

  -------------- next part --------------
  An HTML attachment was scrubbed...
  URL:<http://lists.einsteintoolkit.org/pipermail/users/attachments/20250725/dca9
  8ab5/attachment-0001.htm>

  ------------------------------

  Message: 2
  Date: Fri, 25 Jul 2025 15:05:48 +0000
  From: Luciano Combi <lcombi@perimeterinstitute.ca>
  To: "users@einsteintoolkit.org" <users@einsteintoolkit.org>
  Subject: [Users] Issue with ReadInterpolate
  Message-ID:
         <YQBPR0101MB544409A753F7D8E3ED3097D9CC59A@YQBPR0101MB5444.CANPRD01.PROD.OUT
  LOOK.COM>

  Content-Type: text/plain; charset="iso-8859-1"

  Hi people,

  I have used ReadInterpolate successfully for different
  applications in the past. I'm now trying to use it to
  interpolate onto a new grid that has 5 ghost zones i.e.:

  CoordBase::boundary_size_x_lower     = 5
  CoordBase::boundary_size_y_lower     = 5
  CoordBase::boundary_size_z_lower     = 5
  CoordBase::boundary_size_x_upper     = 5
  CoordBase::boundary_size_y_upper     = 5
  CoordBase::boundary_size_z_upper     = 5
  Driver::ghost_size                   = 5
  GRHydro::GRHydro_stencil             = 5
  But I'm getting this error:

  CCTK_Proc569.err <==
    while executing schedule bin ReadInterpolate_ReadData, routine
  ReadInterpolate::ReadInterpolate_Read
    in thorn ReadInterpolate, file/home/mvxcombi/ET_Zelmani/Cactus/arrangements/EinsteinInitialData/ReadInter
  polate/src/read.c:559:
    -> Skipping integer variable 'SPACEMASK::space_mask it=335925
  tl=0 rl=0 c=569'. Do not know how to interpolate integers.
  ERROR from host bcn1814.usr.hlrn.de process 569
    while executing schedule bin ReadInterpolate_ReadData, routine
  ReadInterpolate::ReadInterpolate_Read
    in thorn ReadInterpolate, file/home/mvxcombi/ET_Zelmani/Cactus/arrangements/EinsteinInitialData/ReadInter
  polate/src/interpolate.cc:130:
    -> There were 70227 points in 27 variables that could not be
  set.
  cactus_etk_devel:/home/mvxcombi/ET_Zelmani/Cactus/arrangements/Carpet/Carpet/src/helpers.cc:
  275: int Carpet::Abort(const cGH*, int): Assertion `0' failed.
  Rank 569 with PID 1675935 received signal 6
  Writing backtrace to AIC_interp/backtrace.569.txt
  The original grid has 3 ghost zones. I'm using the following
  parfile:
  # What to interpolate
  # -------------------
  ReadInterpolate::files                        =
  "checkpoint.chkpt.it_335925"
  ReadInterpolate::max_number_of_read_variables = 30
  ReadInterpolate::only_these_datasets          = ":Y_e, :rho,
  :temperature, :vel, :Ax, :Ay, :Az, :psi6phi, :kxx, :kxy, :kxz,
  :kyy, :kyz, :kzz, :gxx, :gxy, :gxz, :gyy, :gyz, :gzz, :betax,
  :betay, :betaz, :alp"
  ReadInterpolate::minimum_reflevel             = 0     # Default
  ReadInterpolate::maximum_reflevel             = 1000  # Default

  # Interpolator settings
  # ---------------------
  ReadInterpolate::epsilon                 = 1.e-12 #2  # Default
  ReadInterpolate::interpolator_half_width = -1      # Default,
  use cctk_nghostzones
  ReadInterpolate::interpolator_name       = "Lagrange polynomial
  interpolation"
  ReadInterpolate::interpolator_pars       = "
      order = 3
      boundary_off_centering_tolerance = {1. 1. 1. 1. 1. 1.}
      boundary_extrapolation_tolerance = {1.e-12 1.e-12 1.e-12
  1.e-12 1.e-12 1.e-12}
  "

  # Fixing the read-in data
  # -----------------------
  ReadInterpolate::shift_read_datasets_by[0] = 0.
  ReadInterpolate::shift_read_datasets_by[1] = 0.
  ReadInterpolate::shift_read_datasets_by[2] = 0.

  ReadInterpolate::enforce_symmetries_after_reading = no  #
  Default
  ReadInterpolate::fix_cell_centered_origins        = no  #
  Default

  Any suggestions on how to solve this issue?

  Thanks a lot!!
  -------------- next part --------------
  An HTML attachment was scrubbed...
  URL:<http://lists.einsteintoolkit.org/pipermail/users/attachments/20250725/8744
  4213/attachment.htm>

  ------------------------------

  Subject: Digest Footer

  _______________________________________________
  Users mailing list
  Users@einsteintoolkit.org
  http://lists.einsteintoolkit.org/mailman/listinfo/users


  ------------------------------

  End of Users Digest, Vol 184, Issue 14
  **************************************