[Users] Unexpected slowdown when increasing Carpet::max_refinement_levels (no AMR active)

[Rhiannon Silva]

Hi Mahdi,

The iteration numbering is based on  max_refinement_levels, so you would need to adjust e.g.

CarpetRegrid2::regrid_every                   = 512
AHFinderDirect::find_every                               = 1024

and all the outputting too.  The iteration count is based on the finest possible level, not the actual finest level, so, if you go from 12 to 9 max levels, you can divide all these numbers by 8 to make sure your simulation is doing everything at the equivalent intervals!

I only know this because I got stuck on something related to this recently and found a short note about it here: https://einsteintoolkit.org/arrangementguide/Carpet/documentation.html  (near the end of the page).

Rhiannon.

________________________________
From: Users <users-bounces at einsteintoolkit.org> on behalf of Naseri, Mahdi - (mahdinaseri) <mahdinaseri at arizona.edu>
Sent: 25 October 2025 01:00
To: users at einsteintoolkit.org <users at einsteintoolkit.org>
Subject: [Users] Unexpected slowdown when increasing Carpet::max_refinement_levels (no AMR active)

External email to Cardiff University - Take care when replying/opening attachments or links.
Nid ebost mewnol o Brifysgol Caerdydd yw hwn - Cymerwch ofal wrth ateb/agor atodiadau neu ddolenni.


Hello,

I’m seeing an unexpected performance drop in Carpet (ET). I ran two simulations with identical parameter files (attached) except for:

Carpet::max_refinement_levels = 9     (Run A)
Carpet::max_refinement_levels = 12     (Run B)

During the period I conducted this experiment, all the refinement levels were inactive except for the base level. Therefore, I expected to see the same performance in both Runs A and B, because the active grid structure is the same. However, I realized that Run B is almost 4 times slower than Run A; a plot of wall-clock time per simulation time for these two simulations is also attached for comparison.

Is this behavior known/expected (e.g., due to bookkeeping, allocation, or scheduling costs that scale with the allowed number of levels), or does it suggest a configuration issue on my side? Any pointers on what to check, or how to profile this within Carpet/Cactus, to understand the overhead would be greatly appreciated.

Thank you,
Mahdi Naseri
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.einsteintoolkit.org/pipermail/users/attachments/20251025/e040546b/attachment.htm>