Comment (by Liwei Ji):

I’m not surprised with this speed without subcycling.
You can do the following to improve a bit

1. Turn off calc_ADMRHS_vars, it’s not required for BBH simulations.
2. Use 2 OMP instead of 7. It’s the best number for me. You should try different OMP yourself.
3. You can also play a bit with the par max_tile_size_x while keeping max_tile_size_y/z small. I didn’t see big speed difference for myself, but you can still try.

For the memory issue, if you are using Z4c from the development branch of my fork, it has 5 copies of state vector for the purpose of subcycling. You can use the main branch instead.

@Roland Haas could you take a look at the run_cactus.sh and see if there anything suspicions?

--
Ticket URL: https://bitbucket.org/einsteintoolkit/tickets/issues/2882/running-bbh-with-carpetx-with-cpus-high