Hello Luciano,
Given that factor of 3 that you notice, it could be that this is mostly some latency when doing network communication.
If your memory usage allows it you can experiment with the option
CarpetLib::combine_sends = "yes"
see
https://bitbucket.org/eschnett/carpet/src/master/CarpetLib/param.ccl#lines-2...
and see if that makes things go faster.
This will send all data to all ranks at once instead of going a bit slower so may help.
Yours, Roland
Hi, community,
I was doing some benchmark tests for my version of GRHydro with TimerReport and I found that a lot of time in the code is spent in SYNCs for the staggered variables. Analogous to IllinoisGRMHD I have the following lines in the schedule.ccl
{
SYNC:GRHydro::em_Ax,GRHydro::em_Ay,GRHydro::em_Az,GRHydro::em_psi6phi LANG: C } "Schedule symmetries -- Actually just a placeholder function to ensure prolongations / processor syncs are done BEFORE outer boundaries are updated."
According to the timer, this takes 3 times more than other SYNCs. Is there a reason for this? Is it related to the prolongation operators for staggered variables?
Thanks for your help!
*Dr. Luciano Combi* Postdoctoral Researcher Perimeter Institute for Theoretical Physics CITA National Fellow (U. of Guelph)