Hello Luciano,

Given that factor of 3 that you notice, it could be that this is mostly
some latency when doing network communication.

If your memory usage allows it you can experiment with the option

CarpetLib::combine_sends = "yes"

see

https://bitbucket.org/eschnett/carpet/src/master/CarpetLib/param.ccl#lines-223

and see if that makes things go faster.

This will send all data to all ranks at once instead of going a bit
slower so may help.

Yours,
Roland

> Hi, community,
>
> I was doing some benchmark tests for my version of GRHydro with TimerReport
> and I found that a lot of time in the code is spent in SYNCs for the
> staggered variables. Analogous to IllinoisGRMHD I have the following  lines
> in the schedule.ccl
>
>  {
> >       SYNC:
> > GRHydro::em_Ax,GRHydro::em_Ay,GRHydro::em_Az,GRHydro::em_psi6phi
> >       LANG: C
> >     } "Schedule symmetries -- Actually just a placeholder function to
> > ensure prolongations / processor syncs are done BEFORE outer boundaries are
> > updated." 
>
>
>
> According to the timer, this takes 3 times more than other SYNCs. Is there
> a reason for this? Is it related to the prolongation operators for
> staggered variables?
>
> Thanks for your help!
>
>
>
> *Dr. Luciano Combi*
> Postdoctoral Researcher
> Perimeter Institute for Theoretical Physics
> CITA National Fellow (U. of Guelph)
> ---


--
My email is as private as my paper mail. I therefore support encrypting
and signing email messages. Get my PGP key from http://pgp.mit.edu .