[Users] About SYNCs with staggered variables

[Roland Haas]

Hello Luciano,

Given that factor of 3 that you notice, it could be that this is mostly
some latency when doing network communication.

If your memory usage allows it you can experiment with the option

CarpetLib::combine_sends = "yes"

see

https://bitbucket.org/eschnett/carpet/src/master/CarpetLib/param.ccl#lines-223

and see if that makes things go faster.

This will send all data to all ranks at once instead of going a bit
slower so may help.

Yours,
Roland

> Hi, community,
> 
> I was doing some benchmark tests for my version of GRHydro with TimerReport
> and I found that a lot of time in the code is spent in SYNCs for the
> staggered variables. Analogous to IllinoisGRMHD I have the following  lines
> in the schedule.ccl
> 
>  {
> >       SYNC:
> > GRHydro::em_Ax,GRHydro::em_Ay,GRHydro::em_Az,GRHydro::em_psi6phi
> >       LANG: C
> >     } "Schedule symmetries -- Actually just a placeholder function to
> > ensure prolongations / processor syncs are done BEFORE outer boundaries are
> > updated."  
> 
> 
> 
> According to the timer, this takes 3 times more than other SYNCs. Is there
> a reason for this? Is it related to the prolongation operators for
> staggered variables?
> 
> Thanks for your help!
> 
> 
> 
> *Dr. Luciano Combi*
> Postdoctoral Researcher
> Perimeter Institute for Theoretical Physics
> CITA National Fellow (U. of Guelph)
> ---


-- 
My email is as private as my paper mail. I therefore support encrypting
and signing email messages. Get my PGP key from http://pgp.mit.edu .
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 833 bytes
Desc: OpenPGP digital signature
URL: <http://lists.einsteintoolkit.org/pipermail/users/attachments/20240401/1a55ba1d/attachment.sig>