[Users] About SYNCs with staggered variables

[Luciano Combi]

Thanks for the tip Roland.

I have tried it and it seems I'm getting the same efficiency nevertheless.

I was wondering whether, perhaps, the prolongations of the staggered
variables are somehow more expensive/inefficient?

Thanks a lot.


Luciano


On Mon, Apr 1, 2024 at 12:29 PM Roland Haas <rhaas at illinois.edu> wrote:

> Hello Luciano,
>
> Given that factor of 3 that you notice, it could be that this is mostly
> some latency when doing network communication.
>
> If your memory usage allows it you can experiment with the option
>
> CarpetLib::combine_sends = "yes"
>
> see
>
>
> https://bitbucket.org/eschnett/carpet/src/master/CarpetLib/param.ccl#lines-223
>
> and see if that makes things go faster.
>
> This will send all data to all ranks at once instead of going a bit
> slower so may help.
>
> Yours,
> Roland
>
> > Hi, community,
> >
> > I was doing some benchmark tests for my version of GRHydro with
> TimerReport
> > and I found that a lot of time in the code is spent in SYNCs for the
> > staggered variables. Analogous to IllinoisGRMHD I have the following
> lines
> > in the schedule.ccl
> >
> >  {
> > >       SYNC:
> > > GRHydro::em_Ax,GRHydro::em_Ay,GRHydro::em_Az,GRHydro::em_psi6phi
> > >       LANG: C
> > >     } "Schedule symmetries -- Actually just a placeholder function to
> > > ensure prolongations / processor syncs are done BEFORE outer
> boundaries are
> > > updated."
> >
> >
> >
> > According to the timer, this takes 3 times more than other SYNCs. Is
> there
> > a reason for this? Is it related to the prolongation operators for
> > staggered variables?
> >
> > Thanks for your help!
> >
> >
> >
> > *Dr. Luciano Combi*
> > Postdoctoral Researcher
> > Perimeter Institute for Theoretical Physics
> > CITA National Fellow (U. of Guelph)
> > ---
>
>
> --
> My email is as private as my paper mail. I therefore support encrypting
> and signing email messages. Get my PGP key from http://pgp.mit.edu .
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.einsteintoolkit.org/pipermail/users/attachments/20240412/b8996280/attachment.htm>