[Users] Loop over the grid points of the base grid

[Haas, Roland]

Hello Erik,

there is one more mode that you need to enter: LOCAL mode which is the
only mode that actually has access to the grid data. This is because a
single refinement level can contain multiple disjoint regions eg when
there is a binary black hole system where each black hole has a set of
refined region surrounding it. Nothing prevents multiple of those
disjoint regions to exists on a single MPI rank.

Then there is SINGLEMAPMODE sandwiched in between which takes care of
looping of multiple map patches (if you are using Llama).

Please see the file arrangements/Carpet/doc/scheduling.pdf (which you
may have to first generate from scheduling.tex) for an explanation of
the modes.

In a nutshell (see eg
arrangements/Carpet/Carpet/src/CallFunction.cc), you
need to add another two loops inside of your ENTER_LEVEL_MODE region:

--8<--
BEGIN_GLOBAL_MODE(cctkGH) {
  ENTER_LEVEL_MODE(cctkGH, 0) {
    BEGIN_LOCAL_MAP_LOOP(cctkGH) {
      BEGIN_LOCAL_COMPONENT_LOOP(cctkGH, CCTK_GF) {
        DECLARE_CCTK_ARGUMENTS;

        for(int k=0;k<cctk_lsh[2];k++) {
          for(int j=0;j<cctk_lsh[1];j++) {
            for(int i=0;i<cctk_lsh[0];i++) {
              int index= CCTK_GFINDEX3D(cctkGH,i,j,k);
              lres += abs(phi[index]-phi_p[index]);
            }
          }

      } END_LOCAL_COMPONENT_LOOP;
    } END_LOCAL_MAP_LOOP;
  } LEAVE_LEVEL_MODE;
} END_GLOBAL_MODE;
--8<--

where the DECLARE_CCTK_ARGUMENTS inside of the LOCAL_COMPONENTS loop
sets up the variable pointers for each component.

Yours,
Roland

> Dear all
> 
> I want to subtract an array at t-dt and t times only at the base grid
> (refinement level 0),  following
> http://lists.einsteintoolkit.org/pipermail/users/2013-May/003004.html, I
> implemented the suggestion in my code as
> 
> BEGIN_GLOBAL_MODE(cctkGH) {
>   ENTER_LEVEL_MODE(cctkGH, 0) {
> 
>  for(int k=0;k<cctk_lsh[2];k++) {
>         for(int j=0;j<cctk_lsh[1];j++) {
>             for(int i=0;i<cctk_lsh[0];i++) {
>                 int index= CCTK_GFINDEX3D(cctkGH,i,j,k);
>                 lres += abs(phi[index]-phi_p[index]);
>              }
>         }
>   }
>  } LEAVE_LEVEL_MODE;
> } END_GLOBAL_MODE;
> 
> However I got a segmentation fault. I think is because of the array index,
> if I comment
> the line
> 
> lres += abs(phi[index]-phi_p[index]);
> 
>  the code works fine.
> My questions are: 1) How can I perform this operation only at the base
> grid?, 2) How can I get the correct index at the base level? .. any help is
> very welcome. Thanks.
> 
> Erik



-- 
My email is as private as my paper mail. I therefore support encrypting
and signing email messages. Get my PGP key from http://pgp.mit.edu .
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 833 bytes
Desc: OpenPGP digital signature
Url : http://lists.einsteintoolkit.org/pipermail/users/attachments/20191025/223dcfc1/attachment.bin