[Users] global reductions in specified domains

[Roland Haas]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hello Vassili,

> how can i now perform the reduction on this "processor-local" 
> quantity?
You can use the same as for the first example that I gave:

schedule.ccl:
schedule integrate in CCTK_ANALYSIS AFTER setup_integrand
{
  OPTIONS: global
  LANG: C
} "reduce 'helper' grid scalar on all processes"

interface.ccl:
CCTK_REAL rho_total_shell[nradii] TYPE=SCALAR "blah"
CCTK_REAL rho_in_shell[nradii] TYPE=SCALAR "blah"

C-code:
void integrate(CCTK_ARGUMENTS) {
  ...
  CCTK_ReduceArray(cctkGH,-1, op_sum, nradii,
                   CCTK_VARIABLE_REAL, rho_total_shell, 1/*num_dims*/,
                   1/*num_in_arrays*/, CCTK_VARIALBE_REAL, nradii,
                   rho_in_shell);
}

Once intergrate ran then on all processors rho_total_shell will
contain the sums of rho_in_shell (one sum per radius).

> if i write a new function, will it have access to all the instances
>  every processor creates?
I don't understand. Any scheduled function always only has access to
processor local data (since you cannot access remote memory using MPI).

> if i calculate the reduction inside the "processor-local" function,
>  will it work? because the function is global loop-local and i will
>  need a loop to perform the integration on every shell i specify..
Do you mean loop-local function? You cannot call the reduction inside
of the loop-local routine. It will most likely cause an MPI hang.
Instead you have to schedule a separate GLOBAL (no loop-local) routine
to call do the reduction.

> i tried to manually change modes using the carpet c++ macros, but 
> that has led to segmentation faults only so far...
> 
> any idea how i could do that?
The thorn LSUThorns Refluxing does that (in its correct.cc file as far
as I remember). I'd check there. Otherwise Erik has at times explained
how this is done. Most recently here:

http://lists.einsteintoolkit.org/pipermail/users/2013-May/003004.html

Yours,
Roland

- -- 
My email is as private as my paper mail. I therefore support encrypting
and signing email messages. Get my PGP key from http://keys.gnupg.net.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.15 (GNU/Linux)
Comment: Using GnuPG with Icedove - http://www.enigmail.net/

iEYEARECAAYFAlKNUmQACgkQTiFSTN7SboWl2gCgvtMRxvrPty5vBRNF2FhsLc3Z
yAkAoKaEw0CXyE+JggAcimffPswgRGZi
=O+Ks
-----END PGP SIGNATURE-----