On 2012-08-02, at 4:26 , Luca Baiotti baiotti@ile.osaka-u.ac.jp wrote:

On 8/2/12 4:11 PM, Roland Haas wrote:

...

...

CCTK_ReductionArrayHandle is used in several thorns but it is not described in the ReferenceManual. Is it (still) supported? How is it different from CCTK_LocalArrayReductionHandle?

It's part of the "old" reduction interface. Which happens to be the one Carpet supports, so it is widely used :-). I doubt support will go away anytime soon. The calls are documented in the User manual in section C1.7.5 (page 84 in my build).

I see, so CCTK_ReductionArrayHandle is described in the User's Manual but not in the Reference Manual.

...

As for differences I'd consult this section and the reference manual. It seems that one is a reduction over local arrays (ie double a[100]) while the other reduces grid functions (or grid arrays or grid scalars most likely).

I found no reference saying that CCTK_ReductionArrayHandle should be used only for local arrays or only for grid functions.

Which of the two codes below is correct (in a subroutine called by the scheduler)? Why?

For historic reasons, local reductions (where the caller passes in the grid size) and "regular" reductions (which use grid variables) have different operators. In principle, using the wrong operator should lead to an error message at run time.

What happens currently is likely that PUGHReduce and CarpetReduce register the same operators for both local and global reductions in the same order, and (by coincidence!) their handles are thus interchangeable.

Below, number (1) is correct, number (2) is wrong.

Number (2) obtains a handle for a global reduction operation. This handle will have a certain value, e.g. 15. Later, the code uses the _local_ reduction operation with handle 15, which (apparently) is also the local sum reduction.

We should correct this. For example, we could add large, different offsets for local and global handles, so that they would be different. We already do this for other integer IDs in the flesh.

-erik

CCTK_REAL local_array_in,local_array_out;

CCTK_INT handle = CCTK_LocalArrayReductionHandle("sum");

CCTK_ReduceLocArrayToArray1D(cctkGH, -1, handle, &local_array_in, local_array_out, 4, CCTK_VARIABLE_REAL));

CCTK_REAL local_array_in,local_array_out;

CCTK_INT handle_sum = CCTK_ReductionHandle("sum");

CCTK_ReduceLocArrayToArray1D(cctkGH, -1, handle, &local_array_in, local_array_out, 4, CCTK_VARIABLE_REAL));

Best, Luca _______________________________________________ Users mailing list Users@einsteintoolkit.org http://lists.einsteintoolkit.org/mailman/listinfo/users

Luca

I may have been confused. The flesh offers two different reduction APIs, one (the newer one) implemented by PUGH and LocalReduce, the other one (the older one) implemented by Carpet. I may have mixed up the names of the new and the old interfaces...

-erik

On Thu, Aug 2, 2012 at 10:26 PM, Luca Baiotti baiotti@ile.osaka-u.ac.jp wrote:

On 8/2/12 9:19 PM, Erik Schnetter wrote:

...

On 2012-08-02, at 4:26 , Luca Baiotti baiotti@ile.osaka-u.ac.jp wrote:

...

On 8/2/12 4:11 PM, Roland Haas wrote:

...

...

CCTK_ReductionArrayHandle is used in several thorns but it is not described in the ReferenceManual. Is it (still) supported? How is it different from CCTK_LocalArrayReductionHandle?

It's part of the "old" reduction interface. Which happens to be the one Carpet supports, so it is widely used :-). I doubt support will go away anytime soon. The calls are documented in the User manual in section C1.7.5 (page 84 in my build).

I see, so CCTK_ReductionArrayHandle is described in the User's Manual but not in the Reference Manual.

...

As for differences I'd consult this section and the reference manual. It seems that one is a reduction over local arrays (ie double a[100]) while the other reduces grid functions (or grid arrays or grid scalars most likely).

I found no reference saying that CCTK_ReductionArrayHandle should be used only for local arrays or only for grid functions.

Which of the two codes below is correct (in a subroutine called by the scheduler)? Why?

For historic reasons, local reductions (where the caller passes in the grid size) and "regular" reductions (which use grid variables) have different operators. In principle, using the wrong operator should lead to an error message at run time.

What happens currently is likely that PUGHReduce and CarpetReduce register the same operators for both local and global reductions in the same order, and (by coincidence!) their handles are thus interchangeable.

Below, number (1) is correct, number (2) is wrong.

Number (2) obtains a handle for a global reduction operation. This handle will have a certain value, e.g. 15. Later, the code uses the _local_ reduction operation with handle 15, which (apparently) is also the local sum reduction.

We should correct this. For example, we could add large, different offsets for local and global handles, so that they would be different. We already do this for other integer IDs in the flesh.

-erik

...

CCTK_REAL local_array_in,local_array_out;

CCTK_INT handle = CCTK_LocalArrayReductionHandle("sum");

CCTK_ReduceLocArrayToArray1D(cctkGH, -1, handle, &local_array_in, local_array_out, 4, CCTK_VARIABLE_REAL));

CCTK_REAL local_array_in,local_array_out;

CCTK_INT handle = CCTK_ReductionHandle("sum");

CCTK_ReduceLocArrayToArray1D(cctkGH, -1, handle, &local_array_in, local_array_out, 4, CCTK_VARIABLE_REAL));

Thanks for the explanation.

Actually what I have in my code is:

CCTK_REAL local_array_in,local_array_out;

CCTK_INT handle = CCTK_ReductionArrayHandle("sum");

CCTK_ReduceLocArrayToArray1D(cctkGH, -1, handle, &local_array_in, local_array_out, 4, CCTK_VARIABLE_REAL));

Is this correct? If I change the handle call to CCTK_LocalArrayReductionHandle, I get this error at run time:

WARNING level 1 in thorn Cactus processor 0 host mbaiotti2.local (line 1120 of /Users/baiotti/Cactus/src/comm/Reduction.c): -> CCTK_LocalArrayReductionHandle: No handle: '-2' found for reduction operator 'sum'

Luca

The new API requires LocalReduce.

PUGH uses the new API, Carpet uses the old API.

LocalReduce has bugs, e.g. for complex numbers, and possible for some norms.

-erik

On Fri, Aug 3, 2012 at 7:49 AM, Luca Baiotti baiotti@ile.osaka-u.ac.jp wrote:

OK, Erik, but could you give advice on the correct and recommended way to get and use the handle as in the examples I gave?

A related question: do I understand correctly that the new API requires to activate LocalReduce in parfiles (if one uses Carpet), while CarpetReduce is no longer needed and so should not be activated?

Luca

On 8/3/12 8:40 PM, Erik Schnetter wrote:

...

Luca

I may have been confused. The flesh offers two different reduction APIs, one (the newer one) implemented by PUGH and LocalReduce, the other one (the older one) implemented by Carpet. I may have mixed up the names of the new and the old interfaces...

-erik

On Thu, Aug 2, 2012 at 10:26 PM, Luca Baiotti baiotti@ile.osaka-u.ac.jp wrote:

...

On 8/2/12 9:19 PM, Erik Schnetter wrote:

...

On 2012-08-02, at 4:26 , Luca Baiotti baiotti@ile.osaka-u.ac.jp wrote:

...

On 8/2/12 4:11 PM, Roland Haas wrote:

...

> > > CCTK_ReductionArrayHandle is used in several thorns but it is > not described in the ReferenceManual. Is it (still) supported? > How is it different from CCTK_LocalArrayReductionHandle?

It's part of the "old" reduction interface. Which happens to be the one Carpet supports, so it is widely used :-). I doubt support will go away anytime soon. The calls are documented in the User manual in section C1.7.5 (page 84 in my build).

I see, so CCTK_ReductionArrayHandle is described in the User's Manual but not in the Reference Manual.

...

As for differences I'd consult this section and the reference manual. It seems that one is a reduction over local arrays (ie double a[100]) while the other reduces grid functions (or grid arrays or grid scalars most likely).

I found no reference saying that CCTK_ReductionArrayHandle should be used only for local arrays or only for grid functions.

Which of the two codes below is correct (in a subroutine called by the scheduler)? Why?

For historic reasons, local reductions (where the caller passes in the grid size) and "regular" reductions (which use grid variables) have different operators. In principle, using the wrong operator should lead to an error message at run time.

What happens currently is likely that PUGHReduce and CarpetReduce register the same operators for both local and global reductions in the same order, and (by coincidence!) their handles are thus interchangeable.

Below, number (1) is correct, number (2) is wrong.

Number (2) obtains a handle for a global reduction operation. This handle will have a certain value, e.g. 15. Later, the code uses the _local_ reduction operation with handle 15, which (apparently) is also the local sum reduction.

We should correct this. For example, we could add large, different offsets for local and global handles, so that they would be different. We already do this for other integer IDs in the flesh.

-erik

...

CCTK_REAL local_array_in,local_array_out;

CCTK_INT handle = CCTK_LocalArrayReductionHandle("sum");

CCTK_ReduceLocArrayToArray1D(cctkGH, -1, handle, &local_array_in, local_array_out, 4, CCTK_VARIABLE_REAL));

CCTK_REAL local_array_in,local_array_out;

CCTK_INT handle = CCTK_ReductionHandle("sum");

CCTK_ReduceLocArrayToArray1D(cctkGH, -1, handle, &local_array_in, local_array_out, 4, CCTK_VARIABLE_REAL));

Thanks for the explanation.

Actually what I have in my code is:

CCTK_REAL local_array_in,local_array_out;

CCTK_INT handle = CCTK_ReductionArrayHandle("sum");

CCTK_ReduceLocArrayToArray1D(cctkGH, -1, handle, &local_array_in, local_array_out, 4, CCTK_VARIABLE_REAL));

Is this correct? If I change the handle call to CCTK_LocalArrayReductionHandle, I get this error at run time:

WARNING level 1 in thorn Cactus processor 0 host mbaiotti2.local (line 1120 of /Users/baiotti/Cactus/src/comm/Reduction.c): -> CCTK_LocalArrayReductionHandle: No handle: '-2' found for reduction operator 'sum'

Luca