many thanks for this, Roland! i have not yet had the time to try it, i'll do it within the next few days and report back.
thanks, Miguel
On 27/02/20 01:17, Roland Haas wrote:
Hello Miguel, all,
I have not yet gotten to LoopControl proper, but I have at least made the plain Cactus CCTK_LOOP macros work (the one you get if you do not have #include "loopcontrol.h").
They are in the branch "rhaas/loops" of the flesh and I have also updated / fixed / generated the tests in TestLoop where I have checked that Fortran and C code (which I have not touched) produce the same result.
You can use them via:
cd repos/flesh git fetch git checkout rhaas/loops cd ../repos/cactustest git fetch git checkout rhaas/loops
then build (no need to clean).
You will have to take care of making your loop variables private on your own (unless you trust OpenMP + Fortran to be smart enough to do so for you, I believe the standard claims is should be... but that may only apply to the loop directly following the "omp do" directive) and use CCTK_LOOP3_XXX_OMP_PRIVATE(name). See TestLoopFortran.F90 for examples (the default(none) is purely optional and just helped me debug things).
So basically:
!$OMP PARALLEL private(i,j,k) CCTK_LOOP3_INT_OMP_PRIVATE(int3) CCTK_LOOP3_INT(int3, i,j,k) ! do something CCTK_ENDLOOP3_INT(int3) !$OMP END PARALLEL
The calling syntax of the PRIVATE and DECLARE macros may still change. Right now the C LOOP macros internally declare the loop and normal variables (i, ni) while my current set of Fortran DECLARE and PRIVATE variables do not declare them or make them private (they cannot, they do not now know their name).
So the options are to have either (1):
integer :: i,ni CCTK_LOOP1_DECLARE(foo) !$OMP PARALLEL private(i,ni) CCTK_LOOP1_BND_OMP_PRIVATE(int1) CCTK_LOOP1_INT(int1, i,ni) ! do something CCTK_ENDLOOP1_INT(int1) !$OMP END PARALLEL
or (2):
CCTK_LOOP1_DECLARE(foo, i,ni) !$OMP PARALLEL CCTK_LOOP1_BND_OMP_PRIVATE(int1,i,ni) CCTK_LOOP1_INT(int1, i,ni) ! do something CCTK_ENDLOOP1_INT(int1) !$OMP END PARALLEL
ie carry a couple of extra arguments (2*dim) in the DECLARE and PRIVATE macros but rest assured that all variables are properly declared private no matter what.
I believe LoopControl went with (2) but Cactus' cctk_Loop.h (which was broken before as far as I can tell) seems to have aimed for (1).
Having just implemented (1) I am actually leaning towards (2) right now, but then I do not actually use the LOOP macros in Fortran.
Yours, Roland
Hello Miguel,
ah, but this is for when one uses the CCTK_LOOP3_ALL macro... i'm using the CCTK_LOOP3_INT macro, so my understanding is that i need to declare instead
CCTK_LOOP3_INT_DECLARE(int3)which i'm doing (and which works fine without the OMP statements). the tests don't seem to use OMP statements; could this be untested?
Yes, this certainly could very well be be untested.
I had a look at LoopControl itself (carpet/LoopControl/src/loopcontrol_fortran.inc) where there are eg LC_LOOP3STR_NORMAL_OMP_PRIVATE(name, i,j,k) macros.
Then looking at cctk_Loop.h there are also INTERIOR, BOUNDARY and INTBOUNDARIES flavors eg CCTK_LOOP3_INTBND_OMP_PRIVATE(name) which (unfortunately) has incompatible calling syntax with LoopControl's macro (this is a bug...).
So without changing your code you would have to use (see cctk_Loop.h lines 3223, 3468, 3445.
then that would be gladly accepted. Right now those sections do not mention Fortran or OpenMP at all.
sure, i'd be happy to propose some text once i manage to have this working.
Thank you.
Yours, Roland