#2853: AHFinderDirect_import_mask() needs an OpenMP pragma

Reporter: Zach Etienne Status: new Milestone: Version: Type: enhancement Priority: minor Component:

While comparing the performance of **AHFinderDirect** with my new apparent horizon finder thorn, **ET_BHaHAHA** (“the ET implementation of the BlackHoles@Home Apparent Horizon Algorithm”), I observed the following runtime distribution in **AHFinderDirect**:

* **AHFinderDirect_find_horizons**: ~56% of total runtime * **AHFinderDirect_import_mask**: ~31% of total runtime * **AHFinderDirect_store**: ~13% of total runtime

#### Context

These measurements were taken using a slightly modified version of the **GW150914 BBH gallery example**, replacing **ML_BSSN** with **BaikalVacuum** at a slightly higher resolution. Runtime data is based on **TimerReport cumulative runtime**. Granted, the import_mask function takes up 0.2% of the total runtime at this find-horizon cadence, but that it takes a significant fraction of the AHFinderDirect runtime at all is simply unacceptable.

#### Issue: AHFinderDirect_import_mask Performance

The **AHFinderDirect_import_mask** function is particularly inefficient. It consists of a simple **3D loop** that **lacks OpenMP parallelization**, significantly impacting performance. Here’s the entire function (with comments removed):

``` extern "C" void AHFinderDirect_import_mask(CCTK_ARGUMENTS) { DECLARE_CCTK_ARGUMENTS_AHFinderDirect_import_mask DECLARE_CCTK_PARAMETERS

assert(ahmask != 0);

for (int k=0; k<cctk_lsh[2]; ++k) for (int j=0; j<cctk_lsh[1]; ++j) for (int i=0; i<cctk_lsh[0]; ++i) { const int ind = CCTK_GFINDEX3D(cctkGH,i,j,k); ahmask[ind] = 0; if (use_mask) then ahmask[ind] = fabs(emask[ind] - 1.0) > 0.01; } } ```

I would strongly recommend adding an OpenMP pragma, as follows:

``` diff --git a/AHFinderDirect/src/driver/find_horizons.cc b/AHFinderDirect/src/driver/find_horizons.cc index 67838eb..4ca0702 100644 --- a/AHFinderDirect/src/driver/find_horizons.cc +++ b/AHFinderDirect/src/driver/find_horizons.cc @@ -98,6 +98,7 @@ DECLARE_CCTK_PARAMETERS

assert(ahmask != 0);

+#pragma omp parallel for for (int k=0; k<cctk_lsh[2]; ++k) for (int j=0; j<cctk_lsh[1]; ++j) for (int i=0; i<cctk_lsh[0]; ++i) ```

‌

-- Ticket URL: https://bitbucket.org/einsteintoolkit/tickets/issues/2853/ahfinderdirect_imp...