[ET Trac] #2853: AHFinderDirect_import_mask() needs an OpenMP pragma

[Zach Etienne]

#2853: AHFinderDirect_import_mask() needs an OpenMP pragma

 Reporter: Zach Etienne
   Status: new
Milestone: 
  Version: 
     Type: enhancement
 Priority: minor
Component: 

While comparing the performance of **AHFinderDirect** with my new apparent horizon finder thorn, **ET\_BHaHAHA** \(“the ET implementation of the BlackHoles at Home Apparent Horizon Algorithm”\), I observed the following runtime distribution in **AHFinderDirect**:

* **AHFinderDirect\_find\_horizons**: ~56% of total runtime
* **AHFinderDirect\_import\_mask**: ~31% of total runtime
* **AHFinderDirect\_store**: ~13% of total runtime

#### Context

These measurements were taken using a slightly modified version of the **GW150914 BBH gallery example**, replacing **ML\_BSSN** with **BaikalVacuum** at a slightly higher resolution. Runtime data is based on **TimerReport cumulative runtime**. Granted, the import\_mask function takes up 0.2% of the total runtime at this find-horizon cadence, but that it takes a significant fraction of the AHFinderDirect runtime at all is simply unacceptable.

#### Issue: AHFinderDirect\_import\_mask Performance

The **AHFinderDirect\_import\_mask** function is particularly inefficient. It consists of a simple **3D loop** that **lacks OpenMP parallelization**, significantly impacting performance. Here’s the entire function \(with comments removed\):

```
extern "C"
  void AHFinderDirect_import_mask(CCTK_ARGUMENTS)
{
DECLARE_CCTK_ARGUMENTS_AHFinderDirect_import_mask
DECLARE_CCTK_PARAMETERS

assert(ahmask != 0);

for (int k=0; k<cctk_lsh[2]; ++k)
for (int j=0; j<cctk_lsh[1]; ++j)
for (int i=0; i<cctk_lsh[0]; ++i)
{
        const int ind = CCTK_GFINDEX3D(cctkGH,i,j,k);
        ahmask[ind] = 0;
        if (use_mask)
           then ahmask[ind] = fabs(emask[ind] - 1.0) > 0.01;
}
}
```

I would strongly recommend adding an OpenMP pragma, as follows:

```
diff --git a/AHFinderDirect/src/driver/find_horizons.cc b/AHFinderDirect/src/driver/find_horizons.cc
index 67838eb..4ca0702 100644
--- a/AHFinderDirect/src/driver/find_horizons.cc
+++ b/AHFinderDirect/src/driver/find_horizons.cc
@@ -98,6 +98,7 @@ DECLARE_CCTK_PARAMETERS
 
 assert(ahmask != 0);
 
+#pragma omp parallel for
 for (int k=0; k<cctk_lsh[2]; ++k)
 for (int j=0; j<cctk_lsh[1]; ++j)
 for (int i=0; i<cctk_lsh[0]; ++i)
```

‌

--
Ticket URL: https://bitbucket.org/einsteintoolkit/tickets/issues/2853/ahfinderdirect_import_mask-needs-an-openmp
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.einsteintoolkit.org/pipermail/trac/attachments/20250207/d311d4bc/attachment.htm>