Comment (by Erik Schnetter):

Lunan

Cactus (and Carpet) use int to access arrays. This means that the maximum number of grids points is limited to be about 2e9.

Parts of Carpet have been extended to use 64-bit integers. I think that many parts of Carpet will work correctly as long as the process-local grid size is less than 2e9. Unfortunately, there is no good way to flag integer overflow errors in C or C++, and it is thus difficult to debug this.

You might get lucky if you simply disable this check. There are other checks for process-local grid sizes that would trigger if things go wrong. If you don’t get lucky, then there will be weird errors. I assume that most such errors will affect “unimportant” things such as calculating the average over a full grid function, or estimating the current progress rate in terms of number of grid points per second that are evolved.

If you are familiar with C/C++ and integer overflow, then this is worth an attempt. You can also build a version of your code with compiler options that check for signed integer overflow to see whether this triggers an error. I’ll be happy to assist if needed.

If you are not familiar with this, then I would not advise trying this, because the C/C++ standards are not exactly user friendly. In this case, using a smaller domain is the only way to go. You might try using mesh refinement, which generally leads to smaller numbers of grid points.

We’re currently working on a new driver “CarpetX”. We should make sure there are no respective limitations there.

-erik

--
Ticket URL: https://bitbucket.org/einsteintoolkit/tickets/issues/2471/carpet-npoints-d-max-issue