Question about Llama's Thornburg04nc set up

List overview All Threads
Download

newer

older

Einstein Toolkit Meeting Reminder

Configuring SimFactory issue

Luciano Combi

20 Oct 2021 20 Oct '21

9:30 p.m.

Hello everyone!

I'm trying to set up an accretion disk using GRHydro and a Llama grid setup without a Cartesian patch (Thornburg04nc). I'm using a private thorn (TorusID) to set up all grid functions including a fixed Kerr-Schild metric.

I'm following a simple parfile in LlamaToy that uses Thornburg04nc and one that Roland shared a few months ago but I'm having seemingly basic issues. In particular, I'm getting the following error:

cactus_etk_devel_gcc_mpi:

...

/gpfs/fs0/home/d/dsiegel/lcombi/ET_devel/Cactus/arrangements/Llama/Coordinates/src/interpolate.cc:407: CCTK_INT4 Coordinates::Coordinates_SymmetryInterpolate(CCTK_POINTER_TO_CONST, CCTK_INT4, CCTK_INT4, CCTK_INT4, CCTK_INT4, CCTK_INT4, CCTK_INT4, const void* const*, CCTK_INT4, const CCTK_INT4*, CCTK_INT4, const CCTK_INT4*, void* const*, CCTK_INT4): Assertion `thevars.at(num_time_derivs).at(num_derivs).at(time_level).at(basevar).at(derivvar) == -1' failed.

In the code, it's mentioned that this could be a problem from CarpetInterp? but not sure what's going on, honestly. I attach the parfile and the full .out, .err.

Thanks a lot in advance,

Best.

-- *Luc Combi* Instituto Argentino de Radioastronomía CONICET, Buenos Aires, Argentina web page. http://sites.google.com/view/lucianocombi

Attachments:

disk_llama.out (application/octet-stream — 1.5 MB)
Tor_M3a8Mt0016r5-43_nuceos_leak_LLAMA6patch_Ye05.par (application/octet-stream — 25.2 KB)
attachment.html (text/html — 2.2 KB)
disk_llama.err (application/octet-stream — 64.3 KB)
disk_llama.log (text/x-log — 200.1 KB)

Show replies by date

Roland Haas

21 Oct 21 Oct

9:03 a.m.

Hello Luciano,

...

I'm following a simple parfile in LlamaToy that uses Thornburg04nc and one that Roland shared a few months ago but I'm having seemingly basic issues. In particular, I'm getting the following error:

...

In the code, it's mentioned that this could be a problem from CarpetInterp? but not sure what's going on, honestly. I attach the parfile and the full .out, .err.

Based on the error and the comment in the code around the assert, namely:

// This does not hold if the caller requests the same // interpolation to be done into different output arrays. // This may happen e.g. when CarpetInterp needs to // differentiate in time. This is arguably a performance bug // in CarpetInterp. (See whether this goes away now.) (It // should!)

something seems to request data at a time not commensurate with the coarsest timestep.

Since you have no mesh refinement that is somewhat strange.

Could you provide one of the backtrace.txt files? So that one can get some idea which part of the code called the interpolator?

Looking at your parfile (thank you for including it) You should be able to use:

Carpet::prolongation_order_time = 0

since you only have 1 refinement level and also remove

CarpetLib::restriction_order_space = 3

which is not used for unigrid (or vertex centered refinement where restriction is always an exact copy so has infinite order).

Neither of these two should really do anything to your run though. The only effect may be that setting

Carpet::prolongation_order_time = 0

may give you a more useful error message from a routine earlier in the call stack.

Yours, Roland

-- My email is as private as my paper mail. I therefore support encrypting and signing email messages. Get my PGP key from http://pgp.mit.edu .

Luciano Combi

9:39 a.m.

Hi Roland,

thanks for the quick response.

Great points. I wasn't really sure how to use the backtrace, but now following the addresses there I found that the problem was in the Outflow thorn. In fact, paying closer attention to the parfile you shared a while back, I see that you don't use the Outflow thorn. Is it possible to use it within Llama maybe tweaking the "coord_system" parameter or the interpolator? Or should I consider alternatives?

Thanks again.

Cheers.

El jue, 21 oct 2021 a las 13:03, Roland Haas (rhaas@illinois.edu) escribió:

...

Hello Luciano,

...
I'm following a simple parfile in LlamaToy that uses Thornburg04nc and

one

...
that Roland shared a few months ago but I'm having seemingly basic

issues.

...
In particular, I'm getting the following error:

...
In the code, it's mentioned that this could be a problem from

CarpetInterp?

...
but not sure what's going on, honestly. I attach the parfile and the full .out, .err.

Based on the error and the comment in the code around the assert, namely:
    // This does not hold if the caller requests the same
    // interpolation to be done into different output arrays.
    // This may happen e.g. when CarpetInterp needs to
    // differentiate in time.  This is arguably a performance bug
    // in CarpetInterp.  (See whether this goes away now.)  (It
    // should!)
something seems to request data at a time not commensurate with the coarsest timestep.

Since you have no mesh refinement that is somewhat strange.

Could you provide one of the backtrace.txt files? So that one can get some idea which part of the code called the interpolator?

Looking at your parfile (thank you for including it) You should be able to use:

Carpet::prolongation_order_time = 0

since you only have 1 refinement level and also remove

CarpetLib::restriction_order_space = 3

which is not used for unigrid (or vertex centered refinement where restriction is always an exact copy so has infinite order).

Neither of these two should really do anything to your run though. The only effect may be that setting

Carpet::prolongation_order_time = 0

may give you a more useful error message from a routine earlier in the call stack.

Yours, Roland

-- My email is as private as my paper mail. I therefore support encrypting and signing email messages. Get my PGP key from http://pgp.mit.edu .

-- *Luc Combi* Instituto Argentino de Radioastronomía CONICET, Buenos Aires, Argentina web page. http://sites.google.com/view/lucianocombi

Roland Haas

10:03 a.m.

Hello Luciano,

...

Great points. I wasn't really sure how to use the backtrace, but now following the addresses there I found that the problem was in the Outflow thorn. In fact, paying closer attention to the parfile you shared a while back, I see that you don't use the Outflow thorn. Is it possible to use it within Llama maybe tweaking the "coord_system" parameter or the interpolator? Or should I consider alternatives?

I would have expected that Outflow should work with Llama since it only has fairly basics use of the interpolator.

I can see that this may be triggered in your parfile since you set:

Outflow::extra_variables = " HydroBase::rho HydroBase::Y_e HydroBase::entropy HydroBase::temperature GRHydro::poynting_scalar "

and "rho" is already in the set of variables that Outflow interpolates all the time i.e. this matches the description ("same interpolation to be done into different output arrays.") of the trigger in the comment.

So you could try what happens if you make sure that you are not requesting any of the variables that Outflow already interpolates then this may work. I think this is only "rho" right now. You can see the full list of default variables in Outflow's source code einsteinanalysis/Outflow/src/outflow.c:

CCTK_STRING input_array_names[NUM_INPUT_ARRAYS] = { "ADMBase::gxx", "ADMBase::gxy", "ADMBase::gxz", "ADMBase::gyy", "ADMBase::gyz", "ADMBase::gzz",

"HydroBase::vel[0]", "HydroBase::vel[1]", "HydroBase::vel[2]", "HydroBase::rho",

"ADMBase::betax", "ADMBase::betay", "ADMBase::betaz", "ADMBase::alp", };

If you could create a ticket with this parfile and linking to this email conversation (via the email archive http://lists.einsteintoolkit.org/pipermail/users/2021-October/date.html) that would be great. This should be supported by Llama.

Yours, Roland

-- My email is as private as my paper mail. I therefore support encrypting and signing email messages. Get my PGP key from http://pgp.mit.edu .

Luciano Combi

27 Oct 27 Oct

10:04 a.m.

Hi Roland,

Excellent, that makes sense and if I deactivate "rho" as an extra variable it works. I can create a ticket in a couple of days.

Thanks a lot.

El jue, 21 oct 2021 a las 14:04, Roland Haas (rhaas@illinois.edu) escribió:

...

Hello Luciano,

...
Great points. I wasn't really sure how to use the backtrace, but now following the addresses there I found that the problem was in the Outflow thorn. In fact, paying closer attention to the parfile you shared a while back, I see that you don't use the Outflow thorn. Is it possible to use it within Llama maybe tweaking the "coord_system" parameter or the interpolator? Or should I consider alternatives?

I would have expected that Outflow should work with Llama since it only has fairly basics use of the interpolator.

I can see that this may be triggered in your parfile since you set:

Outflow::extra_variables = " HydroBase::rho HydroBase::Y_e HydroBase::entropy HydroBase::temperature GRHydro::poynting_scalar "

and "rho" is already in the set of variables that Outflow interpolates all the time i.e. this matches the description ("same interpolation to be done into different output arrays.") of the trigger in the comment.

So you could try what happens if you make sure that you are not requesting any of the variables that Outflow already interpolates then this may work. I think this is only "rho" right now. You can see the full list of default variables in Outflow's source code einsteinanalysis/Outflow/src/outflow.c:

CCTK_STRING input_array_names[NUM_INPUT_ARRAYS] = { "ADMBase::gxx", "ADMBase::gxy", "ADMBase::gxz", "ADMBase::gyy", "ADMBase::gyz", "ADMBase::gzz",
    "HydroBase::vel[0]",
    "HydroBase::vel[1]",
    "HydroBase::vel[2]",
    "HydroBase::rho",

    "ADMBase::betax",
    "ADMBase::betay",
    "ADMBase::betaz",
    "ADMBase::alp",
  };
If you could create a ticket with this parfile and linking to this email conversation (via the email archive http://lists.einsteintoolkit.org/pipermail/users/2021-October/date.html) that would be great. This should be supported by Llama.

Yours, Roland

-- My email is as private as my paper mail. I therefore support encrypting and signing email messages. Get my PGP key from http://pgp.mit.edu .

-- *Luc Combi* Instituto Argentino de Radioastronomía CONICET, Buenos Aires, Argentina web page. http://sites.google.com/view/lucianocombi

1807

Age (days ago)

1813

Last active (days ago)

users@lists.einsteintoolkit.org

4 comments

2 participants

tags (0)

participants (2)

Luciano Combi
Roland Haas