#163: Assertion `tl>=0 and tl<timelevels' failed error ------------------------------------+--------------------------------------- Reporter: azebrowski@… | Owner: eschnett Type: defect | Status: new Priority: major | Milestone: Component: Carpet | Version: Keywords: | ------------------------------------+--------------------------------------- I'm currently running a Cactus simulation based off the ET mclachlan parameter file. I have some custom thorns which force checkpointing every iteration, and then write new parameter files. The new parameter files are used to run specific functions from the host simulation ("spawning"), but I'm having some problems resuming simulations.
Currently, I get this error when resuming:
INFO (Carpet): GF: rhs: 818k active, 1440k owned (+76%), 1896k total (+32%), 328 steps/time cactus_sim: /home/azebrowski/Cactus/arrangements/Carpet/CarpetLib/src/th.hh:79: double th::get_time(int, int, int) const: Assertion `tl>=0 and tl<timelevels' failed. [cyder:32759] *** Process received signal *** [cyder:32759] Signal: Aborted (6)
I'm guessing there's a parameter I'm not setting properly in my child simulation, could anyone give me a pointer to where I should be looking? I looked for things relating to timelevels in the host/spawned parameter files, but didn't see anything that stood out. My parameter files used and full output are attached to this email, with the disclaimer that I modified the spawned parameter file to run every function instead of skipping some in an attempt to bypass any problems that could be caused by skipping some Carpet function on accident.
I've made a bzipped tarball containing the checkpointed data from the simulation. It contains several parameter files. The parameter file of interest here is spawn.par, as it doesn't use any of my custom code but still causes Cactus to abort with an error. I left the other parameter files in on the off chance that I might need to refer to them later.
Here is the source parameter file, which creates the spawned simulation: http://www.cct.lsu.edu/~azebrowski/ml-ahfinder-spawn.par
Here is the spawned simulation's parameter file: http://www.cct.lsu.edu/~azebrowski/spawn.par
Here is the full checkpointed data and another copy of the spawned parameter file: http://www.cct.lsu.edu/~azebrowski/data.tar.bz2
Other information: I ran the simulation using OpenMP with 12 cores to generate the checkpointed data. I've also tried MPI, but that didn't seem to make a difference.
Thornlist: http://www.cct.lsu.edu/~azebrowski/ThornList
gcc: azebrowski@cyder:~/Cactus$ gcc -v Using built-in specs. Target: x86_64-linux-gnu Configured with: ../src/configure -v --with-pkgversion='Ubuntu 4.4.3-4ubuntu5' --with-bugurl=file:///usr/share/doc/gcc-4.4/README.Bugs --enable-languages=c,c++,fortran,objc,obj-c++ --prefix=/usr --enable- shared --enable-multiarch --enable-linker-build-id --with-system-zlib --libexecdir=/usr/lib --without-included-gettext --enable-threads=posix --with-gxx-include-dir=/usr/include/c++/4.4 --program-suffix=-4.4 --enable-nls --enable-clocale=gnu --enable-libstdcxx-debug --enable-plugin --enable-objc-gc --disable-werror --with-arch-32=i486 --with-tune=generic --enable-checking=release --build=x86_64-linux-gnu --host=x86_64-linux-gnu --target=x86_64-linux-gnu Thread model: posix gcc version 4.4.3 (Ubuntu 4.4.3-4ubuntu5)
Fortran is gfortran-4.4
I'm using the Mercurial version of Carpet, and the ET development thorns.
If you need more information, please let me know.