#1053: CarpetIOHDF5: it doesn't honor out3D_every parameter --------------------------------------+------------------------------------- Reporter: bmundim | Owner: eschnett Type: defect | Status: new Priority: major | Milestone: Component: Carpet | Version: Keywords: CarpetIOHDF5 out3D_every | --------------------------------------+------------------------------------- Hi,
I have noticed a strange behaviour of CarpetIOHDF5 when running a bbh simulation with Lovelace release: it doesn't seem to follow the parameter out3D_every. I have attached a modified version of CarpetWaveToyCheckpointTest.par that reproduces this error. I tried to mimic a bbh simulation setup I am using in terms of number of refinement levels (it also helps to slow down the simulation a bit) and what I was interested to output. The experiment goes as follows:
1) create two directories, 000 and 001, where you can copy the attached par file to. Symlink a cactus executable for ET on each directory. Both development and lovelace releases will show this problem. You should have something like this:
{{{
[bruno@frozenstar wavetoytest]$ ls 0* 000: cactus_einstein@ CarpetWaveToyCheckpointTest.par
001: cactus_einstein@ CarpetWaveToyCheckpointTest.par }}}
2) Run the executable on both directories. First on 000 and then on 001.
{{{ ./cactus_einstein CarpetWaveToyCheckpointTest.par }}}
3) Analyze the results. Everything goes all right on the first directory, 000:
{{{ [bruno@frozenstar CarpetWaveToyCheckpointTest]$ h5ls wavetoy::scalarevolve.xyz.h5 Parameters\ and\ Global\ Attributes Group WAVETOY::phi\ it=0\ tl=0\ rl=0 Dataset {43, 43, 43} WAVETOY::phi\ it=0\ tl=0\ rl=1 Dataset {41, 41, 41} WAVETOY::phi\ it=0\ tl=0\ rl=2 Dataset {33, 33, 33} WAVETOY::phi\ it=0\ tl=0\ rl=3 Dataset {33, 33, 33} WAVETOY::phi\ it=0\ tl=0\ rl=4 Dataset {33, 33, 33} WAVETOY::phi\ it=0\ tl=0\ rl=5 Dataset {33, 33, 33} WAVETOY::phi\ it=0\ tl=0\ rl=6 Dataset {33, 33, 33} WAVETOY::phi\ it=0\ tl=0\ rl=7 Dataset {33, 33, 33} WAVETOY::phi\ it=0\ tl=0\ rl=8 Dataset {33, 33, 33} WAVETOY::phi\ it=128\ tl=0\ rl=1 Dataset {41, 41, 41} WAVETOY::phi\ it=128\ tl=0\ rl=2 Dataset {33, 33, 33} WAVETOY::phi\ it=128\ tl=0\ rl=3 Dataset {33, 33, 33} WAVETOY::phi\ it=128\ tl=0\ rl=4 Dataset {33, 33, 33} WAVETOY::phi\ it=128\ tl=0\ rl=5 Dataset {33, 33, 33} WAVETOY::phi\ it=128\ tl=0\ rl=6 Dataset {33, 33, 33} WAVETOY::phi\ it=128\ tl=0\ rl=7 Dataset {33, 33, 33} WAVETOY::phi\ it=128\ tl=0\ rl=8 Dataset {33, 33, 33} WAVETOY::phi\ it=256\ tl=0\ rl=0 Dataset {43, 43, 43} WAVETOY::phi\ it=256\ tl=0\ rl=1 Dataset {41, 41, 41} WAVETOY::phi\ it=256\ tl=0\ rl=2 Dataset {33, 33, 33} WAVETOY::phi\ it=256\ tl=0\ rl=3 Dataset {33, 33, 33} WAVETOY::phi\ it=256\ tl=0\ rl=4 Dataset {33, 33, 33} WAVETOY::phi\ it=256\ tl=0\ rl=5 Dataset {33, 33, 33} WAVETOY::phi\ it=256\ tl=0\ rl=6 Dataset {33, 33, 33} WAVETOY::phi\ it=256\ tl=0\ rl=7 Dataset {33, 33, 33} WAVETOY::phi\ it=256\ tl=0\ rl=8 Dataset {33, 33, 33}
}}}
You can see above that all refinement levels were correctly output every 128 iterations as it was set on the par file. That doesn't happen anymore after recovery, ie on the second directory, 001:
{{{ [bruno@frozenstar CarpetWaveToyCheckpointTest]$ h5ls wavetoy::scalarevolve.xyz.h5 Parameters\ and\ Global\ Attributes Group WAVETOY::phi\ it=262\ tl=0\ rl=7 Dataset {33, 33, 33} WAVETOY::phi\ it=262\ tl=0\ rl=8 Dataset {33, 33, 33} WAVETOY::phi\ it=390\ tl=0\ rl=7 Dataset {33, 33, 33} WAVETOY::phi\ it=390\ tl=0\ rl=8 Dataset {33, 33, 33}
}}}
neither 262 or 390 are multiples of 128! So something is going really wrong here. Note however that the respective 2D slice is correct:
{{{ [bruno@frozenstar CarpetWaveToyCheckpointTest]$ h5ls wavetoy::scalarevolve.xy.h5 Parameters\ and\ Global\ Attributes Group WAVETOY::phi\ it=384\ tl=0\ rl=1 Dataset {41, 41} WAVETOY::phi\ it=384\ tl=0\ rl=2 Dataset {33, 33} WAVETOY::phi\ it=384\ tl=0\ rl=3 Dataset {33, 33} WAVETOY::phi\ it=384\ tl=0\ rl=4 Dataset {33, 33} WAVETOY::phi\ it=384\ tl=0\ rl=5 Dataset {33, 33} WAVETOY::phi\ it=384\ tl=0\ rl=6 Dataset {33, 33} WAVETOY::phi\ it=384\ tl=0\ rl=7 Dataset {33, 33} WAVETOY::phi\ it=384\ tl=0\ rl=8 Dataset {33, 33} }}}
384 is a multiple of 128 and as you can see all active refinement levels for that iteration were written down.
Any idea on how to fix this?
Thanks!