Hi all,
While examining different results from my colleague's, I found the vectorization option could develop misleading results. I seldom use the vectorization option but in the configuration file but he mixed up with mine and others leaving:
VECTORISE = yes VECTORISE_ALIGNED_ARRAYS = no VECTORISE_INLINE = yes
in the config file. I found these are default config options in recent simfactory optionlists. Instead, I used to use AVX but always adding e.g., "-fp-model precise" for value safeness.
I attached two figures with the vectorization on/off. For the case of no vectorization, I tested it w/wo AVX and got the identical result.
Hee Il
Hee Il
This looks disconcerting. Using vectorization or not should only affect things at the round-off level (the same way parallelization does).
The vectorization settings you show look good.
I'd like to dig deeper.
- Is your source code public, or would you be willing to share it with a few developers? - Is your parameter file public, or would you be willing to share it with a few developers? - The first thing to do is probably to look at other, more fundamental variables (e.g. metric components) in 3d and see where they begin to differ. The hope is that they would begin to differ early during the run. Do you have such 3d output at the same time from both simulations? Or do you have 1d output e.g. along the x axis for some variables?
-erik
On Thu, Jul 8, 2021 at 9:01 AM Hee Il Kim heeilkim@gmail.com wrote:
Hi all,
While examining different results from my colleague's, I found the vectorization option could develop misleading results. I seldom use the vectorization option but in the configuration file but he mixed up with mine and others leaving:
VECTORISE = yes VECTORISE_ALIGNED_ARRAYS = no VECTORISE_INLINE = yes
in the config file. I found these are default config options in recent simfactory optionlists. Instead, I used to use AVX but always adding e.g., "-fp-model precise" for value safeness.
I attached two figures with the vectorization on/off. For the case of no vectorization, I tested it w/wo AVX and got the identical result.
Hee Il
Users mailing list Users@einsteintoolkit.org http://lists.einsteintoolkit.org/mailman/listinfo/users
Hi Erik,
I'm sorry for the late reply. I'm replying to ET as well to hear from others about my short comments below.
On Thu, Jul 8, 2021 at 11:07 PM Erik Schnetter schnetter@cct.lsu.edu wrote:
Hee Il
This looks disconcerting. Using vectorization or not should only affect things at the round-off level (the same way parallelization does).
I agree with you. But I'm not 100% sure myself whether I can ignore any chance of dynamical instabilities owing to the round-off, even though it would be extremely rare.
The vectorization settings you show look good.
I'd like to dig deeper.
- Is your source code public, or would you be willing to share it with
a few developers?
At least for this issue, I'm using public ET codes (mainly Turing version).
- Is your parameter file public, or would you be willing to share it
with a few developers?
Not public. I guess that this vectorization issue would be reproducible in any circumstances if the resolutions are sufficiently high. For example, the resolution used here is dxmin=0.25 num_levels=6, and dtfac=0.4. I can share with you a public EOS version.
- The first thing to do is probably to look at other, more fundamental
variables (e.g. metric components) in 3d and see where they begin to differ. The hope is that they would begin to differ early during the run. Do you have such 3d output at the same time from both simulations? Or do you have 1d output e.g. along the x axis for some variables?
I can make you an account on my machine to access the data. I will let you know if I gather the data.
Hee Il
-erik
On Thu, Jul 8, 2021 at 9:01 AM Hee Il Kim heeilkim@gmail.com wrote:
Hi all,
While examining different results from my colleague's, I found the
vectorization option could develop misleading results. I seldom use the vectorization option but in the configuration file but he mixed up with mine and others leaving:
VECTORISE = yes VECTORISE_ALIGNED_ARRAYS = no VECTORISE_INLINE = yes
in the config file. I found these are default config options in recent
simfactory optionlists. Instead, I used to use AVX but always adding e.g., "-fp-model precise" for value safeness.
I attached two figures with the vectorization on/off. For the case of
no vectorization, I tested it w/wo AVX and got the identical result.
Hee Il
Users mailing list Users@einsteintoolkit.org http://lists.einsteintoolkit.org/mailman/listinfo/users
-- Erik Schnetter schnetter@cct.lsu.edu http://www.perimeterinstitute.ca/personal/eschnetter/
users@lists.einsteintoolkit.org
-
Erik Schnetter -
Hee Il Kim