#1612: reace condition when building Cactus utilities
--------------------+-------------------------------------------------------
Reporter: rhaas | Owner:
Type: defect | Status: new
Priority: minor | Milestone:
Component: Cactus | Version: development version
Keywords: |
--------------------+-------------------------------------------------------
I just compiled on bluewaters and received output
{{{
Done creating cactus_simO3.
All done !
Building utilities for simO3
Building utilities for simO3
...
Creating hdf5_recombiner in /mnt/a/u/sciteam/rhaas/ET_trunk/exe/simO3 from
/mnt/a/u/sciteam/rhaas/ET_trunk/configs/simO3/build/CarpetIOHDF5/hdf5_recombiner.o
mkdir: cannot create directory `/tmp/1399318629': File exists
mkdir: cannot create directory `/tmp/1399318629': File exists
}}}
with the full (last part of) the log in the attached file.
So it seems as if we have a race condition in the make system that causes
it to try and build the utilities twice (in parallel).
On bluewaters simfactory (which I used) builds Cactus using 16 processes.
--
Ticket URL: <https://trac.einsteintoolkit.org/ticket/1612>
Einstein Toolkit <http://einsteintoolkit.org>
The Einstein Toolkit
#1613: provide CCTK_VParamWarm
-------------------------+--------------------------------------------------
Reporter: rhaas | Owner:
Type: enhancement | Status: new
Priority: optional | Milestone:
Component: Cactus | Version: development version
Keywords: |
-------------------------+--------------------------------------------------
it would be nice if there was a CCTK_VParamWarn along with CCTK_PARAMWARN,
analogous to CCTK_VWarn and CCTK_WARN.
--
Ticket URL: <https://trac.einsteintoolkit.org/ticket/1613>
Einstein Toolkit <http://einsteintoolkit.org>
The Einstein Toolkit
#1326: running loopcontrol on strange number of threads fails
-----------------------------------+----------------------------------------
Reporter: rhaas | Owner:
Type: defect | Status: new
Priority: minor | Milestone:
Component: EinsteinToolkit thorn | Version:
Keywords: LoopControl |
-----------------------------------+----------------------------------------
my machine has 8 cores (according to /proc/cpuinfo). Running eg the
trigger test with 3 threads fails inside of loopcontrol.
To reproduce:
{{{
export OMP_NUM_THREADS=3
mpirun -n 2 exe/cactus_bns_all
arrangements/AEIThorns/Trigger/test/trigger.par
}}}
--
Ticket URL: <https://trac.einsteintoolkit.org/ticket/1326>
Einstein Toolkit <http://einsteintoolkit.org>
The Einstein Toolkit
#1361: disable hyperthreading in loopcontrol by default
-----------------------------------+----------------------------------------
Reporter: rhaas | Owner:
Type: defect | Status: new
Priority: major | Milestone:
Component: EinsteinToolkit thorn | Version:
Keywords: LoopControl |
-----------------------------------+----------------------------------------
when running with both openmp and sufficiently many threads that
hyperthreading threads are used, many tests using LoopControl (Cartoon,
RotatingSymmety180, RotatingSymmetry90) fail.
This can be tracked down to disabling hyperthreading support in
LoopControl (ie. turning of hyperthreading makes things work).
In particular on bethe with smt and 8 physical cores:
The Cartoon/test_cartoon_2.par test shows differences from the recorded
results when run with 16 threads (but not with 8 threads). If I then go
ahead and disable OMP in all ML source files but ML_BSSN_enforce *and*
comment out the #include "loopcontrol.h", then the difference goes away.
Adding back #include "loopcontrol.h" brings back the error.
Some further experimenting with LoopControl's options shows that indeed
the use_smt_threads option is what causes problems. If I turn it off
things work fine even with a vanilla source tree. Otherwise relative
differences are on the order 1e-7 and absolute 1e-11 (in
momx_z_[2][2].xg). Without smt the results are identical to the stored
values.
The issue only occurs in combination of OpenMP, vectorization and
hyperthreading. The issue is independent of the compiler (both intel 13
and gcc 4.4 show the same behaviour), and vectorization (sse2) and many
threads (up to 4 times the number of physical cores) works fine on non-smt
machines.
I propose to disable LoopControl::use_smt_threads by default. Note that we
cannot completely remove it since apparently for Vesta (a Blue Gene/Q) smt
is required to get and multi-threading at all.
--
Ticket URL: <https://trac.einsteintoolkit.org/ticket/1361>
Einstein Toolkit <http://einsteintoolkit.org>
The Einstein Toolkit
#1204: carpet bug
-------------------------------------+--------------------------------------
Reporter: abdik@… | Owner: eschnett
Type: defect | Status: new
Priority: major | Milestone:
Component: Carpet | Version:
Keywords: |
-------------------------------------+--------------------------------------
The latest version of carpet seems to contain a bug that affects
AMR+multipatch runs. My stderr and stdout and par file are attached. Found
by Roland and Ernazar.
--
Ticket URL: <https://trac.einsteintoolkit.org/ticket/1204>
Einstein Toolkit <http://einsteintoolkit.org>
The Einstein Toolkit
#997: problem in appending output after recovery
----------------------------------------+-----------------------------------
Reporter: corvino.giovanni@… | Owner: eschnett
Type: defect | Status: new
Priority: major | Milestone:
Component: Carpet | Version:
Keywords: |
----------------------------------------+-----------------------------------
I have a problem in appending output files from Carpet. I used to produce
3d HDF5 output of grid variables
and write the output in the same directory also after recovery from
checkpoint. The new output was automatically
appended to the existing one. Now I am producing h5 output also on 2D
slices but in this case the output is overwritten
so I lost the data for all but the last recovery.
--
Ticket URL: <https://trac.einsteintoolkit.org/ticket/997>
Einstein Toolkit <http://einsteintoolkit.org>
The Einstein Toolkit
#1624: remove MOLDOESCOMPLEX from MoL
-----------------------------------+----------------------------------------
Reporter: knarf | Owner:
Type: enhancement | Status: new
Priority: optional | Milestone:
Component: EinsteinToolkit thorn | Version: development version
Keywords: |
-----------------------------------+----------------------------------------
MOLDOESCOMPLEX is an old #define in MoL, and seems to be unused for quite
some time now. It also comes with the comment "even using it probably
doesn't work" in the commit. I suggest to remove it (removing the code
within).
--
Ticket URL: <https://trac.einsteintoolkit.org/ticket/1624>
Einstein Toolkit <http://einsteintoolkit.org>
The Einstein Toolkit
#1518: Parameter parser and CCTK_ParameterSet interpret leading zeros in numbers
differently
--------------------+-------------------------------------------------------
Reporter: rhaas | Owner:
Type: defect | Status: new
Priority: minor | Milestone:
Component: Cactus | Version: development version
Keywords: |
--------------------+-------------------------------------------------------
the parameter parser allows things like:
{{{
thorn::param1 = 011
thorn::param2 = 012.34
}}}
in parameter files. For floating point values this is a bit unexpected but
otherwise mostly harmless. For integers the situation is a bit more
complex since in C a leading zero is used to indicate a octal number. And
(worse) while the parameter file parser converts the string "011" to the
number 11 the Cactus call CCTK_ParameterSet will convert it to 9. The
difference is ultimately the difference between calling atof (Parser) and
strtol (CCTK_ParameterSet).
To avoid confusion it would likely be good to change CCTK_ParameterSet to
behave the way the Parameter parser does. This is a change in behaviour
compared to the pre-Piraha parser, however I suspect the number of users
that actually used octal (or hexedecimal) notation in their parameter
files is small.
The change is to change {{{inval = strtol (value, &endptr, 0);}}} to
{{{inval = strtol (value, &endptr, 10);}}} in line 2209 of
src/main/Parameters.c and similar in line 2270.
--
Ticket URL: <https://trac.einsteintoolkit.org/ticket/1518>
Einstein Toolkit <http://einsteintoolkit.org>
The Einstein Toolkit
#428: Forbid configuration names ending in -reconfig
----------------------+-----------------------------------------------------
Reporter: eschnett | Owner:
Type: defect | Status: new
Priority: minor | Milestone:
Component: Cactus | Version:
Keywords: |
----------------------+-----------------------------------------------------
I accidentally created a Cactus configuration with a name like "sim-
reconfig". This confuses Cactus, because the command "make sim-reconfig"
can then mean either to build the "sim-reconfig" configuration, or to
reconfigure the "sim" configuration.
Cactus should catch and forbid these cases.
This is probably most cleanly handled by using a script instead of a
Makefile to interpret the user commands.
--
Ticket URL: <https://trac.einsteintoolkit.org/ticket/428>
Einstein Toolkit <http://einsteintoolkit.org>
The Einstein Toolkit
#1610: Switch Cactus to 64 bits
-------------------------+--------------------------------------------------
Reporter: eschnett | Owner:
Type: enhancement | Status: new
Priority: major | Milestone:
Component: Cactus | Version: development version
Keywords: |
-------------------------+--------------------------------------------------
Although pointers can have 64 bits, integers in Cactus are de facto (since
we use "int" in many places) restricted to 32 bits. This limit is easily
reached, and this has in fact been a problem on at least two occasions.
One issue comes from Carpet's numbering of grid points. With 25 refinement
levels, the coarse grid can be at most 127^3, and this limit was reached
in production simulations years ago.
Another issue comes from large 1D arrays. It is easily possible to have a
1D array with 10M (10^7) elements that is replicated across processes
(distrib=const). In this case, one can use at most 200 MPI processes.
We should devise a plan to switch Cactus to 64 bits. One option would be
to replace most "int" by "CCTK_INT", so that those affected can switch
these to 64 bits.
I want to note that using 64-bit loop counters is often slightly faster
than using 32-bit loop counters (on 64-bit architectures), since the
32->64 bit conversion for array indexing can be omitted.
--
Ticket URL: <https://trac.einsteintoolkit.org/ticket/1610>
Einstein Toolkit <http://einsteintoolkit.org>
The Einstein Toolkit