#1757: Stampede needs to be updated
----------------------+-----------------------------------------------------
Reporter: hinder | Owner:
Type: task | Status: new
Priority: critical | Milestone:
Component: Other | Version: development version
Keywords: backport |
----------------------+-----------------------------------------------------
The SimFactory definition for Stampede currently uses the intel/13.1.1.163
compiler. This compiler will be removed next Tuesday (see
https://portal.tacc.utexas.edu/user-guides/stampede/intel15). According
to TACC, the current default is the older intel/13.0.2.146, and this is
the recommended and default compiler, and will remain so. Both compilers
have the "restrict" keyword blacklisted in Cactus (#1276). They will
install intel/15.0.1 at the end of April. There is also intel/14.0.1.106
available, but it is labeled as "limited software stack". It is not clear
why they are removing a newer compiler, or why they chose to do this a
month before installing an even newer one.
Our options are:
1. Upgrade to intel/14.0.1.106, which is labeled as "limited software
stack"
2. Downgrade to intel/13.0.2.146, which is the default and will remain so
for a while
This should be done both for the trunk and the release branch. I suggest
that option 2 is the most conservative, and probably the easiest, as there
might be libraries which are not compiled for intel/14.0.1.106.
--
Ticket URL: <https://trac.einsteintoolkit.org/ticket/1757>
Einstein Toolkit <http://einsteintoolkit.org>
The Einstein Toolkit
#2007: MoL fails to build with Intel compiler
---------------------+------------------------------------------------------
Reporter: hinder | Owner: eschnett
Type: defect | Status: new
Priority: blocker | Milestone:
Component: Cactus | Version: development version
Keywords: |
---------------------+------------------------------------------------------
Compiling MoL gives this error:
{{{
/home/ianhin/Cactus/EinsteinToolkitGit/arrangements/CactusNumerical/MoL/src/RK4-RK2.c(60):
error: unrecognized OpenMP #pragma
#pragma omp /*parallel for*/
^
compilation aborted for
/home/ianhin/Cactus/EinsteinToolkitGit/configs/sim/build/MoL/RK4-RK2.c
(code 2)
}}}
This code was introduced in
https://bitbucket.org/cactuscode/cactusnumerical/commits/09a209266daa281a3e….
Removing the commented portion doesn't help. Replacing it with
{{{
#pragma omp parallel
}}}
allows the code to compile.
This is on Minerva, with icc (ICC) 16.0.1 20151021.
--
Ticket URL: <https://trac.einsteintoolkit.org/ticket/2007>
Einstein Toolkit <http://einsteintoolkit.org>
The Einstein Toolkit
#1458: intel 13 2013.1.117 miss-compiles asserts in TwoPunctures tp_utils.c
-----------------------------------+----------------------------------------
Reporter: rhaas | Owner:
Type: defect | Status: new
Priority: major | Milestone:
Component: EinsteinToolkit thorn | Version: development version
Keywords: TwoPunctures |
-----------------------------------+----------------------------------------
this continues the thread in #1429 in particular
https://trac.einsteintoolkit.org/ticket/1429#comment:14.
Currently we need to determine which versions are affected.
--
Ticket URL: <https://trac.einsteintoolkit.org/ticket/1458>
Einstein Toolkit <http://einsteintoolkit.org>
The Einstein Toolkit
#1599: CarpetInterp/waveinterp_2p tests constant in time initial data
--------------------------+-------------------------------------------------
Reporter: rhaas | Owner: eschnett
Type: defect | Status: new
Priority: minor | Milestone:
Component: Carpet | Version: development version
Keywords: CarpetInterp |
--------------------------+-------------------------------------------------
the test waveinterp_2p sets up initial data using IDWaveMoL (fom
CactusExamples) using kx=ky=kz = 0. and slopet = 0.1. Ie the initial data
is constant in space and has frequency 0.1. It then evolves this using
WaveMoL and interpolates in space and time with CarpetInterp.
Since the function is constant in space (exactly so, see eg the output in
InterpToArray::array1d_vars[0]) the interpolation in space just tests
roundoff errors. The second time derivative (array1d_vars[2]) should be
zero and all we measure in the test is truncation errors on the level of
1e-9. This makes the test very sensitive to -Ofast, -O0, gcc vs. intel
etc. and not a very good test.
I would rather either use a Gausian wave for initial data so that there is
some actual data in the result and not just numerical noise, or leave out
the 2nd time derivative from the test data.
The test fails due to significant errors in a virtual machine using
ubuntu.cfg (which usies -ffast-math and -O2).
--
Ticket URL: <https://trac.einsteintoolkit.org/ticket/1599>
Einstein Toolkit <http://einsteintoolkit.org>
The Einstein Toolkit
#2073: IO corruption on SDSC oasis file systems
-------------------------------+--------------------------------------------
Reporter: rhaas | Owner:
Type: defect | Status: new
Priority: major | Milestone:
Component: Other | Version: development version
Keywords: Comet Gordon SDSC |
-------------------------------+--------------------------------------------
I am experiencing file corruption in ASCII output produced by the Cactus
code on Comet's (and Gordon's as far as I remember) scratch file systems.
This manifests as lines of output being mashed together in the output
file.
I have added strace calls to my job script to capture all arguments to the
OS's write() function and re-created the write() calls based on this.
Those write calls, when replayed on a login node, do produce a correct (no
mashed lines) file.
All output to the file in question was from rank 0 only even though the
code used MPI and ran on two MPI ranks.
The same code and number of MPI ranks produces a correct output file when
run on the $HOME file system.
Thus it seems to me as if there may be an issue with the file system. I
can try and reduce the test case to a more minimal example (right now it
is a full simulation even though it runs only for <1minute) .
You can find the job script (for account, SLURM options etc) here:
/oasis/scratch/comet/rhaas/temp_project/simulations/OSTREAM_2_12/output-0000/SIMFACTORY/SubmitScript
the script that launches the MPI executable here:
/oasis/scratch/comet/rhaas/temp_project/simulations/OSTREAM_2_12/output-0000/SIMFACTORY/RunScript
the strace output here:
/home/rhaas/strace/strace.1882[67].log
and the awk script to recreate the write calls is:
gawk -vFS='"' '/write.*\/grid-coordinates.xy.asc/{print "printf
\""$2"\""}' ~/strace.18826.log >recreate.sh
The corrupted line is eg. line 161 of
/oasis/scratch/comet/rhaas/temp_project/simulations/OSTREAM_2_12/output-0000/TEST/sim/CarpetIOASCII/newsep
/grid-coordinates.xy.asc
which reads
1 4 3 4 1 0.1666666666660.505076272276105285714 etc
but should read
1 4 3 4 1 0.166666666666667 -0.0714285714285714
I can avoid the file corruption by flushing the output file after each
line.
I am wondering if there is anything known about this or if there is a
workaround that does not boil to first writing all data to a file system
local to the compute node and copying to /oasis/scratch after the job is
finished (how much local space would be available since I would also have
to do so for eg checkpoint files and 3d hdf5 output).
--
Ticket URL: <https://trac.einsteintoolkit.org/ticket/2073>
Einstein Toolkit <http://einsteintoolkit.org>
The Einstein Toolkit
#1919: have make newthorn create a functioning thorn
-------------------------+--------------------------------------------------
Reporter: rhaas | Owner:
Type: enhancement | Status: new
Priority: optional | Milestone:
Component: Cactus | Version: development version
Keywords: |
-------------------------+--------------------------------------------------
Currently the newthorn make target does not create a functioning thorn. It
would be good if it produced a skeleton thorn that compiled.
--
Ticket URL: <https://trac.einsteintoolkit.org/ticket/1919>
Einstein Toolkit <http://einsteintoolkit.org>
The Einstein Toolkit
#2022: thorn guide on et website links to pdf pages
-------------------------------------+--------------------------------------
Reporter: rhaas | Owner:
Type: defect | Status: new
Priority: major | Milestone:
Component: EinsteinToolkit website | Version: development version
Keywords: |
-------------------------------------+--------------------------------------
This is a continuation #2010 which was about the thorn guide being
missing. This one is about the link behind "Thorn Guide" on
http://einsteintoolkit.org/documentation.html, namely
http://einsteintoolkit.org/thornguide/ points to an auto-generated index
page of a directory of pdf files. Instead of the correct HTML page for a
table of content pointing to html pages of documentation.
Likely requires web server access to fix.
--
Ticket URL: <https://trac.einsteintoolkit.org/ticket/2022>
Einstein Toolkit <http://einsteintoolkit.org>
The Einstein Toolkit
#2030: Multi-block boundaries leave uninitialized boundary points
-----------------------------------+----------------------------------------
Reporter: eschnett | Owner:
Type: defect | Status: new
Priority: major | Milestone:
Component: EinsteinToolkit thorn | Version: development version
Keywords: |
-----------------------------------+----------------------------------------
Applying multi-block boundaries can leave uninitialized boundary points.
This happens when multi-block boundaries are used in combination with
another boundary condition that looks at the interior of the grid, such as
e.g. a symmetry condition or radiative / extrapolating boundaries.
There is another bug currently in McLachlan that applies its "scalar"
boundary conditions to too few grid points (it uses "BoundaryNoSync"
instead of "Boundary"), which means there are uninitialized boundary
points for {{{initial_boundary_condition = "scalar"}}} as well in multi-
block systems.
--
Ticket URL: <https://trac.einsteintoolkit.org/ticket/2030>
Einstein Toolkit <http://einsteintoolkit.org>
The Einstein Toolkit
#2053: simfactory's hard-coded rsync options override a mdb entry's rsyncopts
------------------------+---------------------------------------------------
Reporter: rhaas | Owner:
Type: defect | Status: new
Priority: minor | Milestone:
Component: SimFactory | Version: development version
Keywords: |
------------------------+---------------------------------------------------
Right now, in line 165 of simfactory/lib/sim-sync.py
{{{
cmd = "%s --rsh=%s --rsync-path=%s %s %s %s" % (rsynccmd,
simlib.QuoteSafe(sshcmd), simlib.QuoteSafe(machineEntry.rsynccmd),
rsyncopts, machineEntry.rsyncopts, arguments)
cmd = "%s %s" % (cmd, " ".join(rsyncoptions))
}}}
the hard-coded set of options in rsycnoptions
{{{
rsyncoptions = [
'--checksum',
'--compress',
'--delete',
'--hard-links',
'--links',
'--partial',
'--perms',
'--progress',
'--recursive',
'--sparse',
'--stats',
#'--times',
'--verbose']
}}}
overwrites the options passed in via the mdb (machineEntry.rsyncopts)
since it
appears later on the commmand line.
This is (currently) an issue for minerva, whose file system does not
support
hard-links, since there is no way to pass in a {{{--no-hard-links}}}
option
for just minerva.
Typically we don't have hard-links in our code trees, though it is
certainly
not something that is expected to never happen (eg I do have some hard
links).
--
Ticket URL: <https://trac.einsteintoolkit.org/ticket/2053>
Einstein Toolkit <http://einsteintoolkit.org>
The Einstein Toolkit
#2060: Parallel checkout fails on systems with threading missing from the perl
installation
---------------------------+------------------------------------------------
Reporter: diener | Owner:
Type: defect | Status: new
Priority: minor | Milestone:
Component: GetComponents | Version: development version
Keywords: |
---------------------------+------------------------------------------------
One of the participants at the EinsteinToolkit workshop tried to checkout
Cactus on a machine she had access to. Apparently threading was missing
from the perl installation and checking out with --parallel failed with
the error:
Can't call method "enqueue" on an undefined value at ./GetComponents line
1025.
Checking out serially works. This suggests that some check for the
threading is missing or not taken into account properly.
--
Ticket URL: <https://trac.einsteintoolkit.org/ticket/2060>
Einstein Toolkit <http://einsteintoolkit.org>
The Einstein Toolkit