Trac December 2017

trac@lists.einsteintoolkit.org

1 participants
66 discussions

[Einstein Toolkit] #2072: Carpet: Disable OpenMP parallelization of transport operators
by Einstein Toolkit 22 Jan '18

22 Jan '18

#2072: Carpet: Disable OpenMP parallelization of transport operators -------------------------+-------------------------------------------------- Reporter: eschnett | Owner: eschnett Type: enhancement | Status: new Priority: major | Milestone: Component: Carpet | Version: development version Keywords: | -------------------------+-------------------------------------------------- This disables the OpenMP parallelization of Carpet's transport operators. I have observed that this leads to a significant speedup when many threads are used. The likely reason is that the regions which are parallelized are typically small. A typical reason would e.g. be the lower x-boundary of one component of one grid variable. The OpenMP thread startup overhead and the cache misses caused by parallelizing this are then larger than any benefit. See <https://bitbucket.org/eschnett/carpet/pull-requests/18/carpet- disable-openmp-parallelization-of/diff>. In a next step (not proposed here), we can parallelize transport operators again, but at a much higher level, e.g. at the level of the loop over all variables that need to be prolongated. However, Carpet currently (and quite unfortunately) uses static variables to hold pointers to timers, and these are not thread-safe. (Neither the static variables nor the timer implementation itself are.) This needs to be either corrected or disabled, which will be the topic of a further pull request. At this time, I ask some of those who are interested in performance in trying this pull request on a few iterations of a production simulation that uses many OpenMP threads and report back here. -- Ticket URL: <https://trac.einsteintoolkit.org/ticket/2072> Einstein Toolkit <http://einsteintoolkit.org> The Einstein Toolkit

1 10

[Einstein Toolkit] #1784: osx-homebrew.cfg does not work out of the box on Yosemite
by Einstein Toolkit 22 Jan '18

22 Jan '18

#1784: osx-homebrew.cfg does not work out of the box on Yosemite ------------------------+--------------------------------------------------- Reporter: anonymous | Owner: Type: task | Status: new Priority: optional | Milestone: Component: SimFactory | Version: ET_2015_05 Keywords: | ------------------------+--------------------------------------------------- When installing ET anew on Yosemite with the instructions in osx-homebrew.cfg, the build fails because homebrew gets the latest version of gcc (5.1), which is currently not the one then specified in the cactus compilation options (4.9). This problem was encountered by a student who approached ET for the first time. -- Ticket URL: <https://trac.einsteintoolkit.org/ticket/1784> Einstein Toolkit <http://einsteintoolkit.org> The Einstein Toolkit

1 3

[Einstein Toolkit] #1757: Stampede needs to be updated
by Einstein Toolkit 22 Jan '18

22 Jan '18

#1757: Stampede needs to be updated ----------------------+----------------------------------------------------- Reporter: hinder | Owner: Type: task | Status: new Priority: critical | Milestone: Component: Other | Version: development version Keywords: backport | ----------------------+----------------------------------------------------- The SimFactory definition for Stampede currently uses the intel/13.1.1.163 compiler. This compiler will be removed next Tuesday (see https://portal.tacc.utexas.edu/user-guides/stampede/intel15). According to TACC, the current default is the older intel/13.0.2.146, and this is the recommended and default compiler, and will remain so. Both compilers have the "restrict" keyword blacklisted in Cactus (#1276). They will install intel/15.0.1 at the end of April. There is also intel/14.0.1.106 available, but it is labeled as "limited software stack". It is not clear why they are removing a newer compiler, or why they chose to do this a month before installing an even newer one. Our options are: 1. Upgrade to intel/14.0.1.106, which is labeled as "limited software stack" 2. Downgrade to intel/13.0.2.146, which is the default and will remain so for a while This should be done both for the trunk and the release branch. I suggest that option 2 is the most conservative, and probably the easiest, as there might be libraries which are not compiled for intel/14.0.1.106. -- Ticket URL: <https://trac.einsteintoolkit.org/ticket/1757> Einstein Toolkit <http://einsteintoolkit.org> The Einstein Toolkit

1 8

[Einstein Toolkit] #2007: MoL fails to build with Intel compiler
by Einstein Toolkit 22 Jan '18

22 Jan '18

#2007: MoL fails to build with Intel compiler ---------------------+------------------------------------------------------ Reporter: hinder | Owner: eschnett Type: defect | Status: new Priority: blocker | Milestone: Component: Cactus | Version: development version Keywords: | ---------------------+------------------------------------------------------ Compiling MoL gives this error: {{{ /home/ianhin/Cactus/EinsteinToolkitGit/arrangements/CactusNumerical/MoL/src/RK4-RK2.c(60): error: unrecognized OpenMP #pragma #pragma omp /*parallel for*/ ^ compilation aborted for /home/ianhin/Cactus/EinsteinToolkitGit/configs/sim/build/MoL/RK4-RK2.c (code 2) }}} This code was introduced in https://bitbucket.org/cactuscode/cactusnumerical/commits/09a209266daa281a3e…. Removing the commented portion doesn't help. Replacing it with {{{ #pragma omp parallel }}} allows the code to compile. This is on Minerva, with icc (ICC) 16.0.1 20151021. -- Ticket URL: <https://trac.einsteintoolkit.org/ticket/2007> Einstein Toolkit <http://einsteintoolkit.org> The Einstein Toolkit

1 10

[Einstein Toolkit] #1458: intel 13 2013.1.117 miss-compiles asserts in TwoPunctures tp_utils.c
by Einstein Toolkit 22 Jan '18

22 Jan '18

#1458: intel 13 2013.1.117 miss-compiles asserts in TwoPunctures tp_utils.c -----------------------------------+---------------------------------------- Reporter: rhaas | Owner: Type: defect | Status: new Priority: major | Milestone: Component: EinsteinToolkit thorn | Version: development version Keywords: TwoPunctures | -----------------------------------+---------------------------------------- this continues the thread in #1429 in particular https://trac.einsteintoolkit.org/ticket/1429#comment:14. Currently we need to determine which versions are affected. -- Ticket URL: <https://trac.einsteintoolkit.org/ticket/1458> Einstein Toolkit <http://einsteintoolkit.org> The Einstein Toolkit

1 4

[Einstein Toolkit] #2091: ET wiki is down
by Einstein Toolkit 08 Jan '18

08 Jan '18

#2091: ET wiki is down -----------------------------------+---------------------------------------- Reporter: hinder | Owner: sbrandt Type: defect | Status: new Priority: blocker | Milestone: Component: Server Infrastructure | Version: development version Keywords: | -----------------------------------+---------------------------------------- The ET wiki went down at 03:12 CET on 10-Dec-2017. -- Ticket URL: <https://trac.einsteintoolkit.org/ticket/2091> Einstein Toolkit <http://einsteintoolkit.org> The Einstein Toolkit

1 3

[Einstein Toolkit] #1599: CarpetInterp/waveinterp_2p tests constant in time initial data
by Einstein Toolkit 27 Dec '17

27 Dec '17

#1599: CarpetInterp/waveinterp_2p tests constant in time initial data --------------------------+------------------------------------------------- Reporter: rhaas | Owner: eschnett Type: defect | Status: new Priority: minor | Milestone: Component: Carpet | Version: development version Keywords: CarpetInterp | --------------------------+------------------------------------------------- the test waveinterp_2p sets up initial data using IDWaveMoL (fom CactusExamples) using kx=ky=kz = 0. and slopet = 0.1. Ie the initial data is constant in space and has frequency 0.1. It then evolves this using WaveMoL and interpolates in space and time with CarpetInterp. Since the function is constant in space (exactly so, see eg the output in InterpToArray::array1d_vars[0]) the interpolation in space just tests roundoff errors. The second time derivative (array1d_vars[2]) should be zero and all we measure in the test is truncation errors on the level of 1e-9. This makes the test very sensitive to -Ofast, -O0, gcc vs. intel etc. and not a very good test. I would rather either use a Gausian wave for initial data so that there is some actual data in the result and not just numerical noise, or leave out the 2nd time derivative from the test data. The test fails due to significant errors in a virtual machine using ubuntu.cfg (which usies -ffast-math and -O2). -- Ticket URL: <https://trac.einsteintoolkit.org/ticket/1599> Einstein Toolkit <http://einsteintoolkit.org> The Einstein Toolkit

1 11

[Einstein Toolkit] #2073: IO corruption on SDSC oasis file systems
by Einstein Toolkit 25 Dec '17

25 Dec '17

#2073: IO corruption on SDSC oasis file systems -------------------------------+-------------------------------------------- Reporter: rhaas | Owner: Type: defect | Status: new Priority: major | Milestone: Component: Other | Version: development version Keywords: Comet Gordon SDSC | -------------------------------+-------------------------------------------- I am experiencing file corruption in ASCII output produced by the Cactus code on Comet's (and Gordon's as far as I remember) scratch file systems. This manifests as lines of output being mashed together in the output file. I have added strace calls to my job script to capture all arguments to the OS's write() function and re-created the write() calls based on this. Those write calls, when replayed on a login node, do produce a correct (no mashed lines) file. All output to the file in question was from rank 0 only even though the code used MPI and ran on two MPI ranks. The same code and number of MPI ranks produces a correct output file when run on the $HOME file system. Thus it seems to me as if there may be an issue with the file system. I can try and reduce the test case to a more minimal example (right now it is a full simulation even though it runs only for <1minute) . You can find the job script (for account, SLURM options etc) here: /oasis/scratch/comet/rhaas/temp_project/simulations/OSTREAM_2_12/output-0000/SIMFACTORY/SubmitScript the script that launches the MPI executable here: /oasis/scratch/comet/rhaas/temp_project/simulations/OSTREAM_2_12/output-0000/SIMFACTORY/RunScript the strace output here: /home/rhaas/strace/strace.1882[67].log and the awk script to recreate the write calls is: gawk -vFS='"' '/write.*\/grid-coordinates.xy.asc/{print "printf \""$2"\""}' ~/strace.18826.log >recreate.sh The corrupted line is eg. line 161 of /oasis/scratch/comet/rhaas/temp_project/simulations/OSTREAM_2_12/output-0000/TEST/sim/CarpetIOASCII/newsep /grid-coordinates.xy.asc which reads 1 4 3 4 1 0.1666666666660.505076272276105285714 etc but should read 1 4 3 4 1 0.166666666666667 -0.0714285714285714 I can avoid the file corruption by flushing the output file after each line. I am wondering if there is anything known about this or if there is a workaround that does not boil to first writing all data to a file system local to the compute node and copying to /oasis/scratch after the job is finished (how much local space would be available since I would also have to do so for eg checkpoint files and 3d hdf5 output). -- Ticket URL: <https://trac.einsteintoolkit.org/ticket/2073> Einstein Toolkit <http://einsteintoolkit.org> The Einstein Toolkit

1 4

[Einstein Toolkit] #2092: make Cactus build system support non-interactive use eg in Jupyter notebooks
by Einstein Toolkit 23 Dec '17

23 Dec '17

#2092: make Cactus build system support non-interactive use eg in Jupyter notebooks -------------------------+-------------------------------------------------- Reporter: rhaas | Owner: Type: enhancement | Status: new Priority: optional | Milestone: Component: Cactus | Version: development version Keywords: | -------------------------+-------------------------------------------------- This refers to #1919:comment:4 which specifically asks about "make newthorn" but really seems to indicate that all interaction should be double checked. -- Ticket URL: <https://trac.einsteintoolkit.org/ticket/2092> Einstein Toolkit <http://einsteintoolkit.org> The Einstein Toolkit

1 0

[Einstein Toolkit] #1919: have make newthorn create a functioning thorn
by Einstein Toolkit 23 Dec '17

23 Dec '17

#1919: have make newthorn create a functioning thorn -------------------------+-------------------------------------------------- Reporter: rhaas | Owner: Type: enhancement | Status: new Priority: optional | Milestone: Component: Cactus | Version: development version Keywords: | -------------------------+-------------------------------------------------- Currently the newthorn make target does not create a functioning thorn. It would be good if it produced a skeleton thorn that compiled. -- Ticket URL: <https://trac.einsteintoolkit.org/ticket/1919> Einstein Toolkit <http://einsteintoolkit.org> The Einstein Toolkit

1 5

2026

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

Trac December 2017