Users May 2012

users@lists.einsteintoolkit.org

21 participants
44 discussions

Start a nNew thread

Einstein Toolkit Meeting Reminder
by Frank Loeffler 14 May '15

14 May '15

10 31

Simulation domain volume and reduction weight sum differ
by Jakob Hansen 25 Sep '12

25 Sep '12

Hi, Using mercurial, I get this warning during simulations : INFO (CarpetReduce): Simulation domain volume: 170368000 INFO (CarpetReduce): Reduction weight sum: 170367999.880642 WARNING level 1 in thorn CarpetReduce processor 0 host tachyon2974 (line 120 of /home01/r632kgw/jakob/CactusP/arrangements/Carpet/CarpetReduce/src/mask_test.c): -> Simulation domain volume and reduction weight sum differ Simulations seem to run okay though. What is causing this warning and is it serious or can I just ignore it? Cheers, Jakob

3 3

support_old_CalcTmunu_mechanism
by Frank Loeffler 23 Sep '12

23 Sep '12

Hi, Are there objections to change the default of TmunuBase::support_old_CalcTmunu_mechanism from "yes" to "no"? Frank

1 1

meeting minutes
by Roland Haas 08 Jun '12

08 Jun '12

Present were: Frank, Philipp, Ian, Erik, Peter, Christian Ott, Josh, Bruno, Roland ET release: * there were still open tickets in review state. Decision made: ** will make timelevels parameters steerable ** will hold GetComponents changes since only "unusual" thorn lists trigger the bug ** slope limiters: leave as they are for the release ** hold patch to handle optional dependencies since it does not affect the distributed thorn lists ** all tickets cleared out now * please look at release notes once they are available * mention failing tests in release notes, warn about broken slope limiters * advertise release to same set of mailing list as last time * make sure to link full release notes on website Testsuites: * there are a number of tests failing. It seems as if they fail on systems using Intel12 based compilers. Need to collect exact build number of compiler (icpc -V) to compare. * failing Exact/KS-tilted test is very likely due to RELTOL=0. Increase RELTOL to 1-12 * ADAxiOdd test failures are in a thorn that is known to produce different results each time it is called (tolerance is 1e-7) * many tests fail on Pandora, Peter is the goto person for this machine * SphericalHarmonicRecon/regression tests fail with 2MPI processes on some machines, mention in release notes * no tests for trestles are present since the machine was down during the last week Misc: * grant Béla Szilágyi write access to PITTNULL code Roland

5 7

Follow-up: How I got Visit 2.4.2 & CarpetHDF5 to work on Mac OS X Lion
by Babiuc-Hamilton, Maria 30 May '12

30 May '12

Scott, I finally got around to check follow your instructions, but I ran into this problem: ----------------------------------------------------------------- Parsed document of type Plugin ----------------------------------------------------------------- Plugin: CarpetHDF5 ("Carpet AMR/curvilinear HDF5 output generated by the Cactus Computational Toolkit", type=database) -- version 2.1 Attribute: () -- The C compiler identification is GNU -- The CXX compiler identification is GNU -- Checking whether C compiler has -isysroot -- Checking whether C compiler has -isysroot - yes -- Checking whether C compiler supports OSX deployment target flag -- Checking whether C compiler supports OSX deployment target flag - yes -- Check for working C compiler: /usr/bin/gcc -- Check for working C compiler: /usr/bin/gcc -- works -- Detecting C compiler ABI info -- Detecting C compiler ABI info - done -- Checking whether CXX compiler has -isysroot -- Checking whether CXX compiler has -isysroot - yes -- Checking whether CXX compiler supports OSX deployment target flag -- Checking whether CXX compiler supports OSX deployment target flag - yes -- Check for working CXX compiler: /usr/bin/c++ -- Check for working CXX compiler: /usr/bin/c++ -- works -- Detecting CXX compiler ABI info -- Detecting CXX compiler ABI info - done CMake Error at CMakeLists.txt:90 (VISIT_PLUGIN_TARGET_RTOD): Unknown CMake command "VISIT_PLUGIN_TARGET_RTOD". -- Configuring incomplete, errors occurred! make: *** No targets specified and no makefile found. Stop. The only difference was that I did a ./configure --enable-cxx first in my hdf5-1.8.9 directory. Could you please help me out? Christian, do you have any suggestions? thanks, Maria ________________________________________ From: Babiuc-Hamilton, Maria Sent: Tuesday, May 15, 2012 8:42 AM To: Scott Hawley Subject: Re: [Users] How I got Visit 2.4.2 & CarpetHDF5 to work on Mac OS X Lion Scott, Fantastic! I'll try it right away! I had problems making Visit work with MacOS last year, and had to do my own little hack, but haven't use it for a while, because I was afraid that if I upgrade I'll run into problems again. Have had any success with ygraph? Cheers, Maria On 5/14/12 5:23 PM, "Scott Hawley" <scott.hawley(a)belmont.edu> wrote: > >Here's what I did: > >1. Downloaded the 64-bit binary of Visit 2.4.2 from the Visit web site >2. Downloaded the CarpetHDF directory via svn. >3. Downloaded hdf5-1.8.9 source via ftp. I just put it in the CarpetHDF5 >directory. Did a make and make installŠ. > "HDF5 Directory" is then .../hdf5-1.8.9/hdf5/ (see below) >4. Made sure "Remote Login" is enabled under "Sharing" in "System >Preferences". Visit uses this. > Went ahead and did a "ssh localhost" just to make sure. I was actually >prompted to confirm an address because I'm on LSU's network right now. > (Presumably Visit needs a clean login with no pesky questions or >password prompts.) >5. Created symbolic links in /usr/lib to whichever MPI libraries I planned >to use. > Older versions of OS X had native MPI support, and this is hard coded >in the binary you get from the visit site. > I use OpenMPI and MacPorts, so I went to /usr/lib and did "sudo ln -s >/opt/local/lib/libmpi* ." >6. Deleted my ~/.visit directory entirely. This may not be necessary. >7. Went in the CarpetHDF5 and ran ./install and Šbasically just did what >was in the readme, that is: > in the xmledit window, clicked on "Cmake" and then updated the flags, as >in > > CXXFLAGS = >-I/Applications/VisIt.app/Contents/Resources/2.4.2/darwin-x86_64/include >-I/Users/shawley/apps/VizTools/CarpetHDF5/hdf5-1.8.9/hdf5/include > LDFLAGS = >-L/Applications/VisIt.app/Contents/Resources/2.4.2/darwin-x86_64/lib >-L/Users/shawley/apps/VizTools/CarpetHDF5/hdf5-1.8.9/hdf5/lib > Saved and closed the window, let it do its thing... >8. Ran visit from the command line, using the -noconfig option. The >latest version of Visit has a problem with "localhost" on Mac, for some >reason. >so >visit -noconfig > >..and then visit came up, I opened a .h5 file and was able to do all the >usual things with it! > >Good luck! >-Scott > > >_______________________________________________ >Users mailing list >Users(a)einsteintoolkit.org >http://lists.einsteintoolkit.org/mailman/listinfo/users

3 6

WaveDemo question
by Comer Duncan 29 May '12

29 May '12

HI, I am trying to get WaveDemo built on my local laptop. I have done the following: make WaveDemo--config make WaveDemo while in the Cactus directory with the result that seemingly _all_ resident thorns have been compiled/configed. As I doubt that all are needed for the WaveDemo example, I find this a bit strange. Can someone please explain? Secondly, at the end of the make WaveDemo it crashes complaining that LSUThorns/Vectors is needed but not present. This is associated with several thorns including CarpetLib, ML_BSSN, ML_BSSN_02, and WeylScal4. I have added the line LSUThorns/Vectors to the Thornlist file and have rerun the make WaveDemo but still I get no find of LSUThorns/Vectors. Can someone please let me know the correct procedure here? Thanks very much for your help. Comer

4 9

Einstein Toolkit Release
by Frank Loeffler 28 May '12

28 May '12

We are pleased to announce the fifth release (code name Lovelace (<http://en.wikipedia.org/wiki/Ada_Lovelace>) of the Einstein Toolkit, an open, community developed software infrastructure for relativistic astrophysics. This release includes beginning support for OpenCL (disabled by default). In addition, bug fixes accumulated since the previous release in October 2011 have been included. The Einstein Toolkit is a collection of software components and tools for simulating and analyzing general relativistic astrophysical systems that builds on numerous software efforts in the numerical relativity community including CactusEinstein, the Carpet AMR infrastructure and the relativistic hydrodynamics code GRHydro (an updated and extended version of the public release of the Whisky code). The Cactus Framework is used as the underlying computational infrastructure providing large-scale parallelization, general computational components, and a model for collaborative, portable code development. The toolkit includes modules to build complete codes for simulating black hole spacetimes as well as systems governed by relativistic hydrodynamics. The Einstein Toolkit uses a distributed software model and its different modules are developed, distributed, and supported either by the core team of Einstein Toolkit Maintainers, or by individual groups. Where modules are provided by external groups, the Einstein Toolkit Maintainers provide quality control for modules for inclusion in the toolkit and help coordinate support. The Einstein Toolkit Maintainers currently involve postdocs and faculty from five different institutions, and host weekly meetings that are open for anyone to join in. Guiding principles for the design and implementation of the toolkit include: open, community-driven software development; well thought out and stable interfaces; separation of physics software from computational science infrastructure; provision of complete working production code; training and education for a new generation of researchers. For more information about using or contributing to the Einstein Toolkit, or to join the Einstein Toolkit Consortium, please visit our web pages at http://einsteintoolkit.org. The Einstein Toolkit is primarily supported by NSF 0903973/0903782/0904015 (CIGR), and also by NSF 0701566/0855892 (XiRel), 0721915 (Alpaca), 0905046/0941653 (PetaCactus), and 0710874 (LONI Grid). The Einstein Toolkit contain over 170 regression test cases. On a large portion of the tested machines, all of these testsuites pass, using both MPI and OpenMP parallelization. The changes between this and the previous release include: - Accelerator Support This release of the Einstein Toolkit adds support for GPUs and other accelerators. This support comprises three levels of abstraction, ranging from merely building and running both CUDA and OpenCL code, to automated code generation targeting GPUs instead of CPUs. As with any other programming paradigm (such as MPI or OpenMP), the performance benefits depend on the particular algorithms used and optimizations that are applied. In addition, the Simulation Factory greatly aids portability to a wide range of computing systems. At the lowest level, Cactus now supports compiling, building, and running with either CUDA or OpenCL. CUDA is supported as new language in addition to C, C++, and Fortran; OpenCL is supported as an external library, and builds and executes compute kernels via run-time calls. Details are described in the user's guide (for CUDA) and in thorn ExternalLibraries/OpenCL (for OpenCL). Many accelerator platforms today separate between host memory and device memory, and require explicit copy or map operations to transfer data. An intermediate level of abstraction aids transferring grid variables between host and device, using schedule declarations to keep track of which data are needed where, and minimizing expensive data transfers. For OpenCL, there is a compact API to build and execute compute kernels at run time. Details are described in thorns CactusUtils/Accelerator and CactusUtils/OpenCLRunTime (with example parameter file). Finally, the code generation system Kranc has been extended to be able to produce either C++ or OpenCL code, based on the infrastructure described above. This allows writing GPU code in a very high-level manner. However, it needs to be stated that the efficiency of the generated code depends on many variables, including e.g. the finite differencing stencil radius and the number of operations in the generated compute kernels. Non-trivial kernels typically require system-dependent tuning to execute efficiently, as GPUs and other accelerators generally show a rather unforgiving performance behavior. The thorns McLachlan/ML_WaveToy and McLachlan/ML_WaveToy_CL are examples, generated from the same Kranc script, showing the generated C++ and OpenCL code. - SimFactory - Machine database and optionlists updated due to system changes on HPC resources - Simfactory's capability of running the testsuites is properly tested on a lot of systems. - IOUtil: checkpoint_dir is now steerable - SphericalSurface: added functionality to name spherical surfaces - Formaline: Support a "local repository" that collects all machine-local repositories - TimerReport: Allow different timers on different processes - WeylScal4: Enable use of LoopControl, and hence OpenMP - EOS_Omni: use C interface for HDF5 to avoid needing Fortran HDF5 bindings - EOSG_*: Support for the so-called 'general EOS interface' has been dropped from the Einstein Toolkit - A new arrangement EinsteinExact has been added to the toolkit, providing a wide range of exact initial data, which will eventually replace the 'Exact' thorn. - The *_O2 versions of McLachlan have been removed from the toolkit. This functionality is already provided by the regular McLachlan thorns now. - A new thorn ADMMass has been added to the Einstein Toolkit, which can calculate approximations of the ADM mass using a finite surface or volume integral. - The old library mechanism in Cactus (e.g. HDF5=yes) is now deprecated. Expect it to be removed in one of the next releases. - The thorns ADM and LegoExcision are deprecated and will be removed in one of the next releases. - GRHydro: - use atmosphere integer mask instead of bitmask - remove (now) unused old Tmunu interface - Implemented enhanced PPM scheme by Colella & Sekora 2008, McCorquodale & Colella 2011. Can be activated by setting use_enhanced_ppm = yes - External Libraries: several updates and configuration improvements - Cactus - implement per-variable tolerances for Cactus testsuites, for long discussion, see ET ticket #114 - Allow arithmetic expression in ParameterSet: parameter files can now contain a limited set of expressions - Handles requirements recursively - A lot of smaller bug fixes - McLachlan: Implement CCZ4 formulation - CarpetMask: Keep track of the volume that is masked out - CarpetLib: Define MPI reduction operators for complex numbers - CarpetIOASCII: Add new "compact" output format - Csrpet: Support accelerator data transfer - CarpetRegrid2: Add periodic boundary conditions - Simfactory - Use OpenMP by default - Make running testsuites using Simfactory possible - Updated a lot of configurations All repositories participating in this release carry a branch ET_2012_05 marking this release. These release branches will be updated if severe errors are found. For more detailed information about the "Lovelace" release please read the long release announcement on the Einstein Toolkit web pages: http://einsteintoolkit.org/about/releases/ET_2012_05_announcement.php. On behalf of the Einstein Toolkit Consortium: the "Lovelace" Release Team Eloisa Bentivegna Tanja Bode Peter Diener Roland Haas Ian Hinder Frank Löffler Bruno Mundim Christian D. Ott Erik Schnetter May 28, 2012

1 0

Carpet/MPI Compilation help
by Carl Rodriguez 28 May '12

28 May '12

Hi all, I've been having a great deal of trouble installing ET(K) on our local cluster at Northwestern. While I've been able to install and run the code, I believe something is wrong with how I compiled it with MPI. I had tried to reverse engineer some of the machine/submit files in simfactory/mdb, but the kludged configurations I've tried either stall after Carpet tries to allocate memory or fails in some of the MPI calls. Sometimes it runs correctly, but only on a power of two number of cores. Could someone who is very familiar with Carpet/MPI and compiling with ICC take a quick look at the configuration I've used and let me know if something is obviously wrong (i.e. incorrect libraries or conflicting compilation flags)? It might be a machine specific problem, but it'd be good to know I've compiled Carpet correctly before moving on to more technical discussions with our sys-admin. The relevant parts of the cfg (compiled with icc associated with this MPI) are MPI = OpenMPI OPENMPI_DIR = /software/mpi/openmpi-1.4.3-intel-2011.3 MPI_INC_DIRS = /software/mpi/openmpi-1.4.3-intel-2011.3/include MPI_LIB_DIRS = /software/mpi/openmpi-1.4.3-intel-2011.3/lib /usr/lib64 MPI_LIBS = mpi_cxx mpi rt torque dl pthread PTHREADS = yes and then submit with time $OPENMPI_DIR/bin/mpirun -np @NUM_PROCS@ -cpus-per-proc @NUM_THREADS@ @EXECUTABLE@ -L 3 @PARFILE@ Thanks! Carl

2 1

Einstein Toolkit Meeting
by Frank Loeffler 27 May '12

27 May '12

Hi Please consider joining the weekly Einstein Toolkit phone call at 10 am US central time on Mondays. As usual, you can find instructions how to join on the following web site: http://einsteintoolkit.org/community/support/ I short: the number is (+1) 225-578-4942 or (+1) 866-573-0359 and the conference id is 118682#. The main topic will be the organization of the next ET release. In particular, this means we will discuss: - Release-relevant tickets (4) [1] - three of these are pending a review: please help here! - Schedule: - hard freeze date of May 21th (evening) - only 'important' bug fixes or documentation improvements ^--> to be decided by two maintainers - release date of May 28th -> can we hold this? - Now most important: running testsuites, write release notes Not necessarily connected to the release: - What happend during last week: 29 issue were opened or changed and are still open, and 10 tickets could be closed. Some of them were: - minor changes to trac interface - openssl security fix - latex warnings for thornguide fixed - simfactory doesn't enable thorns automatically atm - ExternalThorns/GSL fixed - disable VECTORISE_ALIGNED_ARRAYS in all optionlists - ADMBase variable initialization is now thread-safe (uses openmp) - OpenCLRunTime is now part of the ET, as well as ML_WaveToy and ML_WaveToy_CL (although not built by default) - thorn Exact doesn't use openmp anymore (atm) because this was found to be buggy and fixing it would require a larger patch -> no-go now. As usual, feel free to add to this agenda. Frank Loeffler [1] https://trac.einsteintoolkit.org/query?status=accepted&status=assigned&stat…

1 0

Re: [Users] ETK Maxwell checkout failing for McLachlan, KrancNumericalTools
by Kelly, Bernard J. (GSFC-660.0)[UNIVERSITY OF MARYLAND BALTIMORE COUNTY] 24 May '12

24 May '12

[moving back to the list after two off-list mails] Hi Ian. OK, I've tried the bare git commands you mention. Here are my results: ------------------------------- humperdinck:~ prince$ git clone --depth 1 git://carpetcode.org/McLachlan Cloning into 'McLachlan'... remote: Counting objects: 961, done. remote: Compressing objects: 100% (627/627), done. remote: Total 961 (delta 772), reused 428 (delta 327) Receiving objects: 100% (961/961), 897.79 KiB | 397 KiB/s, done. Resolving deltas: 100% (772/772), done. humperdinck:~ prince$ cd McLachlan/ humperdinck:McLachlan prince$ git checkout --track -b ET_2011_10 origin/ET_2011_10 fatal: git checkout: updating paths is incompatible with switching branches. Did you intend to checkout 'origin/ET_2011_10' which can not be resolved as commit? humperdinck:McLachlan prince$ git --version git version 1.7.10.2 ------------------------------- I note that (a) there are far fewer "objects" in my git checkout (perhaps this is normal?) than in yours -- 978 vs 3878; (b) my git version is 1.7.10.2, rather than the 1.7.5.4. This is the current result of an up-to-date MacPorts install of git-core. Bernard On 5/23/12 7:56 AM, "Ian Hinder" <ian.hinder(a)aei.mpg.de> wrote: > >On 23 May 2012, at 13:25, Kelly, Bernard J. (GSFC-660.0)[UNIVERSITY OF >MARYLAND BALTIMORE COUNTY] wrote: > >> Hi Ian. >> >> Yes, I should have explained that this was an additional checkout after >> most of ETK was already checked out (hence the number 12 instead of 175 >>or >> so). But the original checkout was also Maxwell, and was performed only >>a >> day or two earlier, with the same GetComponents and thornlist. >> >> I just moved that whole Cactus aside and redid from scratch. Identical >> failure (but this time saying 175). Almost everything checks out fine; >> just not McLachlan and KrancNumericalTools, and GetComponents itself, >> apparently (?). >> >> I've redone the checkout (again after the main checkout) with the >> "--verbose" option. Result after main text. The option "--verbose 2" >> fails, so presumably the correct syntax is slightly different; I didn't >> hang around to check. >> >> BTW, I don't really think the issue has anything to do with my OS X >> version, though I suppose my MacPorts git might have some funny >>settings. > >Hi Bernard, > >Can we move this back to the list? Others might have suggestions and >will benefit in future from the discussion. I have done this: > >MacBook-2:temp $ git clone --depth 1 git://carpetcode.org/McLachlan >Cloning into McLachlan... >remote: Counting objects: 3878, done. >remote: Compressing objects: 100% (2222/2222), done. >remote: Total 3878 (delta 3229), reused 2050 (delta 1638) >Receiving objects: 100% (3878/3878), 2.38 MiB | 838 KiB/s, done. >Resolving deltas: 100% (3229/3229), done. >MacBook-2:temp $ cd McLachlan/ >MacBook-2:McLachlan (master) $ git checkout --track -b ET_2011_10 >origin/ET_2011_10 >Branch ET_2011_10 set up to track remote branch ET_2011_10 from origin. >Switched to a new branch 'ET_2011_10' >MacBook-2:McLachlan (ET_2011_10) $ git --version >git version 1.7.5.4 > >Those are the commands that GetComponents claims to be using. Can you >try these commands on their own and see what happens? It might be >something to do with old versions of Git, or maybe GetComponents is doing >something different than the commands it outputs. > >Have a look at >http://stackoverflow.com/questions/499316/git-plugin-for-hudson-checkout-p >roblem, and other google hits for the checkout error message. It's >looking like it might be a problem with an old version of git. > >-- >Ian Hinder >http://numrel.aei.mpg.de/people/hinder >

6 11

2026

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

Users May 2012