Users April 2012

users@lists.einsteintoolkit.org

21 participants
31 discussions

Visit & CarpetHDF5 mismatch?
by Kelly, Bernard J. (GSFC-660.0)[UNIVERSITY OF MARYLAND BALTIMORE COUNTY] 17 Apr '12

17 Apr '12

Hi all. I'd like to visualise some 3D data produced by an initial-data run of the EinsteinToolkit (Maxwell). I'm outputting using CarpetIOHDF5; I have an output file "hahndol::hahndol_coulomb.h5", and decided to use Visit, as recommended by cactuscode, etc. I grabbed the CarpetHDF5 plugin code from svn, and installed as per instructions (pointing at the ETK-installed HDF5 in my Cactus/configs/<config-name>/scratch/external directory). Starting Visit (2.4.2) , and trying to load a .h5 file yields the following error: --------------------------------------------------------------------------- ---- The metadata server running on host localhost has exited abnormally. VisIt is trying to restart it. Shortly thereafter, the following occured... VisIt was unable to open "/Users/bjkelly1/Projects/CODES/Cactus/exe/BL_IJS_test/hahndol::hahndol_cou lomb.h5". Each attempt to open it caused VisIt's metadata server to crash. This can occur when the file is corrupted, or when the underlying file format has changed and VisIt's readers have not been updated yet, or when the reader VisIt is using for your file format is not robust. Please check whether the file is corrupted and, if not, contact a VisIt developer. --------------------------------------------------------------------------- ---- I have no idea whether the file is corrupted, but wanted to know: is the CarpetHDF5 plugin up-to-date with the output of Maxwell's CarpetIOHDF5? Any basic ideas where I might have gone wrong? Thanks, Bernard --------------------------------------------------------------------------- -- Bernard Kelly -- CRESST Research Associate, NASA/GSFC Phone: +1 (301) 286-7243 E-Mail: bernard.j.kelly(a)nasa.gov Web: http://science.gsfc.nasa.gov/sed/index.cfm?fuseAction=people.jumpBio&iphone bookid=13052 --------------------------------------------------------------------------- --

7 19

CarpetHDF5 plugin referencing library in wrong spot. How to change?
by Scott Hawley 17 Apr '12

17 Apr '12

The install script is trying to find the file /usr/lib/libmpi_cxx.dylib, but the file is not there. It's actually in /opt/local/lib. How does one change this setting? I've gone through the various tabs in the menu (resulting from running the install script) and I cannot find a place to change this setting. Doing a "grep -r mpi *" in the CarpetHDF5 directory turns up nothing. I went ahead and did a "sudo ln -s /opt/local/lib/libmpi* /usr/lib". That solved the FIRST problem. Now it's looking for `/usr/lib/libopen-rte.dylib'…. make[2]: *** No rule to make target `/usr/lib/libopen-rte.dylib', needed by `/Users/shawley/.visit/2.4.2/darwin-x86_64/plugins/databases/libECarpetHDF5Database_par.dylib'. Stop. make[1]: *** [CMakeFiles/ECarpetHDF5Database_par.dir/all] Error 2 make: *** [all] Error 2 We need to be able to change the directory it looks in. Setting LIB_PATHS and/or LD_LIBRARY_PATH has no effect. Thanks, Scott P.S.- Going back and putting in symbolic links for all the complaining libraries results in a full build! However, when you try to run it in Visit (2.4.2) and load a .h5 file, what you get is…. VisIt was unable to open "/Users/shawley/runs/cactus/bruiser/texmex_test/gxx.h5". Each attempt to open it caused VisIt's metadata server to crash. This can occur when the file is corrupted, or when the underlying file format has changed and VisIt's readers have not been updated yet, or when the reader VisIt is using for your file format is not robust. Please check whether the file is corrupted and, if not, contact a VisIt develop Plugin: CarpetHDF5 ("Carpet AMR/curvilinear HDF5 output generated by the Cactus Computational Toolkit", type=database) -- version 2.1 Attribute: () -- The C compiler identification is GNU -- The CXX compiler identification is Clang -- Checking whether C compiler has -isysroot -- Checking whether C compiler has -isysroot - yes -- Checking whether C compiler supports OSX deployment target flag -- Checking whether C compiler supports OSX deployment target flag - yes -- Check for working C compiler: /usr/bin/gcc -- Check for working C compiler: /usr/bin/gcc -- works -- Detecting C compiler ABI info -- Detecting C compiler ABI info - done -- Check for working CXX compiler: /usr/bin/c++ -- Check for working CXX compiler: /usr/bin/c++ -- works -- Detecting CXX compiler ABI info -- Detecting CXX compiler ABI info - done -- Configuring done -- Generating done -- Build files have been written to: /Users/shawley/apps/VizTools/CarpetHDF5 Scanning dependencies of target ECarpetHDF5Database_par [ 6%] Building CXX object CMakeFiles/ECarpetHDF5Database_par.dir/CarpetHDF5EnginePluginInfo.C.o [ 12%] Building CXX object CMakeFiles/ECarpetHDF5Database_par.dir/CarpetHDF5PluginInfo.C.o [ 18%] Building CXX object CMakeFiles/ECarpetHDF5Database_par.dir/CarpetHDF5CommonPluginInfo.C.o [ 25%] Building CXX object CMakeFiles/ECarpetHDF5Database_par.dir/avtCarpetHDF5FileFormat.C.o [ 31%] Building CXX object CMakeFiles/ECarpetHDF5Database_par.dir/H5Index.C.o make[2]: *** No rule to make target `/usr/lib/libmpi_cxx.dylib', needed by `/Users/shawley/.visit/2.4.2/darwin-x86_64/plugins/databases/libECarpetHDF5Database_par.dylib'. Stop. make[1]: *** [CMakeFiles/ECarpetHDF5Database_par.dir/all] Error 2 make: *** [all] Error 2 [provision:~/apps/VizTools/CarpetHDF5] shawley% ./install

2 3

Meeting Minutes
by Roland Haas 16 Apr '12

16 Apr '12

Present were: Frank, Roland, Peter, Tanja, Josh ET workshop: * Cactus virtual machines should be available for download today, they should include all required software, incl. visualization tools. * please test them in particular if you are handling a hands on session * please try and submit your slides before the APS meeting, using the templates provided to ensure a uniform slide layout * the talks should be aimed at beginning graduate students * the hands on analysis example will be a star tracker as described in the schedule * we currently have 2 confirmed invited speakers and are working on the third. The next phone call will be three (3) weeks from now on April 16th and we will discuss the then upcoming ET release, which is due in May. Yours, Roland

4 5

GetComponents Freezing on Simfactory
by Grayson Carroll 16 Apr '12

16 Apr '12

Hello all, I'm attempting to run GetComponents on a working thorn list, and it consistently freezes when trying to check out the simfactory svn server. My subversion is up to date and I'm not behind a firewall, all the ports associated with the script should be open. Any ideas as to what could be causing the script to hang or how I could debug it? Thanks, Grayson Carroll

2 1

Re: [Users] Howto? Outer loop over coarse maps, inner loop over fine maps
by Scott Hawley 16 Apr '12

16 Apr '12

How about we schedule the call using this Doodle I just madeŠ :-) http://www.doodle.com/q2xqm722ifq98xkk -Scott On 4/16/12 7:36 AM, "Erik Schnetter" <eschnetter(a)perimeterinstitute.ca> wrote: >I am busy today from 14:00 EST on, which is 13:00 CST. I am much less >constrained tomorrow. > >-erik > >On Sun, Apr 15, 2012 at 11:26 PM, Scott Hawley <scott.hawley(a)belmont.edu> >wrote: >> I teach at 9am central, possibly free at 10, and then am advising a >>student at 11 am central. I'll see if I can reschedule with her. >>Otherwise, what about 1pm central? >> Thanks >> >> -- >> Scott H. Hawley, Ph.D. >> Assistant Professor of Physics >> Belmont University. >> 615-460-6206. >> Sent from my iPad >> >> On Apr 15, 2012, at 8:01 PM, "Frank Loeffler" <knarf(a)cct.lsu.edu> wrote: >> >>> On Sun, Apr 15, 2012 at 08:51:49PM -0400, Erik Schnetter wrote: >>>> I'm unfortunately busy at 10:00 EST (9:00 CST). I'm free before that, >>>> or right after the ET call. I'm busy again from 14:00 EST on. >>> >>> Right after the ET call would work best for me (11am central, or >>> whenever we might finish earlier). >>> >>> Frank >>> > > > >-- >Erik Schnetter <eschnetter(a)perimeterinstitute.ca> >http://www.perimeterinstitute.ca/personal/eschnetter/ >AIM: eschnett247, Skype: eschnett, Google Talk: schnetter(a)gmail.com >

2 1

Einstein Toolkit Meeting
by Frank Loeffler 15 Apr '12

15 Apr '12

Hi Please consider joining the weekly Einstein Toolkit phone call at 10 am US central time on Mondays. As usual, you can find instructions how to join on the following web site: http://einsteintoolkit.org/community/support/ I short: the number is (+1) 225-578-4942 or (+1) 866-573-0359 and the conference id is 118682#. We will talk about impressions from the ET workshop at the beginning of the month, future workshop plans, and will start organizing the next ET release. In particular, this means that it would make sense to think about the following items before the meeting: - Is there code which should be included (but which isn't yet)? If so: which? - Should something be removed? - Should something be depreciated (marked as being removed in the fall)? - Time planning (https://docs.einsteintoolkit.org/et-docs/Release_Details) - Test machine planning (https://docs.einsteintoolkit.org/et-docs/Release_Details) - Critical tickets [1] - ChangeLog - Release Name As usual, feel free to add to this agenda. Frank Loeffler [1] https://trac.einsteintoolkit.org/query?status=!closed&milestone=ET_2012_05&…

1 0

Howto? Outer loop over coarse maps, inner loop over fine maps
by Scott Hawley 15 Apr '12

15 Apr '12

Erik, How does one "nest" loops over grids at different levels? In my way of doing things, it was something like... l = level; gblm1 = mgh[l-1]; while (gblm1 != NULL) { Bbox coarse_bbox = gblm1->bbox; gbl = mgh[l]; while (gbl != NULL) { Bbox fine_bbox = gbl->bbox; // On the regions for which the finer grid overlaps the coarse grid, // do some operations... gbl = gbl->next; } gblm1 = gblm1->next; } I have an idea of something to try, below, but I'm worried that I'm abusing the SWITCH_TO_LEVEL macro… //reflevel = finer grid… SWITCH_TO_LEVEL( cctkGH, reflevel - 1 ) { BEGIN_MAP_LOOP (cctkGH, CCTK_GF) { BEGIN_LOCAL_COMPONENT_LOOP (cctkGH, CCTK_GF) { DECLARE_CCTK_ARGUMENTS; SWITCH_TO_LEVEL( cctkGH, reflevel ) { BEGIN_MAP_LOOP (cctkGH, CCTK_GF) { BEGIN_LOCAL_COMPONENT_LOOP (cctkGH, CCTK_GF) { DECLARE_CCTK_ARGUMENTS; // where fine grid bbox overlaps coarse bbox…. // do operations on coarse grid… SWITCH_TO_LEVEL( cctkGH, reflevel - 1 ) { // do operations…, e.g. call "L" operator } END_SWITCH_TO_LEVEL; } END_LOCAL_COMPONENT_LOOP; } END_MAP_LOOP; } END_SWITCH_TO_LEVEL; } END_LOCAL_COMPONENT_LOOP; } END_MAP_LOOP; } END_SWITCH_TO_LEVEL; Or?

3 5

PGI configuration on trestles (maxwell release)
by Bruno C. Mundim 15 Apr '12

15 Apr '12

Hi, I am trying to put together a configuration file using PGI compilers (10.5) on trestles and I am facing a few problems. I am using the Maxwell release as code base and currently my configuration is quitting with an error when compiling Cactus/arrangements/LSUThorns/Vectors /src/test.cc. The error message is the following: ~/Cactus/arrangements/LSUThorns/Vectors/src/test.cc", line 172: error: no suitable user-defined conversion from "const __m128d" to "__m128i" exists VECTEST("kifpos positive", ^ I am using the following vector options: VECTORISE = yes VECTORISE_ALIGNED_ARRAYS = no VECTORISE_INLINE = no Does anyone have any idea on how to avoid/fix this? Actually, was this already fixed in the development branch? I would like to avoid using the development branch until I have a working configuration though. Also, when putting together this configuration file I looked into several ones at simfactory/mdb/optionlists that would use pgi10 compilers. Inspecting in more detail the one for hopper, hopper2-pgi10.cfg, I got puzzled by several options adopted there. I understand that the system there is a Cray one while trestles uses linux Centos5, so some options are really system dependent, but others I found quite intriguing and couldn't justify for. I hope someone with more experience with the pgi compilers could shed some light on the issues I found below. Here they are: 1) FPPFLAGS = -traditional: This is a flag for the fortran preprocessor that seems to be widely used in several config files. Virtually every config file uses cpp as preprocessor and I believe most of them if not all uses the GNU cpp. The trouble is that there is no such a flag for the most recent GNU cpp. What I could find was -traditional-cpp. So my question is: is this flag actually relevant? isn't already being ignored by the preprocessor? 2) CXXFLAGS = -g -DCRAY_XT --exceptions -D_ISOC99_SOURCE -D_BSD_SOURCE Here I see three issues: 2.1) I would use -g option only for debugging, otherwise optimization could be affected with PGI compilers. The manual suggests to use -gopt for optimization. This way the symbols would still be present without impacting optimization. 2.2) PGI optimization manual suggests that the user code do not handle exceptions in order to produce faster codes. I was wondering if it would be ok to move the --exceptions flag to CXX_DEBUG_FLAGS and add --noexceptions to CXX_OPTIMISE_FLAGS. Would Carpet/AHFinderDirect work properly with these changes? What is your opinion on it? 2.3) Why is the -D_ISOC99_SOURCE -D_BSD_SOURCE feature macros for GNU C Library needed here? I checked Cactus code and codes in arrangements and none of them use these macros. Also the counterpart intel config file doesn't use them either. So it doesn't seem to be system dependent options. Could anyone explain their need here? 3) F90FLAGS = -g -Mx,125,0x200 with the explanation: "allow calling C varargs functions from Fortran". Anyone could point to a PGI manual that mention this option? I couldn't find any so far. 4) LDFLAGS = -Wl,--allow-multiple-definition -Bstatic: Isn't the linker option --allow-multiple-definition a little bit dangerous? Right now I can't build GSL 1.14 since it quits with multiple definition of symbols kind of error if I don't use this flag. I would be tempted to build GSL with that flag, but not all the other code base. In any case, your opinion is appreciated here. Thanks, Bruno.

2 1

Simfactory (Maxwell) issue on Lonestar
by P. Ajith 13 Apr '12

13 Apr '12

Hello! I am trying to run the Tutorial for New Users (https://docs.einsteintoolkit.org/et-docs/Tutorial_for_New_Users ) on Lonestar (this is the first time I am using the machine). My job fails with the following error (in the .out) file: .... Simulation name: static_tov_8 Running simulation static_tov_8 Preparing: Checking: /scratch/01489/pajith/simulations/static_tov_8/output-0000-active c324-310.ls4.tacc.utexas.edu Fri Apr 13 01:46:47 CDT 2012 Environment: Starting: TACC: Starting up job 541416 TACC: Setting up parallel environment for OpenMPI mpirun. TACC: Setup complete. Running job script. TACC: starting parallel tasks... TACC: Could not determine MPI stack. Exiting! ... These are the modules that are loaded: login2$ module list Currently Loaded Modules: 1) TACC-paths 2) Linux 3) cluster-paths 4) intel/11.1 5) gzip/ 1.3.12 6) tar/1.22 7) cluster 8) TACC 9) openmpi/1.4.3 And my defs.local.ini file looks like this: # Defaults for all machines [default] user = pajith email = ajith_p(a)ligo.caltech.edu allocation = TG-MCA02N014 thornlist = manifest/einsteintoolkit.th [lonestar] user = pajith email = ajith_p(a)ligo.caltech.edu allocation = TG-MCA02N014 sourcebasedir = /work/01489/@USER@ basedir = /scratch/01489/@USER@/simulations envsetup = module purge && source /etc/profile.d/ tacc_modules.sh && module load TACC && module swap mvapich2 openmpi The extra bits were added to the envsetup (following the advice of Roland Haas) because by default the system was loading the mvapich2 module and *earlier* my jobs were failing with the following error: Error: You can only have one MPI module loaded at time. You already have mvapich2 loaded. To correct the situation, please enter the following command: module swap mvapich2 openmpi/1.4.3 Please let me know if you need any additional information. Thanks a lot! Ajith PS: Sorry for spamming you all. I am not sure if this mailing list is the appropriate place to bring this issue. I tried to submit a bug report at https://trac.einsteintoolkit.org/. But my ticket was rejected as Trac suspected it as spam! -- P. Ajith, LIGO Laboratory and Theoretical Astrophysics California Institute of Technology, MS 100-36, Pasadena, CA 91125, USA. Tel. +1 626 395 6295 (LIGO), 6162 (TApIR), Fax. +1 626 584 5735 What is your impact? http://www.nature.org/initiatives/climatechange/calculator

2 1

Thornlist cleanup
by Frank Loeffler 11 Apr '12

11 Apr '12

Hello, Before going into testing mode for the next release I would like to raise the question whether the following list of thorns should stay within the toolkit thornlist. If nobody speaks up for any of them we should see it as sign that we should think about removing them. Don't see this as suggestion to remove them right now - just as a reminder that we shouldn't keep thorns without purpose within the toolkit and from time to time have to think about possible removals. Any removal should anyway be announced first as depreciated and could then be removed for the fall release. not supported by GRHydro anymore: EinsteinEOS/EOS_Hybrid EinsteinEOS/EOS_IdealFluid EinsteinEOS/EOS_Polytrope EinsteinEOS/EOSG_Hybrid EinsteinEOS/EOSG_IdealFluid EinsteinEOS/EOSG_Polytrope unused, not well tested anymore: ? EinsteinAnalysis/AHFinder EinsteinInitialData/DistortedBHIVP EinsteinInitialData/IDAxiBrillBH EinsteinInitialData/IDAxiOddBrillBH EinsteinInitialData/IDBrillData EinsteinInitialData/RotatingDBHIVP Frank Loeffler

2 1

2026

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

Users April 2012