Users March 2013

users@lists.einsteintoolkit.org

8 participants
19 discussions

Einstein Toolkit Meeting Reminder
by Frank Loeffler 14 May '15

14 May '15

Hi Please consider joining the weekly Einstein Toolkit phone call at 10 am US central time on Mondays. As usual, you can find instructions how to join on the following web site: http://einsteintoolkit.org/community/support/ I short: the number is (+1) 225-578-4942 or (+1) 866-573-0359 and the conference id is 118682#. My agenda contains: - Fall workshop - PLEASE let us know if you come [1]! - Hotel booking - Release planning - release date: Mon Oct 24 2011 - freeze date: Mon Sep 26 - the day of the call! - branch name: ET_2011_10 - List of release-concerning bugs [2] - ET paper: proposed "freeze" by Oct 3rd (Mon after this call) As always: feel free to add to this list. Frank Loeffler [1] https://docs.einsteintoolkit.org/et-docs/ET_Workshop_Fall_2011#Fall_Einstei… [2] https://trac.einsteintoolkit.org/query?status=!closed&milestone=ET_2011_11&…

10 31

strong scaling tests on Stampede (TACC)
by Bruno Giacomazzo 19 Sep '13

19 Sep '13

Hi, I did some strong scaling tests on Stampede (https://www.xsede.org/tacc-stampede) with both Whisky and GRHydro (using the development version of ET in both cases). I used a Carpet par file that Roberto DePietri provided me and that he used for similar tests on an Italian cluster (I have attached the GRHydro version, the Whisky one is similar except for using Whisky instead of GRHydro). I used both Intel MPI and Mvapich and I did both pure MPI and MPI/OpenMP runs. I have attached a text file with my results. The first column is the name of the run (if it starts with mvapich it used mvapich otherwise it used Intel MPI), the second one is the number of cores (option --procs in simfactory), the third one the number of threads (--num-threads), the fourth one the time in seconds spent in CCTK_EVOL, and the fifth one the walltime in seconds (i.e., the total time used by the run as measured on the cluster). I have also attached a couple of figures that show CCTK_EVOL vs #cores and walltime vs #cores (only for Intel MPI runs). First of all, in pure MPI runs (--num-threads=1) I was unable to run on more than 1024 cores using Intel MPI (the run was just crashing before iteration zero or hanging up). No problem instead when using --num-threads=8 or --num-threads=16. I also noticed that scaling was particularly bad in pure MPI runs and that a lot of time was spent outside CCTK_EVOL (both with Intel MPI and MVAPICH). After speaking with Roberto, I found out that the problem is due to 1D ASCII output (which is active in that parfile) and that makes the runs particularly slow above ~100 cores on this machine. In plot_scaling_walltime_all.pdf I plot also two pure MPI runs, but without 1D ASCII output and the scaling is much better in this case (the time spent in CCTK_EVOL is identical to the case with 1D output and hence I didn't plot them in the other figure). I didn't try using 1D hdf5 output instead, does anyone use it? According to my tests, --num-threads=16 performs better than --num-threads=8 (which is the current default value in simfactory) and Intel MPI seems to be better than MVAPICH. Is there a particular reason for using 8 instead of 16 threads as the default simfactory value on Stampede? Let me know if you have any comment or suggestion. Cheers, Bruno Dr. Bruno Giacomazzo JILA - University of Colorado 440 UCB Boulder, CO 80309 USA Tel. : +1 303-492-5170 Fax : +1 303-492-5235 email : bruno.giacomazzo(a)jila.colorado.edu web: http://www.brunogiacomazzo.org ---------------------------------------------------------------------- There are only 10 types of people in the world: Those who understand binary, and those who don't ----------------------------------------------------------------------

6 10

Contamination in restarting using Carpet mesh refinement
by Kentaro Takami 29 Mar '13

29 Mar '13

Dear all, I have the following question. In my project, it is important to compute reproducible results when I use same executable, initial data, parameter file. So I'm debugging and improving our code (as you know, one of them is dissipation thorn.) Now I can get exactly same results every time unless using Carpet refinement being larger than 2 (i.e., only 1 refinement case is OK). When we restart the simulation with several refinement from checkpoint files, the results become slightly change. So I investigated this reason, and I notice the finer grid variables are contaminated between (A) and (B) in the following Cactus scheduling: ========== Related part of Cactus scheduling ========== [CCTK_RECOVER_VARIABLES] IOUtil::IOUtil_RecoverGH: [level] Checkpoint recovery routine [CCTK_POST_RECOVER_VARIABLES] .... (A) < IN THE END OF THIS SCHEDULE BIN, VARIABLES ARE FINE > endif if (checkpoint initial data) [CCTK_CPINITIAL] CarpetIOHDF5::CarpetIOHDF5_InitialDataCheckpoint: [meta] Initial data checkpoint routine endif if (analysis) [CCTK_ANALYSIS] (B) < IN THE TOP OF THIS SCHEDULE BIN, VARIABLES ARE CONTAMINATED > ..... endif Output grid variables do loop over timesteps [CCTK_PREREGRID] ================================================ Apparently there is no thorn between (A) and (B). So what is doing between (A) and (B)? Can I avoid this changing of variables? Do I need certain special parameters? Further information: In the test case, I used the following conditions. * Carpet and CarpetRegrid2 with 2 refinement level. (in this case, RL1 is OK, but RL2 is contaminated.) * Of course I'm using the parameters, Carpet::regrid_during_recovery = "no" and CarpetIOHDF5::use_grid_structure_from_checkpoint = "yes". * Debug options such as O0, -fp-model precise, VECTORISE = no, OPENMP = no in compiling. * Even if we use only 1 MPI process, still there is contamination. Sincerely yours Kentaro TAKAMI

1 0

_No_ Einstein Toolkit Meeting
by Frank Loeffler 22 Mar '13

22 Mar '13

Hi, There will be no weekly Einstein Toolkit meeting on the coming Monday, Mar 25. The next meeting will be one week after that: on April 1st. One reminder especially to developers: if you plan to get the newest, best, hottest feature into the next ET release: hurry up. We are approaching April which means that at some point especially larger changes will not be permitted into the toolkit and have to wait until after the release. Frank Loeffler

1 0

Skip meeting next week?
by Frank Loeffler 21 Mar '13

21 Mar '13

Hi, As it turns out, the whole LSU group _has_ to attend a local meeting next Monday, exactly at the time of the usual ET call. Unless there is something urgent I will cancel next week's call, which would mean that the next call would be on Mon, April 1st. Please let us know by tomorrow if there is the need to have a call next week. Frank Loeffler

1 0

CarpetRegrid2, RotationSymmetry90 and cell centered grid
by Roland Haas 18 Mar '13

18 Mar '13

Hello all, I am trying to run a cell centered run with Carpet and RotatingSymmetry90. Using the attached parfile (cropped from the real one) I get an error cactus_bns_all: /mnt/data/rhaas/postdoc/gr/Zelmani/arrangements/Carpet/CarpetRegrid2/src/property.cc:382: ibset CarpetRegrid2::rotsym90::symmetrised_regions(const gh&, const dh&, const CarpetRegrid2::level_boundary&, const std::vector<bboxset<int, 3> >&, int): Assertion `all (iaxis0 % istr == 0)' failed. Rank 0 with PID 12717 received signal 6 Right now I can just run without the symmetry (slower of course...). Before I eventually try to debug this: is Regrid2 supposed to support 90 degree symmetry when using cell centering or was it only ever tested with vertex centering or without 90 degree symmetry? Yours, Roland -- My email is as private as my paper mail. I therefore support encrypting and signing email messages. Get my PGP key from http://keys.gnupg.net.

2 1

meeting minutes
by Roland Haas 18 Mar '13

18 Mar '13

Present were: Frank, Roland, Peter, Philipp, Ian, Eloisa, Seth, Steve, Tanja Example parameter files: * we would like to have a description on how long the run will run, how many cores are needed, what data to compare * Ian suggests using a qc0 ML file * want to test the parameter files in the thorns' "par" folder before the release Next ET release: * aim for release date of May 23rd * create release branches on May 10th, spend time testing it * have all new thorns in April 8th * create release tag for tarball Yours, Roland

1 0

Einstein Toolkit Meeting Reminder
by Frank Loeffler 17 Mar '13

17 Mar '13

Hi Please consider joining the weekly Einstein Toolkit phone call at 10 am US central time on Mondays. As usual, you can find instructions how to join on the following web site: http://einsteintoolkit.org/community/support/ In short: the number is (+1) 225-578-4942 or (+1) 866-573-0359 and the conference id is 118682#. Some items to discuss: - ET_2013_05 schedule - proposed rough timeline: - May 01: feature freeze of /trunk extensive testing begins (test-suites on all relevant machines) - May 10: release branches are created trunk un-frozen again release-branch code in only-important-bugs-freeze bug-fixing primarily in release branch - May 21: complete freeze, to prepare tar-balls, add tags ect. - May 23: target release date (or earlier if all bugs are fixed) - Example parameter files for ET http://einsteintoolkit.org/about/gallery/ - added TOV parameter file - simple, static TOV - runs in about 2SU: <1h on an old laptop, 10min on 1 cluster node - please add more! Frank Loeffler

1 0

McLachlan, UseSpatialBetaDriver
by Erik Schnetter 14 Mar '13

14 Mar '13

I notice there is a parameter UseSpatialBetaDriver in McLachlan. This parameter's value is never accessed. (1) Is this intentional, or is there some important functionality missing? (2) Can we remove this parameter, or would this break people's parameter files? -erik -- Erik Schnetter <schnetter(a)cct.lsu.edu> http://www.perimeterinstitute.ca/personal/eschnetter/

2 1

How to add external fields to a simulation?
by Ben Bernard 13 Mar '13

13 Mar '13

All, I have been stydying the par files that are provided, but can't seem to figure out how to add an external scalar, spinor, or vector field to the simulation. Specifically, I am interested in periodically sweeping a polarized electromagnetic plane wave across a simulation volume, from a source point that is outside the simulation volume. I am also curious about permeating a volume with a bispinor field. Is there a package that I would have to add? Regards, Ben Bernard

2 1

2026

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

Users March 2013