Users August 2011

users@lists.einsteintoolkit.org

17 participants
39 discussions

Inherits: grid should initialize 1-d coordinate gridfunctions?
by Scott Hawley 05 Aug '11

05 Aug '11

I'm getting a segmentation fault when I try to use, say "x[i]" in my thorn that inherits "grid", because the array x itself is null. Am I doing this wrong? I see that CartGrid3D executes… Thanks, Scott The beginning of the code looks like this... % cat mythorn.cc #include <cassert> #include <cstdio> #include <cstdlib> #include <sstream> #include <string> #include <vector> #include <fcntl.h> #include <unistd.h> #include "cctk.h" #include "cctk_Arguments.h" #include "cctk_Parameters.h" #include "Slab.h" #include "carpet.hh" using namespace std; using namespace Carpet; extern "C" void MyThorn(CCTK_ARGUMENTS) { DECLARE_CCTK_ARGUMENTS; DECLARE_CCTK_PARAMETERS; CCTK_INFO ("Calling MyThorn"); /* total number of points on this processor */ CCTK_INT nx = cctk_lsh[0]; CCTK_INT ny = cctk_lsh[1]; CCTK_INT nz = cctk_lsh[2]; /* print out x coordinates */ for (int i = 0; i < nx; i++) { printf("x[%d] = %g\n",i,x[i]); } ... % more interface.ccl IMPLEMENTS: MyThorn INHERITS: ADMBase StaticConformal grid USES INCLUDE HEADER: Slab.h -- Scott H. Hawley, Ph.D. Asst. Prof. of Physics Chemistry & Physics Dept Office: Hitch 100D Belmont University Tel: +1-615-460-6206 Nashville, TN 37212 USA Fax: +1-615-460-5458 PGP Key at http://sks-keyservers.net

2 2

Schedule corrections to various thorns: Requires update of MoL and other thorns!
by Erik Schnetter 04 Aug '11

04 Aug '11

As discussed a few days ago, I have just committed corrections to the schedule.ccl of various thorns. If you update any of these thorns, then you need to update MoL and several other thorns as well -- otherwise you will have an inconsistent schedule, and time evolution will be gravely inconsistent. (If you fail to update, you are supposed to see an error message.) In particular, I corrected the schedules of - GRHydro - ML_ADMConstraints - ML_AMDQuantities - ML_BSSN - NaNChecker and these corrections crucially depend on the thorns - Carpet - (Kranc -- only when you generate code yourself) - MoL - TmunuBase These schedule corrections ensure consistency during checkpointing and recovery, and correct a few other minor spots. In particular, if you checkpoint and recover, you should now see identical results (except for the pressure, which depends on the con2prim accuracy). Previously, certain operations were missing or were applied twice after recovering, leading e.g. to "weird" constraint values for the next few coarse grid time steps. To my knowledge, the evolution variables themselves were already consistent after recovery up to floating-point round-off (and up to con2prim accuracy). -erik -- Erik Schnetter <schnetter(a)cct.lsu.edu> http://www.cct.lsu.edu/~eschnett/

2 1

ETK tutorial: long runtime for tov example?
by Scott Hawley 03 Aug '11

03 Aug '11

The tutorial says to run ./simfactory/sim create-submit static_tov --parfile=par/static_tov.par --procs=32 --walltime=8:0:0 Does this mean the run is going to take 8 hours to complete? That's what 'sim help' seems to indicate, as does the fact that my job is still "ACTIVE" with no output directory after a few hours. Perhaps an example specifying a shorter run time would be good for tutorial purposes.

2 1

Re: [Users] [Commits] [svn:einsteintoolkit] GRHydro/trunk/ (Rev. 260)
by Roland Haas 02 Aug '11

02 Aug '11

Hello Erik, all, > +schedule group HydroBase_Boundaries AT EVOL AFTER MoL_Evolution AFTER (GRHydro_AtmosphereRese GRHydro_AtmosphereResetM) Is GRHydro_AtmosphereRese a typo here or did we actually release a version of GRHydro with such a scheduled routine? :-) > -schedule GROUP SetTmunu IN MoL_Evolution AFTER HydroBase_Boundaries > -{ > -} "Calculate the stress-energy tensor" > - I missed pointing out that this one is required by GRHydro's AtmosphereReset routine. Since AtmosphereReset changes the primitives and Tmunu depends on the primitives, it has to be recalculated. Sorry about that. If there is no objection I will put this schedule item back in together with comments similar to the original "you did not see this abuse of MoL comment" as to why they are required (even though they are bad). Yours, Roland

1 0

GetComponents Errors for private thorn
by Scott Hawley 02 Aug '11

02 Aug '11

Hi, I'm trying to add an svn repository for my thorn in my Thornlist file, and it doesn't seem to be working or telling me the nature of the problem: % ./GetComponents --debug --verbose ~/ThornLists/ThornListTexMEx Warning: Nothing will be checked out from svn://svn.belmont.edu/svnrepos/texmex TexMEx Is there something besides "debug" and "verbose" that will tell me WHY it won't be checked out? (I was reading the docs at http://cactuscode.org/documentation/GetComponents.php) Thanks. -Scott PS- The lines in question are: # TexMex !TARGET = $ARR !TYPE = svn !ANONYMOUS_USER = anonymous !ANONYMOUS_PASSWORD = anon !URL = svn://svn.belmont.edu/svnrepos/texmex TexMEx -- Scott H. Hawley, Ph.D. Asst. Prof. of Physics Chemistry & Physics Dept Office: Hitch 100D Belmont University Tel: +1-615-460-6206 Nashville, TN 37212 USA Fax: +1-615-460-5458 PGP Key at http://sks-keyservers.net

6 12

Meeting minutes
by Roland Haas 01 Aug '11

01 Aug '11

Meeting minutes: Present were: Tanja, Roland, Frank, Ian, Eloisa, Erik, Peter, Josh, Bruno, Christian Recovery/scheduling cleanup (see http://lists.einsteintoolkit.org/pipermail/users/2011-July/001270.html): * there should be no need to modify existing code to keep them working, though without modifications they (eg. Whisky vs. GRHydro) might display the same inconsistencies that they currently do ET paper: * Ian will expand the section on Kranc to 1 page * Josh and Christian will work on the GRHydro evolution section based on text that Christian will (has) commit(ted) * Josh will work through the introduction section and add bibtex entries for the missing references to einsteintoolkit.bib (from a bibtex file of a review of his own). We want to keep the number of bibtex files used to a minimum. * Christian will work on the EOS section (5.5) to describe EOS_Omni, in particular the tabulated EOS support. Will do so after finishing other work this week. * Peter will move the Tracker description into the analysis section, TmunuBase into the base thorn section. This leaves the relativity tools section short so it will be merged/restructured. * Roland will merge the AMR section in 5.8.3 with the one in the Carpet section 4.3 * Roland will move the Boundary conditions section in 5.8 into the Boundary conditions subsection of the evolution section * collect ideas for future work, and acknowledgments. Frank will compile them into a coherent text near the end of the paper writing process * examples: ** spinning BH: Peter has tapered runs and sees 4th order convergence ** BBH: Bruno does not see 4th order convergence. Tapering might help. For now Bruno will analyze the data he has and it will be decided based upon these if we want tapering. Otherwise list reasons why we do not see 4th order convergence (time interpolation, moving punctures, outer boundary conditions, ...). ** TOV: Frank has evolution data for a strongly perturbed TOV, where he sees the fundamental mode frequency and multiples of it but no higher modes. To see higher modes a weaker perturbation and more accuracy seems required ** TOV collapse: Christian has collapse parfile (and data?) ready, will commit later this week ** all: please commit parameter files and data if possible ** parameter files will be made public on a paper page ** if you have experience with the utility thorns (Timers, Formaline, others that were mentioned after my connections dropped) please consider adding them to you parameter file ** add references to physics papers with simulations similar to the examples * we will have a short status update in the Wednesday Cactus chat this week (see http://cactuscode.org/pipermail/users/2011-July/002868.html for connection details) Yours, Roland -- My email is as private as my paper mail. I therefore support encrypting and signing email messages. Get my PGP key from http://keys.gnupg.net.

1 0

Meeting minutes
by Roland Haas 01 Aug '11

01 Aug '11

Meeting minutes: Present were: Tanja, Roland, Frank, Bruno, Yosef, Peter, Erik, Eloisa, Ian ET paper: MHD: * Christian and Bruno independently looked at PPM and re-split its MHD version from the hydro only version and rearranged code to allow the compiler (ifort) to optimize the routine * Bruno finds about 3% speedup (in PPM) * Erik says that Christian found about 30% speedup * Bruno and Christian will exchange notes * Peter agreed to run some performance tests with hardware counters using a parameter file that Bruno will provide * we still fail the cylindrical blast wave test which many other codes also do not pass (using divergence cleaning) * will try dissipation in the B field * will run white noise test for dissipation to check that divergence cleaning works as expected (see apples with apples paper) CCE: * Yosef committed writeup of sketch of documentation and a sample parameter file based on simfactory's qc0 example * we would like a testsuite, which might however require a large input file for CCE * Yosef is not sure if the PITTNull CCE extractor would work with Multipatch codes since he has no access to any * Christian Reisswigs Worldtube thorn should (there is a Llama version) * Yosef will give a short tutorial at a later date, date TBA testsuites: * see Ian's email http://lists.einsteintoolkit.org/pipermail/users/2011-July/001253.html * test data was regenerated for RotatingSymmetry which was affected by changes in Kranc which affected McLachlan * correct boundary/ghost zones in GRHydro test for second order convergence (2 required for McLachlan due to advective derivatives) * try tapering to find fourth order convergence in constraints paper: * no progress * need to write missing sections (!) * all authors: take half an hour to read sections and make comments * lead author(s) will assign sections to people to work on in the next two weeks, and which sections to keep or drop (depending on their state) * reserve a Monday call for paper talk (with all callers having read the paper) * decide on day for call within the next three days (until Wednesday), Frank will set up a doodle poll Yours, Roland -- My email is as private as my paper mail. I therefore support encrypting and signing email messages. Get my PGP key from http://keys.gnupg.net.

3 3

Checkpoint/recovery inconsistencies
by Erik Schnetter 01 Aug '11

01 Aug '11

Christian Ott, Peter Diener, and I tracked down a set of checkpointing/recovery inconsistencies in the Einstein Toolkit. This means that many variables had different values after checkpointing and then recovering them, which should not be the case. We were careful not to use MPI, OpenMP, or any kind of compiler optimisations, so these inconsistencies represent real changes. We found a series of problems, and we have now one possible correction. Unfortunately, this requires changes to several thorns, mostly to schedule.ccl declarations, but also to Carpet. In addition, we find that it is impossible to re-calculate PseudoEvolution variables after recovery -- they have to be checkpointed. In other words, variables such as ADMBase, TmunuBase, or other variables with 3 timelevels need to be checkpointed to be consistent. (Many of the inconsistencies are small, and are caused by differences in the discretisation error. That is, they will vanish in the continuum limit. However, one of the design principles of the Einstein Toolkit is that results should be as much as possible independent of the number of MPI processes, OpenMP threads, and checkpointing/recovery. Disabling checkpointing of such variables could be implemented depending on a parameter, which would also need to ensure that these variables are then recalculated -- with slightly different values -- after recovery. In particular, this would require a new schedule bin in Carpet that is executed after recovery for all timelevels, or alternatively a new schedule option that requests this behaviour from Carpet.) The necessary changes to correct these inconsistencies are in detail: 1. All variables with 3 timelevels need to be checkpointed. This affects ADMBase, TmunuBase, ML_ADMConstraints, ML_ADMQuantities, and the BSSN constraints in ML_BSSN. Since this depends on the number of timelevels, care has to be taken to make the right decision at run time. 2. The schedule group MoL_PseudoEvolution must not be scheduled in post_recover_variables -- since these variables are checkpointed, they do not need to be recalculated. 3. Carpet cannot execute the post_recover_variables bin on the past timelevels, since applying boundary conditions to past timelevels includes prolongation, and the time interpolation necessary for this would have a different discretisation error. 4. Carpet traversed the PostRestrict bin in the incorrect order. It traversed from finest to coarsest (the same order in which restriction has to be applied), but because fine grid boundaries may be interpolate from coarse grid boundaries, this bin must be traversed from coarsest to finest. 5. The scheduling of GRHydro and HydroBase is off in certain places. 6. The scheduling of NaNChecker leaves the NaNmask uninitialised after recovery. 7. The scheduling of various Kranc generated thorns is off in certain places. 8. Certain schedule items requiring the ADMBase variables are executed in the incorrect order, i.e. before the ADMBase variables were available. I attach a diff of a possible set of corrections. However: - This diff does not correct handling the TmunuBase variables -- they are still not checkpointed if they have multiple timelevels. I believe this should be done by thorn TmunuBase itself. - Similarly, handling checkpointing of the ADMBase variables should be moved from ML_BSSN_Helper to ADMBase itself. - I think a new schedule group MoL_PseudoEvolutionBoundaries would make sense, as this would simplify the schedule of thorns which use MoL_PseudoEvolution. -erik -- Erik Schnetter <schnetter(a)cct.lsu.edu> http://www.cct.lsu.edu/~eschnett/

4 7

Einstein Toolkit Meeting Reminder
by Frank Loeffler 01 Aug '11

01 Aug '11

Hi Please consider joining the weekly Einstein Toolkit phone call at 10 am US central time on Mondays. As usual, you can find instructions how to join on the following web site: http://einsteintoolkit.org/community/support/ I short: the number is (+1) 225-578-4942 or (+1) 866-573-0359 and the conference id is 118682#. This call will be devoted to an effort to make progress with the paper, but I would like to use the first minutes to give everyone the possibility to raise questions or make comments. Next weeks meeting will again have a "usual" agenda. Frank

1 0

2026

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

Users August 2011