Users

users@lists.einsteintoolkit.org

3 participants
3984 discussions

MPI trouble on 64-bit Mac OS X 10.6.7 (fink version)
by Scott Hawley 21 May '11

21 May '11

My code works totally ok with MPI on my linux machine, but as I'm porting to my Mac, I find that MPI isn't working right. I'm using the OpenMPI version from fink) A simple "hello world" MPI program works fine (proof at the bottom of this message), but Cactus does not. That is to say, e.g. an 8-processor run ("/sw/bin om-mpirun -np 8 ....") results in 8 processes, but each processes thinks it's processor 0, and that there are no other processors. Any ideas why this might happen? Thanks, Scott I'm using Cactus version 4.0.b17 (Compile date: May 20 2011 (11:13:49)). Cactus run: > mpirun -np 8 ~/Cactus/exe/cactus_texmex sep20a0.5fmrbbh97_cactus.par -------------------------------------------------------------------------------- 10 1 0101 ************************ 01 1010 10 The Cactus Code V4.0 1010 1101 011 www.cactuscode.org 1001 100101 ************************ 00010101 100011 (c) Copyright The Authors 0100 GNU Licensed. No Warranty 0101 -------------------------------------------------------------------------------- Cactus version: 4.0.b17 Compile date: May 20 2011 (11:13:49) -------------------------------------------------------------------------------- 10 1 0101 ************************ 01 1010 10 The Cactus Code V4.0 1010 1101 011 www.cactuscode.org 1001 100101 ************************ 00010101 100011 (c) Copyright The Authors 0100 GNU Licensed. No Warranty 0101 -------------------------------------------------------------------------------- Cactus version: 4.0.b17 Compile date: May 20 2011 (11:13:49) -------------------------------------------------------------------------------- 10 1 0101 ************************ 01 1010 10 The Cactus Code V4.0 1010 1101 011 www.cactuscode.org 1001 100101 ************************ 00010101 100011 (c) Copyright The Authors 0100 GNU Licensed. No Warranty 0101 -------------------------------------------------------------------------------- Cactus version: 4.0.b17 Compile date: May 20 2011 (11:13:49) -------------------------------------------------------------------------------- 10 1 0101 ************************ 01 1010 10 The Cactus Code V4.0 1010 1101 011 www.cactuscode.org 1001 100101 ************************ 00010101 100011 (c) Copyright The Authors 0100 GNU Licensed. No Warranty 0101 -------------------------------------------------------------------------------- Cactus version: 4.0.b17 Compile date: May 20 2011 (11:13:49) -------------------------------------------------------------------------------- 10 1 0101 ************************ 01 1010 10 The Cactus Code V4.0 1010 1101 011 www.cactuscode.org 1001 100101 ************************ 00010101 100011 (c) Copyright The Authors 0100 GNU Licensed. No Warranty 0101 -------------------------------------------------------------------------------- Cactus version: 4.0.b17 Compile date: May 20 2011 (11:13:49) -------------------------------------------------------------------------------- 10 1 0101 ************************ 01 1010 10 The Cactus Code V4.0 1010 1101 011 www.cactuscode.org 1001 100101 ************************ 00010101 100011 (c) Copyright The Authors 0100 GNU Licensed. No Warranty 0101 -------------------------------------------------------------------------------- Cactus version: 4.0.b17 Compile date: May 20 2011 (11:13:49) -------------------------------------------------------------------------------- 10 1 0101 ************************ 01 1010 10 The Cactus Code V4.0 1010 1101 011 www.cactuscode.org 1001 100101 ************************ 00010101 100011 (c) Copyright The Authors 0100 GNU Licensed. No Warranty 0101 -------------------------------------------------------------------------------- Cactus version: 4.0.b17 Compile date: May 20 2011 (11:13:49) -------------------------------------------------------------------------------- 10 1 0101 ************************ 01 1010 10 The Cactus Code V4.0 1010 1101 011 www.cactuscode.org 1001 100101 ************************ 00010101 100011 (c) Copyright The Authors 0100 GNU Licensed. No Warranty 0101 -------------------------------------------------------------------------------- Cactus version: 4.0.b17 Compile date: May 20 2011 (11:13:49) Run date: May 20 2011 (11:24:38-0500) Run host: provision.local (pid=6067) Working directory: /Users/shawley/runs/cactus Executable: /Users/shawley/Cactus/exe/cactus_texmex Parameter file: sep20a0.5fmrbbh97_cactus.par -------------------------------------------------------------------------------- Run date: May 20 2011 (11:24:38-0500) Run host: provision.local (pid=6071) Working directory: /Users/shawley/runs/cactus Executable: /Users/shawley/Cactus/exe/cactus_texmex Parameter file: sep20a0.5fmrbbh97_cactus.par -------------------------------------------------------------------------------- Activating thorn Cactus...Success -> active implementation Cactus Run date: May 20 2011 (11:24:38-0500) Run host: provision.local (pid=6070) Activating thorn Cactus...Success -> active implementation Cactus Run date: May 20 2011 (11:24:38-0500) Run host: provision.local (pid=6074) Working directory: /Users/shawley/runs/cactus Executable: /Users/shawley/Cactus/exe/cactus_texmex Parameter file: sep20a0.5fmrbbh97_cactus.par -------------------------------------------------------------------------------- Activating thorn Cactus...Success -> active implementation Cactus Run date: May 20 2011 (11:24:38-0500) Run host: provision.local (pid=6073) Working directory: /Users/shawley/runs/cactus Executable: /Users/shawley/Cactus/exe/cactus_texmex Parameter file: sep20a0.5fmrbbh97_cactus.par -------------------------------------------------------------------------------- Working directory: /Users/shawley/runs/cactus Executable: /Users/shawley/Cactus/exe/cactus_texmex Parameter file: sep20a0.5fmrbbh97_cactus.par -------------------------------------------------------------------------------- Activating thorn Cactus...Success -> active implementation Cactus Activating thorn Cactus...Success -> active implementation Cactus Run date: May 20 2011 (11:24:38-0500) Run host: provision.local (pid=6068) Run date: May 20 2011 (11:24:38-0500) Run host: provision.local (pid=6069) Run date: May 20 2011 (11:24:38-0500) Run host: provision.local (pid=6072) Working directory: /Users/shawley/runs/cactus Executable: /Users/shawley/Cactus/exe/cactus_texmex Parameter file: sep20a0.5fmrbbh97_cactus.par ....and so on..... Yet a imple program works: The simple example which works fine is... > cat hello.c #include <stdio.h> #include "mpi.h" int main(argc, argv) int argc; char **argv; { int rank, size; MPI_Init(&argc,&argv); MPI_Comm_rank(MPI_COMM_WORLD, &rank); MPI_Comm_size(MPI_COMM_WORLD, &size); printf("Hello world! I am %d of %d\n",rank,size); MPI_Finalize(); return 0; } > mpicc -o hello hello.c > which mpirun /sw/bin/mpirun > ls -l /sw/bin/mpirun lrwxr-xr-x 1 root shawley 9 May 20 10:55 /sw/bin/mpirun@ -> om-mpirun > mpirun -np 8 ./hello Hello world! I am 0 of 8 Hello world! I am 6 of 8 Hello world! I am 3 of 8 Hello world! I am 1 of 8 Hello world! I am 2 of 8 Hello world! I am 4 of 8 Hello world! I am 5 of 8 Hello world! I am 7 of 8

4 8

outdated version of GetComponents on the einstein toolkit website
by Alexander Beck-Ratzka 21 May '11

21 May '11

Hi Folks, I have just downloaded GetCompents from the official einstein toolkit website via the command wget https://svn.cactuscode.org/Utilities/branches/ET_2011_05/Scripts/GetCompone… and also via cut & paste Calling ./GetComponents <thornlilst> gives me the message: [snip] Use of uninitialized value in pattern match (m//) at ./GetComponents line 135. You are using an outdated version of GetComponents. Please use the version from GitHub (https://github.com/gridaphobe/CRL) (if you are using the Einstein Toolkit, the new version should already be in Cactus/bin/GetComponents [snip] I mean, the website should contain the actual version. If someone is starting with the einstein toolkit, a message as: "if you are using the Einstein Toolkit, the new version should already be in Cactus/bin/GetComponents" is really not helpful. Best wishes Alexander

3 3

Bad link in simfactory python tutorial
by Scott Hawley 21 May '11

21 May '11

In https://docs.einsteintoolkit.org/et-docs/Pysim_Tutorial_for_New_Users "If you don't have an existing account on Queen Bee you can request an account<http://einsteintoolkit.org/info/documentation/request> to use for these examples. " "request an account" goes to 404 Page Not Found, "Please let the Einstein Toolkit team know if there's a bad internal link somewhere" (http://einsteintoolkit.org/info/documentation/request)

2 1

TwoPunctures warning message
by Luca Baiotti 17 May '11

17 May '11

Hello, I am running TwoPunctures for the first time and I see warnings like: WARNING level 1 in thorn Carpet processor 3 host node032.damiana.admin (line 393 of /home/baiotti/Cactus/arrangements/Carpet/Carpet/src/Storage.cc): -> There are not enough time levels for the desired temporal prolongation order in the grid function group "TWOPUNCTURES::PUNCTURE_U". With Carpet::prolongation_order_time=2, you need at least 3 time levels. Since TwoPunctures is run only at initial data, prolongation should not be necessary. Could those variables be tagged as prolongation-none? Cheers, Luca -- Thanks for saving energy! 節電のため、ご協力をお願いいたします。

2 3

Re: [Users] [Commits] [svn:einsteintoolkit] Paper_EinsteinToolkit_2010/examples/bbh/ (Rev. 86)
by Frank Loeffler 17 May '11

17 May '11

On Tue, May 17, 2011 at 11:06:24PM -0400, Roland Haas wrote: > Hmm. There is a new version of Slab/RotatingSymmetry in the development > version (which seemed to work for me when I tested it against the > testsuites). Maybe your run works with the old version of > RotatingSymmetry180? Does that mean the the hg version of Carpet doesn't work with CactusNumerical/RotatingSymmetry180? Frank

1 0

No Meeting tomorrow, Monday May 15th
by Frank Loeffler 17 May '11

17 May '11

Hi, At least Tanja, Roland, myself and Erik cannot join the ET call tomorrow, so let's cancel the meeting for this week. However: in order not to lose focus in the ET paper it would be nice to have some progress report for the largest missing sections, e.g., the examples, on this mailing list. For the TOVs: I ran a couple of resolutions of a standard TOV model and can easily see the fundamental mode. It is pretty clear that in order to get a good measurement of the frequency the resolution cannot be laptop or workstation-friendly, but that is to be expected. I cannot easily see or measure any overtones though - it's a nice, damped sine. I will finish to run the high-res simulations and then include that frequency measurement in the paper, comparing with the expected result. Frank

6 5

double instead of CCTK_REAL in two routines in GRHydro
by Jian Tao 13 May '11

13 May '11

Hi, There are two places in GRHydro code that variables are declared using double. It will be better to use CCTK_REAL instead. It makes it easier to check various things by setting REAL_PRECISION in the config file. GRHydro/src/GRHydro_Con2PrimM_pt.c: register double ftmp,gtmp; GRHydro/src/GRHydro_Con2PrimM_pt.c: double gam_m1_o_gam = ((gammaeos-1.)/gammaeos); -- Regards, Jian

2 1

CarpetMask problem (was: Re: [Commits] [svn:einsteintoolkit] Paper_EinsteinToolkit_2010/examples/bbh/ (Rev. 85))
by Erik Schnetter 12 May '11

12 May '11

Bruno The problem was indeed that CarpetReduce overwrote the mask. While setting up the mask, CarpetReduce uses a temporary integer grid function to store which octant of which cell is active. This is later converted into the real grid function that people use. CarpetMask needed to set up this integer grid function instead of the real grid function. Please update Carpet and try again. -erik On Wed, May 11, 2011 at 6:39 PM, Bruno Coutinho Mundim <bcmsma(a)astro.rit.edu> wrote: > Hi Erik, > > I have attached the schedule output for this run. I suspect CarpetReduce > is somehow overwriting the weight function set by CarpetMask. > > Thanks, > Bruno. > > Erik Schnetter wrote: >> >> That is strange. I assume some other routine overwrites it. Can you >> check the schedule? Is there another CarpetMask or CarpetReduce >> routine acting on the mask afterwards? >> >> -erik >> >> On Wed, May 11, 2011 at 1:30 PM, Bruno C. Mundim <bcmsma(a)astro.rit.edu> >> wrote: >>> >>> Thanks, Erik! On a related note: I checked if CarpetMask was setting >>> weight[ind]=0 inside the horizons and it seems to do so correctly. >>> However CarpetReduce::weight still do not reflect that. Its output >>> still doesn't have the horizons masked out. >>> >>> Thanks, >>> Bruno. >>> >>> Erik Schnetter wrote: >>>> >>>> On Wed, May 11, 2011 at 12:53 PM, Bruno C. Mundim <bcmsma(a)astro.rit.edu> >>>> wrote: >>>>> >>>>> Hi Roland: >>>>> >>>>> Roland Haas wrote: >>>>>> >>>>>> Hello Bruno, >>>>>> >>>>>>> a) Its evolution is stopped with Nans around 8.6M. Nans appear to >>>>>>> come >>>>>>> from the BSSN fields, but still needs further investigation. >>>>>> >>>>>> Hmm I could evolve the qc0 example in McLachlan. Maybe there are >>>>>> differences in the parameter file that make the error "obvious". >>>>>> >>>>> A few obvious changes that I can think of: >>>>> >>>>> 1) I reduced the number of spherical surfaces from 5 to 3. Now the >>>>> puncture tracker uses surfaces 0 and 1, AHFinderDirect track those >>>>> surfaces and load AH related info there. QLM, CarpetTracker and >>>>> CarpetMask uses those 2 surfaces then. I will go back to 5 surfaces >>>>> as it was set before. Maybe I am missing some subtle scheduling order >>>>> here. I don't know. >>>>> >>>>> 2) I added a few general parameters I am used to use to CarpetInterp, >>>>> CarpetLib and CarpetRegrid2. >>>>> >>>>> 3) From a discussion I had with Peter on the ET workshop, I decided >>>>> to change the AHFinderDirect interpolator to Hermite. >>>>> >>>>> So I will switch back to what it was and avoid to do so many changes >>>>> at once! :) However I find strange that these changes could be the >>>>> cause of what I am seeing. >>>>> >>>>> >>>>>>> c) When CarpetReduce::weight is output, its values seem strange to >>>>>>> me. >>>>>>> Most of the domain has 1/2 set. A few points close to the boundary >>>>>>> 1/4 >>>>>>> and the boundaries 0 (physical and ghost). No mask out values inside >>>>>>> AH is shown, ie they still are set to 1/2. These values seem to make >>>>>>> more sense to AMR restriction operations. >>>>>> >>>>>> Are you maybe looking at the z=0 plane? That plane has a weight of 1/2 >>>>>> since only half of the volume of each cell is used when eg. computing >>>>>> the simulation volume. I have never used CarpetMask myself (but have >>>>>> written a similar thorn of my own... would have paid to look into >>>>>> Carpet >>>>>> first I guess :-) ) >>>>>> >>>>> Yes I was looking at the z=0 plane and had refection symmetry set up! >>>>> Thanks for pointing this out. If we go out of the plane then the >>>>> weight=1.0 values show up. A more careful look into the data, I noticed >>>>> the zeros are set in the buffer zones indeed with the exception of the >>>>> third point (ghostzone=3) in the domain (from the AMR boundary) that >>>>> has >>>>> its value set to 1/4. Do you know why? >>>> >>>> For some reason that I currently cannot recall, the ghost zones in the >>>> mask may be set to 1 instead of 0. However, ghost zones are ignored >>>> anyway by a different mechanism when reducing (by using cctk_bbox and >>>> cctk_nghostzones). >>>> >>>> The particular values indicate how much of the "control cell" >>>> surrounding this grid point should be taken into account when >>>> reducing. If the value is 1/4, then you may be looking at an edge >>>> point (e.g. and x-y edge in the z=0 plane), where 3/4 of the control >>>> cell are already covered by the corresponding coarse grid. >>>> >>>> One important test is that the sum of all weights times the grid cell >>>> volume equals the total volume of the simulation domain. If you look >>>> into the simulation log output and search for "volume" there, you >>>> should find a corresponding statement. >>>> >>>> -erik -- Erik Schnetter <schnetter(a)cct.lsu.edu> http://www.cct.lsu.edu/~eschnett/

3 4

Re: [Users] [Commits] [svn:einsteintoolkit] Paper_EinsteinToolkit_2010/examples/bbh/ (Rev. 85)
by Roland Haas 11 May '11

11 May '11

Hello Bruno, > a) Its evolution is stopped with Nans around 8.6M. Nans appear to come > from the BSSN fields, but still needs further investigation. Hmm I could evolve the qc0 example in McLachlan. Maybe there are differences in the parameter file that make the error "obvious". > c) When CarpetReduce::weight is output, its values seem strange to me. > Most of the domain has 1/2 set. A few points close to the boundary 1/4 > and the boundaries 0 (physical and ghost). No mask out values inside > AH is shown, ie they still are set to 1/2. These values seem to make > more sense to AMR restriction operations. Are you maybe looking at the z=0 plane? That plane has a weight of 1/2 since only half of the volume of each cell is used when eg. computing the simulation volume. I have never used CarpetMask myself (but have written a similar thorn of my own... would have paid to look into Carpet first I guess :-) ) Yours Roland

4 5

Meeting minutes (fwd)
by Roland Haas 09 May '11

09 May '11

Meeting minutes: Present were: Frank, Peter, Erik, Ian, Eloisa, Josh, Yosef, Bruno, Tanja ET paper: * single spinning BH example: runs finished, Peter will add results this week * BBH runs have a parameter file that is ready to submit, Bruno will do so today * TOV star was done, can see fundamental mode very easily, higher harmonics are not easily visible. Need other full GR code result to compare to (maybe Christian Evan O'Connor's and Ott's GR1D code). * comsology: figures were missing, have been added since the call * agree to publish in CQG since the intended audience is numerical relativists fall workshop: * target audience is graduate students * will try to get funding for student travel from funding agencies (Josh) * topics will include: introduction to BSSN, general relativistic hydro, MHD, Cactus, Alpaca, VisIt * possible speakers listed in wiki on ET website * time: Oct 14th-18th (fall break for LSU GT) * location: GT or LSU, need room for 25 persons, GT will check if lecture hall can be used. LSU has room available. * accomodation: prefer inexpensive location (since the workshop is rather long). Atlanta should have places in the $40/night range which however require a commute of about 30-45min. Yours, Roland

1 0

Jump to page:

2026

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

Users