Scott
This usually means that the libraries used to build the executable and the mpirun command don't belong together. In other words, either the libraries used for building, or the shared libraries used at run time (if you use shared libraries), or the mpirun executable are not what you think they are.
make texmex SILENT=no shows you (at the end) which libraries are used during the build.
ldd exe/cactus_texmex (or otool -L exe/cactus_texmex) shows which shared libraries will be used. Certain environment variables influence this; make sure you use the same settings that you use when you call mpirun.
"which mpirun" shows which mpirun will be used.
-erik
On Fri, May 20, 2011 at 12:36 PM, Scott Hawley scott.hawley@belmont.edu wrote:
My code works totally ok with MPI on my linux machine, but as I'm porting to my Mac, I find that MPI isn't working right. I'm using the OpenMPI version from fink) A simple "hello world" MPI program works fine (proof at the bottom of this message), but Cactus does not.
That is to say, e.g. an 8-processor run ("/sw/bin om-mpirun -np 8 ....") results in 8 processes, but each processes thinks it's processor 0, and that there are no other processors.
Any ideas why this might happen? Thanks, Scott
I'm using Cactus version 4.0.b17 (Compile date: May 20 2011 (11:13:49)).
Cactus run:
mpirun -np 8 ~/Cactus/exe/cactus_texmex sep20a0.5fmrbbh97_cactus.par
10 1 0101 ************************ 01 1010 10 The Cactus Code V4.0 1010 1101 011 www.cactuscode.org 1001 100101 ************************ 00010101 100011 (c) Copyright The Authors 0100 GNU Licensed. No Warranty 0101
Cactus version: 4.0.b17 Compile date: May 20 2011 (11:13:49)
10 1 0101 ************************ 01 1010 10 The Cactus Code V4.0 1010 1101 011 www.cactuscode.org 1001 100101 ************************ 00010101 100011 (c) Copyright The Authors 0100 GNU Licensed. No Warranty 0101
Cactus version: 4.0.b17 Compile date: May 20 2011 (11:13:49)
10 1 0101 ************************ 01 1010 10 The Cactus Code V4.0 1010 1101 011 www.cactuscode.org 1001 100101 ************************ 00010101 100011 (c) Copyright The Authors 0100 GNU Licensed. No Warranty 0101
Cactus version: 4.0.b17 Compile date: May 20 2011 (11:13:49)
10 1 0101 ************************ 01 1010 10 The Cactus Code V4.0 1010 1101 011 www.cactuscode.org 1001 100101 ************************ 00010101 100011 (c) Copyright The Authors 0100 GNU Licensed. No Warranty 0101
Cactus version: 4.0.b17 Compile date: May 20 2011 (11:13:49)
10 1 0101 ************************ 01 1010 10 The Cactus Code V4.0 1010 1101 011 www.cactuscode.org 1001 100101 ************************ 00010101 100011 (c) Copyright The Authors 0100 GNU Licensed. No Warranty 0101
Cactus version: 4.0.b17 Compile date: May 20 2011 (11:13:49)
10 1 0101 ************************ 01 1010 10 The Cactus Code V4.0 1010 1101 011 www.cactuscode.org 1001 100101 ************************ 00010101 100011 (c) Copyright The Authors 0100 GNU Licensed. No Warranty 0101
Cactus version: 4.0.b17 Compile date: May 20 2011 (11:13:49)
10 1 0101 ************************ 01 1010 10 The Cactus Code V4.0 1010 1101 011 www.cactuscode.org 1001 100101 ************************ 00010101 100011 (c) Copyright The Authors 0100 GNU Licensed. No Warranty 0101
Cactus version: 4.0.b17 Compile date: May 20 2011 (11:13:49)
10 1 0101 ************************ 01 1010 10 The Cactus Code V4.0 1010 1101 011 www.cactuscode.org 1001 100101 ************************ 00010101 100011 (c) Copyright The Authors 0100 GNU Licensed. No Warranty 0101
Cactus version: 4.0.b17 Compile date: May 20 2011 (11:13:49) Run date: May 20 2011 (11:24:38-0500) Run host: provision.local (pid=6067) Working directory: /Users/shawley/runs/cactus Executable: /Users/shawley/Cactus/exe/cactus_texmex Parameter file: sep20a0.5fmrbbh97_cactus.par
Run date: May 20 2011 (11:24:38-0500) Run host: provision.local (pid=6071) Working directory: /Users/shawley/runs/cactus Executable: /Users/shawley/Cactus/exe/cactus_texmex Parameter file: sep20a0.5fmrbbh97_cactus.par
Activating thorn Cactus...Success -> active implementation Cactus Run date: May 20 2011 (11:24:38-0500) Run host: provision.local (pid=6070) Activating thorn Cactus...Success -> active implementation Cactus Run date: May 20 2011 (11:24:38-0500) Run host: provision.local (pid=6074) Working directory: /Users/shawley/runs/cactus Executable: /Users/shawley/Cactus/exe/cactus_texmex Parameter file: sep20a0.5fmrbbh97_cactus.par
Activating thorn Cactus...Success -> active implementation Cactus Run date: May 20 2011 (11:24:38-0500) Run host: provision.local (pid=6073) Working directory: /Users/shawley/runs/cactus Executable: /Users/shawley/Cactus/exe/cactus_texmex Parameter file: sep20a0.5fmrbbh97_cactus.par
Working directory: /Users/shawley/runs/cactus Executable: /Users/shawley/Cactus/exe/cactus_texmex Parameter file: sep20a0.5fmrbbh97_cactus.par
Activating thorn Cactus...Success -> active implementation Cactus
Activating thorn Cactus...Success -> active implementation Cactus Run date: May 20 2011 (11:24:38-0500) Run host: provision.local (pid=6068) Run date: May 20 2011 (11:24:38-0500) Run host: provision.local (pid=6069) Run date: May 20 2011 (11:24:38-0500) Run host: provision.local (pid=6072) Working directory: /Users/shawley/runs/cactus Executable: /Users/shawley/Cactus/exe/cactus_texmex Parameter file: sep20a0.5fmrbbh97_cactus.par
....and so on.....
Yet a imple program works: The simple example which works fine is...
cat hello.c
#include <stdio.h> #include "mpi.h"
int main(argc, argv) int argc; char **argv; { int rank, size; MPI_Init(&argc,&argv); MPI_Comm_rank(MPI_COMM_WORLD, &rank); MPI_Comm_size(MPI_COMM_WORLD, &size); printf("Hello world! I am %d of %d\n",rank,size); MPI_Finalize(); return 0; }
mpicc -o hello hello.c which mpirun
/sw/bin/mpirun
ls -l /sw/bin/mpirun
lrwxr-xr-x 1 root shawley 9 May 20 10:55 /sw/bin/mpirun@ -> om-mpirun
mpirun -np 8 ./hello
Hello world! I am 0 of 8 Hello world! I am 6 of 8 Hello world! I am 3 of 8 Hello world! I am 1 of 8 Hello world! I am 2 of 8 Hello world! I am 4 of 8 Hello world! I am 5 of 8 Hello world! I am 7 of 8
Users mailing list Users@einsteintoolkit.org http://lists.einsteintoolkit.org/mailman/listinfo/users