[Users] MPI trouble on 64-bit Mac OS X 10.6.7 (fink version)
Ian Hinder
ian.hinder at aei.mpg.de
Sat May 21 04:51:02 CDT 2011
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
On 20 May 2011, at 19:08, Scott Hawley wrote:
> Got it to build and run! See below for config "options"
> Yes, it is complicated on the Mac because Apple ships its own messed up version of MPI which you have to be careful not to use, and Apple uses the gnu v4.2.1 compiler but in order to get a fortran 90 compiler you need the straight-GNU compiler which for me is v4.4
I wrote a simfactory optionlist for using 64 bit fink on Mac OS with Cactus. See
https://svn.cct.lsu.edu/repos/numrel/simfactory/branches/PYSIM_2010/mdb/optionlists/macos-fink-gcc.cfg
This seems to only be in the python version of simfactory - we could put it in the perl version as well. There are full instructions at the top of the file for the commands needed to get it working, and you don't need simfactory to use the optionlist. Just replace the @..@ tokens with something sensible.
>
> This, then, is my "options.mac" file, which produces a complete build of my thorn and runs properly with MPI. Consider using it for a "known architectures" file. The /sw directory is where fink puts things. One needs to get gfortran and openmpi from fink. Some of the libraries I'm using will not be necessary for all people, but I'll let you take them out. This works for me! ...Oh, that is to say, with all the "CarpetIO..." thorns not activated. Haven't gotten those working yet.
>
> % cat options.mac
> CC = /sw/bin/mpicc
> CXX = /sw/bin/mpic++
> F77 = /sw/bin/mpif77
> F90 = gfortran
> CFLAGS = -m64 -std=c99
> CXXFLAGS = -m64
> FFLAGS = -m64
> F90FLAGS = -m64
> LIBS = texmex bbhutil sv mfhdf df jpeg z gfortran
> LDFLAGS = -L/sw/lib -L/sw/lib/gcc4.4/lib/ -L/Users/shawley/lib -L/Users/shawley/texmex/lib
> MPI = OPENMPI
> OPENMPI_DIR = /sw
> TEXMEX_DIR = /Users/shawley/texmex
>
>
>
>
>
> On May 20, 2011, at 11:43 AM, Erik Schnetter wrote:
>
>> Scott
>>
>> This usually means that the libraries used to build the executable and
>> the mpirun command don't belong together. In other words, either the
>> libraries used for building, or the shared libraries used at run time
>> (if you use shared libraries), or the mpirun executable are not what
>> you think they are.
>>
>> make texmex SILENT=no shows you (at the end) which libraries are used
>> during the build.
>>
>> ldd exe/cactus_texmex (or otool -L exe/cactus_texmex) shows which
>> shared libraries will be used. Certain environment variables influence
>> this; make sure you use the same settings that you use when you call
>> mpirun.
>>
>> "which mpirun" shows which mpirun will be used.
>>
>> -erik
>>
>> On Fri, May 20, 2011 at 12:36 PM, Scott Hawley <scott.hawley at belmont.edu> wrote:
>>> My code works totally ok with MPI on my linux machine, but as I'm porting to my Mac, I find that MPI isn't working right. I'm using the OpenMPI version from fink) A simple "hello world" MPI program works fine (proof at the bottom of this message), but Cactus does not.
>>>
>>> That is to say, e.g. an 8-processor run ("/sw/bin om-mpirun -np 8 ....") results in 8 processes, but each processes thinks it's processor 0, and that there are no other processors.
>>>
>>> Any ideas why this might happen?
>>> Thanks,
>>> Scott
>>>
>>>
>>> I'm using Cactus version 4.0.b17 (Compile date: May 20 2011 (11:13:49)).
>>>
>>> Cactus run:
>>>> mpirun -np 8 ~/Cactus/exe/cactus_texmex sep20a0.5fmrbbh97_cactus.par
>>> --------------------------------------------------------------------------------
>>>
>>> 10
>>> 1 0101 ************************
>>> 01 1010 10 The Cactus Code V4.0
>>> 1010 1101 011 www.cactuscode.org
>>> 1001 100101 ************************
>>> 00010101
>>> 100011 (c) Copyright The Authors
>>> 0100 GNU Licensed. No Warranty
>>> 0101
>>>
>>> --------------------------------------------------------------------------------
>>>
>>> Cactus version: 4.0.b17
>>> Compile date: May 20 2011 (11:13:49)
>>> --------------------------------------------------------------------------------
>>>
>>> 10
>>> 1 0101 ************************
>>> 01 1010 10 The Cactus Code V4.0
>>> 1010 1101 011 www.cactuscode.org
>>> 1001 100101 ************************
>>> 00010101
>>> 100011 (c) Copyright The Authors
>>> 0100 GNU Licensed. No Warranty
>>> 0101
>>>
>>> --------------------------------------------------------------------------------
>>>
>>> Cactus version: 4.0.b17
>>> Compile date: May 20 2011 (11:13:49)
>>> --------------------------------------------------------------------------------
>>>
>>> 10
>>> 1 0101 ************************
>>> 01 1010 10 The Cactus Code V4.0
>>> 1010 1101 011 www.cactuscode.org
>>> 1001 100101 ************************
>>> 00010101
>>> 100011 (c) Copyright The Authors
>>> 0100 GNU Licensed. No Warranty
>>> 0101
>>>
>>> --------------------------------------------------------------------------------
>>>
>>> Cactus version: 4.0.b17
>>> Compile date: May 20 2011 (11:13:49)
>>> --------------------------------------------------------------------------------
>>>
>>> 10
>>> 1 0101 ************************
>>> 01 1010 10 The Cactus Code V4.0
>>> 1010 1101 011 www.cactuscode.org
>>> 1001 100101 ************************
>>> 00010101
>>> 100011 (c) Copyright The Authors
>>> 0100 GNU Licensed. No Warranty
>>> 0101
>>>
>>> --------------------------------------------------------------------------------
>>>
>>> Cactus version: 4.0.b17
>>> Compile date: May 20 2011 (11:13:49)
>>> --------------------------------------------------------------------------------
>>>
>>> 10
>>> 1 0101 ************************
>>> 01 1010 10 The Cactus Code V4.0
>>> 1010 1101 011 www.cactuscode.org
>>> 1001 100101 ************************
>>> 00010101
>>> 100011 (c) Copyright The Authors
>>> 0100 GNU Licensed. No Warranty
>>> 0101
>>>
>>> --------------------------------------------------------------------------------
>>>
>>> Cactus version: 4.0.b17
>>> Compile date: May 20 2011 (11:13:49)
>>> --------------------------------------------------------------------------------
>>>
>>> 10
>>> 1 0101 ************************
>>> 01 1010 10 The Cactus Code V4.0
>>> 1010 1101 011 www.cactuscode.org
>>> 1001 100101 ************************
>>> 00010101
>>> 100011 (c) Copyright The Authors
>>> 0100 GNU Licensed. No Warranty
>>> 0101
>>>
>>> --------------------------------------------------------------------------------
>>>
>>> Cactus version: 4.0.b17
>>> Compile date: May 20 2011 (11:13:49)
>>> --------------------------------------------------------------------------------
>>>
>>> 10
>>> 1 0101 ************************
>>> 01 1010 10 The Cactus Code V4.0
>>> 1010 1101 011 www.cactuscode.org
>>> 1001 100101 ************************
>>> 00010101
>>> 100011 (c) Copyright The Authors
>>> 0100 GNU Licensed. No Warranty
>>> 0101
>>>
>>> --------------------------------------------------------------------------------
>>>
>>> Cactus version: 4.0.b17
>>> Compile date: May 20 2011 (11:13:49)
>>> --------------------------------------------------------------------------------
>>>
>>> 10
>>> 1 0101 ************************
>>> 01 1010 10 The Cactus Code V4.0
>>> 1010 1101 011 www.cactuscode.org
>>> 1001 100101 ************************
>>> 00010101
>>> 100011 (c) Copyright The Authors
>>> 0100 GNU Licensed. No Warranty
>>> 0101
>>>
>>> --------------------------------------------------------------------------------
>>>
>>> Cactus version: 4.0.b17
>>> Compile date: May 20 2011 (11:13:49)
>>> Run date: May 20 2011 (11:24:38-0500)
>>> Run host: provision.local (pid=6067)
>>> Working directory: /Users/shawley/runs/cactus
>>> Executable: /Users/shawley/Cactus/exe/cactus_texmex
>>> Parameter file: sep20a0.5fmrbbh97_cactus.par
>>> --------------------------------------------------------------------------------
>>> Run date: May 20 2011 (11:24:38-0500)
>>> Run host: provision.local (pid=6071)
>>> Working directory: /Users/shawley/runs/cactus
>>> Executable: /Users/shawley/Cactus/exe/cactus_texmex
>>> Parameter file: sep20a0.5fmrbbh97_cactus.par
>>> --------------------------------------------------------------------------------
>>>
>>>
>>> Activating thorn Cactus...Success -> active implementation Cactus
>>> Run date: May 20 2011 (11:24:38-0500)
>>> Run host: provision.local (pid=6070)
>>> Activating thorn Cactus...Success -> active implementation Cactus
>>> Run date: May 20 2011 (11:24:38-0500)
>>> Run host: provision.local (pid=6074)
>>> Working directory: /Users/shawley/runs/cactus
>>> Executable: /Users/shawley/Cactus/exe/cactus_texmex
>>> Parameter file: sep20a0.5fmrbbh97_cactus.par
>>> --------------------------------------------------------------------------------
>>>
>>> Activating thorn Cactus...Success -> active implementation Cactus
>>> Run date: May 20 2011 (11:24:38-0500)
>>> Run host: provision.local (pid=6073)
>>> Working directory: /Users/shawley/runs/cactus
>>> Executable: /Users/shawley/Cactus/exe/cactus_texmex
>>> Parameter file: sep20a0.5fmrbbh97_cactus.par
>>> --------------------------------------------------------------------------------
>>> Working directory: /Users/shawley/runs/cactus
>>> Executable: /Users/shawley/Cactus/exe/cactus_texmex
>>> Parameter file: sep20a0.5fmrbbh97_cactus.par
>>> --------------------------------------------------------------------------------
>>>
>>> Activating thorn Cactus...Success -> active implementation Cactus
>>>
>>> Activating thorn Cactus...Success -> active implementation Cactus
>>> Run date: May 20 2011 (11:24:38-0500)
>>> Run host: provision.local (pid=6068)
>>> Run date: May 20 2011 (11:24:38-0500)
>>> Run host: provision.local (pid=6069)
>>> Run date: May 20 2011 (11:24:38-0500)
>>> Run host: provision.local (pid=6072)
>>> Working directory: /Users/shawley/runs/cactus
>>> Executable: /Users/shawley/Cactus/exe/cactus_texmex
>>> Parameter file: sep20a0.5fmrbbh97_cactus.par
>>>
>>> ....and so on.....
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>> Yet a imple program works:
>>> The simple example which works fine is...
>>>> cat hello.c
>>> #include <stdio.h>
>>> #include "mpi.h"
>>>
>>> int main(argc, argv)
>>> int argc;
>>> char **argv;
>>> {
>>> int rank, size;
>>> MPI_Init(&argc,&argv);
>>> MPI_Comm_rank(MPI_COMM_WORLD, &rank);
>>> MPI_Comm_size(MPI_COMM_WORLD, &size);
>>> printf("Hello world! I am %d of %d\n",rank,size);
>>> MPI_Finalize();
>>> return 0;
>>> }
>>>> mpicc -o hello hello.c
>>>> which mpirun
>>> /sw/bin/mpirun
>>>> ls -l /sw/bin/mpirun
>>> lrwxr-xr-x 1 root shawley 9 May 20 10:55 /sw/bin/mpirun@ -> om-mpirun
>>>> mpirun -np 8 ./hello
>>> Hello world! I am 0 of 8
>>> Hello world! I am 6 of 8
>>> Hello world! I am 3 of 8
>>> Hello world! I am 1 of 8
>>> Hello world! I am 2 of 8
>>> Hello world! I am 4 of 8
>>> Hello world! I am 5 of 8
>>> Hello world! I am 7 of 8
>>>
>>>
>>> _______________________________________________
>>> Users mailing list
>>> Users at einsteintoolkit.org
>>> http://lists.einsteintoolkit.org/mailman/listinfo/users
>>>
>>>
>>
>>
>>
>> --
>> Erik Schnetter <schnetter at cct.lsu.edu> http://www.cct.lsu.edu/~eschnett/
>>
>
> _______________________________________________
> Users mailing list
> Users at einsteintoolkit.org
> http://lists.einsteintoolkit.org/mailman/listinfo/users
- --
Ian Hinder
ian.hinder at aei.mpg.de
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.8 (Darwin)
iEYEARECAAYFAk3XiwYACgkQF1LN8Zj+Ceh5qwCfSuHFBt55qB7WGsLC+JCE6sdU
PIAAn2+G3xAfON9rzfv7UTlcCu7IZ+0j
=z2SW
-----END PGP SIGNATURE-----
More information about the Users
mailing list