[Users] config file for lonestar

[Erik Schnetter]

Yosef

Yes, there is a "standard" config file, but I have not tested it on
large numbers of cores. I attach it below.

If there is a problem on large numbers of cores, then it is typically
a problem with MPI, either an inconsistency with the MPI libraries
(compiling, linking, and running have to use the same version), or a
problem with the startup mechanism that MPI uses (there may be a
timeout). In the latter case, I hope the system administrators will be
able to help.

-erik

On Fri, Mar 18, 2011 at 11:35 AM, Yosef Zlochower <yosef at astro.rit.edu> wrote:
> Hi,
>
>  I am trying some simulations on lonestar using the config file
> below. I noticed that my simulations hang when I choose large
> numbers of cores. I haven't yet determined where
> they hang (the same executable and parfile combination does
> not hang on fewer cores). Is there a standard config file
> for lonestar?
>
> CPP = cpp
> FPP = cpp
> CC  = /opt/apps/intel/11.1/bin/intel64/icc
> CXX = /opt/apps/intel/11.1/bin/intel64/icpc
> F77 = /opt/apps/intel/11.1/bin/intel64/ifort
> F90 = /opt/apps/intel/11.1/bin/intel64/ifort
>
> # -inline-debug-info leads to compiler crashes when used with optimisation
> CPPFLAGS = -DMPICH_IGNORE_CXX_SEEK
> FPPFLAGS = -traditional
> #CFLAGS   = -g -debug all -traceback -align -std=gnu99 -ansi_alias
> CFLAGS   = -g -debug all -traceback -align -std=c99 -ansi_alias
> CXXFLAGS = -g -debug all -traceback -align -restrict
> F77FLAGS = -g -debug all -traceback -align -pad -w95 -cm
> F90FLAGS = -g -debug all -traceback -align -pad -w95 -cm
>
> C_LINE_DIRECTIVES = yes
> F_LINE_DIRECTIVES = yes
>
> LDFLAGS =  -Wl,-rpath,/opt/apps/intel11_1/mvapich2/1.6/lib
> -Wl,-rpath,/opt/apps/
> intel/11.1/lib/intel64/lib
>
> # Include hypre, SPAI, Chaco, spooles, SuperLU_DIST, MUMPS, ParMetis,
> SCALAPACK,
>  blacs, g2c, and X11 for PETSc
> LIBDIRS = /opt/apps/intel/11.1/lib/intel64/lib
>
> # -check-uninit aborts for uninitialised variables, which is too strict
> DEBUG           = no
> CPP_DEBUG_FLAGS = -DCARPET_DEBUG
> FPP_DEBUG_FLAGS = -DCARPET_DEBUG
> C_DEBUG_FLAGS   = -O0 # -check-uninit
> CXX_DEBUG_FLAGS = -O0 # -check-uninit
> F77_DEBUG_FLAGS = -O0 -check bounds -check format # -check all
> F90_DEBUG_FLAGS = -O0 -check bounds -check format # -check all
>
> # -O3 miscompiles parts of CarpetLib
> # -ip leads to wrong physics when OpenMP is used
> OPTIMISE           = yes
> CPP_OPTIMISE_FLAGS = # -DCARPET_OPTIMISE -DNDEBUG
> FPP_OPTIMISE_FLAGS = # -DCARPET_OPTIMISE -DNDEBUG
> C_OPTIMISE_FLAGS   = -O2 -xW
> CXX_OPTIMISE_FLAGS = -O2 -xW
> F77_OPTIMISE_FLAGS = -O2 -xW
> F90_OPTIMISE_FLAGS = -O2 -xW
>
> PROFILE           = no
> CPP_PROFILE_FLAGS =
> FPP_PROFILE_FLAGS =
> C_PROFILE_FLAGS   = -pg
> CXX_PROFILE_FLAGS = -pg
> F77_PROFILE_FLAGS = -pg
> F90_PROFILE_FLAGS = -pg
>
> OPENMP           = yes
> CPP_OPENMP_FLAGS = -openmp
> FPP_OPENMP_FLAGS = -D_OPENMP
> C_OPENMP_FLAGS   = -openmp
> CXX_OPENMP_FLAGS = -openmp
> F77_OPENMP_FLAGS = -openmp
> F90_OPENMP_FLAGS = -openmp
>
> WARN           = yes
> CPP_WARN_FLAGS =
> FPP_WARN_FLAGS =
> C_WARN_FLAGS   =
> CXX_WARN_FLAGS =
> F77_WARN_FLAGS =
> F90_WARN_FLAGS =
>
> BLAS_DIR  =
>
> HDF5     = yes
> HDF5_DIR = BUILD
>
> #Previously:
> #GSL_DIR = /opt/apps/gsl/1.11
> GSL_DIR = BUILD
>
> LAPACK_DIR  =
> MPI          = CUSTOM
> MPI_INC_DIRS = /opt/apps/intel11_1/mvapich2/1.6/include/
> MPI_LIB_DIRS = /opt/apps/intel11_1/mvapich2/1.6/lib/ /opt/ofed/lib64/
> MPI_LIBS     = mpich rt rdmacm ibverbs ibumad
>
> PTHREADS = yes
>
>
>
> --
> Dr. Yosef Zlochower
> Center for Computational Relativity and Gravitation
> Assistant Professor
> School of Mathematical Sciences
> Rochester Institute of Technology
> 85 Lomb Memorial Drive
> Rochester, NY 14623
>
> Office:74-2067
> Phone: +1 585-475-6103
>
> yosef at astro.rit.edu
>
> CONFIDENTIALITY NOTE: The information transmitted, including
> attachments, is intended only for the person(s) or entity to which it
> is addressed and may contain confidential and/or privileged material.
> Any review, retransmission, dissemination or other use of, or taking
> of any action in reliance upon this information by persons or entities
> other than the intended recipient is prohibited. If you received this
> in error, please contact the sender and destroy any copies of this
> information.
> _______________________________________________
> Users mailing list
> Users at einsteintoolkit.org
> http://lists.einsteintoolkit.org/mailman/listinfo/users
>



-- 
Erik Schnetter <schnetter at cct.lsu.edu>   http://www.cct.lsu.edu/~eschnett/
-------------- next part --------------
A non-text attachment was scrubbed...
Name: lonestar.cfg
Type: application/octet-stream
Size: 4525 bytes
Desc: not available
Url : http://lists.einsteintoolkit.org/pipermail/users/attachments/20110318/025da9f4/attachment.obj