[Users] update new user guide for non-TeraGrid/XSEDE users?

[Friesen, Brian]

Thanks for the help so far.

I imagine (wrongly perhaps?) that a significant portion of ET users run on big machines at, e.g., LONI, XSEDE, NERSC, HLRN, etc. I would be happy to add a few more .cfg files to simfactory's array of optionlists for the NERSC machines. There already exist a few for them, but they're sloppily written and makes explicit calls to the compilers instead of using Cray's wrappers which automatically link to  MPI libraries, math libraries, etc. Also, some compilers (Cray's, for example) enable OpenMP by default and the user must disable it explicitly, while others (pretty much all others actually) do the opposite. Drawing attention to this in the option lists would be very helpful I think.

--
Brian C. Friesen
Graduate Research Assistant
University of Oklahoma
friesen at ou.edu
http://www.nhn.ou.edu/~friesen

Am 24.08.2011 um 08:47 schrieb Ian Hinder:

> 
> On 24 Aug 2011, at 15:32, Erik Schnetter wrote:
> 
>> On Wed, Aug 24, 2011 at 3:36 AM, Ian Hinder <ian.hinder at aei.mpg.de> wrote:
>>> 
>>> On 23 Aug 2011, at 21:29, Erik Schnetter wrote:
>>> 
>>>> Brian
>>>> 
>>>> Cactus uses certain software packages (C compiler, Fortran compiler,
>>>> MPI, queuing system) which differ on basically every HPC system.
>>>> Instructions for building Cactus are therefore either targeted to a
>>>> specific system, or have to say "please read the system's
>>>> documentation to figure out these things". There is nothing in
>>>> particular that makes our instructions target TeraGrid systems; in
>>>> fact, we are targeting LONI systems, which are not part of XSEDE.
>>>> 
>>>> If you follow the instructions at
>>>> <https://docs.einsteintoolkit.org/et-docs/Tutorial_for_New_Users>,
>>>> then these should work almost as is on Hopper. The are two
>>>> differences:
>>>> 1. You have to leave out the "soft add +git" since git is available by default.
>>>> 2. Since Hopper has 24 cores per node, you need to use a multiple of
>>>> 24 cores when submitting a job. Use "--procs=48" instead of
>>>> "--procs=32". You can also add "--num-threads=6" if you want to use
>>>> OpenMP as well (which should work out of the box).
>>> 
>>> Should we have a section in the tutorial with information like this for each of the machines we know how to use?
>> 
>> Should we change SimFactory to do this automatically (as default) when
>> OpenMP is enabled?
> 
> 
> I wasn't referring to this specifically.  I meant "Should we have a section in the tutorial which explains what is needed for each machine to run the ET?".
> 
> Regarding OpenMP, I would like to have at least an option to always use the number of threads per process specified by num-threads in the machine's ini file.  At the moment, that variable is not used as far as I know.  What do you mean by "openmp is enabled"?  SimFactory always compiles with OpenMP, doesn't it?  The reason not to use more than 1 thread by default is that codes which don't support openmp will run several times more slowly.
> 
> Would it be useful for each thorn to declare in its configuration.ccl file whether it supports OpenMP or not?  Then at least Cactus could abort if you try to run a thorn which does not support OpenMP with more than one thread per process.
> 
> -- 
> Ian Hinder
> ian.hinder at aei.mpg.de
>