[Einstein Toolkit] #1928: support make parallel options in simfactory

List overview All Threads
Download

newer

older

[Einstein Toolkit] #1973:...

[Einstein Toolkit] #1975: Cactus:...

Einstein Toolkit

25 Jun 2016 25 Jun '16

11:34 a.m.

#1928: support make parallel options in simfactory -------------------------+-------------------------------------------------- Reporter: rhaas | Owner: Type: enhancement | Status: new Priority: minor | Milestone: Component: SimFactory | Version: development version Keywords: | -------------------------+-------------------------------------------------- Pull request https://bitbucket.org/simfactory/simfactory2/pull-requests/11 /parallel-make/diff adds a --jobs option to sim build that sets a @MAKEJOBS@ replacement so that one can use make -j @MAKEJOBS@ in machine definitons files and let the user choose the number of make jobs when compiling

-- Ticket URL: https://trac.einsteintoolkit.org/ticket/1928 Einstein Toolkit http://einsteintoolkit.org The Einstein Toolkit

Show replies by date

Einstein Toolkit

25 Jun 25 Jun

12:46 p.m.

#1928: support make parallel options in simfactory --------------------------+------------------------------------------------- Reporter: rhaas | Owner: Type: enhancement | Status: review Priority: minor | Milestone: Component: SimFactory | Version: development version Resolution: | Keywords: --------------------------+------------------------------------------------- Changes (by rhaas):

* status: new => review

-- Ticket URL: https://trac.einsteintoolkit.org/ticket/1928#comment:1 Einstein Toolkit http://einsteintoolkit.org The Einstein Toolkit

Einstein Toolkit

2:45 p.m.

Comment (by eschnett):

Currently, Simfactory's machine database contains the preferred number of make jobs. Do you want to override this manually on a case-by-case basis? Is there a case where the default settings are inconvenient?

-- Ticket URL: https://trac.einsteintoolkit.org/ticket/1928#comment:2 Einstein Toolkit http://einsteintoolkit.org The Einstein Toolkit

Einstein Toolkit

2:53 p.m.

Comment (by knarf):

I could imagine that, from time to time, another than the default number of cores might be of interest. Say, if a build isn't time critical, and shouldn't take a large portion of a machine's cores. I didn't look at the patch yet, but I think the request is understandable. I would go as far as to see if a short "-j" option would be possible.

-- Ticket URL: https://trac.einsteintoolkit.org/ticket/1928#comment:3 Einstein Toolkit http://einsteintoolkit.org The Einstein Toolkit

Einstein Toolkit

2:55 p.m.

Comment (by rhaas):

Right now nothing changes since none of the ini files use @MAKEJOBS@. One can add makejobs to the ini files and use this as the default which can be overridden by the user. Eg.

{{{ [zwicky] makejobs = 6 make = make -j @MAKEJOBS@ }}}

which can then be overridden by the user on the command line (not tested yet).

The defaults are often inconvenient on the OSX and generic Linux builds (since one cannot now in advance whether the build is a raspberry pi or a 16 core corei7 workstation), see https://bitbucket.org/simfactory/simfactory2/commits/e6784d678365/ . I occasionally use a different number of make jobs on clusters but mostly for debugging or compile time error fixes when I want "-j1" so that make stops right after the error and error messages are not mixed with regular output from other make jobs.

-- Ticket URL: https://trac.einsteintoolkit.org/ticket/1928#comment:4 Einstein Toolkit http://einsteintoolkit.org The Einstein Toolkit

Einstein Toolkit

2:55 p.m.

Comment (by rhaas):

Replying to [comment:3 knarf]:

...

I could imagine that, from time to time, another than the default number

of cores might be of interest. Say, if a build isn't time critical, and shouldn't take a large portion of a machine's cores. I didn't look at the patch yet, but I think the request is understandable. I would go as far as to see if a short "-j" option would be possible.

-j is already implemented.

-- Ticket URL: https://trac.einsteintoolkit.org/ticket/1928#comment:5 Einstein Toolkit http://einsteintoolkit.org The Einstein Toolkit

Einstein Toolkit

28 Jun 28 Jun

9:52 a.m.

Comment (by knarf):

If I read the code correctly, '-j' (without argument) would imply '-j 1'. Is this correct? If so, is this intended? Usually it means '-j infinity' (for make).

-- Ticket URL: https://trac.einsteintoolkit.org/ticket/1928#comment:6 Einstein Toolkit http://einsteintoolkit.org The Einstein Toolkit

Einstein Toolkit

1:51 p.m.

Comment (by rhaas):

{{{-j}}} without an argument should abort with an error stating that {{{-j}}} requires an argument. The default of "1" is when {{{-j}}} is not specified at all (or at least that it the intention)

-- Ticket URL: https://trac.einsteintoolkit.org/ticket/1928#comment:7 Einstein Toolkit http://einsteintoolkit.org The Einstein Toolkit

Einstein Toolkit

2:11 p.m.

Comment (by eschnett):

If I don't specify an option, then Simfactory should use a system-specific default, not 1.

-- Ticket URL: https://trac.einsteintoolkit.org/ticket/1928#comment:8 Einstein Toolkit http://einsteintoolkit.org The Einstein Toolkit

Einstein Toolkit

2:22 p.m.

Comment (by rhaas):

Sure. The system specific default must be set in the machine.ini file for the machine by specifying a value of makejobs (same as the system specific default number of threads when running the simulation). If none is specified (and make is actually "make -j @MAKEJOBS@" rather than "make -j 12" or so) then @MAKEJOBS@ defaults to 1 unless either overridden by a setting in the machine.ini file or by a -j or --jobs option on the command line.

Obviously if you just hard-code "make" as "make -j 12" in the machine.ini file then no value for {{{--jobs}}} will do anything.

-- Ticket URL: https://trac.einsteintoolkit.org/ticket/1928#comment:9 Einstein Toolkit http://einsteintoolkit.org The Einstein Toolkit

Einstein Toolkit

2:39 p.m.

#1928: support make parallel options in simfactory --------------------------+------------------------------------------------- Reporter: rhaas | Owner: Type: enhancement | Status: reopened Priority: minor | Milestone: Component: SimFactory | Version: development version Resolution: | Keywords: --------------------------+------------------------------------------------- Changes (by rhaas):

* status: review => reopened

Comment:

Replying to [comment:9 rhaas]:

...

Sure. The system specific default must be set in the machine.ini file

for the machine by specifying a value of makejobs (same as the system specific default number of threads when running the simulation). If none is specified (and make is actually "make -j @MAKEJOBS@" rather than "make -j 12" or so) then @MAKEJOBS@ defaults to 1 unless either overridden by a setting in the machine.ini file or by a -j or --jobs option on the command line.

...

Obviously if you just hard-code "make" as "make -j 12" in the

machine.ini file then no value for {{{--jobs}}} will do anything.

However it currently does not do this yet. Needs to be fixed.

-- Ticket URL: https://trac.einsteintoolkit.org/ticket/1928#comment:10 Einstein Toolkit http://einsteintoolkit.org The Einstein Toolkit

Einstein Toolkit

2:43 p.m.

#1928: support make parallel options in simfactory --------------------------+------------------------------------------------- Reporter: rhaas | Owner: Type: enhancement | Status: reopened Priority: minor | Milestone: Component: SimFactory | Version: development version Resolution: | Keywords: --------------------------+-------------------------------------------------

Comment (by knarf):

Replying to [comment:7 rhaas]:

...

{{{-j}}} without an argument should abort with an error stating that

{{{-j}}} requires an argument.

That should be fine. Even 'make -j' is hardly ever useful.

-- Ticket URL: https://trac.einsteintoolkit.org/ticket/1928#comment:11 Einstein Toolkit http://einsteintoolkit.org The Einstein Toolkit

Einstein Toolkit

3:08 p.m.

* status: reopened => review

Comment:

Now it does handle makejobs in ini files and --jobs on the command line properly I believe. This is in git hash 88f3ddb which is the new head of the pull request.

-- Ticket URL: https://trac.einsteintoolkit.org/ticket/1928#comment:12 Einstein Toolkit http://einsteintoolkit.org The Einstein Toolkit

Einstein Toolkit

4 Nov 4 Nov

9:46 a.m.

Comment (by rhaas):

This should still be reviewed. Note that I implemented the requested changes.

-- Ticket URL: https://trac.einsteintoolkit.org/ticket/1928#comment:13 Einstein Toolkit http://einsteintoolkit.org The Einstein Toolkit

Einstein Toolkit

28 Nov 28 Nov

2:18 p.m.

#1928: support make parallel options in simfactory --------------------------+------------------------------------------------- Reporter: rhaas | Owner: Type: enhancement | Status: reviewed_ok Priority: minor | Milestone: Component: SimFactory | Version: development version Resolution: | Keywords: --------------------------+------------------------------------------------- Changes (by eschnett):

* status: review => reviewed_ok

-- Ticket URL: https://trac.einsteintoolkit.org/ticket/1928#comment:14 Einstein Toolkit http://einsteintoolkit.org The Einstein Toolkit

Einstein Toolkit

1 Jan 1 Jan

5:55 p.m.

#1928: support make parallel options in simfactory --------------------------+------------------------------------------------- Reporter: rhaas | Owner: Type: enhancement | Status: closed Priority: minor | Milestone: Component: SimFactory | Version: development version Resolution: fixed | Keywords: --------------------------+------------------------------------------------- Changes (by rhaas):

* status: reviewed_ok => closed * resolution: => fixed

Comment:

Applied as git hash 85946a3 of simfactory.

-- Ticket URL: https://trac.einsteintoolkit.org/ticket/1928#comment:15 Einstein Toolkit http://einsteintoolkit.org The Einstein Toolkit

3568

Age (days ago)

3758

Last active (days ago)

trac@lists.einsteintoolkit.org

15 comments

1 participants

tags (0)

participants (1)

Einstein Toolkit