#1364: be mor careful extracting $MAKE in RunTestSuite -------------------------+-------------------------------------------------- Reporter: rhaas | Owner: eschnett Type: defect | Status: reopened Priority: critical | Milestone: ET_2013_05 Component: SimFactory | Version: Resolution: | Keywords: -------------------------+--------------------------------------------------
Comment (by hinder):
I can see r2101 on the release branch, and r2100 on the trunk, both of which have the subject line "be more careful extraction $MAKE in RunTestSuite", and both of which seem to be the same patch to RunTestSuite. These were both committed (by Roland) on 20th May, but several hours apart. So the fix seems to have been applied both to the trunk and the release branch.
I just tried to run with the version from the release branch again, and get the same error.
RunTestSuite contains
{{{ MAKE=$(@SOURCEDIR@/simfactory/bin/sim print-mdb-entry @MACHINE@ | grep '^make' | sed -e 's/^make *= *//;s/ *#.*$//') }}}
Adding set -x to the script gives:
{{{ + command=/lustre/datura/ianhin/simulations/ettests_1proc/output-0000/SIMFACTORY/RunScript + export 'CCTK_TESTSUITE_RUN_COMMAND=ln -fns . output-0000-active && mkdir -p SIMFACTORY && TESTSUITE_PARFILE=$parfile /lustre/datura/ianhin/simulations/ettests_1proc/output-0000/SIMFACTORY/RunScript' + CCTK_TESTSUITE_RUN_COMMAND='ln -fns . output-0000-active && mkdir -p SIMFACTORY && TESTSUITE_PARFILE=$parfile /lustre/datura/ianhin/simulations/ettests_1proc/output-0000/SIMFACTORY/RunScript' + export CCTK_TESTSUITE_RUN_PROCESSORS=1 + CCTK_TESTSUITE_RUN_PROCESSORS=1 + ulimit -c 0 ++ /home/ianhin/Cactus/EinsteinToolkit/simfactory/bin/sim print-mdb-entry datura ++ grep '^make' ++ sed -e 's/^make *= *//;s/ *#.*$//' + MAKE='INTEL_LICENSE_FILE=/cluster/intel/licenses make -j32' + INTEL_LICENSE_FILE=/cluster/intel/licenses make -j32 ET_2013_06-testsuite PROMPT=no /lustre/datura/ianhin/simulations/ettests_1proc/output-0000/RunTestSuite: line 23: INTEL_LICENSE_FILE=/cluster/intel/licenses: No such file or directory }}}
It seems to be getting the correct value for the MAKE variable, but some interplay between substitution and word-splitting is causing an error. If I do
{{{ [ianhin@login-damiana EinsteinToolkit]$ cmd='ls -l' [ianhin@login-damiana EinsteinToolkit]$ $cmd -a total 156 drwxr-xr-x+ 15 ianhin users 4096 Nov 7 2012 . drwxr-xr-x+ 14 ianhin users 4096 May 9 11:14 .. drwxr-xr-x+ 29 ianhin users 4096 Apr 25 17:20 arrangements }}} then cmd is inserted into the '$cmd -a' string and ls and -l are treated as separate words, just like -a, and the shell then correctly executes the command. However, if I do {{{ [ianhin@login-damiana EinsteinToolkit]$ cmd='var=val ls -l' [ianhin@login-damiana EinsteinToolkit]$ $cmd -a -bash: var=val: command not found }}} this does not happen. I thought that unquoted parameter expansions like $cmd here would be expanded before word-splitting, so the result would be split. But it looks like environment variable assignments are performed before even parameter substitution, meaning that you cannot include these variable assignments in expanded variables. See http://www.gnu.org/software/bash/manual/bashref.html#Executing-Commands:
When a simple command is executed, the shell performs the following
expansions, assignments, and redirections, from left to right.
The words that the parser has marked as variable assignments (those
preceding the command name) and redirections are saved for later processing.
The words that are not variable assignments or redirections are expanded
(see Shell Expansions). If any words remain after expansion, the first word is taken to be the name of the command and the remaining words are the arguments.
Redirections are performed as described above (see Redirections). The text after the ‘=’ in each variable assignment undergoes tilde
expansion, parameter expansion, command substitution, arithmetic expansion, and quote removal before being assigned to the variable.
This suggests that we need to first expand the parameter, then use eval to execute the command as a whole:
{{{ cmd="${MAKE} @CONFIGURATION@-testsuite PROMPT=no" eval $cmd }}}
This works for me on Datura. Is this a good solution?