#382: SimFactory home directory on Kraken is too specific ------------------------+--------------------------------------------------- Reporter: hinder | Owner: mthomas Type: defect | Status: new Priority: major | Milestone: ET_2011_05 Component: SimFactory | Version: Keywords: | ------------------------+--------------------------------------------------- The mdb entry for Kraken in SimFactory has
'sourcebasedir' => '/nics/b/home/@USER@',
My home directory is
'/nics/d/home/@USER@'
Either we could leave the source base dir as unset to force the user to set it, or we could automatically detect the location of the user's home directory (better).
#382: SimFactory home directory on Kraken is too specific -------------------------+-------------------------------------------------- Reporter: hinder | Owner: mthomas Type: defect | Status: new Priority: major | Milestone: ET_2011_11 Component: SimFactory | Version: Resolution: | Keywords: -------------------------+-------------------------------------------------- Changes (by knarf):
* milestone: ET_2011_05 => ET_2011_11
#382: SimFactory home directory on Kraken is too specific -------------------------+-------------------------------------------------- Reporter: hinder | Owner: mthomas Type: defect | Status: review Priority: major | Milestone: ET_2011_11 Component: SimFactory | Version: Resolution: | Keywords: -------------------------+-------------------------------------------------- Changes (by knarf):
* status: new => review
Comment:
First of all, sourcebasedir of kraken is not set to the home-dir in simfactory anymore, at least not in the python version.
Secondly, you should be able to set it to $HOME, assuming that $HOME is defined on the remote machine, which it typically is. We could do the same for SCRATCH and WORK, and possibly PROJECT which some machine environments by default provide.
Should we leave the ticket open to change machines which could use this? Can someone else than me try this mechanism? (works for me)
#382: SimFactory home directory on Kraken is too specific -------------------------+-------------------------------------------------- Reporter: hinder | Owner: mthomas Type: defect | Status: review Priority: major | Milestone: ET_2011_11 Component: SimFactory | Version: Resolution: | Keywords: -------------------------+--------------------------------------------------
Comment (by eschnett):
I don't think that setting it to $HOME will work. Simfactory will look at this directory in Python code, and there is nothing that expands $HOME to the corresponding environment variable. There is a syntax for accessing environment variables maybe @ENV[HOME]@; you could try this. We then also need to check that Simfactory expands this wherever it is used.
The current Kraken setting is "/nics/a/proj/cactus/@USER@/xt5", which is probably not a good default, as it is specific to the LSU relativity group and our collaborators. If someone confirms that using the home directory works for checking out and building the default Einstein Toolkit thornlist, then let's change this.
#382: SimFactory home directory on Kraken is too specific -------------------------+-------------------------------------------------- Reporter: hinder | Owner: mthomas Type: defect | Status: review Priority: major | Milestone: ET_2011_11 Component: SimFactory | Version: Resolution: | Keywords: -------------------------+--------------------------------------------------
Comment (by hinder):
As far as I know, the syntax for expanding environment variables is not implemented. You can see in the code where it was going to go, but it was never actually added.
#382: SimFactory home directory on Kraken is too specific -------------------------+-------------------------------------------------- Reporter: hinder | Owner: mthomas Type: defect | Status: review Priority: major | Milestone: ET_2011_11 Component: SimFactory | Version: Resolution: | Keywords: -------------------------+--------------------------------------------------
Comment (by knarf):
$HOME at least works for syncs, where this is directly given to the command line (with the backslash removed), so it's expanded on the shell of the remote machine. Now of course, if that directory variable should also be used inside python that would be a problem.
#382: SimFactory home directory on Kraken is too specific -------------------------+-------------------------------------------------- Reporter: hinder | Owner: mthomas Type: defect | Status: review Priority: major | Milestone: ET_2011_11 Component: SimFactory | Version: Resolution: | Keywords: -------------------------+--------------------------------------------------
Comment (by rhaas):
the same issue occurs with the $WORK directories on ranger and lonestar. Mine is /work/00945/@USER@ while simfactory uses /work/00507/@USER@. Using $HOME/work or $WORK instead would solve this problem (TACC creates this symbolic link and the variable when they create the account, so everybody should have it).
#382: SimFactory home directory on Kraken is too specific -------------------------+-------------------------------------------------- Reporter: hinder | Owner: mthomas Type: defect | Status: review Priority: major | Milestone: ET_2011_11 Component: SimFactory | Version: Resolution: | Keywords: -------------------------+--------------------------------------------------
Comment (by eschnett):
The Simfactory syntax for accessing environment variables is @ENV(HOME)@. Roland, can you try whether this works instead of $HOME?
#382: SimFactory home directory on Kraken is too specific -------------------------+-------------------------------------------------- Reporter: hinder | Owner: mthomas Type: defect | Status: review Priority: major | Milestone: ET_2011_11 Component: SimFactory | Version: Resolution: | Keywords: -------------------------+--------------------------------------------------
Comment (by rhaas):
using @ENV(HOME)@ does not seem to work (see comment 4 I guess). I get:
{{{ [rhaas@horizon Cactus]$ PYSIM_2010/bin/sim sync lonestar /bin/bash: -c: line 0: syntax error near unexpected token `(' /bin/bash: -c: line 0: `{ :; } && { cd '/home/rhaas/Cactus' && { :; } && ssh -Y rhaas@lonestar.tacc.utexas.edu mkdir -p @ENV(HOME)@/work/Cactus && rsync --rsh='ssh -Y' --rsync- path='/home1/01385/barrywar/software/rsync/bin/rsync' --archive --hard- links --sparse --verbose --progress --partial --delete --stats --compress --filter 'merge /home/rhaas/Cactus/PYSIM_2010/etc/filter.rules' par CONTRIBUTORS COPYRIGHT Makefile arrangements lib manifest repos src utils rhaas@lonestar.tacc.utexas.edu:@ENV(HOME)@/work/Cactus; }' /bin/bash: -c: line 0: syntax error near unexpected token `(' /bin/bash: -c: line 0: `{ :; } && { cd '/home/rhaas/Cactus' && { :; } && ssh -Y rhaas@lonestar.tacc.utexas.edu mkdir -p @ENV(HOME)@/work/Cactus && rsync --rsh='ssh -Y' --rsync- path='/home1/01385/barrywar/software/rsync/bin/rsync' --archive --hard- links --sparse --verbose --progress --partial --delete --stats --compress --filter 'merge /home/rhaas/Cactus/PYSIM_2010/etc/filter.prersync3.rules' par CONTRIBUTORS COPYRIGHT Makefile arrangements lib manifest repos src utils rhaas@lonestar.tacc.utexas.edu:@ENV(HOME)@/work/Cactus; }'
Error while syncing to lonestar.
Errors occurred during synchronisation. }}}
$HOME/work on the other hand works fine (at least for sync).
#382: SimFactory home directory on Kraken is too specific -------------------------+-------------------------------------------------- Reporter: hinder | Owner: mthomas Type: defect | Status: review Priority: major | Milestone: ET_2011_10 Component: SimFactory | Version: Resolution: | Keywords: -------------------------+--------------------------------------------------
Comment (by knarf):
Please review the attached patch. It adds at least support for the @ENV()@ syntax for the sync command. I didn't test anything else, but according to Erik it's the right syntax to use. It works now for ranger to set sourcebasedir=@ENV(WORK)@
#382: SimFactory home directory on Kraken is too specific -------------------------+-------------------------------------------------- Reporter: hinder | Owner: mthomas Type: defect | Status: review Priority: major | Milestone: ET_2011_10 Component: SimFactory | Version: Resolution: | Keywords: -------------------------+--------------------------------------------------
Comment (by eschnett):
How does this compare to ParseEnvCommands? Should this routine now be deleted? (I just grepped for ENV in the source code.)
This is only a partial implementation, since it only works for the sync destination. The general case still needs to be implemented; and this will need to happen in such a way that the special case for sync still works.
Do we actually like the ENV() syntax? I though this was the syntax already implemented; since this is something new, e.g. ENV[] may also work, or ENVIRON[], or requiring quoting the argument...
#382: SimFactory home directory on Kraken is too specific -------------------------+-------------------------------------------------- Reporter: hinder | Owner: mthomas Type: defect | Status: review Priority: major | Milestone: ET_2011_10 Component: SimFactory | Version: Resolution: | Keywords: -------------------------+--------------------------------------------------
Comment (by knarf):
This is reported in ticket:595. In case it will be fully implemented, it should then be used within the sync code.
I don't care much about the ENV syntax. The only possible source of confusion might be that Cactus parameter files use $ENV{'NAME'} as syntax. We could as well adapt simfactory to that. But then, simfactory already uses the @@ syntax in other places.
My only concern is that we should have something working for the release. But that would require not only sync to work, because directories like SCRATCH should be used for job-creation as well. I did not test ENV(NAME) there yet, but given that ParseEnvCommands doesn't seem to replace everything with '' I have doubts that it works.
#382: SimFactory home directory on Kraken is too specific -------------------------+-------------------------------------------------- Reporter: hinder | Owner: mthomas Type: defect | Status: review Priority: major | Milestone: ET_2011_10 Component: SimFactory | Version: Resolution: | Keywords: -------------------------+--------------------------------------------------
Comment (by eschnett):
I suggest to use the simplest possible fix at the moment, which is to use $HOME in the mdb entries. Anything else can be implemented after the release, when we have more freedom to experiment with syntax and functionality.
#382: SimFactory home directory on Kraken is too specific -------------------------+-------------------------------------------------- Reporter: hinder | Owner: mthomas Type: defect | Status: review Priority: major | Milestone: ET_2011_10 Component: SimFactory | Version: Resolution: | Keywords: -------------------------+--------------------------------------------------
Comment (by hinder):
Using $HOME will not solve the problem of the simulation directories, and so the user will have to customise anyway. Since we have just over a week to the release, I propose that we document what is needed for these machines in the release notes and in wherever people go to find out peculiarities of using certain machines with the ET (maybe we need such a place?). After the release, we can solve the problem properly by making a consistent environment variable syntax work everywhere in simfactory.
#382: SimFactory home directory on Kraken is too specific -------------------------+-------------------------------------------------- Reporter: hinder | Owner: mthomas Type: defect | Status: review Priority: major | Milestone: ET_2011_10 Component: SimFactory | Version: Resolution: | Keywords: -------------------------+--------------------------------------------------
Comment (by hinder):
I have updated the [https://docs.einsteintoolkit.org/et- docs/Detailed_Release_Announcement release notes] to include this instruction for LoneStar, Ranger and Kraken, which are the three machines where I have run into this problem. If there is agreement that we can release in this way (the same problem was in the last release, after all), then we can remove the release milestone from this ticket and leave the ticket open for fix in the next cycle.
#382: SimFactory home directory on Kraken is too specific -------------------------+-------------------------------------------------- Reporter: hinder | Owner: mthomas Type: defect | Status: review Priority: major | Milestone: Component: SimFactory | Version: Resolution: | Keywords: -------------------------+-------------------------------------------------- Changes (by hinder):
* milestone: ET_2011_10 =>
Comment:
I am removing the release milestone and leaving the ticket open for a future fix.
#382: SimFactory home directory on Kraken is too specific -------------------------+-------------------------------------------------- Reporter: hinder | Owner: mthomas Type: defect | Status: reopened Priority: major | Milestone: Component: SimFactory | Version: Resolution: | Keywords: -------------------------+-------------------------------------------------- Changes (by knarf):
* status: review => reopened
#382: SimFactory home directory on Kraken is too specific -------------------------+-------------------------------------------------- Reporter: hinder | Owner: mthomas Type: defect | Status: reopened Priority: major | Milestone: Component: SimFactory | Version: Resolution: | Keywords: -------------------------+--------------------------------------------------
Comment (by eschnett):
Should we close this ticket?
#382: SimFactory home directory on Kraken is too specific -------------------------+-------------------------------------------------- Reporter: hinder | Owner: mthomas Type: defect | Status: reopened Priority: major | Milestone: Component: SimFactory | Version: Resolution: | Keywords: -------------------------+--------------------------------------------------
Comment (by hinder):
As far as I know, the problem has not been solved, so the ticket should remain open.
#382: SimFactory home directory on Kraken is too specific -------------------------+-------------------------------------------------- Reporter: hinder | Owner: mthomas Type: defect | Status: closed Priority: major | Milestone: Component: SimFactory | Version: Resolution: wontfix | Keywords: -------------------------+-------------------------------------------------- Changes (by rhaas):
* status: reopened => closed * resolution: => wontfix
Comment:
Kraken was decommissioned in August 27, 2014 allowing us to claim this ticket.
trac@lists.einsteintoolkit.org
-
Einstein Toolkit