#174: Remove cd semaphore from GetComponents ---------------------------+------------------------------------------------ Reporter: eschnett | Owner: eric9 Type: defect | Status: new Priority: major | Milestone: Component: GetComponents | Version: Keywords: | ---------------------------+------------------------------------------------ GetComponents uses a semaphore that decides which task can cd into a subdirectory. This limits parallelism, since in essence only one task can cd into (and presumably thus access) a repository. Furthermore, using semaphores is a tedious and complex way to handle parallelism.
I suggest to remove the semaphore, and instead not use cd within GetComponents at all. Instead, run_command should take care of this. For example, instead of
if ($PARALLEL) {$cd_sem->down()} chdir("$orig_dir/$ROOT/repos/$git_repo"); my $err = run_command("git checkout --track -b $branch origin/$branch"); chdir("$orig_dir"); if ($PARALLEL) {$cd_sem->up()}
I would write
my $err = run_command("cd $orig_dir/$ROOT/repos/$git_repo && git checkout --track -b $branch origin/$branch");
which is four lines shorter, doesn't use semaphores, and doesn't have to handle semaphores and "cd'ing back" in error clean-up code.
I've omitted the error clean-up code in the above; indeed in the example i'm looking at, this cleanup code neither releases the semaphore nor cds back into the original directory.
Using cd this way may also avoid all problems where thorns are accidentally checked out into the wrong directory.
#174: Remove cd semaphore from GetComponents ----------------------------+----------------------------------------------- Reporter: eschnett | Owner: eric9 Type: defect | Status: new Priority: major | Milestone: Component: GetComponents | Version: Resolution: | Keywords: ----------------------------+-----------------------------------------------
Comment (by hinder):
Why is it necessary to change directory anyway? I know that with git you can specify the repository directory as an argument to the command. For svn I think you can just give the path to the repository.
#174: Remove cd semaphore from GetComponents ----------------------------+----------------------------------------------- Reporter: eschnett | Owner: eric9 Type: defect | Status: new Priority: major | Milestone: Component: GetComponents | Version: Resolution: | Keywords: ----------------------------+-----------------------------------------------
Comment (by eschnett):
I tried that with git and couldn't make it work. There are certain subtleties, such as e.g. whether the "repo directory" is the directory containing the .git dir or the .git dir itself, and using "cd" is the easiest way.
#174: Remove cd semaphore from GetComponents ----------------------------+----------------------------------------------- Reporter: eschnett | Owner: eric9 Type: defect | Status: accepted Priority: major | Milestone: Component: GetComponents | Version: Resolution: | Keywords: ----------------------------+----------------------------------------------- Changes (by eric9):
* status: new => accepted
Comment:
I've actually begun doing exactly this, and executing the commands in a subshell so I don't have to worry about the pwd ever changing. As Erik said, there are significant issues to using git outside of the repository; certain commands like "git pull" will not ever work from outside the repository, even if you specify the repository location and the work tree location...
#174: Remove cd semaphore from GetComponents ----------------------------+----------------------------------------------- Reporter: eschnett | Owner: eric9 Type: defect | Status: accepted Priority: major | Milestone: Component: GetComponents | Version: Resolution: | Keywords: ----------------------------+-----------------------------------------------
Comment (by eschnett):
You don't need a subshell for this, a subprocess is good enough. Any time you call system() or popen(), there will automatically be a subprocess created; whether the executed command is wrapped in a shell or not does not matter. That is, the example I gave above is good enough -- there is no need to explicitly call a shell.
#174: Remove cd semaphore from GetComponents ----------------------------+----------------------------------------------- Reporter: eschnett | Owner: eric9 Type: defect | Status: closed Priority: major | Milestone: Component: GetComponents | Version: Resolution: fixed | Keywords: ----------------------------+----------------------------------------------- Changes (by eric9):
* status: accepted => closed * resolution: => fixed
Comment:
I see. I have removed the cd semaphore and changed all the git and ln calls to incorporate the appropriate cd command in the subprocess.
trac@lists.einsteintoolkit.org
-
Einstein Toolkit