[Users] Issues with Downloading/Installing the tool kit
Joe Hellmers
hellmersjl at icloud.com
Mon Sep 3 13:46:36 CDT 2018
OK, I did
git config --global core.preloadIndex false
This allowed me to run GetComponents without error
./GetComponents --verbose https://bitbucket.org/einsteintoolkit/manifest/raw/ET_2018_02/einsteintoolkit.th
When I build
./simfactory/bin/sim build --mdbkey make 'make -j2' --thornlist ../einsteintoolkit.th
all looks well until, I think, the very end when some sort of git clean up seems to be happening.
Done creating cactus_sim.
fatal: bad revision 'HEAD'
Formaline: Committing source tree to git repository...
Formaline: Created git tag build-sim-comet-ln2.sdsc.edu-jhellmer-2018.09.03-17.30.20-18591
Formaline: Updated git branch config-sim-comet-ln2.sdsc.edu-home-jhellmer-etk-Cactus
Formaline: Optimising git repository (slow only the first time)...
Formaline: ERROR during: Garbage collecting git repo
Command was: /usr/bin/git --git-dir='/home/jhellmer/etk/Cactus/configs/sim/configjar.git/.git' gc at /home/jhellmer/etk/Cactus/configs/sim/scratch/Formaline/bin/update-git-repo.pl line 60, <STDIN> line 5090.
make[1]: *** [/home/jhellmer/etk/Cactus/configs/sim/build/Formaline/git-store-source] Error 2
make: *** [sim] Error 2
So I attempted to run the HelloWorld app, and it looks to be trying to submit a job but apparently fails
[jhellmer at comet-ln2 Cactus]$ ./simfactory/bin/sim create-submit helloworld --parfile arrangements/CactusExamples/HelloWorld/par/HelloWorld.par --walltime 0:5:0
Parameter file: /home/jhellmer/comet/Cactus/arrangements/CactusExamples/HelloWorld/par/HelloWorld.par
Skeleton Created
Job directory: "/oasis/scratch/comet/jhellmer/temp_project/simulations/helloworld"
Executable: "/home/jhellmer/comet/Cactus/exe/cactus_sim"
Option list: "/oasis/scratch/comet/jhellmer/temp_project/simulations/helloworld/SIMFACTORY/cfg/OptionList"
Submit script: "/oasis/scratch/comet/jhellmer/temp_project/simulations/helloworld/SIMFACTORY/run/SubmitScript"
Run script: "/oasis/scratch/comet/jhellmer/temp_project/simulations/helloworld/SIMFACTORY/run/RunScript"
Parameter file: "/oasis/scratch/comet/jhellmer/temp_project/simulations/helloworld/SIMFACTORY/par/HelloWorld.par"
Assigned restart id: 0
Warning: Too many threads per process specified: specified num-threads=24 (ppn-used is 24)
Warning: Total number of threads and number of threads per process are inconsistent: procs=1, num-threads=24 (procs*num-smt must be an integer multiple of num-threads)
Warning: Total number of threads and number of cores per node are inconsistent: procs=1, ppn-used=24 (procs must be an integer multiple of ppn-used)
Executing submit command: sbatch /oasis/scratch/comet/jhellmer/temp_project/simulations/helloworld/output-0000/SIMFACTORY/SubmitScript; sleep 5
Warning: submit either failed or could not determine job id, output:
Warning: - Package -----------------------------+- Versions -+- Last mod. ------
Currently Loaded Modulefiles:
intel/2013_sp1.2.144 2017/08/21 20:08:02
mvapich2_ib/2.1 2016/01/20 0:58:02
gnutools/2.69 2016/01/16 0:30:50
hdf5/1.8.14 2016/03/16 19:35:34
netcdf/4.3.2 2016/01/21 22:17:20
ncview/2.1.7 2016/03/31 4:18:21
Unloading compiler-dependent module netcdf/4.3.2
must specify one of the following mpi modules: mvapich2_ib openmpi_ib
- Package -----------------------------+- Versions -+- Last mod. ------
Currently Loaded Modulefiles:
.intel/hdf5/1.8.14 2016/03/16 19:35:34
ncview/2.1.7 2016/03/31 4:18:21
- Package -----------------------------+- Versions -+- Last mod. ------
Currently Loaded Modulefiles:
hdf5/1.8.14 2016/03/16 19:35:34
ncview/2.1.7 2016/03/31 4:18:21
gnutools/2.69 2016/01/16 0:30:50
intel/2015.2.164 2017/08/21 20:08:02
mvapich2_ib/2.1 2016/01/20 0:58:02
fftw/3.3.4 2016/01/20 21:29:46
gsl/2.1 2016/02/25 5:15:41
papi/5.4.1 2016/01/22 0:20:41
scalapack/2.0.2 2016/02/04 8:48:43
parmetis/4.0.3 2016/02/04 7:58:37
superlu/4.2 2016/02/04 7:45:25
petsc/3.6.3 2016/02/04 8:48:36
sbatch: error: bank_limit plugin: expired user, can't submit job
sbatch: error: Batch job submission failed: Invalid account or account/partition combination specified
When I try to execute the TOV I get something similar.
What is the command just to execute without submitting a job? Alternatvely, how do I configure the ETK to submit successfully on Slurm for the Cluster (Comet) I’m using?
Thanks for all your help!
[jhellmer at comet-ln2 Cactus]$ ./simfactory/bin/sim create-run static_tov --parfile=par/static_tov.par --procs=2 --num-threads=1 --ppn-used=2 --walltime=8:0:0 | cat
Parameter file: /home/jhellmer/comet/Cactus/par/static_tov.par
Skeleton Created
Job directory: "/oasis/scratch/comet/jhellmer/temp_project/simulations/static_tov"
Executable: "/home/jhellmer/comet/Cactus/exe/cactus_sim"
Option list: "/oasis/scratch/comet/jhellmer/temp_project/simulations/static_tov/SIMFACTORY/cfg/OptionList"
Submit script: "/oasis/scratch/comet/jhellmer/temp_project/simulations/static_tov/SIMFACTORY/run/SubmitScript"
Run script: "/oasis/scratch/comet/jhellmer/temp_project/simulations/static_tov/SIMFACTORY/run/RunScript"
Parameter file: "/oasis/scratch/comet/jhellmer/temp_project/simulations/static_tov/SIMFACTORY/par/static_tov.par"
Simulation name: static_tov
Assigned restart id: 0
Running simulation static_tov
- Package -----------------------------+- Versions -+- Last mod. ------
Currently Loaded Modulefiles:
intel/2013_sp1.2.144 2017/08/21 20:08:02
mvapich2_ib/2.1 2016/01/20 0:58:02
gnutools/2.69 2016/01/16 0:30:50
hdf5/1.8.14 2016/03/16 19:35:34
netcdf/4.3.2 2016/01/21 22:17:20
ncview/2.1.7 2016/03/31 4:18:21
Unloading compiler-dependent module netcdf/4.3.2
must specify one of the following mpi modules: mvapich2_ib openmpi_ib
- Package -----------------------------+- Versions -+- Last mod. ------
Currently Loaded Modulefiles:
.intel/hdf5/1.8.14 2016/03/16 19:35:34
ncview/2.1.7 2016/03/31 4:18:21
- Package -----------------------------+- Versions -+- Last mod. ------
Currently Loaded Modulefiles:
hdf5/1.8.14 2016/03/16 19:35:34
ncview/2.1.7 2016/03/31 4:18:21
gnutools/2.69 2016/01/16 0:30:50
intel/2015.2.164 2017/08/21 20:08:02
mvapich2_ib/2.1 2016/01/20 0:58:02
fftw/3.3.4 2016/01/20 21:29:46
gsl/2.1 2016/02/25 5:15:41
papi/5.4.1 2016/01/22 0:20:41
scalapack/2.0.2 2016/02/04 8:48:43
parmetis/4.0.3 2016/02/04 7:58:37
superlu/4.2 2016/02/04 7:45:25
petsc/3.6.3 2016/02/04 8:48:36
Preparing:
+ set -e
+ cd /oasis/scratch/comet/jhellmer/temp_project/simulations/static_tov/output-0000-active
+ echo Checking:
+ pwd
Checking:
/oasis/scratch/comet/jhellmer/temp_project/simulations/static_tov/output-0000-active
+ hostname
comet-ln2.sdsc.edu
+ date
Mon Sep 3 11:32:34 PDT 2018
+ echo Environment:
Environment:
+ export CACTUS_NUM_PROCS=2
+ CACTUS_NUM_PROCS=2
+ export CACTUS_NUM_THREADS=1
+ CACTUS_NUM_THREADS=1
+ export GMON_OUT_PREFIX=gmon.out
+ GMON_OUT_PREFIX=gmon.out
+ export KMP_AFFINITY=norespect,compact
+ KMP_AFFINITY=norespect,compact
+ export OMP_NUM_THREADS=1
+ OMP_NUM_THREADS=1
+ sort
+ env
+ echo
+ echo Starting:
Starting:
++ date +%s
+ export CACTUS_STARTTIME=1535999554
+ CACTUS_STARTTIME=1535999554
+ ibrun --npernode 2 -v /oasis/scratch/comet/jhellmer/temp_project/simulations/static_tov/SIMFACTORY/exe/cactus_sim -L 3 /oasis/scratch/comet/jhellmer/temp_project/simulations/static_tov/output-0000/static_tov.par
No SLURM_NODELIST given, run generate_pbs_nodefile inside a Slurm allocation or batch script.
IBRUN ERROR: Unable to obtain number of nodes from resource manager and -N not given.
Contact help at xsede.org for additional assistance.
Mon Sep 3 11:32:34 PDT 2018
Simfactory Done at date: 0
>
>
> Hello Joe,
>
>> Summary of Warnings:
>>
>> Could not update einsteininitialdata. Could not stash local changes. Error message was 'fatal: unable to create threaded lstat
>> fatal: unable to create threaded lstat
>> fatal: unable to create threaded lstat
>> fatal: unable to create threaded lstat
>> fatal: unable to create threaded lstat
>> Cannot save the current worktree state
>> '.
>> Time Elapsed: 0 minutes, 27 seconds
>>
>> This home directory is on a CentOS 6.10 server. I was able to do this successfully on an Ubuntu 16 VM.
> Oha. That does not sound good. The threaded lstat thing is an almost
> internal git error message that could mean that there are issues with
> the file system hosting your $HOME directory. I have seen that message
> once before but the exact context escapes me now. It could have been in
> relation to a BeeGFS file system issue (you can see the file system in
> $HOME by doing "mount' command and looking for the line that
> corresponds to $HOME).
>
> In any case, git lets you avoid the threaded lstat if you set (see
> ):
>
> git config --global core.preloadIndex false
>
> which actually makes my BeeGFS hunch less likely and points to a
> resource limit that the admins set.
>
> Yours,
> Roland
>
> --
> My email is as private as my paper mail. I therefore support encrypting
> and signing email messages. Get my PGP key from http://pgp.mit.edu .
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.einsteintoolkit.org/pipermail/users/attachments/20180903/9ee22a01/attachment-0001.html
More information about the Users
mailing list