Greetings to all.
I've been having some problems running the GW150914 simulation. When following the tutorial on " https://einsteintoolkit.org/gallery/bbh/cr.html " , I run into problems when running the parfile, specifically when running
simfactory/bin/sim create-submit GW150914_28 --define N 28 --parfile par/GW150914/GW150914.rpar --procs 128 --walltime 24:00:00
This leads to the error
Parameter file: /home/enzo/Cactus/par/GW150914/GW150914.rpar Skeleton Created Job directory: "/home/enzo/simulations/GW150914_28" Executable: "/home/enzo/Cactus/exe/cactus_sim" Option list: "/home/enzo/simulations/GW150914_28/SIMFACTORY/cfg/OptionList" Submit script: "/home/enzo/simulations/GW150914_28/SIMFACTORY/run/SubmitScript" Run script: "/home/enzo/simulations/GW150914_28/SIMFACTORY/run/RunScript" Parameter file: "/home/enzo/simulations/GW150914_28/SIMFACTORY/par/GW150914.rpar" Assigned restart id: 0
Error: Too many nodes specified: nodes=16 (maxnodes is 1) Aborting Simfactory.
I am inclined to believe this is because I am running it on my own laptop, but I would nonetheless like to know exactly what does that error mean and if I can fix it somehow.
I would really appreciate your help, thank you :)
Best regards, Enzo
Hi Enzo,
As the website warns: "Note that the simulation requires about 82 GB of RAM, *so most likely you will need to use a cluster.*" As for the error, MPI is probably seeing only one node (your laptop) so requesting 16 is throwing an error.
-Zach
* * * Zachariah Etienne Assoc. Prof. of Physics, U. of Idaho Adjunct Assoc. Prof. of Physics & Astronomy, West Virginia U. https://etienneresearch.com https://blackholesathome.net
On Mon, Oct 9, 2023 at 7:18 AM Enzo Iubini egic.enzo@gmail.com wrote:
Greetings to all.
I've been having some problems running the GW150914 simulation. When following the tutorial on " https://einsteintoolkit.org/gallery/bbh/cr.html " , I run into problems when running the parfile, specifically when running
simfactory/bin/sim create-submit GW150914_28 --define N 28 --parfile par/GW150914/GW150914.rpar --procs 128 --walltime 24:00:00
This leads to the error
Parameter file: /home/enzo/Cactus/par/GW150914/GW150914.rpar Skeleton Created Job directory: "/home/enzo/simulations/GW150914_28" Executable: "/home/enzo/Cactus/exe/cactus_sim" Option list: "/home/enzo/simulations/GW150914_28/SIMFACTORY/cfg/OptionList" Submit script: "/home/enzo/simulations/GW150914_28/SIMFACTORY/run/SubmitScript" Run script: "/home/enzo/simulations/GW150914_28/SIMFACTORY/run/RunScript" Parameter file: "/home/enzo/simulations/GW150914_28/SIMFACTORY/par/GW150914.rpar" Assigned restart id: 0
Error: Too many nodes specified: nodes=16 (maxnodes is 1) Aborting Simfactory.
I am inclined to believe this is because I am running it on my own laptop, but I would nonetheless like to know exactly what does that error mean and if I can fix it somehow.
I would really appreciate your help, thank you :)
Best regards, Enzo _______________________________________________ Users mailing list Users@einsteintoolkit.org http://lists.einsteintoolkit.org/mailman/listinfo/users
Thank you Zach for your reply, that clears it up.
I have one more question then. Since I was not able to run the code in my laptop, I tried downloading the datafiles from the following site.
https://zenodo.org/record/155394
I run into the following problem. When following the tutorial example in ' https://docs.einsteintoolkit.org/et-docs/GW150914_VisIt_Tutorial ' on the section "A visualization example", it shows a long folder of surface01_00xxxxx.vtk datafiles, all of which are different time steps for the black hole horizons. But when looking at the downloaded file, I cannot seem to find all these .vtk files. Instead I only find one .vtk file, and therefore am unable to run the complete simulation. Does anyone understand what's going on?
Thank you so much for your help. Best regards, Enzo
El lun, 9 oct 2023 a las 13:23, Zach Etienne (zachetie@gmail.com) escribió:
Hi Enzo,
As the website warns: "Note that the simulation requires about 82 GB of RAM, *so most likely you will need to use a cluster.*" As for the error, MPI is probably seeing only one node (your laptop) so requesting 16 is throwing an error.
-Zach
Zachariah Etienne Assoc. Prof. of Physics, U. of Idaho Adjunct Assoc. Prof. of Physics & Astronomy, West Virginia U. https://etienneresearch.com https://blackholesathome.net
On Mon, Oct 9, 2023 at 7:18 AM Enzo Iubini egic.enzo@gmail.com wrote:
Greetings to all.
I've been having some problems running the GW150914 simulation. When following the tutorial on " https://einsteintoolkit.org/gallery/bbh/cr.html " , I run into problems when running the parfile, specifically when running
simfactory/bin/sim create-submit GW150914_28 --define N 28 --parfile par/GW150914/GW150914.rpar --procs 128 --walltime 24:00:00
This leads to the error
Parameter file: /home/enzo/Cactus/par/GW150914/GW150914.rpar Skeleton Created Job directory: "/home/enzo/simulations/GW150914_28" Executable: "/home/enzo/Cactus/exe/cactus_sim" Option list: "/home/enzo/simulations/GW150914_28/SIMFACTORY/cfg/OptionList" Submit script: "/home/enzo/simulations/GW150914_28/SIMFACTORY/run/SubmitScript" Run script: "/home/enzo/simulations/GW150914_28/SIMFACTORY/run/RunScript" Parameter file: "/home/enzo/simulations/GW150914_28/SIMFACTORY/par/GW150914.rpar" Assigned restart id: 0
Error: Too many nodes specified: nodes=16 (maxnodes is 1) Aborting Simfactory.
I am inclined to believe this is because I am running it on my own laptop, but I would nonetheless like to know exactly what does that error mean and if I can fix it somehow.
I would really appreciate your help, thank you :)
Best regards, Enzo _______________________________________________ Users mailing list Users@einsteintoolkit.org http://lists.einsteintoolkit.org/mailman/listinfo/users
Hello Enzo,
The example parameter file on the website is not set up to produce VTK output, so you will have to manually enable this output to be able to visualize the horizon. You will also have to yourself run the simulation or possible ask the original authors (see the Zenodo entry https://zenodo.org/records/155394) for it.
To re-run with VTK output, take a look at the GW150915.rpar file linked on the webpage. It contains a line (you can find it be searching for the string "vtk"):
QuasiLocalMeasures::output_vtk_every = $out3d_every
which indicates that VTK files should be output every $out3d_every iterations. This variable itself is set at the beginning of the parameter file:
# out3d_every = rl0_every * 2 out3d_every = 0
to the special value of "0", which disables output. To enable output you must set it to a positive value, and, guessing by the commented out line just above, setting it to
out3d_every = rl0_every * 2
probably produces VTK output with the same frequency as shown in the visualization example. Please note that this will likely produce quite a bit (TB one assumes) of data output.
Yours, Roland
Thank you Zach for your reply, that clears it up.
I have one more question then. Since I was not able to run the code in my laptop, I tried downloading the datafiles from the following site.
https://urldefense.com/v3/__https://zenodo.org/record/155394__;!!DZ3fjg!8Xd1...
I run into the following problem. When following the tutorial example in ' https://urldefense.com/v3/__https://docs.einsteintoolkit.org/et-docs/GW15091... ' on the section "A visualization example", it shows a long folder of surface01_00xxxxx.vtk datafiles, all of which are different time steps for the black hole horizons. But when looking at the downloaded file, I cannot seem to find all these .vtk files. Instead I only find one .vtk file, and therefore am unable to run the complete simulation. Does anyone understand what's going on?
Thank you so much for your help. Best regards, Enzo
El lun, 9 oct 2023 a las 13:23, Zach Etienne (zachetie@gmail.com) escribió:
Hi Enzo,
As the website warns: "Note that the simulation requires about 82 GB of RAM, *so most likely you will need to use a cluster.*" As for the error, MPI is probably seeing only one node (your laptop) so requesting 16 is throwing an error.
-Zach
Zachariah Etienne Assoc. Prof. of Physics, U. of Idaho Adjunct Assoc. Prof. of Physics & Astronomy, West Virginia U. https://urldefense.com/v3/__https://etienneresearch.com__;!!DZ3fjg!8Xd1f_3xr... https://urldefense.com/v3/__https://blackholesathome.net__;!!DZ3fjg!8Xd1f_3x...
On Mon, Oct 9, 2023 at 7:18 AM Enzo Iubini egic.enzo@gmail.com wrote:
Greetings to all.
I've been having some problems running the GW150914 simulation. When following the tutorial on " https://urldefense.com/v3/__https://einsteintoolkit.org/gallery/bbh/cr.html_... " , I run into problems when running the parfile, specifically when running
simfactory/bin/sim create-submit GW150914_28 --define N 28 --parfile par/GW150914/GW150914.rpar --procs 128 --walltime 24:00:00
This leads to the error
Parameter file: /home/enzo/Cactus/par/GW150914/GW150914.rpar Skeleton Created Job directory: "/home/enzo/simulations/GW150914_28" Executable: "/home/enzo/Cactus/exe/cactus_sim" Option list: "/home/enzo/simulations/GW150914_28/SIMFACTORY/cfg/OptionList" Submit script: "/home/enzo/simulations/GW150914_28/SIMFACTORY/run/SubmitScript" Run script: "/home/enzo/simulations/GW150914_28/SIMFACTORY/run/RunScript" Parameter file: "/home/enzo/simulations/GW150914_28/SIMFACTORY/par/GW150914.rpar" Assigned restart id: 0
Error: Too many nodes specified: nodes=16 (maxnodes is 1) Aborting Simfactory.
I am inclined to believe this is because I am running it on my own laptop, but I would nonetheless like to know exactly what does that error mean and if I can fix it somehow.
I would really appreciate your help, thank you :)
Best regards, Enzo _______________________________________________ Users mailing list Users@einsteintoolkit.org https://urldefense.com/v3/__http://lists.einsteintoolkit.org/mailman/listinf...
users@lists.einsteintoolkit.org