Trac

trac@lists.einsteintoolkit.org

8708 discussions

[Einstein Toolkit] #420: newest revision of simfactory 2.0 submit three instead of one job
by Einstein Toolkit 11 Aug '12

11 Aug '12

#420: newest revision of simfactory 2.0 submit three instead of one job ----------------------------------------------+----------------------------- Reporter: alexander.beck-ratzka@… | Owner: mthomas Type: defect | Status: new Priority: minor | Milestone: Component: SimFactory | Version: ET_2010_11 Keywords: | ----------------------------------------------+----------------------------- I am submitting a simulation using simfactory with the command: [snip] simfactory/bin/sim submit test-whisky-openmp --configuration test-whisky- openmp --parfile=whisky-openmp-test-ali.par --verbose --walltime=48:00:00 --procs=16 --ppn=4 --num-threads=4 --machine=damiana --queue=intel.q [snip] sim then comes with the messages: [snip] Info: Simfactory command: simfactory/bin/../lib/sim.py "submit" "test- whisky-openmp" "--configuration" "test-whisky-openmp" "--parfile=whisky- openmp-test-ali.par" "--verbose" "--walltime=48:00:00" "--procs=16" "-- ppn=4" "--num-threads=4" "--machine=damiana" "--queue=intel.q" Info: Version 1331M The Simulation Factory: Manage Cactus simulations Info: defs: /home/alibeck/programme/Cactus- Luca/Cactus/simfactory/etc/defs.ini Info: defs.local: /home/alibeck/programme/Cactus- Luca/Cactus/simfactory/etc/defs.local.ini Info: Cactus Directory: /home/alibeck/programme/Cactus-Luca/Cactus Info: simenv.COMMAND: submit Info: Executing command: submit Info: Assigned restart_id of: 0002 Info: Found the following restart_ids: [0, 1] Info: Maximum restart id determined to be: 0001 Assigned restart id: 2 Info: Simulation is inactive: submitting Info: Job allocation information: Info: System: nodes=170 cores/node=4 threads/process=4 Info: Requested: nodes=4 cores=16 cores/node=4 Info: Run: processes=4 threads=16 threads/process=4 Info: Distribution: processes/node=1 threads/node=4 Info: Ratio: threads/core=1.000 cores/thread=1.000 Info: writing to internalDir: /lustre/AEI/alibeck/simulations/test-whisky- openmp/output-0002/SIMFACTORY Info: saving substituted submitscript contents to: /lustre/AEI/alibeck/simulations/test-whisky- openmp/output-0002/SIMFACTORY/SubmitScript Executing submit command: qsub /lustre/AEI/alibeck/simulations/test- whisky-openmp/output-0002/SIMFACTORY/SubmitScript Submit finished, job id is 259460 Info: Restart 2 is active Info: Assigned restart_id of: 0003 Info: Found the following restart_ids: [0, 1, 2, 2] Info: Maximum restart id determined to be: 0002 Assigned restart id: 3 Info: Simulation is active: presubmitting Info: Job allocation information: Info: System: nodes=170 cores/node=4 threads/process=4 Info: Requested: nodes=4 cores=16 cores/node=4 Info: Run: processes=4 threads=16 threads/process=4 Info: Distribution: processes/node=1 threads/node=4 Info: Ratio: threads/core=1.000 cores/thread=1.000 Info: writing to internalDir: /lustre/AEI/alibeck/simulations/test-whisky- openmp/output-0003/SIMFACTORY Info: saving substituted submitscript contents to: /lustre/AEI/alibeck/simulations/test-whisky- openmp/output-0003/SIMFACTORY/SubmitScript Executing submit command: qsub /lustre/AEI/alibeck/simulations/test- whisky-openmp/output-0003/SIMFACTORY/SubmitScript Submit finished, job id is 259461 Info: Restart 2 is active Info: Assigned restart_id of: 0004 Info: Found the following restart_ids: [0, 1, 2, 2, 3] Info: Maximum restart id determined to be: 0003 Assigned restart id: 4 Info: Simulation is active: presubmitting Info: Job allocation information: Info: System: nodes=170 cores/node=4 threads/process=4 Info: Requested: nodes=4 cores=16 cores/node=4 Info: Run: processes=4 threads=16 threads/process=4 Info: Distribution: processes/node=1 threads/node=4 Info: Ratio: threads/core=1.000 cores/thread=1.000 Info: writing to internalDir: /lustre/AEI/alibeck/simulations/test-whisky- openmp/output-0004/SIMFACTORY Info: saving substituted submitscript contents to: /lustre/AEI/alibeck/simulations/test-whisky- openmp/output-0004/SIMFACTORY/SubmitScript Executing submit command: qsub /lustre/AEI/alibeck/simulations/test- whisky-openmp/output-0004/SIMFACTORY/SubmitScript Submit finished, job id is 259462 [snip] As a result three jobs are queued. [snip] qstat job-ID prior name user state submit/start at queue slots ja-task-ID ----------------------------------------------------------------------------------------------------------------- 259460 0.00000 test-whisk alibeck qw 04/29/2011 10:26:55 16 259461 0.00000 test-whisk alibeck hqw 04/29/2011 10:26:56 16 259462 0.00000 test-whisk alibeck hqw 04/29/2011 10:26:56 16 [snip] What is going wrong here? -- Ticket URL: <https://trac.einsteintoolkit.org/ticket/420> Einstein Toolkit <http://einsteintoolkit.org> The Einstein Toolkit

1 3

[Einstein Toolkit] #1016: External library dependencies not taken into account
by Einstein Toolkit 10 Aug '12

10 Aug '12

#1016: External library dependencies not taken into account ----------------------+----------------------------------------------------- Reporter: eschnett | Owner: Type: defect | Status: new Priority: major | Milestone: Component: Cactus | Version: Keywords: | ----------------------+----------------------------------------------------- I find that dependencies between external libraries are not taken into account when creating the library link order. That is, "REQUIRES" and "OPTIONAL" specifications are ignored. I believe I have isolated the problem and will propose a patch soon. -- Ticket URL: <https://trac.einsteintoolkit.org/ticket/1016> Einstein Toolkit <http://einsteintoolkit.org> The Einstein Toolkit

1 7

[Einstein Toolkit] #1032: change HDF5's file open options to close all sub-file objects when a file is closed
by Einstein Toolkit 10 Aug '12

10 Aug '12

#1032: change HDF5's file open options to close all sub-file objects when a file is closed --------------------------+------------------------------------------------- Reporter: rhaas | Owner: eschnett Type: enhancement | Status: new Priority: minor | Milestone: Component: Carpet | Version: Keywords: CarpetIOHDF5 | --------------------------+------------------------------------------------- this is strictly speaking not necessary, since any need for it is caused by a HDF5 object leak. On the other hand it does not hurt either additionally one could overload H5Fclose and warn the user when there are still open objects when a file is closed, ie: {{{ ////////////////////////////////////////////////////////////////////////////// // Close HDF5 file checking for leaked objects ////////////////////////////////////////////////////////////////////////////// static herr_t H5Fclose (hid_t file) { DECLARE_CCTK_PARAMETERS; herr_t retval; int error_count = 0; hsize_t objectcount; HDF5_ERROR (objectcount = H5Fget_obj_count(file, H5F_OBJ_ALL | H5F_OBJ_LOCAL)); if (objectcount > 1) { std::vector<char> fn; hsize_t sz_fn; HDF5_ERROR (sz_fn = H5Fget_name(file, NULL, 0)); fn.resize(sz_fn+1); HDF5_ERROR (H5Fget_name(file, &fn[0], fn.size())); CCTK_VWarn (1, __LINE__, __FILE__, CCTK_THORNSTRING, "%d open HDF5 objects when closing file '%s'", int(objectcount)-1, &fn[0]); } HDF5_ERROR (retval = ::H5Fclose (file)); return retval; } }}} -- Ticket URL: <https://trac.einsteintoolkit.org/ticket/1032> Einstein Toolkit <http://einsteintoolkit.org> The Einstein Toolkit

1 3

[Einstein Toolkit] #1030: defer (re)opening checkpoint files until we need to
by Einstein Toolkit 10 Aug '12

10 Aug '12

#1030: defer (re)opening checkpoint files until we need to --------------------------+------------------------------------------------- Reporter: rhaas | Owner: eschnett Type: enhancement | Status: new Priority: minor | Milestone: Component: Carpet | Version: Keywords: CarpetIOHDF5 | --------------------------+------------------------------------------------- this patch defers opening checkpoint files until we need to do so because either we need to browse them or open a dataset for reading. This speeds up open_one_file_at_a_time since it will not open a hdf5 file that was already parsed and does not contain useful datasets. -- Ticket URL: <https://trac.einsteintoolkit.org/ticket/1030> Einstein Toolkit <http://einsteintoolkit.org> The Einstein Toolkit

1 3

[Einstein Toolkit] #1029: support indices in sequential chunked output files
by Einstein Toolkit 10 Aug '12

10 Aug '12

#1029: support indices in sequential chunked output files --------------------------+------------------------------------------------- Reporter: rhaas | Owner: eschnett Type: defect | Status: new Priority: minor | Milestone: Component: Carpet | Version: Keywords: CarpetIOHDF5 | --------------------------+------------------------------------------------- this patch makes sure that even the sequential file writer writes correct index files. Right now this writer is (always) used when Carpet runs on a single process. Generated index files are currently invalid since the MetaData group is written but nothing else. -- Ticket URL: <https://trac.einsteintoolkit.org/ticket/1029> Einstein Toolkit <http://einsteintoolkit.org> The Einstein Toolkit

1 3

[Einstein Toolkit] #1028: do not re-parse file when using open_one_input_file_at_a_time
by Einstein Toolkit 10 Aug '12

10 Aug '12

#1028: do not re-parse file when using open_one_input_file_at_a_time -------------------------+-------------------------------------------------- Reporter: rhaas | Owner: eschnett Type: enhancement | Status: new Priority: minor | Milestone: Component: Carpet | Version: Keywords: | -------------------------+-------------------------------------------------- this patch retains the patch map created when reading a file even when the file itself is closed due to open_one_input_file_at_a_time. Since parsing the file is slow this can considerably speed up open_one_input_file_at_a_time. -- Ticket URL: <https://trac.einsteintoolkit.org/ticket/1028> Einstein Toolkit <http://einsteintoolkit.org> The Einstein Toolkit

1 6

[Einstein Toolkit] #1027: close small memory leak in CarpetIOHDF5
by Einstein Toolkit 10 Aug '12

10 Aug '12

#1027: close small memory leak in CarpetIOHDF5 --------------------------+------------------------------------------------- Reporter: rhaas | Owner: eschnett Type: defect | Status: new Priority: minor | Milestone: Component: Carpet | Version: Keywords: CarpetIOHDF5 | --------------------------+------------------------------------------------- this patches closes a memory leak in CarpetIOHDF5 when using open_one_input_file_at_a_time, where the HDF5 file would be closed, but the data read out out it was not freed which would cause storage for it to be reallocated again when the file is opened again for the next refinement level. It also properly closes a HDF5 property that was left dangling otherwise. -- Ticket URL: <https://trac.einsteintoolkit.org/ticket/1027> Einstein Toolkit <http://einsteintoolkit.org> The Einstein Toolkit

1 4

[Einstein Toolkit] #380: AHFinderDirect failure with qc0-mclachlan.par
by Einstein Toolkit 10 Aug '12

10 Aug '12

#380: AHFinderDirect failure with qc0-mclachlan.par --------------------+------------------------------------------------------- Reporter: hinder | Owner: Type: defect | Status: new Priority: major | Milestone: ET_2011_05 Component: Other | Version: Keywords: | --------------------+------------------------------------------------------- Trying to run the par/qc0-mclachlan.par in the ET trunk gives the following error: INFO (AHFinderDirect): proc 0: searching for horizons 1,5/6 WARNING level 0 in thorn CarpetInterp processor 0 host kop193.datura.admin (line 1683 of /home/ianhin/Cactus/etrelease/arrangements/Carpet/CarpetInterp/src/interp.cc): -> Grid function "AHFINDERDIRECT::ahmask" has only 1 active time levels on refinement level 1; this is not en ough for time interpolation See also #373, which might be related. -- Ticket URL: <https://trac.einsteintoolkit.org/ticket/380> Einstein Toolkit <http://einsteintoolkit.org> The Einstein Toolkit

1 10

[Einstein Toolkit] #676: GRHydro/MHD ist not fully multipatch aware yet
by Einstein Toolkit 10 Aug '12

10 Aug '12

#676: GRHydro/MHD ist not fully multipatch aware yet -----------------------------------+---------------------------------------- Reporter: rhaas | Owner: Type: defect | Status: new Priority: minor | Milestone: Component: EinsteinToolkit thorn | Version: Keywords: GRHydro | -----------------------------------+---------------------------------------- eg. ReconstructPoly uses Bvec and vel directly. -- Ticket URL: <https://trac.einsteintoolkit.org/ticket/676> Einstein Toolkit <http://einsteintoolkit.org> The Einstein Toolkit

1 3

[Einstein Toolkit] #488: Reduce space taken by Formaline tarballs
by Einstein Toolkit 10 Aug '12

10 Aug '12

#488: Reduce space taken by Formaline tarballs -------------------------+-------------------------------------------------- Reporter: hinder | Owner: Type: enhancement | Status: new Priority: minor | Milestone: Component: Cactus | Version: Keywords: Formaline | -------------------------+-------------------------------------------------- When syncing lightweight data of simulations from a cluster, much of the time and disk space on the local system is taken with Formaline tarballs. It would be good to reduce this. I propose opportunistically replacing the generated tarballs with hard- links to existing tarballs on the same filesystem after Formaline has written them to the new output directory if the files compare equal. These could be from previous restarts of the same simulation when using simfactory. Then, if an entire simulation is rsynced with the appropriate options, hard-links will be transferred as hard-links, and the overall transfer time and disk space used will be significantly reduced. -- Ticket URL: <https://trac.einsteintoolkit.org/ticket/488> Einstein Toolkit <http://einsteintoolkit.org> The Einstein Toolkit

1 15

Jump to page:

2026

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

Trac