From users at einsteintoolkit.org Wed Oct 1 17:15:01 2025 From: users at einsteintoolkit.org (users at einsteintoolkit.org) Date: Wed, 01 Oct 2025 17:15:01 -0500 Subject: [Users] Einstein Toolkit Meeting Reminder Message-ID: <68dda7e5.vysYqFpcgCRcrFMG%users@einsteintoolkit.org> Hello, Please consider joining the weekly Einstein Toolkit phone call at 9:00 am US central time on Thursdays. For details on how to connect and what agenda items are to be discussed, use the link below. https://docs.einsteintoolkit.org/et-docs/Main_Page#Weekly_Users_Call --The Maintainers From oevelasco at astro.unam.mx Thu Oct 2 14:10:04 2025 From: oevelasco at astro.unam.mx (=?UTF-8?Q?Omar_El=C3=ADas_Velasco_Castillo?=) Date: Thu, 2 Oct 2025 13:10:04 -0600 Subject: [Users] Errors when submitting runs in remote machine In-Reply-To: References: Message-ID: Hello, This is a followup for retaking my questions about my failing to submit and run simulations in a remote cluster. Can you show us what the error message(s) are? Yes, the tail of my err file reads: + set - ERROR: ld.so: object '/lib64/libpapi.so.5.2.0.0' from LD_PRELOAD cannot be preloaded: ignored. ERROR: ld.so: object '/lib64/libpapi.so.5.2.0.0' from LD_PRELOAD cannot be preloaded: ignored. -------------------------------------------------------------------------- Primary job terminated normally, but 1 process returned a non-zero exit code. Per user-direction, the job has been aborted. -------------------------------------------------------------------------- ERROR: ld.so: object '/lib64/libpapi.so.5.2.0.0' from LD_PRELOAD cannot be preloaded: ignored. /home/ia/ovelasco/simulations/tov_ET_decisiva/SIMFACTORY/exe/cactus_sim: error while loading shared libraries: libpapi.so.5.2.0.0: cannot open shared object file: No such file or directory =>> PBS: job killed: walltime 864033 exceeded limit 864000 mpirun: abort is already in progress...hit ctrl-c again to forcibly terminate Maybe. I'm not 100% sure what you are doing. Can you be clearer about how > you are running the ET? Sure, as I mentioned in a previous email, my intention is to run and submit the ET in a queue of a remote machine using Simfactory using either PBS or SLURM, as I work in different remote machines from time to time. The key point is that, I notice that while the sim and cactus_sim build is being done, it seems that some lines printed on the shell indicate that when a module is not found in the machine, the ET compiler "builds a bundle" for those modules that weren't found in the compilation from some thorns (examples below showing creation of bundles for GSL and HDF5): ******************************************************************************** Running configuration script for thorn GSL: GSL selected, but GSL_DIR not set. Checking pkg-config ... GSL not found. Checking standard paths ... GSL not found. Using bundled GSL... Finished running configuration script for thorn GSL. ******************************************************************************** Running configuration script for thorn HDF5: Additional requested language support: Fortran HDF5 selected, but HDF5_DIR not set. Checking pkg-config ... HDF5 not found. Checking standard paths ... HDF5 not found. Using bundled HDF5... Finished running configuration script for thorn HDF5. First, these messages about GSL and HDF5 not being found surprise me because both modules are located in the /usr directory and I pointed to them in the configurations file. Second. If this build was completed succesfully in the home or login shell of that machine, then I assume I can run and submit simulations with the configuration and runscript files of my machine in PBS or SLURM queues, am I right? If yes then, why I still haven't been capable of doing so? My simulations die within some seconds after being started. I include here the optionlist and runscript that I use for a machine that uses PBS as an example. Optionlist: VERSION = 2025 #For Einstein Toolkit 2022_11 CPP = cpp #CC = gcc #CXX = g++ CC = mpicc CXX = mpic++ FPP = cpp #F90 = gfortran F90 = mpif90 CPPFLAGS = -I/usr/include CPPFLAGS += -DCCTK_VECTOR_DISABLE_TESTS CPPFLAGS = -D__USE_ISOC99 PPFLAGS = -D_GLIBCXX_USE_C99_MATH LDFLAGS = -L/usr/lib64 -lssl -lcrypto -rdynamic FPPFLAGS = -traditional CFLAGS = -g -std=gnu99 CXXFLAGS = -g -std=gnu++11 -fpermissive F90FLAGS = -g -fcray-pointer -ffixed-line-length-none DEBUG = no CPP_DEBUG_FLAGS = C_DEBUG_FLAGS = CXX_DEBUG_FLAGS = OPTIMISE = yes CPP_OPTIMISE_FLAGS = C_OPTIMISE_FLAGS = -O2 CXX_OPTIMISE_FLAGS = -O2 F90_OPTIMISE_FLAGS = -O2 PROFILE = no CPP_PROFILE_FLAGS = C_PROFILE_FLAGS = -pg CXX_PROFILE_FLAGS = -pg F90_PROFILE_FLAGS = -pg WARN = yes CPP_WARN_FLAGS = -Wall C_WARN_FLAGS = -Wall CXX_WARN_FLAGS = -Wall F90_WARN_FLAGS = -Wall OPENMP = yes CPP_OPENMP_FLAGS = -fopenmp FPP_OPENMP_FLAGS = -D_OPENMP C_OPENMP_FLAGS = -fopenmp CXX_OPENMP_FLAGS = -fopenmp F90_OPENMP_FLAGS = -fopenmp VECTORISE = no VECTORISE_ALIGNED_ARRAYS = no VECTORISE_INLINE = yes PTHREADS_DIR = NO_BUILD # Para hallar todas estas opciones, hacemos: ldconfig -p | grep nombredelpaquete MPI_DIR = /software/TEST/local # LAPACK_DIR = /usr #/usr/lib64/liblapack.so.3 # lapack-3.4.2-8.el7.x86_64 BLAS_DIR = /usr #/usr/lib64/libblas.so.3 # blas-3.4.2-8.el7.x86_64 HDF5_DIR = /usr #/usr/lib64/libhdf5.so.8 # hdf5-1.8.12-13.el7.x86_64 HWLOC_DIR = /usr #/usr/lib64/libhwloc.so.5 # hwloc-1.11.8-4.el7.x86_64 JPEG_DIR = /usr #/usr/lib64/libjpeg.so.62 # YAML_DIR = /usr #/usr/lib64/libyaml-0.so.2 # /usr/lib64/libyaml-0.so.2.0. ZLIB_DIR = /usr #/usr/lib64/imlib2/loaders/zlib.so # zlib-1.2.7-21.el7_9.x86_64 GSL_DIR = /usr #/usr/lib64/libgsl.so # gsl-1.15-13.el7.x86_64 FFTW3_DIR = /usr #/usr/lib64/libfftw3.so # fftw-3.3.3-8.el7.x86_64 PAPI_DIR = /usr #/usr/lib64/libpapi.so.5.2.0.0 # papi-5.2.0-26.el7.x86_64 XML2_DIR = /usr #/usr/lib64/libxml2.so.2 # xml2-0.5-7.el7.x86_64 NUMA_DIR = /usr #/usr/lib64/libnuma.so.1 OPENSSL_DIR = /usr #/usr/lib64/libssl3.so # openssl-1.0.2k-26.el7_9.x86_64 Runscript: #!/bin/bash set -x set -e cd @SIMULATION_DIR@ # Environment setup source /opt/rh/devtoolset-8/enable module purge module load lamod/cmake/3.17 module load lamod/fftw/gnu/3.3.8 module load lamod/openmpi/gnu/4.1.0 module load libraries/gsl/2.6_gnu module load libraries/hdf5/1.10.5_gnu echo "Environment diagnostics:" date hostname env ldd @EXECUTABLE@ | grep -E "lapack|blas|openssl|stdc++|gfortran" # Set runtime parameters export CACTUS_NUM_PROCS=64 export CACTUS_NUM_THREADS=1 export OMP_NUM_THREADS=1 export GMON_OUT_PREFIX=gmon.out env | sort > SIMFACTORY/ENVIRONMENT echo "Starting simulation at $(date)" export CACTUS_STARTTIME=$(date +%s) mpirun -np $CACTUS_NUM_PROCS -x LD_LIBRARY_PATH @EXECUTABLE@ -L 3 @PARFILE@ echo "Simulation finished at $(date)" touch segment.done El jue, 18 sept 2025 a las 14:04, Steven Brandt () escribi?: > > On 9/17/2025 12:11 PM, Omar El?as Velasco Castillo wrote: > > Dear Einstein Toolkit team, > > I hope this message finds you well. I am a beginner with the Einstein > Toolkit. On personal workstations I have been able to compile and run > tutorial simulations at low resolution, but I am facing problems on two > different clusters. I would like to ask two questions: > > 1. *Are there ET versions prior to 2022_05 (e.g. 2019?2020 releases) that > can still be downloaded and compiled successfully?* When I try to fetch > them from the website using ./GetComponents, the process fails > (CactusSourceJar.git is not created and some components do not download). > Since some of the nodes I use have older GCC versions (8 or 10) and limited > modules, a stable older release might be more practical. > > 2. During compilation, I notice that thorns (such as GSL and HDF5, for > example) fall back to using the bundled versions because system modules are > not found. The build completes successfully, but jobs fail immediately > after submission to PBS or SLURM queues. > > Can you show us what the error message(s) are? > > > *What is the role of the bundled versions in this case*?* If the build > uses bundled GSL/HDF5, do I still need to load corresponding, compatible > modules in the runscript?* > > Could this mismatch explain why jobs die right after submission? > > Maybe. I'm not 100% sure what you are doing. Can you be clearer about how > you are running the ET? > > --Steve > > > Any advice on handling these issues would be very helpful. Thank you very > much for your time and support. > > Greetings, > > O.V. > > > > > > _______________________________________________ > Users mailing listUsers at einsteintoolkit.orghttp://lists.einsteintoolkit.org/mailman/listinfo/users > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mewesv at ornl.gov Fri Oct 3 16:11:57 2025 From: mewesv at ornl.gov (Mewes, Vassilios) Date: Fri, 3 Oct 2025 21:11:57 +0000 Subject: [Users] Post Doctoral Associate - Computational Astrophysics, The University of Tennessee, Knoxville Message-ID: Hi all, See the link below a Post Doctoral Associate - Computational Astrophysics opening at the nearby University of Tennessee in Knoxville. https://aas.org/jobregister/ad/56817693 Please share this with any interested candidates. Best wishes, Vassili -------------- next part -------------- An HTML attachment was scrubbed... URL: From keithdow at keithdow.net Sat Oct 4 10:14:09 2025 From: keithdow at keithdow.net (keithdow@keithdow.net keithdow@keithdow.net) Date: Sat, 4 Oct 2025 10:14:09 -0500 (EST) Subject: [Users] Meeting Minutes 2025-10-02 Message-ID: <2074204422.390149.1759590849550@webmail-oxcs.register.com> Minutes for the Einstein Toolkit Meeting, 20251002. https://docs.einsteintoolkit.org/et-docs/Main_Page#Weekly_Users_Call 9am CDT Thursdays Present: Peter D, Steve B, Deborah F, Roland H, Lucas S, Nikolai W, Zach E, Keith D. Chair: Peter D Minutes: Keith D * BBH gallery example ? Peter finished his part. Zach will follow up. * Github <-- Bitbucket (Cactus Code) Roland will contact the owner of the location Cactuscode on Github to see if we can obtain it to transfer CC from Bitbucket to Github. * ETK has made progress in becoming part of the SPEC Benchmark. Roland and Steve. * GW150914 Roland is to run again. * An issue with 56 threads being specified, from Iosif. Roland will respond. * Open tickets sorted https://bitbucket.org/einsteintoolkit/tickets/issues?status=open&status=new&sort=-updated_on #2866 Problem while building the ET on MN5: OpenCL Roland ?If you don?t need OpenCL I?d disable ExternalLibraries/OpenCL? #2889 Outflow uses incorrect ?sf centroid? There is a pull request. It is ready for review. Roland. #2597 Simfactory on Bridges2 No activity. Its from 2022 so it should be closed. #2888 Race condition writing properties.ini in simfactory No activity. #127 Restarts should have hard link to executable No activity. #1721 Simfactory?s test mechanism should use softlinks instead of rsync No activity. #1321 Collisions in executable cache directory No activity. #1798 add backtrace script to cactus utils No activity. #2706 Update default TwoPunctures partameters, or at least default parameters in BBH gallery example Zach is working on it. #2886 support cell centered directions when calling Fortran scheduled functions No activity. #2874 NRPyElliptic lacks regeneration instructions Zach is making progress. #2858 SF files for MN5 (was: Compiling Carpet X: Issues with PDESolvers) Not resolved. #2882 Running BBH with CarpetX with CPUs: High memory consumption and low performance ? #1847 FFTW3 fortran interface not working for system installation ? #2878 CarpetX: Add multipatch output support to Silo Waiting for review, Roland. #2764 PUGH tries to free memory not allocated by malloc Requires changes, Roland. #2848 updates to FLRWSolver No activity. #963 Improve McLachlan accuracy Peter is working on it. #1370 Provide a framework for simulation metadata ? #2877 ET_BHaHAHA Features Ticket Reminder for Zach. Work in progress. Needs formal review by Peter. #2855 Noise:X Improve noise generation in CarpetX No update. #2862 Update ?SpacetimeX/Z4c? robust stability parameter file. Needs review. #266 Peter to look at. #2364 Roland to look at closing. #2846 Steve to look at. #2814 Lucas to look at. #2179 Roland, needs review. #2786 Roland did work. Steve to review. New Item >From Roland. Volume integrals use a spherical surface. The issue is complicated. Zach and Roland to discuss. Next Meeting, Thursday, 9 October 2025. https://docs.einsteintoolkit.org/et-docs/Meeting_agenda -------------- next part -------------- An HTML attachment was scrubbed... URL: From deborah.ferguson at uri.edu Fri Oct 3 13:08:13 2025 From: deborah.ferguson at uri.edu (Deborah Ferguson) Date: Fri, 3 Oct 2025 18:08:13 +0000 Subject: [Users] Meeting minutes for 2025-09-11 Message-ID: September 11, 2025 Minutes Chair: Roland Present: Peter, Deborah, Lucas, Roland, Leo, Keith, Beyhan, Zach, Nikolai, Steve BBH example * Peter checked the machine he ran it on and realized the data was purged. Will rerun it. Zach and Peter looking into modernizing the gallery example * Including updates to mchlachlan * Roland asks whether we can update the data stored on Zenodo * Uncertain answer * Could create a new Zenodo * Increase resolution * We want it to either be production ready or at least be easy to modify to make it production ready * Likely leave it at low resolution but then document to ?change this value to XXX to make a production level run? Unanswered questions * Help with Spritz and FUKA * seems to have been solved * Roland will open a ticket to address it * add parameter in info file Open Tickets * 2706 * 2887 * Zach addressed Roland?s comments * Went through the changes live on the call * Roland approved it * 2886 * support cell centered directions in Fortran schedule functions * 2874 * NRPyElliptic lacks regeneration instructions * May need to address before new release depending on Zach?s NRPy changes * 2858 * Lucas working on it Tickets requesting review * 2887 * Roland just approved * 2878 * Roland still has to do AOB * Deborah shared video she made for LIGO press release * https://www.youtube.com/watch?v=hnzWNkjKkPU -------------------------------------- Deborah Ferguson Assistant Professor The University of Rhode Island -------------- next part -------------- An HTML attachment was scrubbed... URL: From users at einsteintoolkit.org Mon Oct 6 15:18:01 2025 From: users at einsteintoolkit.org (users at einsteintoolkit.org) Date: Mon, 06 Oct 2025 15:18:01 -0500 Subject: [Users] Agenda for Thursday's Meeting Message-ID: <68e423f9.KnqEErjI8fX1buSJ%users@einsteintoolkit.org> Please update the Wiki with agenda items for Thursday's meeting. Thanks! https://docs.einsteintoolkit.org/et-docs/meeting_agenda --The Maintainers