Users June 2023

users@lists.einsteintoolkit.org

13 participants
14 discussions

running TOVSolver standalone
by CJ Osakwe 07 Jul '23

07 Jul '23

Hello, I am a Ph.D. student working at the University of Calgary. I recently started using the Einstein Toolkit to conduct numerical relativity simulations for my thesis. As of now I am familiarizing myself with the software. In particular, I am trying to model a perturbed TOV neutron star. To this end, I used the TOVSolver thorn for my initial data (based off of the single, stable neutron star example). Where does the TOVSolver thorn output its data? I have so far been unable to determine this myself. I want to run this thorn by itself and am not sure how. Any help would be greatly appreciated. Cheers, CJ Osakwe

2 1

Only getting partial Waveform while running that parameter file in the gallery example for BBH GW150914
by Vinod Pisharody 07 Jul '23

07 Jul '23

Hello, I have been trying to run the simulation of GW150914 given in gallery .The simulation ran succesfully .However I am only getting a partial waveform .The merger and the ringdown phase is not being captured .I am attaching both the waveform and a gnuplot showing the completing of merger . There is also another doubt I have .In the gallery while creating the job for this simulation there is a command called --define N which is set to 28 .Could someone tell me whats the purpose of that and help me understand whats going wrong ? Thanks and Regards, Vinod

2 1

Failure to run static_tov simulation using more than 1 node on cluster
by Wei Sun 07 Jul '23

07 Jul '23

Hello, I am trying to run the static_tov simulation on my university's cluster (CRC Notre Dame <https://docs.crc.nd.edu/new_user/quick_start.html>). However, I got this error when acquiring 48 cores: Error: Too many nodes specified: nodes=2 (maxnodes is 1) I am assuming this is due to the *#Source tree management* module in the Cactus/simfactory/mdb/machines/<host>.ini file. In my case, the automatic *.ini file I got has these configuration: ppn = 24 max-num-threads = 24 num-threads = 12 nodes = 1 However, the machine I use has 24 cores for each node, and the number of threads for each core is 1. That means if I run the simulation using 48 cores, I need 2 nodes. Although I didn't get 2 hosts successfully(I got only 1), I am wondering if I have to edit the <host>.ini file after the command ./simfactory/bin/sim setup-silent by hand, for example: num-threads=1, nodes=2? Or is there any other correct way for getting more than 1 node? I would be appreciate it if you could give me some ideas on this problem. Best regards, Wei

3 2

PROBLEM RUNNING GW150914
by Jose Luis Rativa Medina 06 Jul '23

06 Jul '23

Hi, I'm trying to run the GW150914 <https://einsteintoolkit.org/gallery/bbh/index.html> simulation on my MSI GF63 9sc but the parameter file contains python code and I don't know how to run it. Thanks -- *José Luis Rativa Medina* *Estudiante ing. sistemas y computación* *Universidad Nacional de Colombia* -- *Aviso legal:* El contenido de este mensaje y los archivos adjuntos son confidenciales y de uso exclusivo de la Universidad Nacional de Colombia. Se encuentran dirigidos sólo para el uso del destinatario al cual van enviados. La reproducción, lectura y/o copia se encuentran prohibidas a cualquier persona diferente a este y puede ser ilegal. Si usted lo ha recibido por error, infórmenos y elimínelo de su correo. Los Datos Personales serán tratados conforme a la Ley 1581 de 2012 y a nuestra Política de Datos Personales que podrá consultar en la página web www.unal.edu.co <http://www.unal.edu.co/>.* *Las opiniones, informaciones, conclusiones y cualquier otro tipo de dato contenido en este correo electrónico, no relacionados con la actividad de la Universidad Nacional de Colombia, se entenderá como personales y de ninguna manera son avaladas por la Universidad.

2 1

Meeting Minutes for 23-06-29
by Cupp, Samuel D. 29 Jun '23

29 Jun '23

Chair: Leo Minutes: Sam Present: Leo Sam Roland Yosef ET_2023_11 release People can begin considering possible inclusions for the ET_2023_11 release. There is still ~1 month before finalizing this list, but it would be good to start proposing these sooner rather than later. Roland: Will the Elliptica reader be proposed again? Leo sent email during previous release detailing goals for inclusion. We don't know if they plan to propose it again for this release. Ticket 2497 Leo and Sam made some suggestions, but it is still unclear what the desired behavior is. Because of how TmunuBase schedules things, it appears that setting stress_energy_at_RHS to "no" will never set the Tmunu variables, meaning they will be nan. SetTmunu should probably also be scheduled in CCTK_INITIAL to allow for fixed Tmunu. The scheduling would likely be similar to HydroBase_Prim2ConInitial. Roland will propose a possible solution in the ticket. Ticket 2738 Canuda tests are now working ET US Meeting: RIT Inn answered ambiguously about whether they provide a shuttle to/from airport or just to/from university. Yosef will clarify this with the hotel.

1 0

Fwd: SphericalSurfaces + QuasiLocalMeasures + AHFinderDirect
by Leonardo Werneck 28 Jun '23

28 Jun '23

Dear all, I am trying to create a simple parfile to help a PhD student. The goal is to set up puncture initial data with NRPyEllipticET and then use QuasiLocalMeasures/AHFinderDirect to compute properties of the punctures and compare with the expected results. No evolution thorn is expected/required and the simulation ends after the initial data has been set and the first round of analysis has finished. The parfile attached partially works. The puncture initial data is set up and NRPyEllipticET's output is what I'd expect. However, the diagnostics are clearly wrong since: 1) AHFinderDirect fails to find the puncture (in this example I am setting the mass of one of the punctures to zero); and 2) the QuasiLocalMeasures diagnostics is never triggered, despite being scheduled. Could someone provide some guidance on how to set up these thorns correctly, specifically SphericalSurfaces, QuasiLocalMeasures, and AHFinderDirect? Am I missing additional thorns? Also, if someone has a working TwoPuncture example parfile instead, please share it with me as I'd be happy to adapt it! Thanks in advance! Leo -- *------------------------------* *Dr. Leonardo R. Werneck, Ph.D.*

3 4

Re: [Users] Compilation Error in cluster ParamGanga
by Steven R. Brandt 19 Jun '23

19 Jun '23

I would talk to the system administrators about that. There is definitely something wrong with your compiler install if you can't find math.h. --Steve On 5/11/2023 2:02 AM, Shamim Haque 1910511 wrote: > Dear Steve, > > I just rechecked the cpp program to output M_PI in the computing > nodes, and that is not working right now, which was somehow working > fine a few days ago. I also tried different gcc versions (4.8, 10.2, > 11.1, gnu8/8.3) available in the cluster. All report similar issue: > > prog.cpp:1:18: fatal error: math.h: No such file or directory > #include <math.h> > ^ > compilation terminated. > > ----OR----- > > /home/apps/gcc-10.2.0/include/c++/10.2.0/x86_64-pc-linux-gnu/bits/os_defines.h:39:10: > fatal error: features.h: No such file or directory > 39 | #include <features.h> > | ^~~~~~~~~~~~ > compilation terminated. > > The cpp code seems to work fine in login node, as it gives out the > value of pi upon execution. I am contacting the hpc admin to resolve > this issue. > > I am attaching the test code (prog.cpp), the sbatch jobscript > (prog.sh) and the output file (slurn-117473.out), if you would like to > have a look at it. > > Please let me know if I should try something different to get this > working. > > Regards > Shamim Haque > Senior Research Fellow (SRF) > Department of Physics > IISER Bhopal > > ᐧ > > On Mon, May 8, 2023 at 7:25 PM Shamim Haque 1910511 > <shamims(a)iiserb.ac.in> wrote: > > Thanks Steve. I'll make the changes and try again. I'll let you > know the outcome. > > Regards > Shamim Haque > Senior Research Fellow (SRF) > Department of Physics > IISER Bhopal > > ᐧ > > On Mon, May 8, 2023 at 7:21 PM Steven R. Brandt > <sbrandt(a)cct.lsu.edu> wrote: > > That tends to make me think you aren't using the compiler you > think you're using. You could put full paths in the .cfg files > and recompile from scratch. > > i.e. > > CXX = /full/path/to/icc > > --Steve > > On 5/3/2023 3:26 PM, Shamim Haque 1910511 wrote: >> Yes, this gives the value of Pi successfully, before starting >> the compilation: >> >> //home/vkerni.iitr/sam/trial >> 3.141593 >> Current machine: paramganga3 >> Using configuration: sim >> Reconfiguring sim >> Writing configuration to: >> /home/vkerni.iitr/sam/ET_debug/Cactus/configs/sim/OptionList >> Cactus - version: 4.12.0 >> Reconfiguring sim. >> / >> /./ >> /./ >> /./ >> /checking for M_PI... no >> configure: error: M_PI not defined. Try adding >> -D_XOPEN_SOURCE to CPPFLAGS. >> >> Error reconfiguring sim-config >> make: *** [sim-config] Error 2 >> / >> >> >> Shamim Haque >> Senior Research Fellow (SRF) >> Department of Physics >> IISER Bhopal >> >> ᐧ >> >> On Thu, May 4, 2023 at 12:29 AM Steven Brandt >> <sbrandt(a)cct.lsu.edu> wrote: >> >> So, sanity check, can you compile the following program >> outside of Cactus? >> >> #include <math.h> >> #include <stdio.h> >> >> int main() { >> printf("%f\n", M_PI); >> return 0; >> } >> >> On 5/3/2023 1:50 PM, Shamim Haque 1910511 wrote: >>> Dear Steve, >>> >>> I tried gcc (10 and 11) as well and I am stuck with the >>> following error: >>> /configure: error: M_PI not defined. Try adding >>> -D_XOPEN_SOURCE to CPPFLAGS./ >>> / >>> / >>> This also happened for Intel (2020 and 2019). I >>> have attached both the output files here. >>> >>> Please ignore the out file in the first email, I have >>> uploaded another out file by mistake. The rest of the >>> files are correct in the first mail. >>> >>> Regards >>> Shamim Haque >>> Senior Research Fellow (SRF) >>> Department of Physics >>> IISER Bhopal >>> >>> ᐧ >>> >>> On Wed, May 3, 2023 at 8:04 PM Steven Brandt >>> <sbrandt(a)cct.lsu.edu> wrote: >>> >>> I notice from the output that your compiler claims >>> "limits.h" is missing. Possibly there's something >>> wrong with your icc setup? Maybe you should try >>> using gcc / gfortran instead. >>> >>> --Steve >>> >>> On 5/3/2023 8:20 AM, Shamim Haque 1910511 wrote: >>>> Dear Frank, >>>> >>>> I added the alias pattern for the computing nodes >>>> and now ETK can detect the cluster (checked by >>>> asking "whoami") during the batch queuing. I'll get >>>> in touch with the admins to include 'patch'. >>>> >>>> However, the problem still persists, having assured >>>> that the correct machine file and option list are >>>> being used during the compilation. >>>> >>>> Regards >>>> Shamim Haque >>>> Senior Research Fellow (SRF) >>>> Department of Physics >>>> IISER Bhopal >>>> >>>> ᐧ >>>> >>>> On Wed, May 3, 2023 at 3:49 PM Frank Loeffler >>>> <frank.loeffler(a)uni-jena.de> wrote: >>>> >>>> Hi Shamim >>>> >>>> On Wed, May 03, 2023 at 03:11:49PM +0530, >>>> Shamim Haque 1910511 wrote: >>>> >We are trying to compile ETK on ParamGanga at >>>> IIT Roorkee. The >>>> >compilation >>>> >stops at a very initial step and gives out the >>>> error: >>>> > >>>> >*checking whether the Fortran compiler >>>> (gfortran ) works... yeschecking >>>> >whether the Fortran compiler (gfortran ) is a >>>> cross-compiler... nochecking >>>> >whether the Fortran compiler (gfortran ) >>>> supports TYPE(*) for >>>> >CCTK_PointerTo... yeschecking how to run the C >>>> preprocessor... cppchecking >>>> >for ANSI C header files... nochecking for C99 >>>> features... yeschecking for >>>> >M_PI... noconfigure: error: M_PI not defined. >>>> Try adding -D_XOPEN_SOURCE to >>>> >CPPFLAGS.* >>>> > >>>> >We tried adding -D_XOPEN_SOURCE to CPPFLAGS, >>>> but it does not help. This >>>> >error is consistent if we set up using Intel >>>> Compiler (2020 or 2019), or >>>> >gcc (11 or 10). I have attached the outfile, >>>> error file, machine script, >>>> >and option lists for both cases (Intel and >>>> gcc) for any reference. >>>> > >>>> >Secondly, we are not allowed to compile ETK on >>>> login nodes. So we are >>>> >queuing a job script, which upon execution, >>>> enters the Cactus directory, >>>> >does "setup-silent", and then executes the >>>> build command. Is this the right >>>> >way to compile ETK using the batch script? Or >>>> should we do something >>>> >different? I have attached the shell script >>>> (compile.sh) used for sbatch >>>> >queue for reference. >>>> >>>> I might be wrong, but setup-silent will not use >>>> your machine file which >>>> contains all the compiler flags, but will setup >>>> something "from scratch" >>>> instead. According to the logs, this will be, >>>> e.g., in >>>> >>>> /home/shamims.iiserb/ET_debug/Cactus/repos/simfactory2/mdb/machines/cn139.iitr.ac.in.ini >>>> >>>> My guess would be that this misses the compiler >>>> flags necessary to have >>>> M_PI defined. The best way to get this working >>>> might be to let >>>> simfactory also detect the cluster >>>> configuration on the compute nodes >>>> (where you compile), such that setup-silent is >>>> not needed, but instead >>>> the correct cluster configuration is found and >>>> used automatically. In >>>> order to do that, look at examples of >>>> 'aliaspattern' in the mdb/machines >>>> directory. An alternative would be to tell >>>> simfactory specifically that >>>> you want to use your machine configuration file. >>>> >>>> As a side-note: I noticed the command 'patch' >>>> is missing too. This is a >>>> tool so common that it should be installed >>>> everywhere. Your admin is >>>> probably the best person to ask for advise >>>> here. It should not be too >>>> hard to install yourself, but that should not >>>> be necessary either, >>>> especially when you are told to compile on >>>> compute nodes. >>>> >>>> Frank >>>> >>>> >>>> _______________________________________________ >>>> Users mailing list >>>> Users(a)einsteintoolkit.org >>>> http://lists.einsteintoolkit.org/mailman/listinfo/users >>> _______________________________________________ >>> Users mailing list >>> Users(a)einsteintoolkit.org >>> http://lists.einsteintoolkit.org/mailman/listinfo/users >>>

3 11

meeting minutes for 2023-06-15
by Roland Haas 15 Jun '23

15 Jun '23

Present: Peter, Roland, Leo, Sam, Steve, Yosef open tickets: Ticket 2735 =========== * proposed simplification of schedule.ccl for ADMBase, HydroBase, and TmunuBase * Sam simplified the schedule making use of more modern STORAGE foo[timelevels] constructs * there is also checks if intial_FOO is "none" in which case no storage is allocated for FOO * Roland says that these should be retained Ticket 2742 =========== * proposed restructuring of Seed_Magnetic_Fields * Sam described the two Seed_Magnetic_Fields thorns in WVUThorns and WVUThorns_Diagnostics and desire to merge them into a single thorn * added schedule statements to make thorns use with just HydroBase rather than IllinoisGRMHD * PR is about to be created * plan is to deprecated Seed_Magnetic_Fields_BNS in the next release and remove the release after next ticket 2627 =========== * announce removal of TmunuBase inherting from ADMBase and StaticConformal * would remove more uses of StaticConformal and simplify code * will proceed with deprecation warning ticket 2497 =========== * Sam and Leo are in conversation with Vikram Manikantan about issues they face running IllinoisGRMHD code * Leo suggests to use magnetized TOV test as parfile to study this ticket 2741 =========== * some thorns seem to incorrectly use CCTK_GFINDEX4D * Sam replace some more instances of CCTK_GFINDEX4D with CCTK_VECTGFINDEX3D * Sam will close ticket ticket 2526 =========== * McLachlan has incorrect READ declarations for presync. * Ian is busy right now and is asking for others that could volunteer to review change to Kranc ET release: gallery examples ================ * BBH gallery example updated by Steve * Peter will update gallery example GRHayL library inclusion proposal in ET_2023_11 =============================================== * Ticket 2497, as scheduling doesn't mix con2prim and updating Tmunu * IllinoisGRMHD would no longer see feature updates, be marked as deprecated and eventually archived. * Peter suggests that a "transition guide" from IllinoisGRMHD to GRHayL be provided when IllinoisGRMHD is no longer updated. Sam has a draft of such a document written already. * Roland mentioned to carry IllinoisGRMHD along in the ET even after it no longer sees updates until maintaining it becomes prohibitive. At which point it would move to a archivedthorns-IllinoisGRMHD repository * this will not affect GiRaFFE process to deprecate features ============================= * Sam had asked about procedure * Release announcement will be official notification * process https://docs.einsteintoolkit.org/et-docs/Policies_to_retire_functionality release process updates ideas ============================= * Sam reported on issues he encountered some issues when creating the new release * Sam will update "unshallow" commands to download instructions * notes added to https://docs.einsteintoolkit.org/et-docs/Release_Process_Updates timing for ET_2023_11 meeting ============================= * schedule: https://docs.einsteintoolkit.org/et-docs/Release_Details * proposal deadline for new features would be weeks of RIT meeting * Sam (release manager) will shift choose features date back by one week, keeping other dates the same ET meetings: EU ET meeting ============= * Steve has been working on making the tutorial server a multi-node setup, will test after meeting * will skip ET call next week US ET meeting ============= * please check that you have received registration confirmation email if you registered. If no email was received then the registration is not complete. * RIT Inn reservation code sent at registration, if you already reserved the room without the reservation code, contact the organizers to have the code applied (and be able to check out on Saturday if so desired) * the reserved block will go away soon, please reserve your room soon * Yosef will check if hotel shuttle runs to the airport * Shuttle from RIT Inn to RIT is provided during extended business hours, shuttle runs by request chair week after next (June 29): Leo minutes week after next (June 29): Sam Yours, Roland -- My email is as private as my paper mail. I therefore support encrypting and signing email messages. Get my PGP key from http://pgp.mit.edu .

1 0

Meeting minutes for 2023-06-08
by Leonardo Rosa Werneck 08 Jun '23

08 Jun '23

Hi all, Here are the minutes for today’s meeting: Chair : Steve Minutes: Leo Present: Steve Brandt, Peter Diener, Zach Etienne, Roland Haas, Leo Werneck, Yosef Zlochower Nothing specific added to the agenda. * Next ET release manager search: - Sam Cupp is currently expected to be the lead release manager of the ET_11_2023 release, but he was not in today's telecon. Leo will ask him in person. * ET Summer School and Workshop: - Yosef said a discount code that allows folks to book their room at the RIT Inn has been sent to everyone who already registered. Participants are encouraged to book their rooms as soon as possible. As a reminder, the RIT Inn is the only hotel that offers free ground transportation to/from the RIT. - If someone booked their hotel earlier without using their code, they should reach out to Yosef so that they can see what can be done. - There are people who began their registration but never finished. Unfortunately the organizers do not have access to any information that identifies these people. They are encouraging everyone who wants to register to do so by completing the registration forms all the way through. * NRPy+ modernization: - Zach wants to announce that NRPy+ is undergoing an extensive modernization that will allow it to be pip installed. - Steve suggests all core NRPy+ modules should be moved inside a nrpy/ directory as soon as possible to avoid future issues. - Steve suggests a 'project class' that would hold all global variables (e.g., gridfunction list), but Zach does not think it is necessary since modules containing global variables contain very few of them. - This new version of NRPy+ (still under development) requires Python >=3.6, and thus supports f-strings and type hinting/mypy. Support for Python 2 will be dropped once this new version becomes the official version. * Unanswered question on the mailing list: - Bernard Kelly asked a question regarding whether or not ADMMacros should be removed from parameter files that use GRHydro for the MHD evolution. Steve is writing a reply to that question, which likely involves removing the ADMMacros thorn from the parameter file but updating the value of a new GRHydro parameter that replaces the one from ADMMacros. Next week chair : Peter Next week minutes: Roland Cheers, Leo ------ Leonardo R. Werneck, Ph.D. Postdoctoral researcher Office EP 314 | Department of Physics | University of Idaho 875 Perimeter Dr. MS 0903 Moscow, ID 83844-0903, USA leonardo(a)uidaho.edu <mailto:leonardo@uidaho.edu> https://leowerneck.github.io <https://leowerneck.github.io/>

1 0

ET_2023_05: Hydro_RNSID evolution example parfiles should drop ADMMacros
by Bernard Kelly 08 Jun '23

08 Jun '23

Hi. I was just looking at the Hydro_RNSID thorn in the latest ET release, and the test cases in par (all Perl-generated), which use GRHydro. Running one of these (e.g. speedA0_ppm_r12.par), I immediately hit an error: ------- PARAMETER ERROR (GRHydro): ADMMacros::spatial_order (4) differs from GRHydro::sources_spatial_order (2), this is almost certainly incorrect since GRHydro no longer uses ADMMacros PARAMETER ERROR (GRHydro): ADMMacros::spatial_order (4) differs from GRHydro::sources_spatial_order (2), this is almost certainly incorrect since GRHydro no longer uses ADMMacros -------- So can ADMMacros be explicitly removed from these Hydro_RNSID parameter files? Thanks, Bernard -- ------------------------------------------------------------------ Bernard Kelly -- CRESST Assistant Research Scientist, NASA/GSFC Gravitational Astrophysics Laboratory --- Code 663 Phone: +1 (301) 286-7243 *** Fax: +1 (301) 286-2226 Web: http://science.gsfc.nasa.gov/sed/bio/bernard.j.kelly ORCID: orcid.org/0000-0002-3326-4454 ------------------------------------------------------------------

2 1

2026

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

Users June 2023