Users

users@lists.einsteintoolkit.org

3 participants
3984 discussions

Meeting Minutes for 23-06-29
by Cupp, Samuel D. 29 Jun '23

29 Jun '23

Chair: Leo Minutes: Sam Present: Leo Sam Roland Yosef ET_2023_11 release People can begin considering possible inclusions for the ET_2023_11 release. There is still ~1 month before finalizing this list, but it would be good to start proposing these sooner rather than later. Roland: Will the Elliptica reader be proposed again? Leo sent email during previous release detailing goals for inclusion. We don't know if they plan to propose it again for this release. Ticket 2497 Leo and Sam made some suggestions, but it is still unclear what the desired behavior is. Because of how TmunuBase schedules things, it appears that setting stress_energy_at_RHS to "no" will never set the Tmunu variables, meaning they will be nan. SetTmunu should probably also be scheduled in CCTK_INITIAL to allow for fixed Tmunu. The scheduling would likely be similar to HydroBase_Prim2ConInitial. Roland will propose a possible solution in the ticket. Ticket 2738 Canuda tests are now working ET US Meeting: RIT Inn answered ambiguously about whether they provide a shuttle to/from airport or just to/from university. Yosef will clarify this with the hotel.

1 0

Fwd: SphericalSurfaces + QuasiLocalMeasures + AHFinderDirect
by Leonardo Werneck 28 Jun '23

28 Jun '23

Dear all, I am trying to create a simple parfile to help a PhD student. The goal is to set up puncture initial data with NRPyEllipticET and then use QuasiLocalMeasures/AHFinderDirect to compute properties of the punctures and compare with the expected results. No evolution thorn is expected/required and the simulation ends after the initial data has been set and the first round of analysis has finished. The parfile attached partially works. The puncture initial data is set up and NRPyEllipticET's output is what I'd expect. However, the diagnostics are clearly wrong since: 1) AHFinderDirect fails to find the puncture (in this example I am setting the mass of one of the punctures to zero); and 2) the QuasiLocalMeasures diagnostics is never triggered, despite being scheduled. Could someone provide some guidance on how to set up these thorns correctly, specifically SphericalSurfaces, QuasiLocalMeasures, and AHFinderDirect? Am I missing additional thorns? Also, if someone has a working TwoPuncture example parfile instead, please share it with me as I'd be happy to adapt it! Thanks in advance! Leo -- *------------------------------* *Dr. Leonardo R. Werneck, Ph.D.*

3 4

Re: [Users] Compilation Error in cluster ParamGanga
by Steven R. Brandt 19 Jun '23

19 Jun '23

I would talk to the system administrators about that. There is definitely something wrong with your compiler install if you can't find math.h. --Steve On 5/11/2023 2:02 AM, Shamim Haque 1910511 wrote: > Dear Steve, > > I just rechecked the cpp program to output M_PI in the computing > nodes, and that is not working right now, which was somehow working > fine a few days ago. I also tried different gcc versions (4.8, 10.2, > 11.1, gnu8/8.3) available in the cluster. All report similar issue: > > prog.cpp:1:18: fatal error: math.h: No such file or directory > #include <math.h> > ^ > compilation terminated. > > ----OR----- > > /home/apps/gcc-10.2.0/include/c++/10.2.0/x86_64-pc-linux-gnu/bits/os_defines.h:39:10: > fatal error: features.h: No such file or directory > 39 | #include <features.h> > | ^~~~~~~~~~~~ > compilation terminated. > > The cpp code seems to work fine in login node, as it gives out the > value of pi upon execution. I am contacting the hpc admin to resolve > this issue. > > I am attaching the test code (prog.cpp), the sbatch jobscript > (prog.sh) and the output file (slurn-117473.out), if you would like to > have a look at it. > > Please let me know if I should try something different to get this > working. > > Regards > Shamim Haque > Senior Research Fellow (SRF) > Department of Physics > IISER Bhopal > > ᐧ > > On Mon, May 8, 2023 at 7:25 PM Shamim Haque 1910511 > <shamims(a)iiserb.ac.in> wrote: > > Thanks Steve. I'll make the changes and try again. I'll let you > know the outcome. > > Regards > Shamim Haque > Senior Research Fellow (SRF) > Department of Physics > IISER Bhopal > > ᐧ > > On Mon, May 8, 2023 at 7:21 PM Steven R. Brandt > <sbrandt(a)cct.lsu.edu> wrote: > > That tends to make me think you aren't using the compiler you > think you're using. You could put full paths in the .cfg files > and recompile from scratch. > > i.e. > > CXX = /full/path/to/icc > > --Steve > > On 5/3/2023 3:26 PM, Shamim Haque 1910511 wrote: >> Yes, this gives the value of Pi successfully, before starting >> the compilation: >> >> //home/vkerni.iitr/sam/trial >> 3.141593 >> Current machine: paramganga3 >> Using configuration: sim >> Reconfiguring sim >> Writing configuration to: >> /home/vkerni.iitr/sam/ET_debug/Cactus/configs/sim/OptionList >> Cactus - version: 4.12.0 >> Reconfiguring sim. >> / >> /./ >> /./ >> /./ >> /checking for M_PI... no >> configure: error: M_PI not defined. Try adding >> -D_XOPEN_SOURCE to CPPFLAGS. >> >> Error reconfiguring sim-config >> make: *** [sim-config] Error 2 >> / >> >> >> Shamim Haque >> Senior Research Fellow (SRF) >> Department of Physics >> IISER Bhopal >> >> ᐧ >> >> On Thu, May 4, 2023 at 12:29 AM Steven Brandt >> <sbrandt(a)cct.lsu.edu> wrote: >> >> So, sanity check, can you compile the following program >> outside of Cactus? >> >> #include <math.h> >> #include <stdio.h> >> >> int main() { >> printf("%f\n", M_PI); >> return 0; >> } >> >> On 5/3/2023 1:50 PM, Shamim Haque 1910511 wrote: >>> Dear Steve, >>> >>> I tried gcc (10 and 11) as well and I am stuck with the >>> following error: >>> /configure: error: M_PI not defined. Try adding >>> -D_XOPEN_SOURCE to CPPFLAGS./ >>> / >>> / >>> This also happened for Intel (2020 and 2019). I >>> have attached both the output files here. >>> >>> Please ignore the out file in the first email, I have >>> uploaded another out file by mistake. The rest of the >>> files are correct in the first mail. >>> >>> Regards >>> Shamim Haque >>> Senior Research Fellow (SRF) >>> Department of Physics >>> IISER Bhopal >>> >>> ᐧ >>> >>> On Wed, May 3, 2023 at 8:04 PM Steven Brandt >>> <sbrandt(a)cct.lsu.edu> wrote: >>> >>> I notice from the output that your compiler claims >>> "limits.h" is missing. Possibly there's something >>> wrong with your icc setup? Maybe you should try >>> using gcc / gfortran instead. >>> >>> --Steve >>> >>> On 5/3/2023 8:20 AM, Shamim Haque 1910511 wrote: >>>> Dear Frank, >>>> >>>> I added the alias pattern for the computing nodes >>>> and now ETK can detect the cluster (checked by >>>> asking "whoami") during the batch queuing. I'll get >>>> in touch with the admins to include 'patch'. >>>> >>>> However, the problem still persists, having assured >>>> that the correct machine file and option list are >>>> being used during the compilation. >>>> >>>> Regards >>>> Shamim Haque >>>> Senior Research Fellow (SRF) >>>> Department of Physics >>>> IISER Bhopal >>>> >>>> ᐧ >>>> >>>> On Wed, May 3, 2023 at 3:49 PM Frank Loeffler >>>> <frank.loeffler(a)uni-jena.de> wrote: >>>> >>>> Hi Shamim >>>> >>>> On Wed, May 03, 2023 at 03:11:49PM +0530, >>>> Shamim Haque 1910511 wrote: >>>> >We are trying to compile ETK on ParamGanga at >>>> IIT Roorkee. The >>>> >compilation >>>> >stops at a very initial step and gives out the >>>> error: >>>> > >>>> >*checking whether the Fortran compiler >>>> (gfortran ) works... yeschecking >>>> >whether the Fortran compiler (gfortran ) is a >>>> cross-compiler... nochecking >>>> >whether the Fortran compiler (gfortran ) >>>> supports TYPE(*) for >>>> >CCTK_PointerTo... yeschecking how to run the C >>>> preprocessor... cppchecking >>>> >for ANSI C header files... nochecking for C99 >>>> features... yeschecking for >>>> >M_PI... noconfigure: error: M_PI not defined. >>>> Try adding -D_XOPEN_SOURCE to >>>> >CPPFLAGS.* >>>> > >>>> >We tried adding -D_XOPEN_SOURCE to CPPFLAGS, >>>> but it does not help. This >>>> >error is consistent if we set up using Intel >>>> Compiler (2020 or 2019), or >>>> >gcc (11 or 10). I have attached the outfile, >>>> error file, machine script, >>>> >and option lists for both cases (Intel and >>>> gcc) for any reference. >>>> > >>>> >Secondly, we are not allowed to compile ETK on >>>> login nodes. So we are >>>> >queuing a job script, which upon execution, >>>> enters the Cactus directory, >>>> >does "setup-silent", and then executes the >>>> build command. Is this the right >>>> >way to compile ETK using the batch script? Or >>>> should we do something >>>> >different? I have attached the shell script >>>> (compile.sh) used for sbatch >>>> >queue for reference. >>>> >>>> I might be wrong, but setup-silent will not use >>>> your machine file which >>>> contains all the compiler flags, but will setup >>>> something "from scratch" >>>> instead. According to the logs, this will be, >>>> e.g., in >>>> >>>> /home/shamims.iiserb/ET_debug/Cactus/repos/simfactory2/mdb/machines/cn139.iitr.ac.in.ini >>>> >>>> My guess would be that this misses the compiler >>>> flags necessary to have >>>> M_PI defined. The best way to get this working >>>> might be to let >>>> simfactory also detect the cluster >>>> configuration on the compute nodes >>>> (where you compile), such that setup-silent is >>>> not needed, but instead >>>> the correct cluster configuration is found and >>>> used automatically. In >>>> order to do that, look at examples of >>>> 'aliaspattern' in the mdb/machines >>>> directory. An alternative would be to tell >>>> simfactory specifically that >>>> you want to use your machine configuration file. >>>> >>>> As a side-note: I noticed the command 'patch' >>>> is missing too. This is a >>>> tool so common that it should be installed >>>> everywhere. Your admin is >>>> probably the best person to ask for advise >>>> here. It should not be too >>>> hard to install yourself, but that should not >>>> be necessary either, >>>> especially when you are told to compile on >>>> compute nodes. >>>> >>>> Frank >>>> >>>> >>>> _______________________________________________ >>>> Users mailing list >>>> Users(a)einsteintoolkit.org >>>> http://lists.einsteintoolkit.org/mailman/listinfo/users >>> _______________________________________________ >>> Users mailing list >>> Users(a)einsteintoolkit.org >>> http://lists.einsteintoolkit.org/mailman/listinfo/users >>>

3 11

meeting minutes for 2023-06-15
by Roland Haas 15 Jun '23

15 Jun '23

Present: Peter, Roland, Leo, Sam, Steve, Yosef open tickets: Ticket 2735 =========== * proposed simplification of schedule.ccl for ADMBase, HydroBase, and TmunuBase * Sam simplified the schedule making use of more modern STORAGE foo[timelevels] constructs * there is also checks if intial_FOO is "none" in which case no storage is allocated for FOO * Roland says that these should be retained Ticket 2742 =========== * proposed restructuring of Seed_Magnetic_Fields * Sam described the two Seed_Magnetic_Fields thorns in WVUThorns and WVUThorns_Diagnostics and desire to merge them into a single thorn * added schedule statements to make thorns use with just HydroBase rather than IllinoisGRMHD * PR is about to be created * plan is to deprecated Seed_Magnetic_Fields_BNS in the next release and remove the release after next ticket 2627 =========== * announce removal of TmunuBase inherting from ADMBase and StaticConformal * would remove more uses of StaticConformal and simplify code * will proceed with deprecation warning ticket 2497 =========== * Sam and Leo are in conversation with Vikram Manikantan about issues they face running IllinoisGRMHD code * Leo suggests to use magnetized TOV test as parfile to study this ticket 2741 =========== * some thorns seem to incorrectly use CCTK_GFINDEX4D * Sam replace some more instances of CCTK_GFINDEX4D with CCTK_VECTGFINDEX3D * Sam will close ticket ticket 2526 =========== * McLachlan has incorrect READ declarations for presync. * Ian is busy right now and is asking for others that could volunteer to review change to Kranc ET release: gallery examples ================ * BBH gallery example updated by Steve * Peter will update gallery example GRHayL library inclusion proposal in ET_2023_11 =============================================== * Ticket 2497, as scheduling doesn't mix con2prim and updating Tmunu * IllinoisGRMHD would no longer see feature updates, be marked as deprecated and eventually archived. * Peter suggests that a "transition guide" from IllinoisGRMHD to GRHayL be provided when IllinoisGRMHD is no longer updated. Sam has a draft of such a document written already. * Roland mentioned to carry IllinoisGRMHD along in the ET even after it no longer sees updates until maintaining it becomes prohibitive. At which point it would move to a archivedthorns-IllinoisGRMHD repository * this will not affect GiRaFFE process to deprecate features ============================= * Sam had asked about procedure * Release announcement will be official notification * process https://docs.einsteintoolkit.org/et-docs/Policies_to_retire_functionality release process updates ideas ============================= * Sam reported on issues he encountered some issues when creating the new release * Sam will update "unshallow" commands to download instructions * notes added to https://docs.einsteintoolkit.org/et-docs/Release_Process_Updates timing for ET_2023_11 meeting ============================= * schedule: https://docs.einsteintoolkit.org/et-docs/Release_Details * proposal deadline for new features would be weeks of RIT meeting * Sam (release manager) will shift choose features date back by one week, keeping other dates the same ET meetings: EU ET meeting ============= * Steve has been working on making the tutorial server a multi-node setup, will test after meeting * will skip ET call next week US ET meeting ============= * please check that you have received registration confirmation email if you registered. If no email was received then the registration is not complete. * RIT Inn reservation code sent at registration, if you already reserved the room without the reservation code, contact the organizers to have the code applied (and be able to check out on Saturday if so desired) * the reserved block will go away soon, please reserve your room soon * Yosef will check if hotel shuttle runs to the airport * Shuttle from RIT Inn to RIT is provided during extended business hours, shuttle runs by request chair week after next (June 29): Leo minutes week after next (June 29): Sam Yours, Roland -- My email is as private as my paper mail. I therefore support encrypting and signing email messages. Get my PGP key from http://pgp.mit.edu .

1 0

Meeting minutes for 2023-06-08
by Leonardo Rosa Werneck 08 Jun '23

08 Jun '23

Hi all, Here are the minutes for today’s meeting: Chair : Steve Minutes: Leo Present: Steve Brandt, Peter Diener, Zach Etienne, Roland Haas, Leo Werneck, Yosef Zlochower Nothing specific added to the agenda. * Next ET release manager search: - Sam Cupp is currently expected to be the lead release manager of the ET_11_2023 release, but he was not in today's telecon. Leo will ask him in person. * ET Summer School and Workshop: - Yosef said a discount code that allows folks to book their room at the RIT Inn has been sent to everyone who already registered. Participants are encouraged to book their rooms as soon as possible. As a reminder, the RIT Inn is the only hotel that offers free ground transportation to/from the RIT. - If someone booked their hotel earlier without using their code, they should reach out to Yosef so that they can see what can be done. - There are people who began their registration but never finished. Unfortunately the organizers do not have access to any information that identifies these people. They are encouraging everyone who wants to register to do so by completing the registration forms all the way through. * NRPy+ modernization: - Zach wants to announce that NRPy+ is undergoing an extensive modernization that will allow it to be pip installed. - Steve suggests all core NRPy+ modules should be moved inside a nrpy/ directory as soon as possible to avoid future issues. - Steve suggests a 'project class' that would hold all global variables (e.g., gridfunction list), but Zach does not think it is necessary since modules containing global variables contain very few of them. - This new version of NRPy+ (still under development) requires Python >=3.6, and thus supports f-strings and type hinting/mypy. Support for Python 2 will be dropped once this new version becomes the official version. * Unanswered question on the mailing list: - Bernard Kelly asked a question regarding whether or not ADMMacros should be removed from parameter files that use GRHydro for the MHD evolution. Steve is writing a reply to that question, which likely involves removing the ADMMacros thorn from the parameter file but updating the value of a new GRHydro parameter that replaces the one from ADMMacros. Next week chair : Peter Next week minutes: Roland Cheers, Leo ------ Leonardo R. Werneck, Ph.D. Postdoctoral researcher Office EP 314 | Department of Physics | University of Idaho 875 Perimeter Dr. MS 0903 Moscow, ID 83844-0903, USA leonardo(a)uidaho.edu <mailto:leonardo@uidaho.edu> https://leowerneck.github.io <https://leowerneck.github.io/>

1 0

ET_2023_05: Hydro_RNSID evolution example parfiles should drop ADMMacros
by Bernard Kelly 08 Jun '23

08 Jun '23

Hi. I was just looking at the Hydro_RNSID thorn in the latest ET release, and the test cases in par (all Perl-generated), which use GRHydro. Running one of these (e.g. speedA0_ppm_r12.par), I immediately hit an error: ------- PARAMETER ERROR (GRHydro): ADMMacros::spatial_order (4) differs from GRHydro::sources_spatial_order (2), this is almost certainly incorrect since GRHydro no longer uses ADMMacros PARAMETER ERROR (GRHydro): ADMMacros::spatial_order (4) differs from GRHydro::sources_spatial_order (2), this is almost certainly incorrect since GRHydro no longer uses ADMMacros -------- So can ADMMacros be explicitly removed from these Hydro_RNSID parameter files? Thanks, Bernard -- ------------------------------------------------------------------ Bernard Kelly -- CRESST Assistant Research Scientist, NASA/GSFC Gravitational Astrophysics Laboratory --- Code 663 Phone: +1 (301) 286-7243 *** Fax: +1 (301) 286-2226 Web: http://science.gsfc.nasa.gov/sed/bio/bernard.j.kelly ORCID: orcid.org/0000-0002-3326-4454 ------------------------------------------------------------------

2 1

2-year postdoc position in numerical relativity in Milan, Italy
by Bruno Giacomazzo 08 Jun '23

08 Jun '23

At the University of Milano-Bicocca <https://en.unimib.it/> (Milan, Italy), we have an opening for two postdoc positions "on the development of algorithms, codes and computational strategies for theoretical physics, theoretical astrophysics, and high-energy physics". Candidates with experience in *numerical relativity* are very strongly encouraged to apply. Candidates at the end of their PhD can also apply. *The deadline is June 28*. The candidate is expected to start by October. The salary will be around 2000 EUR per month (after taxes) and the position is for 2 years. Reference letters are not needed and the application has to be submitted on line at https://pica.cineca.it/unimib/23a058/ See the call document at https://www.unimib.it/ateneo/gare-e-concorsi/cod-23a058 for the instructions and the details of the selection procedure. Make sure to attach also the pdf of the published version (journal version) of all your papers with the application. Candidates that will be selected for the interview will be interviewed on 05/07/2023 at 10:00 (Rome Time). When submitting their application, candidates shall enter their e-mail/telematic contact (Skype, Cisco Webex Meetings, Google Hangouts or Google Meet) for the configuration of the online interview. The astrophysics group at Milano-Bicocca <https://astro.fisica.unimib.it/> provides a vibrant environment with expertise covering all aspects of gravitational-wave astronomy, relativistic astrophysics, galactic dynamics, and numerical relativity. This is embedded in a wider astronomical context including both observational and experimental activities. Our group has tight connections with the LISA Consortium, the Virgo Collaboration, the Einstein Telescope Science Board, the European Pulsar Timing Array, and the Italian National Institute for Nuclear Physics (INFN) via the TEONGRAV national initiative. Staff members in Gravitational Waves include Colpi, Dotti, Gerosa, Giacomazzo <https://www.brunogiacomazzo.org/>, and Sesana. Milan <https://www.yesmilano.it/en>is a beautiful, international city in the north of Italy. Mountains and lakes are just around the corner. Art, culture, food, and nightlife are outstanding. The city hosts three international airports with worldwide connections. Please forward this email to potential candidates. Thanks, Bruno -- Prof. Bruno Giacomazzo Department of Physics University of Milano-Bicocca Piazza della Scienza 3 20126 Milano Italy email: bruno.giacomazzo(a)unimib.it phone: (+39) 02 6448 2321 web: http://www.brunogiacomazzo.org --------------------------------------------------------------------- There are only 10 types of people in the world: Those who understand binary, and those who don't ----------------------------------------------------------------------

1 0

Thorn setup taking too much time in cluster
by Shamim Haque 1910511 06 Jun '23

06 Jun '23

Hello, I am trying to run BNSM using IllinoisGRMHD on HPC Kanad at IISER Bhopal. While I have tested the parfile to be running fine on debug queue (1 node) and high memory queue (3 nodes), I am unable to run the simulation in a queue with 9 nodes (144 cores). The output file suggests that the setup of listed thorns is not complete within 24 hours, which is the max walltime for this queue. Is there a way to sort out this issue? I have attached the parfile and outfile for reference. Regards Shamim Haque Senior Research Fellow (SRF) Department of Physics IISER Bhopal ᐧ

3 9

Today: NR community call at 9am Pacific / noon NYC / 5pm London / 6pm Berlin
by Nils Vu 05 Jun '23

05 Jun '23

1 0

meeting minutes for 2023-06-01
by Roland Haas 01 Jun '23

01 Jun '23

Present: Zach, Keith, Roland, Steve, Leo, Peter SPECv8 contribution =================== * no specific news, progressing ET release coordinator search ============================= * had planned to include CarpetX and AsterX * no documentation exists for it yet * Roland estimates that CarpetX is not closer to inclusion ready than 6 months ago. CarpetX reviewers * paired coordinator approach was useful for Leo in the 2023_05 release * will pick this up again next week due to small number of participants Unanswered question =================== * https://lists.einsteintoolkit.org/pipermail/users/2023-May/008930.html Roland will forward similar answer about grid structure consistency * https://lists.einsteintoolkit.org/pipermail/users/2023-May/008935.html Roland will respond suggesting that a full C++11 stack is required Error message shown about missing http://lists.einsteintoolkit.org/pipermail/users/2023-June/date.html is likely due there not yet being a posting to the mailing list for June and the script cannot distinguish "no posting yet" from "server is down or cannot access the file". Galley examples =============== * Steve will update the BBH example (https://www.einsteintoolkit.org/gallery/bbh/index.html) ** Zach mentioned ticket https://bitbucket.org/einsteintoolkit/tickets/issues/2706/increase-twopunct… * Peter had added convergence plot to TOV example (https://www.einsteintoolkit.org/gallery/ns/index.html). Convergence order is between 1 and 2. Zach brought up the issue that gallery examples are typically very low resolution and one often has to increase resolution significantly observed for example good convergence. This typically clashes with the desire to keep the gallery examples small enough so that they can be easily run. Zach noted that the TOV example is not featured in the cycler on the main page. Steve and Zach had a discussion about how one could create a more visually interesting image using VisIt. Steve brought up work on openPMD reader in Python. Roland will work with summer student in summer to add mesh refined openPMD support to VisIt. Will meet with Steve and (NCSA) local visualization expert. Updated tickets =============== * https://bitbucket.org/einsteintoolkit/tickets/issues/2734/consider-adding-i… ** Steve mentioned current situation in CarpetX where storage is always one. ** Steve considers that he may be able to add auto-allocation of storage to the WRITE statements, READ would result in invalid data to read from * https://bitbucket.org/einsteintoolkit/tickets/issues/2733/twopunctures-cont… ** we should fix this Open topics =========== * Clang and Flang support. Zach mentioned benchmarks that have clang outperform gcc, asks about support for clang and flang in simfactory and the Einstein Toolkit. ** Roland says no clang support in the past due to lack of Fortran compiler. Steve and Zach point out that gfortran can be used and the there is now also flang. ** Roland mentions that there are "old" flang based on PGI and "new" flang based on llvm. Also AMD compiler and Cray compiler on Crusher / Frontier are llvm based, so are modern version of the Intel compiler suite. ** currently have not knowledge about clang / flang in the Linux known-architecture files, only in Darwin files for (older) clang Next Chair: Steve Next Minutes Taker: Leo Yours, Roland -- My email is as private as my paper mail. I therefore support encrypting and signing email messages. Get my PGP key from http://pgp.mit.edu .

1 0

Jump to page:

2026

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

Users