From rhaas at illinois.edu Mon Jan 6 15:18:01 2025 From: rhaas at illinois.edu (rhaas at illinois.edu) Date: Mon, 06 Jan 2025 15:18:01 -0600 Subject: [Users] Agenda for Thursday's Meeting Message-ID: Please update the Wiki with agenda items for Thursday's meeting. Thanks! https://docs.einsteintoolkit.org/et-docs/meeting_agenda --The Maintainers From rhaas at illinois.edu Wed Jan 8 17:15:05 2025 From: rhaas at illinois.edu (rhaas at illinois.edu) Date: Wed, 08 Jan 2025 17:15:05 -0600 Subject: [Users] Einstein Toolkit Meeting Reminder Message-ID: Hello, Please consider joining the weekly Einstein Toolkit phone call at 9:00 am US central time on Thursdays. For details on how to connect and what agenda items are to be discussed, use the link below. https://docs.einsteintoolkit.org/et-docs/Main_Page#Weekly_Users_Call --The Maintainers From rhaas at illinois.edu Thu Jan 9 10:02:33 2025 From: rhaas at illinois.edu (Roland Haas) Date: Thu, 9 Jan 2025 11:02:33 -0500 Subject: [Users] meeting minutes for 2025-01-09 Message-ID: <20250109110233.0fedf88b@illinois.edu> Present: Leo, Lucas, Maxwell, Roland, Steve, Zach simfactory issues on anvil ========================== * Maxwell had pointed out slowness of official files by a factor of 70 * Roland had tried to understand the root cause of the failure by modifying Maxwell's working files. So far is not exactly sure what is causing the issue * merge changes by Maxwell in to release simfactory LIBS issue in released Silo and PETSc ===================================== * Roland explained issue that causes build failures when configuring ExternalLibraries multiple times. Fixes exist in ticket: but needs testing upcoming ET possible inclusions =============================== * NewRadX boundary should be about ready, poke Cheng-Hsin * Z4c: want this in, Erik and Roland are champions, Steve and Peter are reviewers, needs testsuites * BHaHAHA Zach is still fine-tuning things, might have paper ready soon * CCE code should be in the works * Roland has to contact Hayley Macpherson release cycle ============= Zach brought up frequency of release, Steve suggests one could extend release cycle to 9 months in instead of 6 months. Roland mentions that time is fairly tight in between releases, even taking a "break" of 1 months means on is running late. Release preparation is quite a bit of work and makes it difficult to find volunteers. Lucas suggested reviewing the release process, trying to remove work items that are not always needed. Lucas suggests soliciting input and having a vote in the call, Roland will forward the suggestion to the mailing list Options for sequences are months of: 8 - 5 - 2 - 11 9 - 6 - 3 - 12 <-- vetoed by multiples 10 - 7 - 4 - 1 tickets ======= * https://bitbucket.org/einsteintoolkit/tickets/issues/2837 building-the-einstein-toolkit is help request * https://bitbucket.org/einsteintoolkit/tickets/issues/2843 update-fftw3-to-3310 looks correct to Leo * https://bitbucket.org/einsteintoolkit/tickets/issues/2840 supplement-rather-than-supercese-auto Roland reports that this helps with issues where either gsl-config or pkg-config report incorrect values * https://bitbucket.org/einsteintoolkit/tickets/issues/2839 look-for-multiple-flavors-of-hdf5-library is useful on macOS version tracking in ExternalLibraries ===================================== * Zach brought up the issue if tracking and guaranteeing specific minimum version of libraries provided by the ExternalLibraries * Roland points out that extracting the version number for an installed library is not always easy since it may involve having to parse a textual version string. Tools like gsl-info sometimes are lying. * An option to test for specific versions in C code are #if and #error preprocessor constructs Yours, Roland -- My email is as private as my paper mail. I therefore support encrypting and signing email messages. Get my PGP key from http://pgp.mit.edu . -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From rhaas at illinois.edu Thu Jan 9 13:57:04 2025 From: rhaas at illinois.edu (Roland Haas) Date: Thu, 9 Jan 2025 14:57:04 -0500 Subject: [Users] proposal to change Einstein Toolkit release cycle to be every 9 months instead every 6 months Message-ID: <20250109145704.50fad8f9@illinois.edu> Hello all, During today's Einstein Toolkit call the proposal to reduce the Einstein Toolkit release frequency from every 6 months to every 9 months has been made. This would reduce the workload for the release manager and avoid the current situation where there is hardly any time between work for releases. Some discussion on relative merits and disadvantages of the 6 and 9 month release cycle were discussed but, also due to the small number of participants, no final decision has been reached. As such, and in accordance with the Einstein Toolkit guiding principles http://einsteintoolkit.org/guidelines.html the topic will be discussed in the Einstein Toolkit call next week, Thu Jan 16th, 2025 and a vote for acceptance or rejection will be held at that point. Please see the meeting minutes for details on the discussion: https://lists.einsteintoolkit.org/pipermail/users/2025-January/009535.html Yours, Roland -- My email is as private as my paper mail. I therefore support encrypting and signing email messages. Get my PGP key from http://pgp.mit.edu . -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From rhaas at illinois.edu Mon Jan 13 15:18:01 2025 From: rhaas at illinois.edu (rhaas at illinois.edu) Date: Mon, 13 Jan 2025 15:18:01 -0600 Subject: [Users] Agenda for Thursday's Meeting Message-ID: Please update the Wiki with agenda items for Thursday's meeting. Thanks! https://docs.einsteintoolkit.org/et-docs/meeting_agenda --The Maintainers From rhaas at illinois.edu Wed Jan 15 17:15:07 2025 From: rhaas at illinois.edu (rhaas at illinois.edu) Date: Wed, 15 Jan 2025 17:15:07 -0600 Subject: [Users] Einstein Toolkit Meeting Reminder Message-ID: Hello, Please consider joining the weekly Einstein Toolkit phone call at 9:00 am US central time on Thursdays. For details on how to connect and what agenda items are to be discussed, use the link below. https://docs.einsteintoolkit.org/et-docs/Main_Page#Weekly_Users_Call --The Maintainers From lucas.t.s.carneiro at gmail.com Thu Jan 16 12:05:23 2025 From: lucas.t.s.carneiro at gmail.com (Lucas Timotheo Sanches) Date: Thu, 16 Jan 2025 12:05:23 -0600 Subject: [Users] Meeting minutes for 2025-01-16 Message-ID: Present: Lucas (Minutes), Steven R Brandt, Johnny Tsao, Maxwell Rizzo, Roland Hass (Chair), Erik Schnetter. # Vote to change the cadence of Einstein Toolkit releases. * After intense discussions it was decided (by the voting of those who were present) that the releases shall now be **every 12 months, every May** * It was agreed that this decision may be revisited in the future, in case the community finds it necessary. # Discussions on minor releases and continuous integration * The change of cadence prompted intense discussions on whether we should do point releases with important bug fixes in between releases. Several points were proposed, in particular the following ideas were proposed 1. Do not perform point releases, as this would be equivalent to having two releases a year. 2. Leave the decision to do point releases to the current release manager. 3. Adopt a CI (continuous integration) scheme that would allow us to know if any of the changes on the master branch are breaking in the machines that we support. 4. In order to ameliorate difficulties of running tests (even if automated) on certain machines, it was proposed that we adopt a system of champions and tiers. Each machine would have a champion, responsible for making sure that the ETK works correctly on that system. These would be Tier 1 machines, with full support and working guarantees. Other machines, where champions were out of touch or unable to get the toolkit working in time, would be called Tier 2 machines. * Erik suggested that we postpone this discussion for the next meetings, as the topic is broad and requires more thought. All members agreed. # Reproducibility issues with BNS * Wolfgang Tichy has obtained different results with different MPI ranks on BNS runs and reported them to Roland. Roland also did a bit of testing with the BNS gallery example, writing checkpoints at iteration 0 and 1. He sees differences between runs with 4 and 8 MPI ranks. Zach reminded Roland that Newton's method exhibits chaotic behavior, which his group observed on Illinois GMHD and may explain the results. Erik countered the argument saying we should assume that a bug is more likely. Roland says he needs to perform more tests, but he suspects that boundary condition interactions may be causing the problems. Zach suggested that Roland determines if the hydro quantities change before the metric changes, as hydro changes would take some time to propagate to metric components. Erik suggested checking different time levels. Erik also suggests comparing RHSs. Finally, Zach recommends trying running it through Valgrind, to check accidental memory issues. Roland took note of all suggestions and will implement them. # Upcoming Einstein Toolkit Release * NewRadX: Even though the code is completed, it still needs reviews and patches to be approved. * Z4C: Pending review by reviewers. * Cauchy characteristic extraction for Spectre CCE: Deborah Ferguson, who is the champion of this code, says that work on it is on her agenda but she is currently busy finishing a paper for her group. * Cosmology codes: Roland has reached out to Hayley Macpherson to see if she has any code that she would like to include. She has not reached back yet. * Gallery examples for more modules in the ETK: Roland reports that there are two such examples in the works based on Canuda for evolving the Einstein + various fields systems. Roland reports summer student projects for new gallery examples may not happen as the summer student budget was cut. # Unanswered question on the mailing list No questions. # Open tickets sorted by update time 2845: Small change for fixing C++ namespaces. 2837: Help request with not much progress since last week. # Tickets ready for review * Roland requests for reviewers, particularly those interested in CarpetX, to help on the review of the first few tickets such as 2845, which are relatively small and easy to review. From rhaas at illinois.edu Thu Jan 16 12:58:41 2025 From: rhaas at illinois.edu (Roland Haas) Date: Thu, 16 Jan 2025 13:58:41 -0500 Subject: [Users] Meeting minutes for 2025-01-16 In-Reply-To: References: Message-ID: <20250116135841.7d6fcf09@illinois.edu> Hello all, > # Reproducibility issues with BNS > * Wolfgang Tichy has obtained different results with different MPI ranks on Sorry, my bad. Correction: Michal Pirog (who is working with Wolfgang) noticed these. Yours, Roland -- My email is as private as my paper mail. I therefore support encrypting and signing email messages. Get my PGP key from http://pgp.mit.edu . -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From bruno.giacomazzo at unimib.it Sat Jan 18 05:23:13 2025 From: bruno.giacomazzo at unimib.it (Bruno Giacomazzo) Date: Sat, 18 Jan 2025 12:23:13 +0100 Subject: [Users] Meeting minutes for 2025-01-16 In-Reply-To: <20250116135841.7d6fcf09@illinois.edu> References: <20250116135841.7d6fcf09@illinois.edu> Message-ID: Regarding the BNS issues, is it something due to when you recover from checkpoint? In Whisky we had an issue related to that due to Con2Prim running after recovery from checkpoint. E.g., you run a simulation from iteration 0 to iteration 24 (made up numbers just to explain myself). You try the same simulation, but this time you run from 0 to 12, checkpoint, recover from checkpoint and run from 12 to 24. The results between these two simulations at iteration 24 were slightly different The issue was in the Newton Raphson being called by Con2Prim and using a different initial guess when recovering from checkpoint. During a run it uses the previous values of the primitive variables as an initial guess, but these are not available when restarting from checkpoint. The solution, as far as I remember, was not to run Con2Prim during recovery from checkpoint (since the primitive variables are anyway recovered from the checkpoint files). Cheers, Bruno Il giorno gio 16 gen 2025 alle ore 19:59 Roland Haas ha scritto: > Hello all, > > > # Reproducibility issues with BNS > > * Wolfgang Tichy has obtained different results with different MPI > ranks on > Sorry, my bad. Correction: Michal Pirog (who is working with Wolfgang) > noticed these. > > Yours, > Roland > > -- > My email is as private as my paper mail. I therefore support encrypting > and signing email messages. Get my PGP key from http://pgp.mit.edu . > _______________________________________________ > Users mailing list > Users at einsteintoolkit.org > http://lists.einsteintoolkit.org/mailman/listinfo/users > -- Prof. Bruno Giacomazzo Deputy Director of the Department of Physics University of Milano-Bicocca Piazza della Scienza 3 20126 Milano Italy email: bruno.giacomazzo at unimib.it phone: (+39) 02 6448 2321 web: http://www.brunogiacomazzo.org --------------------------------------------------------------------- There are only 10 types of people in the world: Those who understand binary, and those who don't ---------------------------------------------------------------------- -------------- next part -------------- An HTML attachment was scrubbed... URL: From rhaas at illinois.edu Sat Jan 18 09:56:15 2025 From: rhaas at illinois.edu (Roland Haas) Date: Sat, 18 Jan 2025 10:56:15 -0500 Subject: [Users] Meeting minutes for 2025-01-16 In-Reply-To: References: <20250116135841.7d6fcf09@illinois.edu> Message-ID: <20250118105615.4ca56667@illinois.edu> Hello all, Bruno, Before this takes on a live of its own: I re-did my own tests and found no more differences after a single Euler step. The issue that caused me to think there were differences before were: * I had looked at the wrong column in norms output and mistaken an "average" norm for a "maximum" norm (and the average will change due to different orders of values being summed up) * I still think I see a difference in values for points that are at the same location on different refinement levels (which should also not be in a vertex centered run), but those seem the same no matter whether I use 4 or 8 MPI ranks (also a bug if true, but of different kind) > Regarding the BNS issues, is it something due to when you recover from > checkpoint? Maybe, I don't know. > In Whisky we had an issue related to that due to Con2Prim running after > recovery from checkpoint. Yes, I had pointed it out in the past to Michal. For those one can set: MoL::run_MoL_PostStep_in_Post_Recover_Variables = "no" which as Bruno said, disables the extra con2prim after checkpoint recovery. Yours, Roland -- My email is as private as my paper mail. I therefore support encrypting and signing email messages. Get my PGP key from http://pgp.mit.edu . -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From rhaas at illinois.edu Mon Jan 20 15:18:01 2025 From: rhaas at illinois.edu (rhaas at illinois.edu) Date: Mon, 20 Jan 2025 15:18:01 -0600 Subject: [Users] Agenda for Thursday's Meeting Message-ID: Please update the Wiki with agenda items for Thursday's meeting. Thanks! https://docs.einsteintoolkit.org/et-docs/meeting_agenda --The Maintainers From rhaas at illinois.edu Wed Jan 22 17:15:01 2025 From: rhaas at illinois.edu (rhaas at illinois.edu) Date: Wed, 22 Jan 2025 17:15:01 -0600 Subject: [Users] Einstein Toolkit Meeting Reminder Message-ID: Hello, Please consider joining the weekly Einstein Toolkit phone call at 9:00 am US central time on Thursdays. For details on how to connect and what agenda items are to be discussed, use the link below. https://docs.einsteintoolkit.org/et-docs/Main_Page#Weekly_Users_Call --The Maintainers From wernecklr at gmail.com Thu Jan 23 09:52:01 2025 From: wernecklr at gmail.com (Leo Rosa Werneck) Date: Thu, 23 Jan 2025 07:52:01 -0800 Subject: [Users] Meeting minutes for 2025-01-23 Message-ID: <6B9D3B50-2E87-4F38-B27B-FAEF68855854@gmail.com> Hi all, Here are the minutes for today?s meeting. Chair: Lucas Minutes: Leo Present: Johnny, Leo, Lucas, Roland, Steve, Zach * Reproducibility of Data from Binary Neutron Star Simulations - Roland sees binary differences in Space Mask when comparing BNS gallery example with different MPI ranks, but otherwise they seem identical. - Observes issues in the post-merger, but inspiral looks the same. - Option MoL::run_MoL_PostStep_in_Post_Recover_Variables = "no" should make checkpoints bitwise identical. * Possible ET Inclusions - Z4c: no updates. - BHaHAHAH: needs reviewer. - CCE code to hook up with Specter: needs reviewer. - Updated FLRWSolver thorn: needs reviewer. - New gallery examples: need champions and reviewers. Leo will work on IllinoisGRMHD's and GRHayL's with David's help. * Unanswered Questions on the Mailing List None. * Open Tickets - #2282: Erik pointed out a potential double-counting issue, but Zach says that, upon reviewing, there should be no issue because the code assumes cell centering instead of vertex centering. Zach is opposed to using higher order integration methods on potentially noisy data. * Tickets Ready for Review - #2776: this helps on macOS using the system installed zlib. - #2364: avoid opening and closing HDF5 files multiple times during output. - #2846: added PreSync support in regrid. - #2842: Silo captures incorrect LIBS values on second CST run. - #2845: support (again) filesystem in std::experimental in Silo output. - #2843: update FFTW3 to 3.3.10. - #2821: update PETSc to 3.21.5. - #2841: update GSL tar file to version 2.8. - #2840: supplement rather than supersede auto-guessed data with gsl-config. - #2839: look for multiple flavors of hdf5 library names (Leo will look). * Lucas is experiencing an issue in CarpetX, but with suggestions form Steve and Roland he has a good idea of how to fix it. Next Chair: Peter Next Minutes: Roland Cheers, Leo ------ Leonardo R. Werneck, Ph.D. Postdoctoral researcher Office EP 314 | Department of Physics | University of Idaho 875 Perimeter Dr. MS 0903 Moscow, ID 83844-0903, USA leonardo at uidaho.edu https://leowerneck.github.io -------------- next part -------------- An HTML attachment was scrubbed... URL: From emost at caltech.edu Thu Jan 23 22:27:31 2025 From: emost at caltech.edu (Most, Elias R.) Date: Fri, 24 Jan 2025 04:27:31 +0000 Subject: [Users] Announcement: Caltech Relativistic Astrophysics Summer School 2025 Message-ID: Dear all, We are pleased to announce the Caltech Relativistic Astrophysics Summer School 2025, which will take place in Pasadena, CA from June 23-27, 2025. https://relastro-school-2025.caltech.edu High-energy astrophysics has made remarkable progress in describing some of the most extreme phenomena in the universe. Recent discoveries such as compact binary mergers and multi-messenger gravitational wave events, fast radio bursts, ultra-high energy cosmic rays, and advances in our understanding of the transient high energy sky (radio, X-ray, gamma-ray) open up new possibilities for forefront research in these rapidly evolving fields. This summer school aims to provide beginning and advanced graduate students in astrophysics, physics, and adjacent fields with the knowledge and tools to fully take advantage of these exciting developments. A particular focus of the school will be the application of emerging computational tools to provide high-fidelity models of high-energy astrophysical phenomena in strong gravity environments, e.g., around black holes or neutron stars. The one-week long summer school will include lectures on a broad range of topics: * Black hole astrophysics * Dynamo and turbulence * Magnetohydrodynamics * Multi-messenger gravitational wave events * Neutrino astrophysics * Neutron star magnetospheres * Numerical Relativity * Plasma kinetics Applications are open now. To receive full consideration, applications should be submitted by April 1, 2025. Partial financial support is available on a request basis. For the organizers, Elias R. Most, Sasha Philippov -------------- next part -------------- An HTML attachment was scrubbed... URL: From rhaas at illinois.edu Mon Jan 27 15:18:01 2025 From: rhaas at illinois.edu (rhaas at illinois.edu) Date: Mon, 27 Jan 2025 15:18:01 -0600 Subject: [Users] Agenda for Thursday's Meeting Message-ID: Please update the Wiki with agenda items for Thursday's meeting. Thanks! https://docs.einsteintoolkit.org/et-docs/meeting_agenda --The Maintainers From rhaas at illinois.edu Wed Jan 29 17:15:01 2025 From: rhaas at illinois.edu (rhaas at illinois.edu) Date: Wed, 29 Jan 2025 17:15:01 -0600 Subject: [Users] Einstein Toolkit Meeting Reminder Message-ID: Hello, Please consider joining the weekly Einstein Toolkit phone call at 9:00 am US central time on Thursdays. For details on how to connect and what agenda items are to be discussed, use the link below. https://docs.einsteintoolkit.org/et-docs/Main_Page#Weekly_Users_Call --The Maintainers From rhaas at illinois.edu Thu Jan 30 11:22:07 2025 From: rhaas at illinois.edu (Roland Haas) Date: Thu, 30 Jan 2025 12:22:07 -0500 Subject: [Users] meeting minutes for 2025-01-30 Message-ID: <20250130122207.78727300@illinois.edu> Present: Roland, Erik, Bill, Peter, Steve, Lucas, Zach reproducibility issues of data in BNS simulation ================================================ * Zach reports that IllinoisGRMHD also runs con2prim in post-recover variables * reports are that results depend on number of MPI ranks, maybe in IllinoisGRMHD or DNSData (SGRID) reader * Roland now has access to initial data files and code used, will test if initial data is read in differently on different MPI ranks and if checkpoint recovery is bit identical upcoming ET possible inclusions =============================== * Z4c, high priority ** test cases required ** https://bitbucket.org/einsteintoolkit/tickets/issues/2748/inclusion-of-z4c-in-einstein-toolkit * BHaHAHA horizon finder ** timeline will depend on workload for APS April meeting ** so far code is closed source and cannot yet be reviewed ** volunteers for reviewer are looked for, Peter may volunteer ** https://bitbucket.org/einsteintoolkit/tickets/issues/2850/add-bhahaha-to-einstein-toolkit * CCE code to hook up to Spectre ** work is restarting in earnest this week ** no ticket yet * possibly more modules for FLRWSolver ** Hayley is interested and has been provided with a timeline and will have items ready for initial review early March * gallery example for Canuda (Axi-dilaton) ** Lucas completed the review, currently online at: http://einsteintoolkit.org/gallery/axidilaton/index.html ** example is complete * gallery examples for Canuda, Fishbone-Moncrief, IllinoisGRMHD ** TBD Yours, Roland -- My email is as private as my paper mail. I therefore support encrypting and signing email messages. Get my PGP key from http://pgp.mit.edu . -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From bill.gabella at gmail.com Fri Jan 31 15:09:30 2025 From: bill.gabella at gmail.com (Bill Gabella) Date: Fri, 31 Jan 2025 16:09:30 -0500 Subject: [Users] libopenPMD missing, OpenPMD question Message-ID: <2a7a7628-d4c8-4cd9-9847-626c188b436e@gmail.com> Okay ran into a problem with "HelloWorld" with Simfactory when checking my ET install of master: ./simfactory/bin/sim create-run helloworld --parfile arrangements/CactusExamples/HelloWorld/par/HelloWorld.par [lots of stuff] /scratch/gabella/simulations/helloworld/SIMFACTORY/exe/cactus_sim: error while loading shared libraries: libopenPMD.so: cannot open shared object file: No such file or directory I see that that carpetX seems to have OpenPMD readers, etc.? I also found the github, if it is the right OpenPMD at https://github.com/openPMD/openPMD-api but expected ET to handle this or to see some text about installing OpenPMD as a pre-req. Any suggestions?? I never saw a file format that I did not like, or at least install, so should I install this, or should ET? thanks, bill e.g. ps--Did a find and there is no libopenPMD.so under Cactus. From rhaas at illinois.edu Fri Jan 31 15:53:22 2025 From: rhaas at illinois.edu (Roland Haas) Date: Fri, 31 Jan 2025 16:53:22 -0500 Subject: [Users] libopenPMD missing, OpenPMD question In-Reply-To: <2a7a7628-d4c8-4cd9-9847-626c188b436e@gmail.com> References: <2a7a7628-d4c8-4cd9-9847-626c188b436e@gmail.com> Message-ID: <20250131165322.627f9028@illinois.edu> Hello Bill, > [lots of stuff] > /scratch/gabella/simulations/helloworld/SIMFACTORY/exe/cactus_sim: error while loading shared libraries: libopenPMD.so: cannot open shared object file: No such file or directory Hmm, ExternalLibraries/openPMD which provides this (most likely, unless you are on a rare cluster that has it pre-installed) should set up an --rpath so that libopenPMD is found at runtime. Though admittedly it would also usually try to build a static library. The library would be in configs/sim/scratch/external/ eg: ET_trunk/configs/sim/scratch/external/openPMD/lib/libopenPMD.a in my case (note that this is the statically linked one). You you provide the file configs/sim/bindings/Configuration/Capabilities/make.OPENPMD_API.defn and you option list, please? > I see that that carpetX seems to have OpenPMD readers, etc.? I also found the github, if it is the right OpenPMD at > > https://urldefense.com/v3/__https://github.com/openPMD/openPMD-api__;!!DZ3f > jg!6chXrSK2Aob-tgiRz5UdTmcqIHLoXkWcH-hNaZhQhXY6V12RaAdQm2bSWB98IEudpHvF1rm5 > dBJcWQ5-QLCMww$ but expected ET to handle this or to see some text about installing OpenPMD as a pre-req. > > Any suggestions?? I never saw a file format that I did not like, or at > least install, so should I install this, or should ET? If you are not using CarpetX, then you can comment out the ExternalLibraries/OpenPMD line (and all CarpetX lines if you like) and thing will be fine. Yours, Roland -- My email is as private as my paper mail. I therefore support encrypting and signing email messages. Get my PGP key from http://pgp.mit.edu . -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From bill.gabella at gmail.com Fri Jan 31 16:28:47 2025 From: bill.gabella at gmail.com (Bill Gabella) Date: Fri, 31 Jan 2025 17:28:47 -0500 Subject: [Users] libopenPMD missing, OpenPMD question In-Reply-To: <20250131165322.627f9028@illinois.edu> References: <2a7a7628-d4c8-4cd9-9847-626c188b436e@gmail.com> <20250131165322.627f9028@illinois.edu> Message-ID: Roland, ? Thanks for the quick response. On 1/31/25 4:53 PM, Roland Haas wrote: > Hello Bill, > >> [lots of stuff] >> /scratch/gabella/simulations/helloworld/SIMFACTORY/exe/cactus_sim: error while loading shared libraries: libopenPMD.so: cannot open shared object file: No such file or directory > Hmm, ExternalLibraries/openPMD which provides this (most likely, unless > you are on a rare cluster that has it pre-installed) should set up an > --rpath so that libopenPMD is found at runtime. Though admittedly it > would also usually try to build a static library. > > The library would be in configs/sim/scratch/external/ eg: > > ET_trunk/configs/sim/scratch/external/openPMD/lib/libopenPMD.a Nope, I see in the Cactus/configs/sim/scratch/external archimedes/gabe :ls ./? ../? ADIOS2/? AMReX/? LORENE/? NSIMD/? SGRID/? Silo/ > > in my case (note that this is the statically linked one). > > You you provide the file > > configs/sim/bindings/Configuration/Capabilities/make.OPENPMD_API.defn > > and you option list, please? Attached make.OPENPMD_API.defn, oddly it has a line HAVE_CAPABILITY_openPMD_api = 1 Oh, I do seem to have it installed, in /usr/local/lib64, I think from a previous ET effort, dated 4 Sep 2024.? But I do not ever sudo installs with ET.? Odd.? Must have been hacking away trying to fix this problem then:)? But likely this is the problem. Also attaching the OptionList in Cactus/configs/sim. Was running helloworld with the create-submit in the Jupyter notebook, ./simfactory/bin/sim create-run helloworld \ ??? --parfile arrangements/CactusExamples/HelloWorld/par/HelloWorld.par > >> ./simfactory/bin/sim create-run helloworld \ >> --parfile arrangements/CactusExamples/HelloWorld/par/HelloWorld.parI see that that carpetX seems to have OpenPMD readers, etc.? I also found the github, if it is the right OpenPMD at >> >> https://urldefense.com/v3/__https://github.com/openPMD/openPMD-api__;!!DZ3f >> jg!6chXrSK2Aob-tgiRz5UdTmcqIHLoXkWcH-hNaZhQhXY6V12RaAdQm2bSWB98IEudpHvF1rm5 >> dBJcWQ5-QLCMww$ but expected ET to handle this or to see some text about installing OpenPMD as a pre-req. >> >> Any suggestions?? I never saw a file format that I did not like, or at >> least install, so should I install this, or should ET? > If you are not using CarpetX, then you can comment out the > ExternalLibraries/OpenPMD line (and all CarpetX lines if you like) and > thing will be fine. If I become motivated, I may want to play with CarpetX. Thanks, bill e.g. > > Yours, > Roland > -------------- next part -------------- include /scratch/gabella/Code/etoolkit/Cactus/configs/sim/bindings/Configuration/Capabilities/make.ADIOS2.defn include /scratch/gabella/Code/etoolkit/Cactus/configs/sim/bindings/Configuration/Capabilities/make.HDF5.defn include /scratch/gabella/Code/etoolkit/Cactus/configs/sim/bindings/Configuration/Capabilities/make.MPI.defn INC_DIRS += $(OPENPMD_INC_DIRS) INC_DIRS_F += $(OPENPMD_INC_DIRS) OPENPMD_BUILD = OPENPMD_ENABLE_FORTRAN = OFF OPENPMD_INSTALL_DIR = OPENPMD_DIR = /usr/local OPENPMD_ENABLE_FORTRAN = OFF OPENPMD_INC_DIRS = /usr/local/lib64 OPENPMD_LIB_DIRS = OPENPMD_LIBS = openPMD HAVE_CAPABILITY_openPMD_api = 1 -------------- next part -------------- # generic # This option list is used internally by simfactory as a template during the # sim setup and sim setup-silent commands # Edit at your own risk # Whenever this version string changes, the application is configured # and rebuilt from scratch VERSION = 2018-12-13 CPP = cpp CC = gcc CXX = g++ CUCC = nvcc FPP = cpp F90 = gfortran # for CarpetX GPU support # subset CUCCFLAGS without "-x cu" (must no be used here) and options used only to compile code # -fno-lto can avoid errors connected to fat binaries # LD = nvcc --compiler-bindir g++ -g -fno-lto --forward-unknown-to-host-compiler --relocatable-device-code=true --objdir-as-tempdir # LIBS = gfortran curand # enable for CarpetX GPU suport # CPPFLAGS = -DSIMD_CPU FPPFLAGS = -traditional # use gnu99 and not c99 since FFTW3 relies on gcc's "asm" function CFLAGS = -g -std=gnu99 # if compiler is old and you do not need AMReX, can change to c++11 # leave as c++17 for easier CUDA support CXXFLAGS = -g -std=c++17 -D_GNU_SOURCE CUCCFLAGS = --compiler-bindir g++ -x cu -g -std=c++17 -D_GNU_SOURCE --expt-relaxed-constexpr --extended-lambda --forward-unknown-to-host-compiler --Werror cross-execution-space-call --Werror ext-lambda-captures-this --relocatable-device-code=true --objdir-as-tempdir F90FLAGS = -g -fcray-pointer -ffixed-line-length-none LDFLAGS = -rdynamic # for CUDA support # LIBS = gfortran nvToolsExt curand DEBUG = no CPP_DEBUG_FLAGS = C_DEBUG_FLAGS = CXX_DEBUG_FLAGS = CUCC_DEBUG_FLAGS= OPTIMISE = yes CPP_OPTIMISE_FLAGS = C_OPTIMISE_FLAGS = -O2 CXX_OPTIMISE_FLAGS = -O2 CUCC_OPTIMISE_FLAGS= -O2 F90_OPTIMISE_FLAGS = -O2 PROFILE = no CPP_PROFILE_FLAGS = C_PROFILE_FLAGS = -pg CXX_PROFILE_FLAGS = -pg CUCC_PROFILE_FLAGS= -pg F90_PROFILE_FLAGS = -pg WARN = yes CPP_WARN_FLAGS = -Wall C_WARN_FLAGS = -Wall CXX_WARN_FLAGS = -Wall CUCC_WARN_FLAGS= -Wall F90_WARN_FLAGS = -Wall OPENMP = yes CPP_OPENMP_FLAGS = -fopenmp FPP_OPENMP_FLAGS = -D_OPENMP C_OPENMP_FLAGS = -fopenmp CXX_OPENMP_FLAGS = -fopenmp CUCC_OPENMP_FLAGS= -fopenmp F90_OPENMP_FLAGS = -fopenmp LD_OPENMP_FLAGS = -fopenmp # for CUDA support DISABLE_INT16=yes DISABLE_REAL16=yes VECTORISE = yes VECTORISE_ALIGNED_ARRAYS = no VECTORISE_INLINE = yes AMREX_ENABLE_CUDA = no PTHREADS_DIR = NO_BUILD From rhaas at illinois.edu Fri Jan 31 17:06:47 2025 From: rhaas at illinois.edu (Roland Haas) Date: Fri, 31 Jan 2025 18:06:47 -0500 Subject: [Users] libopenPMD missing, OpenPMD question In-Reply-To: References: <2a7a7628-d4c8-4cd9-9847-626c188b436e@gmail.com> <20250131165322.627f9028@illinois.edu> Message-ID: <20250131180647.6d530223@illinois.edu> Hello Bill, > > The library would be in configs/sim/scratch/external/ eg: > > > > ET_trunk/configs/sim/scratch/external/openPMD/lib/libopenPMD.a > > Nope, I see in the Cactus/configs/sim/scratch/external > > archimedes/gabe :ls > ./? ../? ADIOS2/? AMReX/? LORENE/? NSIMD/? SGRID/? Silo/ > > > > > in my case (note that this is the statically linked one). > > > > You you provide the file > > > > configs/sim/bindings/Configuration/Capabilities/make.OPENPMD_API.defn > > > > and you option list, please? > > Attached make.OPENPMD_API.defn, oddly it has a line > HAVE_CAPABILITY_openPMD_api = 1 Ok, so it left OPENPMD_LIBS_DIRS empty b/c /usr/local/lib64 is one of the "well known" directories that one assumes the system will automatically search in. Might not true on all systems and OSs (macOS comes to mind as usual) though. OPENMP_INC_DIRS = /usr/local/lib64 sounds a bit fishy to me, but seems be there for Fortran support. > Also attaching the OptionList in Cactus/configs/sim. Thank you. My suggestion would be to add -Wl,--rpath,/usr/local/lib64 to LDFLAGS or /usr/local/lib64 to the LD_LIBRARY_PATH environment variable. Are there any other libraries in /usr/local/lib64 ? You could try and see what directories are configured in /etc/ld.so.conf which on my Debian box ends up being: /usr/local/lib /usr/local/lib/x86_64-linux-gnu /lib/x86_64-linux-gnu /usr/lib/x86_64-linux-gnu > If I become motivated, I may want to play with CarpetX. ok. Yours, Roland -- My email is as private as my paper mail. I therefore support encrypting and signing email messages. Get my PGP key from http://pgp.mit.edu . -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: