From rhaas at illinois.edu Mon Jan 6 15:18:01 2025 From: rhaas at illinois.edu (rhaas at illinois.edu) Date: Mon, 06 Jan 2025 15:18:01 -0600 Subject: [Users] Agenda for Thursday's Meeting Message-ID: Please update the Wiki with agenda items for Thursday's meeting. Thanks! https://docs.einsteintoolkit.org/et-docs/meeting_agenda --The Maintainers From rhaas at illinois.edu Wed Jan 8 17:15:05 2025 From: rhaas at illinois.edu (rhaas at illinois.edu) Date: Wed, 08 Jan 2025 17:15:05 -0600 Subject: [Users] Einstein Toolkit Meeting Reminder Message-ID: Hello, Please consider joining the weekly Einstein Toolkit phone call at 9:00 am US central time on Thursdays. For details on how to connect and what agenda items are to be discussed, use the link below. https://docs.einsteintoolkit.org/et-docs/Main_Page#Weekly_Users_Call --The Maintainers From rhaas at illinois.edu Thu Jan 9 10:02:33 2025 From: rhaas at illinois.edu (Roland Haas) Date: Thu, 9 Jan 2025 11:02:33 -0500 Subject: [Users] meeting minutes for 2025-01-09 Message-ID: <20250109110233.0fedf88b@illinois.edu> Present: Leo, Lucas, Maxwell, Roland, Steve, Zach simfactory issues on anvil ========================== * Maxwell had pointed out slowness of official files by a factor of 70 * Roland had tried to understand the root cause of the failure by modifying Maxwell's working files. So far is not exactly sure what is causing the issue * merge changes by Maxwell in to release simfactory LIBS issue in released Silo and PETSc ===================================== * Roland explained issue that causes build failures when configuring ExternalLibraries multiple times. Fixes exist in ticket: but needs testing upcoming ET possible inclusions =============================== * NewRadX boundary should be about ready, poke Cheng-Hsin * Z4c: want this in, Erik and Roland are champions, Steve and Peter are reviewers, needs testsuites * BHaHAHA Zach is still fine-tuning things, might have paper ready soon * CCE code should be in the works * Roland has to contact Hayley Macpherson release cycle ============= Zach brought up frequency of release, Steve suggests one could extend release cycle to 9 months in instead of 6 months. Roland mentions that time is fairly tight in between releases, even taking a "break" of 1 months means on is running late. Release preparation is quite a bit of work and makes it difficult to find volunteers. Lucas suggested reviewing the release process, trying to remove work items that are not always needed. Lucas suggests soliciting input and having a vote in the call, Roland will forward the suggestion to the mailing list Options for sequences are months of: 8 - 5 - 2 - 11 9 - 6 - 3 - 12 <-- vetoed by multiples 10 - 7 - 4 - 1 tickets ======= * https://bitbucket.org/einsteintoolkit/tickets/issues/2837 building-the-einstein-toolkit is help request * https://bitbucket.org/einsteintoolkit/tickets/issues/2843 update-fftw3-to-3310 looks correct to Leo * https://bitbucket.org/einsteintoolkit/tickets/issues/2840 supplement-rather-than-supercese-auto Roland reports that this helps with issues where either gsl-config or pkg-config report incorrect values * https://bitbucket.org/einsteintoolkit/tickets/issues/2839 look-for-multiple-flavors-of-hdf5-library is useful on macOS version tracking in ExternalLibraries ===================================== * Zach brought up the issue if tracking and guaranteeing specific minimum version of libraries provided by the ExternalLibraries * Roland points out that extracting the version number for an installed library is not always easy since it may involve having to parse a textual version string. Tools like gsl-info sometimes are lying. * An option to test for specific versions in C code are #if and #error preprocessor constructs Yours, Roland -- My email is as private as my paper mail. I therefore support encrypting and signing email messages. Get my PGP key from http://pgp.mit.edu . -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From rhaas at illinois.edu Thu Jan 9 13:57:04 2025 From: rhaas at illinois.edu (Roland Haas) Date: Thu, 9 Jan 2025 14:57:04 -0500 Subject: [Users] proposal to change Einstein Toolkit release cycle to be every 9 months instead every 6 months Message-ID: <20250109145704.50fad8f9@illinois.edu> Hello all, During today's Einstein Toolkit call the proposal to reduce the Einstein Toolkit release frequency from every 6 months to every 9 months has been made. This would reduce the workload for the release manager and avoid the current situation where there is hardly any time between work for releases. Some discussion on relative merits and disadvantages of the 6 and 9 month release cycle were discussed but, also due to the small number of participants, no final decision has been reached. As such, and in accordance with the Einstein Toolkit guiding principles http://einsteintoolkit.org/guidelines.html the topic will be discussed in the Einstein Toolkit call next week, Thu Jan 16th, 2025 and a vote for acceptance or rejection will be held at that point. Please see the meeting minutes for details on the discussion: https://lists.einsteintoolkit.org/pipermail/users/2025-January/009535.html Yours, Roland -- My email is as private as my paper mail. I therefore support encrypting and signing email messages. Get my PGP key from http://pgp.mit.edu . -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From rhaas at illinois.edu Mon Jan 13 15:18:01 2025 From: rhaas at illinois.edu (rhaas at illinois.edu) Date: Mon, 13 Jan 2025 15:18:01 -0600 Subject: [Users] Agenda for Thursday's Meeting Message-ID: Please update the Wiki with agenda items for Thursday's meeting. Thanks! https://docs.einsteintoolkit.org/et-docs/meeting_agenda --The Maintainers From rhaas at illinois.edu Wed Jan 15 17:15:07 2025 From: rhaas at illinois.edu (rhaas at illinois.edu) Date: Wed, 15 Jan 2025 17:15:07 -0600 Subject: [Users] Einstein Toolkit Meeting Reminder Message-ID: Hello, Please consider joining the weekly Einstein Toolkit phone call at 9:00 am US central time on Thursdays. For details on how to connect and what agenda items are to be discussed, use the link below. https://docs.einsteintoolkit.org/et-docs/Main_Page#Weekly_Users_Call --The Maintainers From lucas.t.s.carneiro at gmail.com Thu Jan 16 12:05:23 2025 From: lucas.t.s.carneiro at gmail.com (Lucas Timotheo Sanches) Date: Thu, 16 Jan 2025 12:05:23 -0600 Subject: [Users] Meeting minutes for 2025-01-16 Message-ID: Present: Lucas (Minutes), Steven R Brandt, Johnny Tsao, Maxwell Rizzo, Roland Hass (Chair), Erik Schnetter. # Vote to change the cadence of Einstein Toolkit releases. * After intense discussions it was decided (by the voting of those who were present) that the releases shall now be **every 12 months, every May** * It was agreed that this decision may be revisited in the future, in case the community finds it necessary. # Discussions on minor releases and continuous integration * The change of cadence prompted intense discussions on whether we should do point releases with important bug fixes in between releases. Several points were proposed, in particular the following ideas were proposed 1. Do not perform point releases, as this would be equivalent to having two releases a year. 2. Leave the decision to do point releases to the current release manager. 3. Adopt a CI (continuous integration) scheme that would allow us to know if any of the changes on the master branch are breaking in the machines that we support. 4. In order to ameliorate difficulties of running tests (even if automated) on certain machines, it was proposed that we adopt a system of champions and tiers. Each machine would have a champion, responsible for making sure that the ETK works correctly on that system. These would be Tier 1 machines, with full support and working guarantees. Other machines, where champions were out of touch or unable to get the toolkit working in time, would be called Tier 2 machines. * Erik suggested that we postpone this discussion for the next meetings, as the topic is broad and requires more thought. All members agreed. # Reproducibility issues with BNS * Wolfgang Tichy has obtained different results with different MPI ranks on BNS runs and reported them to Roland. Roland also did a bit of testing with the BNS gallery example, writing checkpoints at iteration 0 and 1. He sees differences between runs with 4 and 8 MPI ranks. Zach reminded Roland that Newton's method exhibits chaotic behavior, which his group observed on Illinois GMHD and may explain the results. Erik countered the argument saying we should assume that a bug is more likely. Roland says he needs to perform more tests, but he suspects that boundary condition interactions may be causing the problems. Zach suggested that Roland determines if the hydro quantities change before the metric changes, as hydro changes would take some time to propagate to metric components. Erik suggested checking different time levels. Erik also suggests comparing RHSs. Finally, Zach recommends trying running it through Valgrind, to check accidental memory issues. Roland took note of all suggestions and will implement them. # Upcoming Einstein Toolkit Release * NewRadX: Even though the code is completed, it still needs reviews and patches to be approved. * Z4C: Pending review by reviewers. * Cauchy characteristic extraction for Spectre CCE: Deborah Ferguson, who is the champion of this code, says that work on it is on her agenda but she is currently busy finishing a paper for her group. * Cosmology codes: Roland has reached out to Hayley Macpherson to see if she has any code that she would like to include. She has not reached back yet. * Gallery examples for more modules in the ETK: Roland reports that there are two such examples in the works based on Canuda for evolving the Einstein + various fields systems. Roland reports summer student projects for new gallery examples may not happen as the summer student budget was cut. # Unanswered question on the mailing list No questions. # Open tickets sorted by update time 2845: Small change for fixing C++ namespaces. 2837: Help request with not much progress since last week. # Tickets ready for review * Roland requests for reviewers, particularly those interested in CarpetX, to help on the review of the first few tickets such as 2845, which are relatively small and easy to review. From rhaas at illinois.edu Thu Jan 16 12:58:41 2025 From: rhaas at illinois.edu (Roland Haas) Date: Thu, 16 Jan 2025 13:58:41 -0500 Subject: [Users] Meeting minutes for 2025-01-16 In-Reply-To: References: Message-ID: <20250116135841.7d6fcf09@illinois.edu> Hello all, > # Reproducibility issues with BNS > * Wolfgang Tichy has obtained different results with different MPI ranks on Sorry, my bad. Correction: Michal Pirog (who is working with Wolfgang) noticed these. Yours, Roland -- My email is as private as my paper mail. I therefore support encrypting and signing email messages. Get my PGP key from http://pgp.mit.edu . -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From bruno.giacomazzo at unimib.it Sat Jan 18 05:23:13 2025 From: bruno.giacomazzo at unimib.it (Bruno Giacomazzo) Date: Sat, 18 Jan 2025 12:23:13 +0100 Subject: [Users] Meeting minutes for 2025-01-16 In-Reply-To: <20250116135841.7d6fcf09@illinois.edu> References: <20250116135841.7d6fcf09@illinois.edu> Message-ID: Regarding the BNS issues, is it something due to when you recover from checkpoint? In Whisky we had an issue related to that due to Con2Prim running after recovery from checkpoint. E.g., you run a simulation from iteration 0 to iteration 24 (made up numbers just to explain myself). You try the same simulation, but this time you run from 0 to 12, checkpoint, recover from checkpoint and run from 12 to 24. The results between these two simulations at iteration 24 were slightly different The issue was in the Newton Raphson being called by Con2Prim and using a different initial guess when recovering from checkpoint. During a run it uses the previous values of the primitive variables as an initial guess, but these are not available when restarting from checkpoint. The solution, as far as I remember, was not to run Con2Prim during recovery from checkpoint (since the primitive variables are anyway recovered from the checkpoint files). Cheers, Bruno Il giorno gio 16 gen 2025 alle ore 19:59 Roland Haas ha scritto: > Hello all, > > > # Reproducibility issues with BNS > > * Wolfgang Tichy has obtained different results with different MPI > ranks on > Sorry, my bad. Correction: Michal Pirog (who is working with Wolfgang) > noticed these. > > Yours, > Roland > > -- > My email is as private as my paper mail. I therefore support encrypting > and signing email messages. Get my PGP key from http://pgp.mit.edu . > _______________________________________________ > Users mailing list > Users at einsteintoolkit.org > http://lists.einsteintoolkit.org/mailman/listinfo/users > -- Prof. Bruno Giacomazzo Deputy Director of the Department of Physics University of Milano-Bicocca Piazza della Scienza 3 20126 Milano Italy email: bruno.giacomazzo at unimib.it phone: (+39) 02 6448 2321 web: http://www.brunogiacomazzo.org --------------------------------------------------------------------- There are only 10 types of people in the world: Those who understand binary, and those who don't ---------------------------------------------------------------------- -------------- next part -------------- An HTML attachment was scrubbed... URL: From rhaas at illinois.edu Sat Jan 18 09:56:15 2025 From: rhaas at illinois.edu (Roland Haas) Date: Sat, 18 Jan 2025 10:56:15 -0500 Subject: [Users] Meeting minutes for 2025-01-16 In-Reply-To: References: <20250116135841.7d6fcf09@illinois.edu> Message-ID: <20250118105615.4ca56667@illinois.edu> Hello all, Bruno, Before this takes on a live of its own: I re-did my own tests and found no more differences after a single Euler step. The issue that caused me to think there were differences before were: * I had looked at the wrong column in norms output and mistaken an "average" norm for a "maximum" norm (and the average will change due to different orders of values being summed up) * I still think I see a difference in values for points that are at the same location on different refinement levels (which should also not be in a vertex centered run), but those seem the same no matter whether I use 4 or 8 MPI ranks (also a bug if true, but of different kind) > Regarding the BNS issues, is it something due to when you recover from > checkpoint? Maybe, I don't know. > In Whisky we had an issue related to that due to Con2Prim running after > recovery from checkpoint. Yes, I had pointed it out in the past to Michal. For those one can set: MoL::run_MoL_PostStep_in_Post_Recover_Variables = "no" which as Bruno said, disables the extra con2prim after checkpoint recovery. Yours, Roland -- My email is as private as my paper mail. I therefore support encrypting and signing email messages. Get my PGP key from http://pgp.mit.edu . -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From rhaas at illinois.edu Mon Jan 20 15:18:01 2025 From: rhaas at illinois.edu (rhaas at illinois.edu) Date: Mon, 20 Jan 2025 15:18:01 -0600 Subject: [Users] Agenda for Thursday's Meeting Message-ID: Please update the Wiki with agenda items for Thursday's meeting. Thanks! https://docs.einsteintoolkit.org/et-docs/meeting_agenda --The Maintainers