<html xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:Helvetica;
panose-1:0 0 0 0 0 0 0 0 0 0;}
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Verdana;
panose-1:2 11 6 4 3 5 4 4 2 4;}
@font-face
{font-family:Aptos;
panose-1:2 11 0 4 2 2 2 2 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
font-size:12.0pt;
font-family:"Aptos",sans-serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
span.EmailStyle19
{mso-style-type:personal-reply;
font-family:"Aptos",sans-serif;
color:windowtext;}
p.gmail-p1, li.gmail-p1, div.gmail-p1
{mso-style-name:gmail-p1;
mso-margin-top-alt:auto;
margin-right:0in;
mso-margin-bottom-alt:auto;
margin-left:0in;
font-size:12.0pt;
font-family:"Aptos",sans-serif;}
span.gmail-apple-converted-space
{mso-style-name:gmail-apple-converted-space;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;
mso-ligatures:none;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style>
</head>
<body lang="EN-US" link="blue" vlink="purple" style="word-wrap:break-word">
<div class="WordSection1">
<p class="MsoNormal"><b><span style="font-family:"Verdana",sans-serif">Meeting minutes – Feb 20, 2025 (12PM ET)<o:p></o:p></span></b></p>
<p class="MsoNormal"><span style="font-family:"Verdana",sans-serif"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Verdana",sans-serif">Present: Cheng-Hsin, Erik, Jay, Liwei, Lucas, Roland, Swapnil, Yosef, Zachariah<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Verdana",sans-serif"><o:p> </o:p></span></p>
<p class="MsoNormal"><b><span style="font-family:"Verdana",sans-serif">Roland</span></b><span style="font-family:"Verdana",sans-serif">:
<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Verdana",sans-serif">*Merged the pull request for symmetry BCs in NewRadX. Test the thorn. <o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Verdana",sans-serif">*Subcycling mostly done except for writing the paper. See how to get these to main branch -> would be good to have multiple commits with separate feature implementations. Liwei says it is already
the case. <o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Verdana",sans-serif">*Roland asks to scrutinize the optimizations to make sure they are safe.
<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Verdana",sans-serif">*Roland commented that Liwei's pull request looks gigantic -> Liwei says it is due to unit tests. <o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Verdana",sans-serif">*Roland trying to craft test cases for Z4c to include in ET. Will try and see if same ones as McLachlan can be used. Also a qc0 test in Z4c that curretly fails. Some work that needs to be done
for subsyling -> Granularity. Ronald is not running Z4c from spacetimeX (which works), but directly from ET. <o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Verdana",sans-serif"><o:p> </o:p></span></p>
<p class="MsoNormal"><b><span style="font-family:"Verdana",sans-serif">Cheng-Hsin</span></b><span style="font-family:"Verdana",sans-serif">: Could not provide updates due to audio issues. <o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Verdana",sans-serif"><o:p> </o:p></span></p>
<p class="MsoNormal"><b><span style="font-family:"Verdana",sans-serif">Zachariah</span></b><span style="font-family:"Verdana",sans-serif">:
<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Verdana",sans-serif">*Roland says try and see if BHaHAHA thorn works with CarpetX. Zach testing it.
<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Verdana",sans-serif">*In the BBH mode -> finds specific horizons and when BHs get closer, starts searching for a common horizon and disables searching for different horizons. Seems to be working correctly. <o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Verdana",sans-serif"><o:p> </o:p></span></p>
<p class="MsoNormal"><b><span style="font-family:"Verdana",sans-serif">Lucas</span></b><span style="font-family:"Verdana",sans-serif">:
<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Verdana",sans-serif">*Roland asks Lucas about the curvilinear coordinate work and Hybrid time integrators. <o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Verdana",sans-serif">*Lucas says that Multipatch seems to be working correctly in CapyrX. Successfully ran qc0 and extracted multipole. Ran for 200-300 timesteps, need to run longer for useful results. Interpatch
communication seems to be working. Exciting! <o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Verdana",sans-serif">*There is some trouble with openpmd output. CPU clusters with spack give warning related to BP5 and produce corrupted files. Suspect something wrong with adios that comes with spack. Try to
use Silo, Erik will look into it. <o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Verdana",sans-serif">*No big updates on hybrid time integrators. Currently, tuning methods to maximize CFL while keeping acceptable results. <o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Verdana",sans-serif"> <o:p></o:p></span></p>
<p class="MsoNormal"><b><span style="font-family:"Verdana",sans-serif">Jay</span></b><span style="font-family:"Verdana",sans-serif">:
<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Verdana",sans-serif">*Doing scaling test with asterX on frontier using subcyling. Sometimes get output (e.g cell updates per second) which is not consistent across different iterations. Even an order of magnitude
difference at times. <o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Verdana",sans-serif">*Could be due to processor mismatch. Roland suggests using barrier at the end of every timestep. Erik says carpetX already has barriers at the end of every timestep, and function in schedule.cxx
does MPI_AllReduce which is a global reduction. Outputs in the evolution loop:<o:p></o:p></span></p>
<p class="gmail-p1" style="margin:0in;font-variant-numeric: normal;font-variant-east-asian: normal;font-variant-alternates: normal;font-size-adjust: none;font-kerning: auto;font-optical-sizing: auto;font-feature-settings: normal;font-variation-settings: normal;font-variant-position: normal;font-stretch: normal">
<span style="font-size:9.0pt;font-family:Helvetica">CCTK_VINFO("Simulation time: %g <span class="gmail-apple-converted-space"> </span>"<o:p></o:p></span></p>
<p class="gmail-p1" style="margin:0in;font-variant-numeric: normal;font-variant-east-asian: normal;font-variant-alternates: normal;font-size-adjust: none;font-kerning: auto;font-optical-sizing: auto;font-feature-settings: normal;font-variation-settings: normal;font-variant-position: normal;font-stretch: normal">
<span class="gmail-apple-converted-space"><span style="font-size:9.0pt;font-family:Helvetica"> </span></span><span style="font-size:9.0pt;font-family:Helvetica">"Iterations per second: %g <span class="gmail-apple-converted-space"> </span>"<o:p></o:p></span></p>
<p class="MsoNormal"><span class="gmail-apple-converted-space"><span style="font-size:9.0pt;font-family:Helvetica"> </span></span><span style="font-size:9.0pt;font-family:Helvetica">"Simulation time per second: %g",</span><span style="font-family:"Verdana",sans-serif"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Verdana",sans-serif"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Verdana",sans-serif">*Erik suggests to use parameter </span><span style="font-size:9.0pt;font-family:Helvetica">out_performance. </span><span style="font-family:"Verdana",sans-serif">Discussed the code further. <o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Verdana",sans-serif"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Verdana",sans-serif">*Roland updated on the UIUC workshop 1st-2nd week of August. <o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Verdana",sans-serif"><o:p> </o:p></span></p>
<p class="MsoNormal"><b><span style="font-size:11.0pt"><o:p> </o:p></span></b></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<div id="mail-editor-reference-message-container">
<div>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal" style="margin-bottom:12.0pt"><b><span style="color:black">From:
</span></b><span style="color:black">Carpetx-developers <carpetx-developers-bounces@einsteintoolkit.org> on behalf of rhaas@illinois.edu <rhaas@illinois.edu><br>
<b>Date: </b>Wednesday, February 19, 2025 at 5:08</span><span style="font-family:"Arial",sans-serif;color:black"> </span><span style="color:black">PM<br>
<b>To: </b>carpetx-developers@lists.einsteintoolkit.org <carpetx-developers@lists.einsteintoolkit.org>, rhaas80@gmail.com <rhaas80@gmail.com><br>
<b>Subject: </b>[Carpetx-developers] Weekly call at 12pm Thursday Eastern time<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt">Title: CarpetX developer call<br>
When: 12pm Thursday Eastern time<br>
<br>
Join Zoom Meeting<br>
<a href="https://pitp.zoom.us/j/93861697232">https://eur04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fpitp.zoom.us%2Fj%2F93861697232&data=05%7C02%7Cs.shankar%40uva.nl%7C8a3097a1869c40189a0808dd5131eda4%7Ca0f1cacd618c4403b94576fb3d6874e5%7C0%7C0%7C638755997111044324%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=5IT5miftPeYyGokHcRflyQrE9Q1mqbQv9HUFiyvdARI%3D&reserved=0</a><br>
<br>
Join by phone<br>
(CA) +1 647-558-0588<br>
<br>
Join using SIP<br>
93861697232@zoomcrc.com<br>
_______________________________________________<br>
Carpetx-developers mailing list<br>
Carpetx-developers@einsteintoolkit.org<br>
<a href="http://lists.einsteintoolkit.org/mailman/listinfo/carpetx-developers">https://eur04.safelinks.protection.outlook.com/?url=http%3A%2F%2Flists.einsteintoolkit.org%2Fmailman%2Flistinfo%2Fcarpetx-developers&data=05%7C02%7Cs.shankar%40uva.nl%7C8a3097a1869c40189a0808dd5131eda4%7Ca0f1cacd618c4403b94576fb3d6874e5%7C0%7C0%7C638755997111066883%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=PpqxCWk9G3BQ9tiHRoz0ppcjsKtXaigz%2B9AWMfihCRc%3D&reserved=0</a><o:p></o:p></span></p>
</div>
</div>
</div>
</div>
</body>
</html>