From sbrandt at cct.lsu.edu Tue Jul 1 10:48:46 2025 From: sbrandt at cct.lsu.edu (Steven Brandt) Date: Tue, 1 Jul 2025 10:48:46 -0500 Subject: [Users] Problems with output in CarpetX In-Reply-To: <40E50047-069B-4E89-B913-37D46EB9711D@gmail.com> References:

<40E50047-069B-4E89-B913-37D46EB9711D@gmail.com> Message-ID: <3a9c37e9-35bd-4065-ab97-e274ca115f9d@cct.lsu.edu> The Tutorial server has the full current release of the ETK, including SILO and OpenPMD. --Steve On 6/30/2025 11:53 AM, Erik Schnetter wrote: > Javiera > > Unfortunately I don't recall what software is installed on the ETK > tutorial server... I'm sure someone else can help us out here? > > -erik > >> On Jun 30, 2025, at 12:17, Javiera Hern?ndez Morales >> wrote: >> >> Hi Erik, >> >> Thanks for your answer. Maybe for more context I should mention that >> I am starting to learn how to use the ETK, and for that I am using >> the notebooks provided in the tutorial server (etk.cct.lsu.edu >> ), together with some of the ones used for >> the last ETK workshop. >> >> To answer your last question, I am not doing anything specific with >> the output, I wanted to get 3D data to see if that could help me >> visualise the output of the examples I did with the WaveToyX thorn. >> >> Unfortunately I am not familiar with SILO, so I tried to use the 1D >> .tsv files, but they only provide the values in a slice (for example >> the wave as a function of x, for y=z=0), but I wanted to create a 3D >> plot similar to the one found in the CreatingANewThorn-WaveEqn >> notebook from the tutorial server (I leave a screenshot attached), >> that is why I thought of outputting 3D. >> >> To use SILO files I understand that one has to have VisIt installed, >> however when I wanted to install it in the tutorial server via >> terminal, I got the error that I am not allowed to install software, >> which I suppose makes sense. >> >> In principle all I want to do is experiment with the WaveToyX thorn >> (and the other examples, like SpacetimeWaveToyX or FluxWaveToyX), >> changing parameters and see how things behave, but I am struggling a >> lot with the visualisation part. Do you have any recommendations that >> I could implement in the environment of the ETK Tutorial server? >> >> Thanks in advance. >> >> Best regards, >> >> Javiera H.M. >> >> >> >>> El 30.06.2025, a las 16:45, Erik Schnetter >>> escribi?: >>> >>> Javiera >>> >>> I recommend using 3D output in the way others are. CarpetX supports >>> many parameters, but not all parameter combinations are well tested, >>> or are intended for production runs. >>> >>> (1, 2) Yes, there are two ways of 3D ASCII (tsv) output. This is >>> very slow and is thus only intended for debugging. Therefore it >>> doesn't have all the features needed for comfortable output in >>> production runs. >>> >>> (3) Either OpenPMD or Silo output are the preferred ways for >>> production runs. These formats are also supported by post-processing >>> and visualization tools. These are not file formats, but they only >>> describe how the metadata are arranged (i.e. what names the >>> attributes have) in the output files. The actual file formats are >>> ADIOS2 (preferred, faster) or HDF5. OpenPMD uses ADIOS2 by default, >>> Silo uses HDF5 by default. >>> >>> I do not know why the OpenPMD/HDF5 output is not working. The error >>> message doesn't show enough detail. >>> >>> If you want HDF5 output then I recommend using Silo. This will >>> produce an HDF5 file. I'm not sure that's what you want because, >>> whatever you are going to do with the file, you will need to >>> understand the metadata in the file, and thus that tool or script >>> will need to understand either the OpenMPD or Silo layout. Otherwise >>> your HDF5 will look like a bag of 3D arrays, each array describing a >>> Fortran array, and it will be very difficult to find out which >>> component of which variable and which time step on which refinement >>> level is stored in what array: That's what the metadata describes. >>> >>> On the other hand, if you want to have an easy way to read the file, >>> then you could use e.g. the Julia or Python of C++ bindings of the >>> OpenMPD library, and you then wouldn't need to care whether the >>> low-level file format is ADIOS2 or HDF5. >>> >>> What do you want to do with the 3D output? >>> >>> -erik >>> >>>> On Jun 27, 2025, at 12:54, Javiera Hern?ndez Morales >>>> wrote: >>>> >>>> Hi all, >>>> >>>> I am starting to learn the use of CarpetX and for this I started >>>> using the WaveToyX thorn. I wanted to ask for guidance for three >>>> problems that I have encountered so far, all of them related to the >>>> output data: >>>> >>>> 1) I wanted to get 3D data, for example in .tsv format. Here I >>>> realised that there is an ambiguous definition in the documentation >>>> for the parameter ?out_tsv?, which in section 12.1 it says to >>>> control TSV output and have default value ?yes?, but in section 14 >>>> it says to control the 3D output in TSV and have default value >>>> ?no?. This last behaviour is the one that I encounter when running >>>> the parfiles. >>>> >>>> 2) When I set up ?out_tsv" to ?yes? and ?out_tsv_every? to a >>>> number, for example = 16, I get 1D output every 16 iterations, but >>>> 3D output in every iteration, therefore I suspect >>>> that??out_tsv_every??is only controlling the frequency of the 1D >>>> output, and I cannot find a way to control the frequency of 3D output. >>>> >>>> 3) I attempted to output data in HDF5 format but I did not succeed. >>>> I first set the parameter ?openpmd_format? to ?HDF5? and also set >>>> ?out_openpmd_vars? to the variables that I wanted, but I got an >>>> error in the first iteration: >>>> >>>> INFO (CarpetX): OutputGH: iteration 0, time 0.000000, run time 1 s >>>> INFO (CarpetX): OutputOpenPMD... >>>> INFO (CarpetX): Creating openPMD object... >>>> INFO (CarpetX): options: >>>> { >>>> "adios2": { >>>> "dataset": { >>>> "operators": [ >>>> ] >>>> } >>>> } >>>> } >>>> >>>> [et-juphub:699479] *** An error occurred in MPI_Comm_create_keyval >>>> [et-juphub:699479] *** reported by process [211877889,2] >>>> [et-juphub:699479] *** on communicator MPI_COMM_WORLD >>>> [et-juphub:699479] *** MPI_ERR_ARG: invalid argument of some other kind >>>> [et-juphub:699479] *** MPI_ERRORS_ARE_FATAL (processes in this communicator will now abort, >>>> [et-juphub:699479] *** and potentially your MPI job) >>>> [et-juphub:699473] PMIX ERROR: UNREACHABLE in file ../../../src/server/pmix_server.c at line 2193 >>>> [et-juphub:699473] 3 more processes have sent help message help-mpi-errors.txt / mpi_errors_are_fatal >>>> [et-juphub:699473] Set MCA parameter "orte_base_help_aggregate" to 0 to see all help / error messages >>>> Simfactory Done at date: Thu 26 Jun 2025 09:19:08 AM UTC >>>> And I did not know how to move on from there. >>>> I am working on the ETK Tutorial server, in case that is important. >>>> Thanks in advance to anyone who took the time to read until here, I >>>> will be looking forward to any piece of advice :) >>>> Best, >>>> Javiera H.M. >>>> _______________________________________________ >>>> Users mailing list >>>> Users at einsteintoolkit.org >>>> http://lists.einsteintoolkit.org/mailman/listinfo/users >>> >> > > > _______________________________________________ > Users mailing list > Users at einsteintoolkit.org > http://lists.einsteintoolkit.org/mailman/listinfo/users -------------- next part -------------- An HTML attachment was scrubbed... URL: From sbrandt at cct.lsu.edu Tue Jul 1 10:49:27 2025 From: sbrandt at cct.lsu.edu (Steven Brandt) Date: Tue, 1 Jul 2025 10:49:27 -0500 Subject: [Users] Problems with output in CarpetX In-Reply-To: <20250630102048.4b986a9b@fdea4908> References:

<40E50047-069B-4E89-B913-37D46EB9711D@gmail.com> <20250630102048.4b986a9b@fdea4908> Message-ID: <811e7662-3773-4980-8841-3ed340b8108e@cct.lsu.edu> I could add Visit. I included it on etk1 for the workshop. The only catch is it has to run headless. --Steve On 6/30/2025 12:20 PM, Roland Haas wrote: > Hello all, > > There's no VisIt instance installed on the tutorial server (it only > provides an interface using jupyter notebooks, no X11 server access). > > I believe Steve has written some Python code to visualize CarpetX data. > > Yours, > Roland > >> [CAUTION: Non-UBC Email] >> >> Javiera >> >> Unfortunately I don't recall what software is installed on the ETK tutorial server... I'm sure someone else can help us out here? >> >> -erik >> >>> On Jun 30, 2025, at 12:17, Javiera Hern?ndez Morales wrote: >>> >>> Hi Erik, >>> >>> Thanks for your answer. Maybe for more context I should mention that I am starting to learn how to use the ETK, and for that I am using the notebooks provided in the tutorial server (etk.cct.lsu.edu ), together with some of the ones used for the last ETK workshop. >>> >>> To answer your last question, I am not doing anything specific with the output, I wanted to get 3D data to see if that could help me visualise the output of the examples I did with the WaveToyX thorn. >>> >>> Unfortunately I am not familiar with SILO, so I tried to use the 1D .tsv files, but they only provide the values in a slice (for example the wave as a function of x, for y=z=0), but I wanted to create a 3D plot similar to the one found in the CreatingANewThorn-WaveEqn notebook from the tutorial server (I leave a screenshot attached), that is why I thought of outputting 3D. >>> >>> To use SILO files I understand that one has to have VisIt installed, however when I wanted to install it in the tutorial server via terminal, I got the error that I am not allowed to install software, which I suppose makes sense. >>> >>> In principle all I want to do is experiment with the WaveToyX thorn (and the other examples, like SpacetimeWaveToyX or FluxWaveToyX), changing parameters and see how things behave, but I am struggling a lot with the visualisation part. Do you have any recommendations that I could implement in the environment of the ETK Tutorial server? >>> >>> Thanks in advance. >>> >>> Best regards, >>> >>> Javiera H.M. >>> >>> >>> >>>> El 30.06.2025, a las 16:45, Erik Schnetter escribi?: >>>> >>>> Javiera >>>> >>>> I recommend using 3D output in the way others are. CarpetX supports many parameters, but not all parameter combinations are well tested, or are intended for production runs. >>>> >>>> (1, 2) Yes, there are two ways of 3D ASCII (tsv) output. This is very slow and is thus only intended for debugging. Therefore it doesn't have all the features needed for comfortable output in production runs. >>>> >>>> (3) Either OpenPMD or Silo output are the preferred ways for production runs. These formats are also supported by post-processing and visualization tools. These are not file formats, but they only describe how the metadata are arranged (i.e. what names the attributes have) in the output files. The actual file formats are ADIOS2 (preferred, faster) or HDF5. OpenPMD uses ADIOS2 by default, Silo uses HDF5 by default. >>>> >>>> I do not know why the OpenPMD/HDF5 output is not working. The error message doesn't show enough detail. >>>> >>>> If you want HDF5 output then I recommend using Silo. This will produce an HDF5 file. I'm not sure that's what you want because, whatever you are going to do with the file, you will need to understand the metadata in the file, and thus that tool or script will need to understand either the OpenMPD or Silo layout. Otherwise your HDF5 will look like a bag of 3D arrays, each array describing a Fortran array, and it will be very difficult to find out which component of which variable and which time step on which refinement level is stored in what array: That's what the metadata describes. >>>> >>>> On the other hand, if you want to have an easy way to read the file, then you could use e.g. the Julia or Python of C++ bindings of the OpenMPD library, and you then wouldn't need to care whether the low-level file format is ADIOS2 or HDF5. >>>> >>>> What do you want to do with the 3D output? >>>> >>>> -erik >>>> >>>>> On Jun 27, 2025, at 12:54, Javiera Hern?ndez Morales wrote: >>>>> >>>>> Hi all, >>>>> >>>>> I am starting to learn the use of CarpetX and for this I started using the WaveToyX thorn. I wanted to ask for guidance for three problems that I have encountered so far, all of them related to the output data: >>>>> >>>>> 1) I wanted to get 3D data, for example in .tsv format. Here I realised that there is an ambiguous definition in the documentation for the parameter ?out_tsv?, which in section 12.1 it says to control TSV output and have default value ?yes?, but in section 14 it says to control the 3D output in TSV and have default value ?no?. This last behaviour is the one that I encounter when running the parfiles. >>>>> >>>>> 2) When I set up ?out_tsv" to ?yes? and ?out_tsv_every? to a number, for example = 16, I get 1D output every 16 iterations, but 3D output in every iteration, therefore I suspect that ?out_tsv_every? is only controlling the frequency of the 1D output, and I cannot find a way to control the frequency of 3D output. >>>>> >>>>> 3) I attempted to output data in HDF5 format but I did not succeed. I first set the parameter ?openpmd_format? to ?HDF5? and also set ?out_openpmd_vars? to the variables that I wanted, but I got an error in the first iteration: >>>>> >>>>> INFO (CarpetX): OutputGH: iteration 0, time 0.000000, run time 1 s >>>>> INFO (CarpetX): OutputOpenPMD... >>>>> INFO (CarpetX): Creating openPMD object... >>>>> INFO (CarpetX): options: >>>>> { >>>>> "adios2": { >>>>> "dataset": { >>>>> "operators": [ >>>>> ] >>>>> } >>>>> } >>>>> } >>>>> >>>>> [et-juphub:699479] *** An error occurred in MPI_Comm_create_keyval >>>>> [et-juphub:699479] *** reported by process [211877889,2] >>>>> [et-juphub:699479] *** on communicator MPI_COMM_WORLD >>>>> [et-juphub:699479] *** MPI_ERR_ARG: invalid argument of some other kind >>>>> [et-juphub:699479] *** MPI_ERRORS_ARE_FATAL (processes in this communicator will now abort, >>>>> [et-juphub:699479] *** and potentially your MPI job) >>>>> [et-juphub:699473] PMIX ERROR: UNREACHABLE in file ../../../src/server/pmix_server.c at line 2193 >>>>> [et-juphub:699473] 3 more processes have sent help message help-mpi-errors.txt / mpi_errors_are_fatal >>>>> [et-juphub:699473] Set MCA parameter "orte_base_help_aggregate" to 0 to see all help / error messages >>>>> Simfactory Done at date: Thu 26 Jun 2025 09:19:08 AM UTC >>>>> And I did not know how to move on from there. >>>>> I am working on the ETK Tutorial server, in case that is important. >>>>> Thanks in advance to anyone who took the time to read until here, I will be looking forward to any piece of advice :) >>>>> Best, >>>>> Javiera H.M. >>>>> _______________________________________________ >>>>> Users mailing list >>>>> Users at einsteintoolkit.org >>>>> http://lists.einsteintoolkit.org/mailman/listinfo/users >>>> >>> > > Yours, > Roland > From bozzola.gabriele at gmail.com Wed Jul 2 09:41:56 2025 From: bozzola.gabriele at gmail.com (Gabriele Bozzola) Date: Wed, 2 Jul 2025 07:41:56 -0700 Subject: [Users] Problems with output in CarpetX In-Reply-To: <811e7662-3773-4980-8841-3ed340b8108e@cct.lsu.edu> References:

<40E50047-069B-4E89-B913-37D46EB9711D@gmail.com> <20250630102048.4b986a9b@fdea4908> <811e7662-3773-4980-8841-3ed340b8108e@cct.lsu.edu> Message-ID: Hi Javiera (and all), Last year, we added experimental support to reading OpenPMD files to kuibit . I haven't tried it for anything more than toy data, but the example we were working with is precisely the output generated with WaveToyX, as seen here . You could give it a try. kuibit will read the data and represent it with its HierarchicalGridData data structure, which can be sampled to a uniform grid and visualized with matplotlib (or you can slice it a given dimension to get a 1D/2D cut and visualize those). Best, Gabriele On Tue, Jul 1, 2025 at 8:49?AM Steven Brandt wrote: > I could add Visit. I included it on etk1 for the workshop. The only > catch is it has to run headless. > > --Steve > > On 6/30/2025 12:20 PM, Roland Haas wrote: > > Hello all, > > > > There's no VisIt instance installed on the tutorial server (it only > > provides an interface using jupyter notebooks, no X11 server access). > > > > I believe Steve has written some Python code to visualize CarpetX data. > > > > Yours, > > Roland > > > >> [CAUTION: Non-UBC Email] > >> > >> Javiera > >> > >> Unfortunately I don't recall what software is installed on the ETK > tutorial server... I'm sure someone else can help us out here? > >> > >> -erik > >> > >>> On Jun 30, 2025, at 12:17, Javiera Hern?ndez Morales < > javihernandezm at posteo.net> wrote: > >>> > >>> Hi Erik, > >>> > >>> Thanks for your answer. Maybe for more context I should mention that I > am starting to learn how to use the ETK, and for that I am using the > notebooks provided in the tutorial server (etk.cct.lsu.edu < > http://etk.cct.lsu.edu/>), together with some of the ones used for the > last ETK workshop. > >>> > >>> To answer your last question, I am not doing anything specific with > the output, I wanted to get 3D data to see if that could help me visualise > the output of the examples I did with the WaveToyX thorn. > >>> > >>> Unfortunately I am not familiar with SILO, so I tried to use the 1D > .tsv files, but they only provide the values in a slice (for example the > wave as a function of x, for y=z=0), but I wanted to create a 3D plot > similar to the one found in the CreatingANewThorn-WaveEqn notebook from the > tutorial server (I leave a screenshot attached), that is why I thought of > outputting 3D. > >>> > >>> To use SILO files I understand that one has to have VisIt installed, > however when I wanted to install it in the tutorial server via terminal, I > got the error that I am not allowed to install software, which I suppose > makes sense. > >>> > >>> In principle all I want to do is experiment with the WaveToyX thorn > (and the other examples, like SpacetimeWaveToyX or FluxWaveToyX), changing > parameters and see how things behave, but I am struggling a lot with the > visualisation part. Do you have any recommendations that I could implement > in the environment of the ETK Tutorial server? > >>> > >>> Thanks in advance. > >>> > >>> Best regards, > >>> > >>> Javiera H.M. > >>> > >>> > >>> > >>>> El 30.06.2025, a las 16:45, Erik Schnetter > escribi?: > >>>> > >>>> Javiera > >>>> > >>>> I recommend using 3D output in the way others are. CarpetX supports > many parameters, but not all parameter combinations are well tested, or are > intended for production runs. > >>>> > >>>> (1, 2) Yes, there are two ways of 3D ASCII (tsv) output. This is very > slow and is thus only intended for debugging. Therefore it doesn't have all > the features needed for comfortable output in production runs. > >>>> > >>>> (3) Either OpenPMD or Silo output are the preferred ways for > production runs. These formats are also supported by post-processing and > visualization tools. These are not file formats, but they only describe how > the metadata are arranged (i.e. what names the attributes have) in the > output files. The actual file formats are ADIOS2 (preferred, faster) or > HDF5. OpenPMD uses ADIOS2 by default, Silo uses HDF5 by default. > >>>> > >>>> I do not know why the OpenPMD/HDF5 output is not working. The error > message doesn't show enough detail. > >>>> > >>>> If you want HDF5 output then I recommend using Silo. This will > produce an HDF5 file. I'm not sure that's what you want because, whatever > you are going to do with the file, you will need to understand the metadata > in the file, and thus that tool or script will need to understand either > the OpenMPD or Silo layout. Otherwise your HDF5 will look like a bag of 3D > arrays, each array describing a Fortran array, and it will be very > difficult to find out which component of which variable and which time step > on which refinement level is stored in what array: That's what the metadata > describes. > >>>> > >>>> On the other hand, if you want to have an easy way to read the file, > then you could use e.g. the Julia or Python of C++ bindings of the OpenMPD > library, and you then wouldn't need to care whether the low-level file > format is ADIOS2 or HDF5. > >>>> > >>>> What do you want to do with the 3D output? > >>>> > >>>> -erik > >>>> > >>>>> On Jun 27, 2025, at 12:54, Javiera Hern?ndez Morales < > javihernandezm at posteo.net> wrote: > >>>>> > >>>>> Hi all, > >>>>> > >>>>> I am starting to learn the use of CarpetX and for this I started > using the WaveToyX thorn. I wanted to ask for guidance for three problems > that I have encountered so far, all of them related to the output data: > >>>>> > >>>>> 1) I wanted to get 3D data, for example in .tsv format. Here I > realised that there is an ambiguous definition in the documentation for the > parameter ?out_tsv?, which in section 12.1 it says to control TSV output > and have default value ?yes?, but in section 14 it says to control the 3D > output in TSV and have default value ?no?. This last behaviour is the one > that I encounter when running the parfiles. > >>>>> > >>>>> 2) When I set up ?out_tsv" to ?yes? and ?out_tsv_every? to a number, > for example = 16, I get 1D output every 16 iterations, but 3D output in > every iteration, therefore I suspect that ?out_tsv_every? is only > controlling the frequency of the 1D output, and I cannot find a way to > control the frequency of 3D output. > >>>>> > >>>>> 3) I attempted to output data in HDF5 format but I did not succeed. > I first set the parameter ?openpmd_format? to ?HDF5? and also set > ?out_openpmd_vars? to the variables that I wanted, but I got an error in > the first iteration: > >>>>> > >>>>> INFO (CarpetX): OutputGH: iteration 0, time 0.000000, run time 1 s > >>>>> INFO (CarpetX): OutputOpenPMD... > >>>>> INFO (CarpetX): Creating openPMD object... > >>>>> INFO (CarpetX): options: > >>>>> { > >>>>> "adios2": { > >>>>> "dataset": { > >>>>> "operators": [ > >>>>> ] > >>>>> } > >>>>> } > >>>>> } > >>>>> > >>>>> [et-juphub:699479] *** An error occurred in MPI_Comm_create_keyval > >>>>> [et-juphub:699479] *** reported by process [211877889,2] > >>>>> [et-juphub:699479] *** on communicator MPI_COMM_WORLD > >>>>> [et-juphub:699479] *** MPI_ERR_ARG: invalid argument of some other > kind > >>>>> [et-juphub:699479] *** MPI_ERRORS_ARE_FATAL (processes in this > communicator will now abort, > >>>>> [et-juphub:699479] *** and potentially your MPI job) > >>>>> [et-juphub:699473] PMIX ERROR: UNREACHABLE in file > ../../../src/server/pmix_server.c at line 2193 > >>>>> [et-juphub:699473] 3 more processes have sent help message > help-mpi-errors.txt / mpi_errors_are_fatal > >>>>> [et-juphub:699473] Set MCA parameter "orte_base_help_aggregate" to 0 > to see all help / error messages > >>>>> Simfactory Done at date: Thu 26 Jun 2025 09:19:08 AM UTC > >>>>> And I did not know how to move on from there. > >>>>> I am working on the ETK Tutorial server, in case that is important. > >>>>> Thanks in advance to anyone who took the time to read until here, I > will be looking forward to any piece of advice :) > >>>>> Best, > >>>>> Javiera H.M. > >>>>> _______________________________________________ > >>>>> Users mailing list > >>>>> Users at einsteintoolkit.org > >>>>> http://lists.einsteintoolkit.org/mailman/listinfo/users > >>>> > >>> > > > > Yours, > > Roland > > > _______________________________________________ > Users mailing list > Users at einsteintoolkit.org > http://lists.einsteintoolkit.org/mailman/listinfo/users > -------------- next part -------------- An HTML attachment was scrubbed... URL: From sbrandt at cct.lsu.edu Wed Jul 2 13:50:11 2025 From: sbrandt at cct.lsu.edu (Steven Brandt) Date: Wed, 2 Jul 2025 13:50:11 -0500 Subject: [Users] Problems with output in CarpetX In-Reply-To: References:

<40E50047-069B-4E89-B913-37D46EB9711D@gmail.com> <20250630102048.4b986a9b@fdea4908> <811e7662-3773-4980-8841-3ed340b8108e@cct.lsu.edu> Message-ID: <1dc0f5bb-f772-4220-b0d6-98266d78228f@cct.lsu.edu> Some of the tutorials on ETK1 use Kuibit to plot data from CarpetX runs, including the basic one on building and running with the ET. --Steve On 7/2/2025 9:41 AM, Gabriele Bozzola wrote: > Hi Javiera (and all), > > Last year, we added experimental support > > to reading OpenPMD files to kuibit . > > I haven't tried it for anything more than toy data, but the example we > were working with is precisely the output generated with WaveToyX, as > seen here > . > > You could give it a try. kuibit will read the data and represent it > with its HierarchicalGridData data structure, which can be sampled to > a uniform grid and visualized with matplotlib (or you can slice it a > given dimension to get a 1D/2D cut and visualize those). > > Best, > Gabriele > > > On Tue, Jul 1, 2025 at 8:49?AM Steven Brandt wrote: > > I could add Visit. I included it on etk1 for the workshop. The only > catch is it has to run headless. > > --Steve > > On 6/30/2025 12:20 PM, Roland Haas wrote: > > Hello all, > > > > There's no VisIt instance installed on the tutorial server (it only > > provides an interface using jupyter notebooks, no X11 server > access). > > > > I believe Steve has written some Python code to visualize > CarpetX data. > > > > Yours, > > Roland > > > >> [CAUTION: Non-UBC Email] > >> > >> Javiera > >> > >> Unfortunately I don't recall what software is installed on the > ETK tutorial server... I'm sure someone else can help us out here? > >> > >> -erik > >> > >>> On Jun 30, 2025, at 12:17, Javiera Hern?ndez Morales > wrote: > >>> > >>> Hi Erik, > >>> > >>> Thanks for your answer. Maybe for more context I should > mention that I am starting to learn how to use the ETK, and for > that I am using the notebooks provided in the tutorial server > (etk.cct.lsu.edu > ), together with some of the ones used > for the last ETK workshop. > >>> > >>> To answer your last question, I am not doing anything specific > with the output, I wanted to get 3D data to see if that could help > me visualise the output of the examples I did with the WaveToyX thorn. > >>> > >>> Unfortunately I am not familiar with SILO, so I tried to use > the 1D .tsv files, but they only provide the values in a slice > (for example the wave as a function of x, for y=z=0), but I wanted > to create a 3D plot similar to the one found in the > CreatingANewThorn-WaveEqn notebook from the tutorial server (I > leave a screenshot attached), that is why I thought of outputting 3D. > >>> > >>> To use SILO files I understand that one has to have VisIt > installed, however when I wanted to install it in the tutorial > server via terminal, I got the error that I am not allowed to > install software, which I suppose makes sense. > >>> > >>> In principle all I want to do is experiment with the WaveToyX > thorn (and the other examples, like SpacetimeWaveToyX or > FluxWaveToyX), changing parameters and see how things behave, but > I am struggling a lot with the visualisation part. Do you have any > recommendations that I could implement in the environment of the > ETK Tutorial server? > >>> > >>> Thanks in advance. > >>> > >>> Best regards, > >>> > >>> Javiera H.M. > >>> > >>> > >>> > >>>> El 30.06.2025, a las 16:45, Erik Schnetter > escribi?: > >>>> > >>>> Javiera > >>>> > >>>> I recommend using 3D output in the way others are. CarpetX > supports many parameters, but not all parameter combinations are > well tested, or are intended for production runs. > >>>> > >>>> (1, 2) Yes, there are two ways of 3D ASCII (tsv) output. This > is very slow and is thus only intended for debugging. Therefore it > doesn't have all the features needed for comfortable output in > production runs. > >>>> > >>>> (3) Either OpenPMD or Silo output are the preferred ways for > production runs. These formats are also supported by > post-processing and visualization tools. These are not file > formats, but they only describe how the metadata are arranged > (i.e. what names the attributes have) in the output files. The > actual file formats are ADIOS2 (preferred, faster) or HDF5. > OpenPMD uses ADIOS2 by default, Silo uses HDF5 by default. > >>>> > >>>> I do not know why the OpenPMD/HDF5 output is not working. The > error message doesn't show enough detail. > >>>> > >>>> If you want HDF5 output then I recommend using Silo. This > will produce an HDF5 file. I'm not sure that's what you want > because, whatever you are going to do with the file, you will need > to understand the metadata in the file, and thus that tool or > script will need to understand either the OpenMPD or Silo layout. > Otherwise your HDF5 will look like a bag of 3D arrays, each array > describing a Fortran array, and it will be very difficult to find > out which component of which variable and which time step on which > refinement level is stored in what array: That's what the metadata > describes. > >>>> > >>>> On the other hand, if you want to have an easy way to read > the file, then you could use e.g. the Julia or Python of C++ > bindings of the OpenMPD library, and you then wouldn't need to > care whether the low-level file format is ADIOS2 or HDF5. > >>>> > >>>> What do you want to do with the 3D output? > >>>> > >>>> -erik > >>>> > >>>>> On Jun 27, 2025, at 12:54, Javiera Hern?ndez Morales > wrote: > >>>>> > >>>>> Hi all, > >>>>> > >>>>> I am starting to learn the use of CarpetX and for this I > started using the WaveToyX thorn. I wanted to ask for guidance for > three problems that I have encountered so far, all of them related > to the output data: > >>>>> > >>>>> 1) I wanted to get 3D data, for example in .tsv format. Here > I realised that there is an ambiguous definition in the > documentation for the parameter ?out_tsv?, which in section 12.1 > it says to control TSV output and have default value ?yes?, but in > section 14 it says to control the 3D output in TSV and have > default value ?no?. This last behaviour is the one that I > encounter when running the parfiles. > >>>>> > >>>>> 2) When I set up ?out_tsv" to ?yes? and ?out_tsv_every? to a > number, for example = 16, I get 1D output every 16 iterations, but > 3D output in every iteration, therefore I suspect that > ?out_tsv_every? is only controlling the frequency of the 1D > output, and I cannot find a way to control the frequency of 3D output. > >>>>> > >>>>> 3) I attempted to output data in HDF5 format but I did not > succeed. I first set the parameter ?openpmd_format? to ?HDF5? and > also set ?out_openpmd_vars? to the variables that I wanted, but I > got an error in the first iteration: > >>>>> > >>>>> INFO (CarpetX): OutputGH: iteration 0, time 0.000000, run > time 1 s > >>>>> INFO (CarpetX): OutputOpenPMD... > >>>>> INFO (CarpetX): Creating openPMD object... > >>>>> INFO (CarpetX):? ?options: > >>>>>? ? { > >>>>>? ? ? "adios2": { > >>>>>? ? ? ? "dataset": { > >>>>>? ? ? ? ? "operators": [ > >>>>>? ? ? ? ? ] > >>>>>? ? ? ? } > >>>>>? ? ? } > >>>>>? ? } > >>>>> > >>>>> [et-juphub:699479] *** An error occurred in > MPI_Comm_create_keyval > >>>>> [et-juphub:699479] *** reported by process [211877889,2] > >>>>> [et-juphub:699479] *** on communicator MPI_COMM_WORLD > >>>>> [et-juphub:699479] *** MPI_ERR_ARG: invalid argument of some > other kind > >>>>> [et-juphub:699479] *** MPI_ERRORS_ARE_FATAL (processes in > this communicator will now abort, > >>>>> [et-juphub:699479] ***? ? and potentially your MPI job) > >>>>> [et-juphub:699473] PMIX ERROR: UNREACHABLE in file > ../../../src/server/pmix_server.c at line 2193 > >>>>> [et-juphub:699473] 3 more processes have sent help message > help-mpi-errors.txt / mpi_errors_are_fatal > >>>>> [et-juphub:699473] Set MCA parameter > "orte_base_help_aggregate" to 0 to see all help / error messages > >>>>> Simfactory Done at date: Thu 26 Jun 2025 09:19:08 AM UTC > >>>>> And I did not know how to move on from there. > >>>>> I am working on the ETK Tutorial server, in case that is > important. > >>>>> Thanks in advance to anyone who took the time to read until > here, I will be looking forward to any piece of advice :) > >>>>> Best, > >>>>> Javiera H.M. > >>>>> _______________________________________________ > >>>>> Users mailing list > >>>>> Users at einsteintoolkit.org > >>>>> http://lists.einsteintoolkit.org/mailman/listinfo/users > >>>> > >>> > > > > Yours, > > Roland > > > _______________________________________________ > Users mailing list > Users at einsteintoolkit.org > http://lists.einsteintoolkit.org/mailman/listinfo/users > -------------- next part -------------- An HTML attachment was scrubbed... URL: From users at einsteintoolkit.org Wed Jul 2 17:15:02 2025 From: users at einsteintoolkit.org (users at einsteintoolkit.org) Date: Wed, 02 Jul 2025 17:15:02 -0500 Subject: [Users] Einstein Toolkit Meeting Reminder Message-ID: <6865af66.UXzNK7SyEF0dlsmd%users@einsteintoolkit.org> Hello, Please consider joining the weekly Einstein Toolkit phone call at 9:00 am US central time on Thursdays. For details on how to connect and what agenda items are to be discussed, use the link below. https://docs.einsteintoolkit.org/et-docs/Main_Page#Weekly_Users_Call --The Maintainers From javihernandezm at posteo.net Thu Jul 3 03:36:38 2025 From: javihernandezm at posteo.net (=?utf-8?Q?Javiera_Hern=C3=A1ndez_Morales?=) Date: Thu, 3 Jul 2025 08:36:38 +0000 Subject: [Users] Problems with output in CarpetX In-Reply-To: <1dc0f5bb-f772-4220-b0d6-98266d78228f@cct.lsu.edu> References:

<40E50047-069B-4E89-B913-37D46EB9711D@gmail.com> <20250630102048.4b986a9b@fdea4908> <811e7662-3773-4980-8841-3ed340b8108e@cct.lsu.edu> <1dc0f5bb-f772-4220-b0d6-98266d78228f@cct.lsu.edu> Message-ID: Hi all, I used the code from the tutorials from the last workshop that use kuibit and now I can indeed visualise the results. Thanks a lot for all the help! :) Best, Javiera > El 02.07.2025, a las 20:50, Steven Brandt escribi?: > > Some of the tutorials on ETK1 use Kuibit to plot data from CarpetX runs, including the basic one on building and running with the ET. > > --Steve > > On 7/2/2025 9:41 AM, Gabriele Bozzola wrote: >> Hi Javiera (and all), >> >> Last year, we added experimental support to reading OpenPMD files to kuibit . >> >> I haven't tried it for anything more than toy data, but the example we were working with is precisely the output generated with WaveToyX, as seen here . >> >> You could give it a try. kuibit will read the data and represent it with its HierarchicalGridData data structure, which can be sampled to a uniform grid and visualized with matplotlib (or you can slice it a given dimension to get a 1D/2D cut and visualize those). >> >> Best, >> Gabriele >> >> >> On Tue, Jul 1, 2025 at 8:49?AM Steven Brandt > wrote: >>> I could add Visit. I included it on etk1 for the workshop. The only >>> catch is it has to run headless. >>> >>> --Steve >>> >>> On 6/30/2025 12:20 PM, Roland Haas wrote: >>> > Hello all, >>> > >>> > There's no VisIt instance installed on the tutorial server (it only >>> > provides an interface using jupyter notebooks, no X11 server access). >>> > >>> > I believe Steve has written some Python code to visualize CarpetX data. >>> > >>> > Yours, >>> > Roland >>> > >>> >> [CAUTION: Non-UBC Email] >>> >> >>> >> Javiera >>> >> >>> >> Unfortunately I don't recall what software is installed on the ETK tutorial server... I'm sure someone else can help us out here? >>> >> >>> >> -erik >>> >> >>> >>> On Jun 30, 2025, at 12:17, Javiera Hern?ndez Morales > wrote: >>> >>> >>> >>> Hi Erik, >>> >>> >>> >>> Thanks for your answer. Maybe for more context I should mention that I am starting to learn how to use the ETK, and for that I am using the notebooks provided in the tutorial server (etk.cct.lsu.edu ), together with some of the ones used for the last ETK workshop. >>> >>> >>> >>> To answer your last question, I am not doing anything specific with the output, I wanted to get 3D data to see if that could help me visualise the output of the examples I did with the WaveToyX thorn. >>> >>> >>> >>> Unfortunately I am not familiar with SILO, so I tried to use the 1D .tsv files, but they only provide the values in a slice (for example the wave as a function of x, for y=z=0), but I wanted to create a 3D plot similar to the one found in the CreatingANewThorn-WaveEqn notebook from the tutorial server (I leave a screenshot attached), that is why I thought of outputting 3D. >>> >>> >>> >>> To use SILO files I understand that one has to have VisIt installed, however when I wanted to install it in the tutorial server via terminal, I got the error that I am not allowed to install software, which I suppose makes sense. >>> >>> >>> >>> In principle all I want to do is experiment with the WaveToyX thorn (and the other examples, like SpacetimeWaveToyX or FluxWaveToyX), changing parameters and see how things behave, but I am struggling a lot with the visualisation part. Do you have any recommendations that I could implement in the environment of the ETK Tutorial server? >>> >>> >>> >>> Thanks in advance. >>> >>> >>> >>> Best regards, >>> >>> >>> >>> Javiera H.M. >>> >>> >>> >>> >>> >>> >>> >>>> El 30.06.2025, a las 16:45, Erik Schnetter > escribi?: >>> >>>> >>> >>>> Javiera >>> >>>> >>> >>>> I recommend using 3D output in the way others are. CarpetX supports many parameters, but not all parameter combinations are well tested, or are intended for production runs. >>> >>>> >>> >>>> (1, 2) Yes, there are two ways of 3D ASCII (tsv) output. This is very slow and is thus only intended for debugging. Therefore it doesn't have all the features needed for comfortable output in production runs. >>> >>>> >>> >>>> (3) Either OpenPMD or Silo output are the preferred ways for production runs. These formats are also supported by post-processing and visualization tools. These are not file formats, but they only describe how the metadata are arranged (i.e. what names the attributes have) in the output files. The actual file formats are ADIOS2 (preferred, faster) or HDF5. OpenPMD uses ADIOS2 by default, Silo uses HDF5 by default. >>> >>>> >>> >>>> I do not know why the OpenPMD/HDF5 output is not working. The error message doesn't show enough detail. >>> >>>> >>> >>>> If you want HDF5 output then I recommend using Silo. This will produce an HDF5 file. I'm not sure that's what you want because, whatever you are going to do with the file, you will need to understand the metadata in the file, and thus that tool or script will need to understand either the OpenMPD or Silo layout. Otherwise your HDF5 will look like a bag of 3D arrays, each array describing a Fortran array, and it will be very difficult to find out which component of which variable and which time step on which refinement level is stored in what array: That's what the metadata describes. >>> >>>> >>> >>>> On the other hand, if you want to have an easy way to read the file, then you could use e.g. the Julia or Python of C++ bindings of the OpenMPD library, and you then wouldn't need to care whether the low-level file format is ADIOS2 or HDF5. >>> >>>> >>> >>>> What do you want to do with the 3D output? >>> >>>> >>> >>>> -erik >>> >>>> >>> >>>>> On Jun 27, 2025, at 12:54, Javiera Hern?ndez Morales > wrote: >>> >>>>> >>> >>>>> Hi all, >>> >>>>> >>> >>>>> I am starting to learn the use of CarpetX and for this I started using the WaveToyX thorn. I wanted to ask for guidance for three problems that I have encountered so far, all of them related to the output data: >>> >>>>> >>> >>>>> 1) I wanted to get 3D data, for example in .tsv format. Here I realised that there is an ambiguous definition in the documentation for the parameter ?out_tsv?, which in section 12.1 it says to control TSV output and have default value ?yes?, but in section 14 it says to control the 3D output in TSV and have default value ?no?. This last behaviour is the one that I encounter when running the parfiles. >>> >>>>> >>> >>>>> 2) When I set up ?out_tsv" to ?yes? and ?out_tsv_every? to a number, for example = 16, I get 1D output every 16 iterations, but 3D output in every iteration, therefore I suspect that ?out_tsv_every? is only controlling the frequency of the 1D output, and I cannot find a way to control the frequency of 3D output. >>> >>>>> >>> >>>>> 3) I attempted to output data in HDF5 format but I did not succeed. I first set the parameter ?openpmd_format? to ?HDF5? and also set ?out_openpmd_vars? to the variables that I wanted, but I got an error in the first iteration: >>> >>>>> >>> >>>>> INFO (CarpetX): OutputGH: iteration 0, time 0.000000, run time 1 s >>> >>>>> INFO (CarpetX): OutputOpenPMD... >>> >>>>> INFO (CarpetX): Creating openPMD object... >>> >>>>> INFO (CarpetX): options: >>> >>>>> { >>> >>>>> "adios2": { >>> >>>>> "dataset": { >>> >>>>> "operators": [ >>> >>>>> ] >>> >>>>> } >>> >>>>> } >>> >>>>> } >>> >>>>> >>> >>>>> [et-juphub:699479] *** An error occurred in MPI_Comm_create_keyval >>> >>>>> [et-juphub:699479] *** reported by process [211877889,2] >>> >>>>> [et-juphub:699479] *** on communicator MPI_COMM_WORLD >>> >>>>> [et-juphub:699479] *** MPI_ERR_ARG: invalid argument of some other kind >>> >>>>> [et-juphub:699479] *** MPI_ERRORS_ARE_FATAL (processes in this communicator will now abort, >>> >>>>> [et-juphub:699479] *** and potentially your MPI job) >>> >>>>> [et-juphub:699473] PMIX ERROR: UNREACHABLE in file ../../../src/server/pmix_server.c at line 2193 >>> >>>>> [et-juphub:699473] 3 more processes have sent help message help-mpi-errors.txt / mpi_errors_are_fatal >>> >>>>> [et-juphub:699473] Set MCA parameter "orte_base_help_aggregate" to 0 to see all help / error messages >>> >>>>> Simfactory Done at date: Thu 26 Jun 2025 09:19:08 AM UTC >>> >>>>> And I did not know how to move on from there. >>> >>>>> I am working on the ETK Tutorial server, in case that is important. >>> >>>>> Thanks in advance to anyone who took the time to read until here, I will be looking forward to any piece of advice :) >>> >>>>> Best, >>> >>>>> Javiera H.M. >>> >>>>> _______________________________________________ >>> >>>>> Users mailing list >>> >>>>> Users at einsteintoolkit.org >>> >>>>> http://lists.einsteintoolkit.org/mailman/listinfo/users >>> >>>> >>> >>> >>> > >>> > Yours, >>> > Roland >>> > >>> _______________________________________________ >>> Users mailing list >>> Users at einsteintoolkit.org >>> http://lists.einsteintoolkit.org/mailman/listinfo/users -------------- next part -------------- An HTML attachment was scrubbed... URL: From rhaas at phas.ubc.ca Thu Jul 3 19:32:59 2025 From: rhaas at phas.ubc.ca (Roland Haas) Date: Thu, 3 Jul 2025 17:32:59 -0700 Subject: [Users] meeting minutes for 2025-07-03 Message-ID: <20250703173250.16a785f4@haengie2.phas.ubc.ca> Present: Leo, Roland, Peter, Beyhan, Keith, Wei Sun, Zach gallery examples ================ * still lacking updates for BBH gallery example for 2025_05 and 2024_11 gcc15 failures ============== * Zach suggested to compile ET with newest flavor of compilers regularly * Roland mentions that we could have a weekly build using the new compiler using github, Ubuntu 25.04 has gcc15, Zach suggests there may even be a gcc-latest or so image Benchmarking GRHayLHD simulations ================================= * Wei, Zach and Leo discussed results of benchmarking a GRHayLHD simulation for optimal speed varying ranks and threads * tests were done on NERSC Perlmutter (CPU only) * suggest to try both with and without IO enabled * suggest to check mpirun options to have correct affinity to cores / numa-nodes etc. Yours, Roland -- My email is as private as my paper mail. I therefore support encrypting and signing email messages. Get my PGP key from http://pgp.mit.edu . -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From users at einsteintoolkit.org Mon Jul 7 15:18:01 2025 From: users at einsteintoolkit.org (users at einsteintoolkit.org) Date: Mon, 07 Jul 2025 15:18:01 -0500 Subject: [Users] Agenda for Thursday's Meeting Message-ID: <686c2b79./fHTYz/PPRW2QbUL%users@einsteintoolkit.org> Please update the Wiki with agenda items for Thursday's meeting. Thanks! https://docs.einsteintoolkit.org/et-docs/meeting_agenda --The Maintainers From sebastiano.bernuzzi at uni-jena.de Tue Jul 8 09:37:19 2025 From: sebastiano.bernuzzi at uni-jena.de (Sebastiano Bernuzzi) Date: Tue, 8 Jul 2025 16:37:19 +0200 Subject: [Users] Postdoc position in gravitational waves theory, Jena, Germany Message-ID: Dear Colleague, there is a postdoc opportunity in Jena; I would appreciate it if you forward this message to potential candidates. Applications are invited for a postdoctoral research position at the Friedrich-Schiller-Universitaet Jena. The appointment is associated with the research group of Prof. Sebastiano Bernuzzi at the Theoretical Physics Institute. The selected candidate is expected to work on the project ?GROOVHY: Gravitational radiation from black holes? hyperbolic mergers?, which is funded by the Deutsche Forschungsgemeinschaft (DFG). The project?s workplan spans effective-one-body models for hyperbolic and generic-orbit binary black hole mergers, numerical-relativity and test-body perturbative simulations, and Bayesian data analysis of non-circularized black hole mergers. The project is embedded in an ongoing scientific effort on gravitational-wave science and modeling of multimessenger signals. The successful candidate has the opportunity to join the international Virgo, LISA and/or Einstein Telescope collaborations. The Jena group is also part of the computational relativity (CoRe) collaboration. Applications should be received by *August 1st, 2025*, but will be considered until the position is filled. The appointment is for one year and should start in April 2026. An extension may be possible depending on finding availability. Electronic applications with subject ?Application for PD position ? SURNAME NAME? should be send to: sebastiano.bernuzzi[AT]uni-jena.de Please attach a single PDF with a cover letter, a curriculum vitae with a list of publications, a brief description of research interests, and a list of three potential referees, who may be contacted separately. LINK: https://www.physik.uni-jena.de/en/2405/jobs S.Bernuzzi -------------- next part -------------- An HTML attachment was scrubbed... URL: From users at einsteintoolkit.org Wed Jul 9 17:15:01 2025 From: users at einsteintoolkit.org (users at einsteintoolkit.org) Date: Wed, 09 Jul 2025 17:15:01 -0500 Subject: [Users] Einstein Toolkit Meeting Reminder Message-ID: <686ee9e5.QWhYXPi8lD0bs/cx%users@einsteintoolkit.org> Hello, Please consider joining the weekly Einstein Toolkit phone call at 9:00 am US central time on Thursdays. For details on how to connect and what agenda items are to be discussed, use the link below. https://docs.einsteintoolkit.org/et-docs/Main_Page#Weekly_Users_Call --The Maintainers From users at einsteintoolkit.org Mon Jul 14 15:18:02 2025 From: users at einsteintoolkit.org (users at einsteintoolkit.org) Date: Mon, 14 Jul 2025 15:18:02 -0500 Subject: [Users] Agenda for Thursday's Meeting Message-ID: <687565fa.pl5y8UIjBpWrPKR6%users@einsteintoolkit.org> Please update the Wiki with agenda items for Thursday's meeting. Thanks! https://docs.einsteintoolkit.org/et-docs/meeting_agenda --The Maintainers From users at einsteintoolkit.org Wed Jul 16 17:15:01 2025 From: users at einsteintoolkit.org (users at einsteintoolkit.org) Date: Wed, 16 Jul 2025 17:15:01 -0500 Subject: [Users] Einstein Toolkit Meeting Reminder Message-ID: <68782465.gm0fsdVgL3HRUrfP%users@einsteintoolkit.org> Hello, Please consider joining the weekly Einstein Toolkit phone call at 9:00 am US central time on Thursdays. For details on how to connect and what agenda items are to be discussed, use the link below. https://docs.einsteintoolkit.org/et-docs/Main_Page#Weekly_Users_Call --The Maintainers From wernecklr at gmail.com Thu Jul 17 08:56:14 2025 From: wernecklr at gmail.com (Leo Rosa Werneck) Date: Thu, 17 Jul 2025 06:56:14 -0700 Subject: [Users] Meeting Minutes for 2025-07-10 Message-ID: <009D3D59-479C-40FE-838C-1B37363A478C@gmail.com> Hi all, Here are the minutes from last week?s ET call. Cheers, Leo Present: Steve (Chair), Leo (Minutes), Roland, Beyhan, Lucas, Peter, Keith, Maxwell **ET Release** - BBH gallery example, past 2 releases: no update. - MacLachlan gallery example needs updating, but not ready yet. **Unanswered Questions on Mailing List** None. **Open Tickets & Tickets Ready for Review** - #2878: Pending Erik's review and approval. - #2875: Steve will review. - #2874: Leo will work on it. ------ Leonardo R. Werneck, Ph.D. Postdoctoral researcher Office EP 314 | Department of Physics | University of Idaho 875 Perimeter Dr. MS 0903 Moscow, ID 83844-0903, USA leonardo at uidaho.edu https://leowerneck.github.io -------------- next part -------------- An HTML attachment was scrubbed... URL: From lucas.t.s.carneiro at gmail.com Thu Jul 17 09:56:06 2025 From: lucas.t.s.carneiro at gmail.com (Lucas Timotheo Sanches) Date: Thu, 17 Jul 2025 09:56:06 -0500 Subject: [Users] Meeting Minutes for 2025-06-17 Message-ID: Present: Steven Brandt, Lucas T. Sanches (minutes), Beyhan Karakas, Zach Etienne (chair), Roland Haas, Peter Diener, Leo Rosa Werneck, Keith Dow. # Remaining BBH gallery example - Peter reports no progress yet. # Tickets - 2877: Zach reports having created this ticket as a TODO list for BHaHAHA, including but not limited to horizon pickup: This finds the horizon again if it is lost. - 2874: Zach suggests assigning this ticket to Leo, since he created the code. - 2879: Petsc update is now ready. - 963: Zach will create a mirror ticket to have Baikal's BSSN improvements as well. - 2282: Zach reports that this is fixed. Using the midpoint rule is the way. - 2878: Lucas was asked to add a reviewer to the ticket. - 2848: Zach suggested that this should be closed. Roland said that maybe not, since there are still issues with the random number generator. Roland suggested closing this and opening a new one ticket for the RNG. From users at einsteintoolkit.org Mon Jul 21 15:18:01 2025 From: users at einsteintoolkit.org (users at einsteintoolkit.org) Date: Mon, 21 Jul 2025 15:18:01 -0500 Subject: [Users] Agenda for Thursday's Meeting Message-ID: <687ea079.E8yQeSifEeXDnS3T%users@einsteintoolkit.org> Please update the Wiki with agenda items for Thursday's meeting. Thanks! https://docs.einsteintoolkit.org/et-docs/meeting_agenda --The Maintainers From users at einsteintoolkit.org Wed Jul 23 17:15:01 2025 From: users at einsteintoolkit.org (users at einsteintoolkit.org) Date: Wed, 23 Jul 2025 17:15:01 -0500 Subject: [Users] Einstein Toolkit Meeting Reminder Message-ID: <68815ee5.f7n9CCPkbMiKfBlO%users@einsteintoolkit.org> Hello, Please consider joining the weekly Einstein Toolkit phone call at 9:00 am US central time on Thursdays. For details on how to connect and what agenda items are to be discussed, use the link below. https://docs.einsteintoolkit.org/et-docs/Main_Page#Weekly_Users_Call --The Maintainers From noreply at qemailserver.com Wed Jul 23 12:43:48 2025 From: noreply at qemailserver.com (Maytal Dahan) Date: Wed, 23 Jul 2025 11:43:48 -0600 (MDT) Subject: [Users] Share Your Feedback: SGX3 Gateway Catalog Survey Message-ID: <511133917.98667.1753292628229@jwm-7fdf6cd45f-vrl5x> Dear Gateway Community Member, SGX3 leadership would like to invite you to participate in a brief survey to help shape the future of the SGX3 Gateway Catalog ( https://catalog.sciencegateways.org ). Your input will guide improvements in features, support services, and outreach efforts to better meet the needs of the science gateway community. Follow this link to the Survey: Take the survey Or copy and paste the URL below into your internet browser: https://utexas.qualtrics.com/jfe/form/SV_0Nis2b79nMn0je6?Q_DL=RGG7o0qI5V6cIbE_0Nis2b79nMn0je6_CGC_6tY5megpUm6XOqW&Q_CHL=email The survey should take no more than 5 minutes to complete. Thank you for contributing to the growth and impact of the SGX3 Gateway Catalog! For any additional questions, please email help at sciencegateways.org Best regards, Maytal Dahan Co-PI SGX3 Follow the link to opt out of future emails: Click here to unsubscribe -------------- next part -------------- An HTML attachment was scrubbed... URL: From rhaas at mail.ubc.ca Thu Jul 24 11:35:07 2025 From: rhaas at mail.ubc.ca (Roland Haas) Date: Thu, 24 Jul 2025 09:35:07 -0700 Subject: [Users] meeting minutes for 2025-07-24 Message-ID: <20250724093443.21a8069a@haengie2.phas.ubc.ca> Present: Peter, Roland, Steve, Beyhan, Keith, Bill, Leo, Kruskal ET release ================== * no objections to Roland's proposal to merge changes required to build back to release branch * issues collected in https://bitbucket.org/einsteintoolkit/tickets/issues/2880/backport-externallibraries-changes-for-gcc BBH gallery example =================== * no updates from Peter or Zach Open tickets ============ * Leo will look into https://bitbucket.org/einsteintoolkit/tickets/issues/2874 * Yosef will look into https://bitbucket.org/einsteintoolkit/tickets/issues/2872 * Roland will try and contact Barry about https://bitbucket.org/einsteintoolkit/tickets/issues/2308 Yours, Roland -- My email is as private as my paper mail. I therefore support encrypting and signing email messages. Get my PGP key from http://pgp.mit.edu . From PANAGIOTIS.IOSIF at units.it Fri Jul 25 05:43:36 2025 From: PANAGIOTIS.IOSIF at units.it (IOSIF PANAGIOTIS) Date: Fri, 25 Jul 2025 10:43:36 +0000 Subject: [Users] SimFactory placeholder macros Message-ID: Hi, I have been following this document, https://einsteintoolkit.org/seminars/2022_02_24/index.html , to set up ET on my cluster, working also on some example configuration files. While preparing the submit and run scripts, I wanted to check the description of the @FOO@ macros that SimFactory uses. I checked these places but found no information. * https://simfactory.bitbucket.io/simfactory2/userguide/ (no mention of macros) * https://docs.einsteintoolkit.org/et-docs/Simulation_Factory_Advanced_Tutorial (1 mention of the @USER@ macro and a pointer to a section #Macros that seems missing) * https://docs.einsteintoolkit.org/et-docs/Configuring_a_new_machine (there is no information on the submitscript and runscript) So I would like to ask: If e.g. I have this line in my submitscript: #SBATCH -A @ALLOCATION@ Where is @ALLOCATION@ "read" from? My guess is that this is specified in the .ini file in this line: allocation = Could you confirm if this is the case? Is there another source of documentation (that I might have missed) describing what these @FOO@ placeholders do? Best regards, Panagiotis ------------------- Panagiotis Iosif postdoctoral researcher Department of Physics, University of Trieste Via Alfonso Valerio 2, Trieste 34127 Italy ------------------- -------------- next part -------------- An HTML attachment was scrubbed... URL: From lcombi at perimeterinstitute.ca Fri Jul 25 10:05:48 2025 From: lcombi at perimeterinstitute.ca (Luciano Combi) Date: Fri, 25 Jul 2025 15:05:48 +0000 Subject: [Users] Issue with ReadInterpolate Message-ID: Hi people, I have used ReadInterpolate successfully for different applications in the past. I'm now trying to use it to interpolate onto a new grid that has 5 ghost zones i.e.: CoordBase::boundary_size_x_lower = 5 CoordBase::boundary_size_y_lower = 5 CoordBase::boundary_size_z_lower = 5 CoordBase::boundary_size_x_upper = 5 CoordBase::boundary_size_y_upper = 5 CoordBase::boundary_size_z_upper = 5 Driver::ghost_size = 5 GRHydro::GRHydro_stencil = 5 But I'm getting this error: CCTK_Proc569.err <== while executing schedule bin ReadInterpolate_ReadData, routine ReadInterpolate::ReadInterpolate_Read in thorn ReadInterpolate, file /home/mvxcombi/ET_Zelmani/Cactus/arrangements/EinsteinInitialData/ReadInterpolate/src/read.c:559: -> Skipping integer variable 'SPACEMASK::space_mask it=335925 tl=0 rl=0 c=569'. Do not know how to interpolate integers. ERROR from host bcn1814.usr.hlrn.de process 569 while executing schedule bin ReadInterpolate_ReadData, routine ReadInterpolate::ReadInterpolate_Read in thorn ReadInterpolate, file /home/mvxcombi/ET_Zelmani/Cactus/arrangements/EinsteinInitialData/ReadInterpolate/src/interpolate.cc:130: -> There were 70227 points in 27 variables that could not be set. cactus_etk_devel: /home/mvxcombi/ET_Zelmani/Cactus/arrangements/Carpet/Carpet/src/helpers.cc:275: int Carpet::Abort(const cGH*, int): Assertion `0' failed. Rank 569 with PID 1675935 received signal 6 Writing backtrace to AIC_interp/backtrace.569.txt The original grid has 3 ghost zones. I'm using the following parfile: # What to interpolate # ------------------- ReadInterpolate::files = "checkpoint.chkpt.it_335925" ReadInterpolate::max_number_of_read_variables = 30 ReadInterpolate::only_these_datasets = ":Y_e, :rho, :temperature, :vel, :Ax, :Ay, :Az, :psi6phi, :kxx, :kxy, :kxz, :kyy, :kyz, :kzz, :gxx, :gxy, :gxz, :gyy, :gyz, :gzz, :betax, :betay, :betaz, :alp" ReadInterpolate::minimum_reflevel = 0 # Default ReadInterpolate::maximum_reflevel = 1000 # Default # Interpolator settings # --------------------- ReadInterpolate::epsilon = 1.e-12 #2 # Default ReadInterpolate::interpolator_half_width = -1 # Default, use cctk_nghostzones ReadInterpolate::interpolator_name = "Lagrange polynomial interpolation" ReadInterpolate::interpolator_pars = " order = 3 boundary_off_centering_tolerance = {1. 1. 1. 1. 1. 1.} boundary_extrapolation_tolerance = {1.e-12 1.e-12 1.e-12 1.e-12 1.e-12 1.e-12} " # Fixing the read-in data # ----------------------- ReadInterpolate::shift_read_datasets_by[0] = 0. ReadInterpolate::shift_read_datasets_by[1] = 0. ReadInterpolate::shift_read_datasets_by[2] = 0. ReadInterpolate::enforce_symmetries_after_reading = no # Default ReadInterpolate::fix_cell_centered_origins = no # Default Any suggestions on how to solve this issue? Thanks a lot!! -------------- next part -------------- An HTML attachment was scrubbed... URL: From rhaas at mail.ubc.ca Fri Jul 25 10:31:33 2025 From: rhaas at mail.ubc.ca (Roland Haas) Date: Fri, 25 Jul 2025 08:31:33 -0700 Subject: [Users] Issue with ReadInterpolate In-Reply-To: References: Message-ID: <20250725083133.6918e245@haengie2.phas.ubc.ca> Hello Luciano, I am not quite clear what's going on, ideally one would need the new grid structure and the old. My guess is that you may need to make the new coarsest grid a bit smaller than the old one (by the difference in boundary sizes would be my guess). If I recall correctly then ReadInterpolate will (have to) fill in all boundary points and the boundary points stick out beyond the grid size you give with CoordBase::[xyz]{min,max}. Yours, Roland PS I should probably add code to avoid the "cannot interpolate integers" warning. > [CAUTION: Non-UBC Email] > > Hi people, > > I have used ReadInterpolate successfully for different applications in the past. I'm now trying to use it to interpolate onto a new grid that has 5 ghost zones i.e.: > > CoordBase::boundary_size_x_lower = 5 > CoordBase::boundary_size_y_lower = 5 > CoordBase::boundary_size_z_lower = 5 > CoordBase::boundary_size_x_upper = 5 > CoordBase::boundary_size_y_upper = 5 > CoordBase::boundary_size_z_upper = 5 > Driver::ghost_size = 5 > GRHydro::GRHydro_stencil = 5 > But I'm getting this error: > > CCTK_Proc569.err <== > while executing schedule bin ReadInterpolate_ReadData, routine ReadInterpolate::ReadInterpolate_Read > in thorn ReadInterpolate, file /home/mvxcombi/ET_Zelmani/Cactus/arrangements/EinsteinInitialData/ReadInterpolate/src/read.c:559: > -> Skipping integer variable 'SPACEMASK::space_mask it=335925 tl=0 rl=0 c=569'. Do not know how to interpolate integers. > ERROR from host bcn1814.usr.hlrn.de process 569 > while executing schedule bin ReadInterpolate_ReadData, routine ReadInterpolate::ReadInterpolate_Read > in thorn ReadInterpolate, file /home/mvxcombi/ET_Zelmani/Cactus/arrangements/EinsteinInitialData/ReadInterpolate/src/interpolate.cc:130: > -> There were 70227 points in 27 variables that could not be set. > cactus_etk_devel: /home/mvxcombi/ET_Zelmani/Cactus/arrangements/Carpet/Carpet/src/helpers.cc:275: int Carpet::Abort(const cGH*, int): Assertion `0' failed. > Rank 569 with PID 1675935 received signal 6 > Writing backtrace to AIC_interp/backtrace.569.txt > The original grid has 3 ghost zones. I'm using the following parfile: > # What to interpolate > # ------------------- > ReadInterpolate::files = "checkpoint.chkpt.it_335925" > ReadInterpolate::max_number_of_read_variables = 30 > ReadInterpolate::only_these_datasets = ":Y_e, :rho, :temperature, :vel, :Ax, :Ay, :Az, :psi6phi, :kxx, :kxy, :kxz, :kyy, :kyz, :kzz, :gxx, :gxy, :gxz, :gyy, :gyz, :gzz, :betax, :betay, :betaz, :alp" > ReadInterpolate::minimum_reflevel = 0 # Default > ReadInterpolate::maximum_reflevel = 1000 # Default > > # Interpolator settings > # --------------------- > ReadInterpolate::epsilon = 1.e-12 #2 # Default > ReadInterpolate::interpolator_half_width = -1 # Default, use cctk_nghostzones > ReadInterpolate::interpolator_name = "Lagrange polynomial interpolation" > ReadInterpolate::interpolator_pars = " > order = 3 > boundary_off_centering_tolerance = {1. 1. 1. 1. 1. 1.} > boundary_extrapolation_tolerance = {1.e-12 1.e-12 1.e-12 1.e-12 1.e-12 1.e-12} > " > > # Fixing the read-in data > # ----------------------- > ReadInterpolate::shift_read_datasets_by[0] = 0. > ReadInterpolate::shift_read_datasets_by[1] = 0. > ReadInterpolate::shift_read_datasets_by[2] = 0. > > ReadInterpolate::enforce_symmetries_after_reading = no # Default > ReadInterpolate::fix_cell_centered_origins = no # Default > > Any suggestions on how to solve this issue? > > Thanks a lot!! -- My email is as private as my paper mail. I therefore support encrypting and signing email messages. Get my PGP key from http://pgp.mit.edu . From users at einsteintoolkit.org Mon Jul 28 15:18:02 2025 From: users at einsteintoolkit.org (users at einsteintoolkit.org) Date: Mon, 28 Jul 2025 15:18:02 -0500 Subject: [Users] Agenda for Thursday's Meeting Message-ID: <6887dafa.Cnt3NXeEigaS1xZm%users@einsteintoolkit.org> Please update the Wiki with agenda items for Thursday's meeting. Thanks! https://docs.einsteintoolkit.org/et-docs/meeting_agenda --The Maintainers From users at einsteintoolkit.org Wed Jul 30 17:15:01 2025 From: users at einsteintoolkit.org (users at einsteintoolkit.org) Date: Wed, 30 Jul 2025 17:15:01 -0500 Subject: [Users] Einstein Toolkit Meeting Reminder Message-ID: <688a9965.v5aDmaVkVb+q7g05%users@einsteintoolkit.org> Hello, Please consider joining the weekly Einstein Toolkit phone call at 9:00 am US central time on Thursdays. For details on how to connect and what agenda items are to be discussed, use the link below. https://docs.einsteintoolkit.org/et-docs/Main_Page#Weekly_Users_Call --The Maintainers From yovqochevp at gmail.com Thu Jul 31 07:09:11 2025 From: yovqochevp at gmail.com (Pahlavon Yovqochev) Date: Thu, 31 Jul 2025 17:09:11 +0500 Subject: [Users] Issues Downloading einsteintoolkit.th and Running GetComponents Message-ID: Dear Einstein Toolkit Support Team, I hope this message finds you well. My name is Pakhlavon Yovkochev, and I am a PhD student in astrophysics at the *Institute of Fundamental and Applied Research* in Tashkent, Uzbekistan. I am currently trying to install and learn how to use the Einstein Toolkit, as I plan to use it for modeling magnetic fields around black holes in my research. I am currently trying to set up the Einstein Toolkit on my system (Ubuntu, running via WSL2). However, I?ve encountered some issues during the process and would greatly appreciate your assistance. Steps I followed: 1. Cloned the CRL repository: git clone https://github.com/gridaphobe/CRL.gitcd CRL 2. Attempted to run: ./GetComponents --parallel https://raw.githubusercontent.com/gridaphobe/CRL/main/einsteintoolkit.th But I received the following error: Error: Couldn't download url https://raw.githubusercontent.com/gridaphobe/CRL/main/einsteintoolkit.th correctly 3. I confirmed that the file einsteintoolkit.th does not exist at that URL (returns 404). I also tried other versions, like ET_2024_05, but still encountered problems initially. 4. I then tried downloading the .th file directly using wget, but again received: ERROR 404: Not Found. 5. Additionally, when I created an empty einsteintoolkit.th file and ran: ./GetComponents --parallel einsteintoolkit.th I received: 0 components checked out successfully. My Question: Could you point me to the correct and currently supported .th file URL for the latest stable Einstein Toolkit release? Also, is there any change in the recommended installation procedure or repository structure that I should be aware of? Thank you very much in advance for your time and support. Best regards, *Pakhlavon Yovkochev* -------------- next part -------------- An HTML attachment was scrubbed... URL: From rhaas at mail.ubc.ca Thu Jul 31 08:40:30 2025 From: rhaas at mail.ubc.ca (Roland Haas) Date: Thu, 31 Jul 2025 06:40:30 -0700 Subject: [Users] Issues Downloading einsteintoolkit.th and Running GetComponents In-Reply-To: References: Message-ID: <20250731064030.316d91f7@haengie2.phas.ubc.ca> Hello Pahlavon, > I am currently trying to set up the Einstein Toolkit on my system (Ubuntu, > running via WSL2). However, I?ve encountered some issues during the process > and would greatly appreciate your assistance. I will try and suggest some steps. > Steps I followed: > > 1. > > Cloned the CRL repository: > > git clone https://github.com/gridaphobe/CRL.gitcd CRL It is normally sufficient to download only the GetComponents script. > 2. > > Attempted to run: > > ./GetComponents --parallel > https://raw.githubusercontent.com/gridaphobe/CRL/main/einsteintoolkit.th This is the wrong URL, the thornlist lives in the "manifest" repository of the Einstein Toolkit. Please take a look at the CactusTutorial.ipynb notebook likned on the "First Steps" page of the Einstein Toolkit website (https://www.einsteintoolkit.org): https://github.com/einsteintoolkit/jupyter-et/blob/master/tutorial-server/notebooks/CactusTutorial.ipynb where we provide detailed instructions on how to download and compile. Note that most likely you will need something more than a laptop for magnetic field simulations of sufficient resolution. The relevant lines for you are: $(sudo -l sudo) su -c 'apt-get update' $(sudo -l sudo) su -c 'apt-get install -y gcc git numactl libgsl-dev libpapi-dev python3 python-is-python3 python3-pip libhwloc-dev libudev-dev cmake make rsync libopenmpi-dev libhdf5-openmpi-dev libfftw3-dev libssl-dev liblapack-dev g++ curl gfortran patch pkg-config libhdf5-dev libjpeg-turbo?-dev libyaml-cpp-dev' curl -kLO https://raw.githubusercontent.com/gridaphobe/CRL/ET_2025_05/GetComponents chmod a+x GetComponents ./GetComponents https://bitbucket.org/einsteintoolkit/manifest/raw/ET_2025_05/einsteintoolkit.th where the first two install the required software packages and the latter three download the code. > > But I received the following error: > > Error: Couldn't download url > https://raw.githubusercontent.com/gridaphobe/CRL/main/einsteintoolkit.th > correctly Did you encounter that URL on some (official) website or wiki or did you guess this on your own (in the former case I'd like to correct the website or wiki). > 5. > > Additionally, when I created an empty einsteintoolkit.th file and ran: > > ./GetComponents --parallel einsteintoolkit.th > > I received: > > 0 components checked out successfully. That is the correct output for an empty file. > Could you point me to the correct and currently supported .th file URL for > the latest stable Einstein Toolkit release? Please see above. > Also, is there any change in the recommended installation procedure or > repository structure that I should be aware of? No change. Yours, Roland -- My email is as private as my paper mail. I therefore support encrypting and signing email messages. Get my PGP key from http://pgp.mit.edu . From bill.gabella at gmail.com Thu Jul 31 10:32:59 2025 From: bill.gabella at gmail.com (Bill Gabella) Date: Thu, 31 Jul 2025 11:32:59 -0400 Subject: [Users] Meeting Minutes 2025-07-31 Message-ID: <3354b7bf-d50c-494f-8e8f-ab1a40ded788@gmail.com> Minutes for the Einstein Toolkit Meeting, 20250731. https://docs.einsteintoolkit.org/et-docs/Main_Page#Weekly_Users_Call 9am CDT Thursdays Present: Peter D, Leo W, Bill G, Deborah F, Maxwell R, Johnny T, Beyhan K, Hector I, Keith D, Yosef Z Chair? Leo, Minutes? Bill * BBH Gallery example (Zach and Peter) ?? Peter, still not yet done, been too busy. * Unanswered Questions on the mailing list https://www.einsteintoolkit.org/tools/unanswered.php ** 25 July SimFactory placeholder macros. Confusion with the @XXX@ macros and where they are set and changed. ** 23 July Share Your Feedback... Seems that it should not be in our mailing list, refers to software and network tools(?). * Open Tickets https://bitbucket.org/einsteintoolkit/tickets/issues?status=open&status=new&sort=-updated_on #2773, CarpetX thorndoc confused in InterLatex.pl.? Assigned to Steve. #2867, WaveToyX examples have HDF5 output error.? Not assigned. #2866, Problem building ET on MN5: OpenCL (Marenostrum 5).?? Not assigned. #2764, PUGH tries to free memory not allocated by malloc. Assigned to Beyhan K.? More work to be done. #963, Improve McLachlan accuracy.? Original post 2012.? Peter, we are trying to compare and see how it compares with NS simulations as well. * Tickets for Review https://bitbucket.org/einsteintoolkit/tickets/issues?status=new&status=open&sort=-updated_on&q=Please%20review Not much discussion. * Any Other Business ** Leo, question maybe for Steve, does anyone use the ET in clusters but through apptainer or singularity images?? Steve does this.? Leo is trying to do that and working with his cluster folks.? Bill is also interested in that.? Leo trying to setup with base layer that is operating system and then compilers and then external libraries, say external Lorene, and last a thornlist or par file.? The last image would behave as ET would.? Has it working locally but when tries with MPI on cluster with multiple nodes it hangs, problem with internode communication.? Peter, Steve has been able to run MPI jobs with containers...maybe using the right MPI tools with Slurm to run them. ** Johnny, question also for Hector, variables needed for CCE Exporter.? Want to use the CCE Export thorn, but do not write the time derivatives Lapse and Shift.? Deborah, MacLachlan does write it back.? Lean at Illinois, folks were working on something similar, Roland knows how to do this.? Something about the time stepping.? Hector, tried adding tensors to the Kranc file and stored them, whatever was done to write them to ADMBase, but see NaNs even though the evolution code does run normally.? Maxwell, Baikal has a similar problem with dt_alpha and use the right hand side, some subtlety on the iteration, because what was saved is a fourth-order RK step. https://einsteintoolkit.org/thornguide/EinsteinAnalysis/CCE_Export/documentation.html ** When is the developer hack-a-thon?? Peter, maybe in two weeks. Next Meeting, Thursday, 7 August 2025. https://docs.einsteintoolkit.org/et-docs/Meeting_agenda fyi, bill gabella From rhaas at mail.ubc.ca Thu Jul 31 10:45:56 2025 From: rhaas at mail.ubc.ca (Roland Haas) Date: Thu, 31 Jul 2025 08:45:56 -0700 Subject: [Users] Meeting Minutes 2025-07-31 In-Reply-To: <3354b7bf-d50c-494f-8e8f-ab1a40ded788@gmail.com> References: <3354b7bf-d50c-494f-8e8f-ab1a40ded788@gmail.com> Message-ID: <20250731084556.3a1681fe@haengie2.phas.ubc.ca> Hello all, > ** 23 July Share Your Feedback... > Seems that it should not be in our mailing list, refers to software and network tools(?). My fault, actually sorry. I thought this was going to the maintainer's list when I let it go through. Yours, Roland -- My email is as private as my paper mail. I therefore support encrypting and signing email messages. Get my PGP key from http://pgp.mit.edu . From wernecklr at gmail.com Thu Jul 31 12:26:08 2025 From: wernecklr at gmail.com (Leonardo Werneck) Date: Thu, 31 Jul 2025 10:26:08 -0700 Subject: [Users] Users Digest, Vol 184, Issue 14 In-Reply-To: References: Message-ID: Dear Iosif, I hope this message finds you well. I'm not sure you already got a reply, but in case you haven't, my understanding is that placeholders like @FOO@ are there to indicate things you need to change yourself, manually. So when you see something like: #SBATCH -A @ALLOCATION@ then it means you should replace @ALLOCATION@ with the name of your allocation in the specific cluster. Someone else can correct me if this is not correct, but that's how I usually handle these. Cheers, Leo On Fri, Jul 25, 2025 at 8:06?AM wrote: > Send Users mailing list submissions to > users at einsteintoolkit.org > > To subscribe or unsubscribe via the World Wide Web, visit > http://lists.einsteintoolkit.org/mailman/listinfo/users > or, via email, send a message with subject or body 'help' to > users-request at einsteintoolkit.org > > You can reach the person managing the list at > users-owner at einsteintoolkit.org > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of Users digest..." > > > Today's Topics: > > 1. SimFactory placeholder macros (IOSIF PANAGIOTIS) > 2. Issue with ReadInterpolate (Luciano Combi) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Fri, 25 Jul 2025 10:43:36 +0000 > From: IOSIF PANAGIOTIS > To: Einstein Toolkit Users > Subject: [Users] SimFactory placeholder macros > Message-ID: > < > AS8P250MB08637A69A316C8763E533B5FF959A at AS8P250MB0863.EURP250.PROD.OUTLOOK.COM > > > > Content-Type: text/plain; charset="iso-8859-1" > > Hi, > > I have been following this document, > https://einsteintoolkit.org/seminars/2022_02_24/index.html , to set up ET > on my cluster, working also on some example configuration files. > > While preparing the submit and run scripts, I wanted to check the > description of the @FOO@ macros that SimFactory uses. > > I checked these places but found no information. > > * > https://simfactory.bitbucket.io/simfactory2/userguide/ (no mention of > macros) > * > > https://docs.einsteintoolkit.org/et-docs/Simulation_Factory_Advanced_Tutorial > (1 mention of the @USER@ macro and a pointer to a section #Macros< > https://docs.einsteintoolkit.org/et-docs/Simulation_Factory_Advanced_Tutorial#Macros> > that seems missing) > * > https://docs.einsteintoolkit.org/et-docs/Configuring_a_new_machine (there > is no information on the submitscript and runscript) > > So I would like to ask: > > If e.g. I have this line in my submitscript: > #SBATCH -A @ALLOCATION@ > > Where is @ALLOCATION@ "read" from? > > My guess is that this is specified in the .ini file in this line: > allocation = > > Could you confirm if this is the case? > > Is there another source of documentation (that I might have missed) > describing what these @FOO@ placeholders do? > > Best regards, > Panagiotis > > > ------------------- > Panagiotis Iosif > postdoctoral researcher > Department of Physics, University of Trieste > Via Alfonso Valerio 2, Trieste 34127 > Italy > ------------------- > > -------------- next part -------------- > An HTML attachment was scrubbed... > URL: < > http://lists.einsteintoolkit.org/pipermail/users/attachments/20250725/dca98ab5/attachment-0001.htm > > > > ------------------------------ > > Message: 2 > Date: Fri, 25 Jul 2025 15:05:48 +0000 > From: Luciano Combi > To: "users at einsteintoolkit.org" > Subject: [Users] Issue with ReadInterpolate > Message-ID: > < > YQBPR0101MB544409A753F7D8E3ED3097D9CC59A at YQBPR0101MB5444.CANPRD01.PROD.OUTLOOK.COM > > > > Content-Type: text/plain; charset="iso-8859-1" > > Hi people, > > I have used ReadInterpolate successfully for different applications in the > past. I'm now trying to use it to interpolate onto a new grid that has 5 > ghost zones i.e.: > > CoordBase::boundary_size_x_lower = 5 > CoordBase::boundary_size_y_lower = 5 > CoordBase::boundary_size_z_lower = 5 > CoordBase::boundary_size_x_upper = 5 > CoordBase::boundary_size_y_upper = 5 > CoordBase::boundary_size_z_upper = 5 > Driver::ghost_size = 5 > GRHydro::GRHydro_stencil = 5 > But I'm getting this error: > > CCTK_Proc569.err <== > while executing schedule bin ReadInterpolate_ReadData, routine > ReadInterpolate::ReadInterpolate_Read > in thorn ReadInterpolate, file > /home/mvxcombi/ET_Zelmani/Cactus/arrangements/EinsteinInitialData/ReadInterpolate/src/read.c:559: > -> Skipping integer variable 'SPACEMASK::space_mask it=335925 tl=0 rl=0 > c=569'. Do not know how to interpolate integers. > ERROR from host bcn1814.usr.hlrn.de process 569 > while executing schedule bin ReadInterpolate_ReadData, routine > ReadInterpolate::ReadInterpolate_Read > in thorn ReadInterpolate, file > /home/mvxcombi/ET_Zelmani/Cactus/arrangements/EinsteinInitialData/ReadInterpolate/src/interpolate.cc:130: > -> There were 70227 points in 27 variables that could not be set. > cactus_etk_devel: > /home/mvxcombi/ET_Zelmani/Cactus/arrangements/Carpet/Carpet/src/helpers.cc:275: > int Carpet::Abort(const cGH*, int): Assertion `0' failed. > Rank 569 with PID 1675935 received signal 6 > Writing backtrace to AIC_interp/backtrace.569.txt > The original grid has 3 ghost zones. I'm using the following parfile: > # What to interpolate > # ------------------- > ReadInterpolate::files = > "checkpoint.chkpt.it_335925" > ReadInterpolate::max_number_of_read_variables = 30 > ReadInterpolate::only_these_datasets = ":Y_e, :rho, :temperature, > :vel, :Ax, :Ay, :Az, :psi6phi, :kxx, :kxy, :kxz, :kyy, :kyz, :kzz, :gxx, > :gxy, :gxz, :gyy, :gyz, :gzz, :betax, :betay, :betaz, :alp" > ReadInterpolate::minimum_reflevel = 0 # Default > ReadInterpolate::maximum_reflevel = 1000 # Default > > # Interpolator settings > # --------------------- > ReadInterpolate::epsilon = 1.e-12 #2 # Default > ReadInterpolate::interpolator_half_width = -1 # Default, use > cctk_nghostzones > ReadInterpolate::interpolator_name = "Lagrange polynomial > interpolation" > ReadInterpolate::interpolator_pars = " > order = 3 > boundary_off_centering_tolerance = {1. 1. 1. 1. 1. 1.} > boundary_extrapolation_tolerance = {1.e-12 1.e-12 1.e-12 1.e-12 1.e-12 > 1.e-12} > " > > # Fixing the read-in data > # ----------------------- > ReadInterpolate::shift_read_datasets_by[0] = 0. > ReadInterpolate::shift_read_datasets_by[1] = 0. > ReadInterpolate::shift_read_datasets_by[2] = 0. > > ReadInterpolate::enforce_symmetries_after_reading = no # Default > ReadInterpolate::fix_cell_centered_origins = no # Default > > Any suggestions on how to solve this issue? > > Thanks a lot!! > -------------- next part -------------- > An HTML attachment was scrubbed... > URL: < > http://lists.einsteintoolkit.org/pipermail/users/attachments/20250725/87444213/attachment.htm > > > > ------------------------------ > > Subject: Digest Footer > > _______________________________________________ > Users mailing list > Users at einsteintoolkit.org > http://lists.einsteintoolkit.org/mailman/listinfo/users > > > ------------------------------ > > End of Users Digest, Vol 184, Issue 14 > ************************************** > -------------- next part -------------- An HTML attachment was scrubbed... URL: