From sbrandt at cct.lsu.edu Tue Jul 1 10:48:46 2025 From: sbrandt at cct.lsu.edu (Steven Brandt) Date: Tue, 1 Jul 2025 10:48:46 -0500 Subject: [Users] Problems with output in CarpetX In-Reply-To: <40E50047-069B-4E89-B913-37D46EB9711D@gmail.com> References:

<40E50047-069B-4E89-B913-37D46EB9711D@gmail.com> Message-ID: <3a9c37e9-35bd-4065-ab97-e274ca115f9d@cct.lsu.edu> The Tutorial server has the full current release of the ETK, including SILO and OpenPMD. --Steve On 6/30/2025 11:53 AM, Erik Schnetter wrote: > Javiera > > Unfortunately I don't recall what software is installed on the ETK > tutorial server... I'm sure someone else can help us out here? > > -erik > >> On Jun 30, 2025, at 12:17, Javiera Hern?ndez Morales >> wrote: >> >> Hi Erik, >> >> Thanks for your answer. Maybe for more context I should mention that >> I am starting to learn how to use the ETK, and for that I am using >> the notebooks provided in the tutorial server (etk.cct.lsu.edu >> ), together with some of the ones used for >> the last ETK workshop. >> >> To answer your last question, I am not doing anything specific with >> the output, I wanted to get 3D data to see if that could help me >> visualise the output of the examples I did with the WaveToyX thorn. >> >> Unfortunately I am not familiar with SILO, so I tried to use the 1D >> .tsv files, but they only provide the values in a slice (for example >> the wave as a function of x, for y=z=0), but I wanted to create a 3D >> plot similar to the one found in the CreatingANewThorn-WaveEqn >> notebook from the tutorial server (I leave a screenshot attached), >> that is why I thought of outputting 3D. >> >> To use SILO files I understand that one has to have VisIt installed, >> however when I wanted to install it in the tutorial server via >> terminal, I got the error that I am not allowed to install software, >> which I suppose makes sense. >> >> In principle all I want to do is experiment with the WaveToyX thorn >> (and the other examples, like SpacetimeWaveToyX or FluxWaveToyX), >> changing parameters and see how things behave, but I am struggling a >> lot with the visualisation part. Do you have any recommendations that >> I could implement in the environment of the ETK Tutorial server? >> >> Thanks in advance. >> >> Best regards, >> >> Javiera H.M. >> >> >> >>> El 30.06.2025, a las 16:45, Erik Schnetter >>> escribi?: >>> >>> Javiera >>> >>> I recommend using 3D output in the way others are. CarpetX supports >>> many parameters, but not all parameter combinations are well tested, >>> or are intended for production runs. >>> >>> (1, 2) Yes, there are two ways of 3D ASCII (tsv) output. This is >>> very slow and is thus only intended for debugging. Therefore it >>> doesn't have all the features needed for comfortable output in >>> production runs. >>> >>> (3) Either OpenPMD or Silo output are the preferred ways for >>> production runs. These formats are also supported by post-processing >>> and visualization tools. These are not file formats, but they only >>> describe how the metadata are arranged (i.e. what names the >>> attributes have) in the output files. The actual file formats are >>> ADIOS2 (preferred, faster) or HDF5. OpenPMD uses ADIOS2 by default, >>> Silo uses HDF5 by default. >>> >>> I do not know why the OpenPMD/HDF5 output is not working. The error >>> message doesn't show enough detail. >>> >>> If you want HDF5 output then I recommend using Silo. This will >>> produce an HDF5 file. I'm not sure that's what you want because, >>> whatever you are going to do with the file, you will need to >>> understand the metadata in the file, and thus that tool or script >>> will need to understand either the OpenMPD or Silo layout. Otherwise >>> your HDF5 will look like a bag of 3D arrays, each array describing a >>> Fortran array, and it will be very difficult to find out which >>> component of which variable and which time step on which refinement >>> level is stored in what array: That's what the metadata describes. >>> >>> On the other hand, if you want to have an easy way to read the file, >>> then you could use e.g. the Julia or Python of C++ bindings of the >>> OpenMPD library, and you then wouldn't need to care whether the >>> low-level file format is ADIOS2 or HDF5. >>> >>> What do you want to do with the 3D output? >>> >>> -erik >>> >>>> On Jun 27, 2025, at 12:54, Javiera Hern?ndez Morales >>>> wrote: >>>> >>>> Hi all, >>>> >>>> I am starting to learn the use of CarpetX and for this I started >>>> using the WaveToyX thorn. I wanted to ask for guidance for three >>>> problems that I have encountered so far, all of them related to the >>>> output data: >>>> >>>> 1) I wanted to get 3D data, for example in .tsv format. Here I >>>> realised that there is an ambiguous definition in the documentation >>>> for the parameter ?out_tsv?, which in section 12.1 it says to >>>> control TSV output and have default value ?yes?, but in section 14 >>>> it says to control the 3D output in TSV and have default value >>>> ?no?. This last behaviour is the one that I encounter when running >>>> the parfiles. >>>> >>>> 2) When I set up ?out_tsv" to ?yes? and ?out_tsv_every? to a >>>> number, for example = 16, I get 1D output every 16 iterations, but >>>> 3D output in every iteration, therefore I suspect >>>> that??out_tsv_every??is only controlling the frequency of the 1D >>>> output, and I cannot find a way to control the frequency of 3D output. >>>> >>>> 3) I attempted to output data in HDF5 format but I did not succeed. >>>> I first set the parameter ?openpmd_format? to ?HDF5? and also set >>>> ?out_openpmd_vars? to the variables that I wanted, but I got an >>>> error in the first iteration: >>>> >>>> INFO (CarpetX): OutputGH: iteration 0, time 0.000000, run time 1 s >>>> INFO (CarpetX): OutputOpenPMD... >>>> INFO (CarpetX): Creating openPMD object... >>>> INFO (CarpetX): options: >>>> { >>>> "adios2": { >>>> "dataset": { >>>> "operators": [ >>>> ] >>>> } >>>> } >>>> } >>>> >>>> [et-juphub:699479] *** An error occurred in MPI_Comm_create_keyval >>>> [et-juphub:699479] *** reported by process [211877889,2] >>>> [et-juphub:699479] *** on communicator MPI_COMM_WORLD >>>> [et-juphub:699479] *** MPI_ERR_ARG: invalid argument of some other kind >>>> [et-juphub:699479] *** MPI_ERRORS_ARE_FATAL (processes in this communicator will now abort, >>>> [et-juphub:699479] *** and potentially your MPI job) >>>> [et-juphub:699473] PMIX ERROR: UNREACHABLE in file ../../../src/server/pmix_server.c at line 2193 >>>> [et-juphub:699473] 3 more processes have sent help message help-mpi-errors.txt / mpi_errors_are_fatal >>>> [et-juphub:699473] Set MCA parameter "orte_base_help_aggregate" to 0 to see all help / error messages >>>> Simfactory Done at date: Thu 26 Jun 2025 09:19:08 AM UTC >>>> And I did not know how to move on from there. >>>> I am working on the ETK Tutorial server, in case that is important. >>>> Thanks in advance to anyone who took the time to read until here, I >>>> will be looking forward to any piece of advice :) >>>> Best, >>>> Javiera H.M. >>>> _______________________________________________ >>>> Users mailing list >>>> Users at einsteintoolkit.org >>>> http://lists.einsteintoolkit.org/mailman/listinfo/users >>> >> > > > _______________________________________________ > Users mailing list > Users at einsteintoolkit.org > http://lists.einsteintoolkit.org/mailman/listinfo/users -------------- next part -------------- An HTML attachment was scrubbed... URL: From sbrandt at cct.lsu.edu Tue Jul 1 10:49:27 2025 From: sbrandt at cct.lsu.edu (Steven Brandt) Date: Tue, 1 Jul 2025 10:49:27 -0500 Subject: [Users] Problems with output in CarpetX In-Reply-To: <20250630102048.4b986a9b@fdea4908> References:

<40E50047-069B-4E89-B913-37D46EB9711D@gmail.com> <20250630102048.4b986a9b@fdea4908> Message-ID: <811e7662-3773-4980-8841-3ed340b8108e@cct.lsu.edu> I could add Visit. I included it on etk1 for the workshop. The only catch is it has to run headless. --Steve On 6/30/2025 12:20 PM, Roland Haas wrote: > Hello all, > > There's no VisIt instance installed on the tutorial server (it only > provides an interface using jupyter notebooks, no X11 server access). > > I believe Steve has written some Python code to visualize CarpetX data. > > Yours, > Roland > >> [CAUTION: Non-UBC Email] >> >> Javiera >> >> Unfortunately I don't recall what software is installed on the ETK tutorial server... I'm sure someone else can help us out here? >> >> -erik >> >>> On Jun 30, 2025, at 12:17, Javiera Hern?ndez Morales wrote: >>> >>> Hi Erik, >>> >>> Thanks for your answer. Maybe for more context I should mention that I am starting to learn how to use the ETK, and for that I am using the notebooks provided in the tutorial server (etk.cct.lsu.edu ), together with some of the ones used for the last ETK workshop. >>> >>> To answer your last question, I am not doing anything specific with the output, I wanted to get 3D data to see if that could help me visualise the output of the examples I did with the WaveToyX thorn. >>> >>> Unfortunately I am not familiar with SILO, so I tried to use the 1D .tsv files, but they only provide the values in a slice (for example the wave as a function of x, for y=z=0), but I wanted to create a 3D plot similar to the one found in the CreatingANewThorn-WaveEqn notebook from the tutorial server (I leave a screenshot attached), that is why I thought of outputting 3D. >>> >>> To use SILO files I understand that one has to have VisIt installed, however when I wanted to install it in the tutorial server via terminal, I got the error that I am not allowed to install software, which I suppose makes sense. >>> >>> In principle all I want to do is experiment with the WaveToyX thorn (and the other examples, like SpacetimeWaveToyX or FluxWaveToyX), changing parameters and see how things behave, but I am struggling a lot with the visualisation part. Do you have any recommendations that I could implement in the environment of the ETK Tutorial server? >>> >>> Thanks in advance. >>> >>> Best regards, >>> >>> Javiera H.M. >>> >>> >>> >>>> El 30.06.2025, a las 16:45, Erik Schnetter escribi?: >>>> >>>> Javiera >>>> >>>> I recommend using 3D output in the way others are. CarpetX supports many parameters, but not all parameter combinations are well tested, or are intended for production runs. >>>> >>>> (1, 2) Yes, there are two ways of 3D ASCII (tsv) output. This is very slow and is thus only intended for debugging. Therefore it doesn't have all the features needed for comfortable output in production runs. >>>> >>>> (3) Either OpenPMD or Silo output are the preferred ways for production runs. These formats are also supported by post-processing and visualization tools. These are not file formats, but they only describe how the metadata are arranged (i.e. what names the attributes have) in the output files. The actual file formats are ADIOS2 (preferred, faster) or HDF5. OpenPMD uses ADIOS2 by default, Silo uses HDF5 by default. >>>> >>>> I do not know why the OpenPMD/HDF5 output is not working. The error message doesn't show enough detail. >>>> >>>> If you want HDF5 output then I recommend using Silo. This will produce an HDF5 file. I'm not sure that's what you want because, whatever you are going to do with the file, you will need to understand the metadata in the file, and thus that tool or script will need to understand either the OpenMPD or Silo layout. Otherwise your HDF5 will look like a bag of 3D arrays, each array describing a Fortran array, and it will be very difficult to find out which component of which variable and which time step on which refinement level is stored in what array: That's what the metadata describes. >>>> >>>> On the other hand, if you want to have an easy way to read the file, then you could use e.g. the Julia or Python of C++ bindings of the OpenMPD library, and you then wouldn't need to care whether the low-level file format is ADIOS2 or HDF5. >>>> >>>> What do you want to do with the 3D output? >>>> >>>> -erik >>>> >>>>> On Jun 27, 2025, at 12:54, Javiera Hern?ndez Morales wrote: >>>>> >>>>> Hi all, >>>>> >>>>> I am starting to learn the use of CarpetX and for this I started using the WaveToyX thorn. I wanted to ask for guidance for three problems that I have encountered so far, all of them related to the output data: >>>>> >>>>> 1) I wanted to get 3D data, for example in .tsv format. Here I realised that there is an ambiguous definition in the documentation for the parameter ?out_tsv?, which in section 12.1 it says to control TSV output and have default value ?yes?, but in section 14 it says to control the 3D output in TSV and have default value ?no?. This last behaviour is the one that I encounter when running the parfiles. >>>>> >>>>> 2) When I set up ?out_tsv" to ?yes? and ?out_tsv_every? to a number, for example = 16, I get 1D output every 16 iterations, but 3D output in every iteration, therefore I suspect that ?out_tsv_every? is only controlling the frequency of the 1D output, and I cannot find a way to control the frequency of 3D output. >>>>> >>>>> 3) I attempted to output data in HDF5 format but I did not succeed. I first set the parameter ?openpmd_format? to ?HDF5? and also set ?out_openpmd_vars? to the variables that I wanted, but I got an error in the first iteration: >>>>> >>>>> INFO (CarpetX): OutputGH: iteration 0, time 0.000000, run time 1 s >>>>> INFO (CarpetX): OutputOpenPMD... >>>>> INFO (CarpetX): Creating openPMD object... >>>>> INFO (CarpetX): options: >>>>> { >>>>> "adios2": { >>>>> "dataset": { >>>>> "operators": [ >>>>> ] >>>>> } >>>>> } >>>>> } >>>>> >>>>> [et-juphub:699479] *** An error occurred in MPI_Comm_create_keyval >>>>> [et-juphub:699479] *** reported by process [211877889,2] >>>>> [et-juphub:699479] *** on communicator MPI_COMM_WORLD >>>>> [et-juphub:699479] *** MPI_ERR_ARG: invalid argument of some other kind >>>>> [et-juphub:699479] *** MPI_ERRORS_ARE_FATAL (processes in this communicator will now abort, >>>>> [et-juphub:699479] *** and potentially your MPI job) >>>>> [et-juphub:699473] PMIX ERROR: UNREACHABLE in file ../../../src/server/pmix_server.c at line 2193 >>>>> [et-juphub:699473] 3 more processes have sent help message help-mpi-errors.txt / mpi_errors_are_fatal >>>>> [et-juphub:699473] Set MCA parameter "orte_base_help_aggregate" to 0 to see all help / error messages >>>>> Simfactory Done at date: Thu 26 Jun 2025 09:19:08 AM UTC >>>>> And I did not know how to move on from there. >>>>> I am working on the ETK Tutorial server, in case that is important. >>>>> Thanks in advance to anyone who took the time to read until here, I will be looking forward to any piece of advice :) >>>>> Best, >>>>> Javiera H.M. >>>>> _______________________________________________ >>>>> Users mailing list >>>>> Users at einsteintoolkit.org >>>>> http://lists.einsteintoolkit.org/mailman/listinfo/users >>>> >>> > > Yours, > Roland > From bozzola.gabriele at gmail.com Wed Jul 2 09:41:56 2025 From: bozzola.gabriele at gmail.com (Gabriele Bozzola) Date: Wed, 2 Jul 2025 07:41:56 -0700 Subject: [Users] Problems with output in CarpetX In-Reply-To: <811e7662-3773-4980-8841-3ed340b8108e@cct.lsu.edu> References:

<40E50047-069B-4E89-B913-37D46EB9711D@gmail.com> <20250630102048.4b986a9b@fdea4908> <811e7662-3773-4980-8841-3ed340b8108e@cct.lsu.edu> Message-ID: Hi Javiera (and all), Last year, we added experimental support to reading OpenPMD files to kuibit . I haven't tried it for anything more than toy data, but the example we were working with is precisely the output generated with WaveToyX, as seen here . You could give it a try. kuibit will read the data and represent it with its HierarchicalGridData data structure, which can be sampled to a uniform grid and visualized with matplotlib (or you can slice it a given dimension to get a 1D/2D cut and visualize those). Best, Gabriele On Tue, Jul 1, 2025 at 8:49?AM Steven Brandt wrote: > I could add Visit. I included it on etk1 for the workshop. The only > catch is it has to run headless. > > --Steve > > On 6/30/2025 12:20 PM, Roland Haas wrote: > > Hello all, > > > > There's no VisIt instance installed on the tutorial server (it only > > provides an interface using jupyter notebooks, no X11 server access). > > > > I believe Steve has written some Python code to visualize CarpetX data. > > > > Yours, > > Roland > > > >> [CAUTION: Non-UBC Email] > >> > >> Javiera > >> > >> Unfortunately I don't recall what software is installed on the ETK > tutorial server... I'm sure someone else can help us out here? > >> > >> -erik > >> > >>> On Jun 30, 2025, at 12:17, Javiera Hern?ndez Morales < > javihernandezm at posteo.net> wrote: > >>> > >>> Hi Erik, > >>> > >>> Thanks for your answer. Maybe for more context I should mention that I > am starting to learn how to use the ETK, and for that I am using the > notebooks provided in the tutorial server (etk.cct.lsu.edu < > http://etk.cct.lsu.edu/>), together with some of the ones used for the > last ETK workshop. > >>> > >>> To answer your last question, I am not doing anything specific with > the output, I wanted to get 3D data to see if that could help me visualise > the output of the examples I did with the WaveToyX thorn. > >>> > >>> Unfortunately I am not familiar with SILO, so I tried to use the 1D > .tsv files, but they only provide the values in a slice (for example the > wave as a function of x, for y=z=0), but I wanted to create a 3D plot > similar to the one found in the CreatingANewThorn-WaveEqn notebook from the > tutorial server (I leave a screenshot attached), that is why I thought of > outputting 3D. > >>> > >>> To use SILO files I understand that one has to have VisIt installed, > however when I wanted to install it in the tutorial server via terminal, I > got the error that I am not allowed to install software, which I suppose > makes sense. > >>> > >>> In principle all I want to do is experiment with the WaveToyX thorn > (and the other examples, like SpacetimeWaveToyX or FluxWaveToyX), changing > parameters and see how things behave, but I am struggling a lot with the > visualisation part. Do you have any recommendations that I could implement > in the environment of the ETK Tutorial server? > >>> > >>> Thanks in advance. > >>> > >>> Best regards, > >>> > >>> Javiera H.M. > >>> > >>> > >>> > >>>> El 30.06.2025, a las 16:45, Erik Schnetter > escribi?: > >>>> > >>>> Javiera > >>>> > >>>> I recommend using 3D output in the way others are. CarpetX supports > many parameters, but not all parameter combinations are well tested, or are > intended for production runs. > >>>> > >>>> (1, 2) Yes, there are two ways of 3D ASCII (tsv) output. This is very > slow and is thus only intended for debugging. Therefore it doesn't have all > the features needed for comfortable output in production runs. > >>>> > >>>> (3) Either OpenPMD or Silo output are the preferred ways for > production runs. These formats are also supported by post-processing and > visualization tools. These are not file formats, but they only describe how > the metadata are arranged (i.e. what names the attributes have) in the > output files. The actual file formats are ADIOS2 (preferred, faster) or > HDF5. OpenPMD uses ADIOS2 by default, Silo uses HDF5 by default. > >>>> > >>>> I do not know why the OpenPMD/HDF5 output is not working. The error > message doesn't show enough detail. > >>>> > >>>> If you want HDF5 output then I recommend using Silo. This will > produce an HDF5 file. I'm not sure that's what you want because, whatever > you are going to do with the file, you will need to understand the metadata > in the file, and thus that tool or script will need to understand either > the OpenMPD or Silo layout. Otherwise your HDF5 will look like a bag of 3D > arrays, each array describing a Fortran array, and it will be very > difficult to find out which component of which variable and which time step > on which refinement level is stored in what array: That's what the metadata > describes. > >>>> > >>>> On the other hand, if you want to have an easy way to read the file, > then you could use e.g. the Julia or Python of C++ bindings of the OpenMPD > library, and you then wouldn't need to care whether the low-level file > format is ADIOS2 or HDF5. > >>>> > >>>> What do you want to do with the 3D output? > >>>> > >>>> -erik > >>>> > >>>>> On Jun 27, 2025, at 12:54, Javiera Hern?ndez Morales < > javihernandezm at posteo.net> wrote: > >>>>> > >>>>> Hi all, > >>>>> > >>>>> I am starting to learn the use of CarpetX and for this I started > using the WaveToyX thorn. I wanted to ask for guidance for three problems > that I have encountered so far, all of them related to the output data: > >>>>> > >>>>> 1) I wanted to get 3D data, for example in .tsv format. Here I > realised that there is an ambiguous definition in the documentation for the > parameter ?out_tsv?, which in section 12.1 it says to control TSV output > and have default value ?yes?, but in section 14 it says to control the 3D > output in TSV and have default value ?no?. This last behaviour is the one > that I encounter when running the parfiles. > >>>>> > >>>>> 2) When I set up ?out_tsv" to ?yes? and ?out_tsv_every? to a number, > for example = 16, I get 1D output every 16 iterations, but 3D output in > every iteration, therefore I suspect that ?out_tsv_every? is only > controlling the frequency of the 1D output, and I cannot find a way to > control the frequency of 3D output. > >>>>> > >>>>> 3) I attempted to output data in HDF5 format but I did not succeed. > I first set the parameter ?openpmd_format? to ?HDF5? and also set > ?out_openpmd_vars? to the variables that I wanted, but I got an error in > the first iteration: > >>>>> > >>>>> INFO (CarpetX): OutputGH: iteration 0, time 0.000000, run time 1 s > >>>>> INFO (CarpetX): OutputOpenPMD... > >>>>> INFO (CarpetX): Creating openPMD object... > >>>>> INFO (CarpetX): options: > >>>>> { > >>>>> "adios2": { > >>>>> "dataset": { > >>>>> "operators": [ > >>>>> ] > >>>>> } > >>>>> } > >>>>> } > >>>>> > >>>>> [et-juphub:699479] *** An error occurred in MPI_Comm_create_keyval > >>>>> [et-juphub:699479] *** reported by process [211877889,2] > >>>>> [et-juphub:699479] *** on communicator MPI_COMM_WORLD > >>>>> [et-juphub:699479] *** MPI_ERR_ARG: invalid argument of some other > kind > >>>>> [et-juphub:699479] *** MPI_ERRORS_ARE_FATAL (processes in this > communicator will now abort, > >>>>> [et-juphub:699479] *** and potentially your MPI job) > >>>>> [et-juphub:699473] PMIX ERROR: UNREACHABLE in file > ../../../src/server/pmix_server.c at line 2193 > >>>>> [et-juphub:699473] 3 more processes have sent help message > help-mpi-errors.txt / mpi_errors_are_fatal > >>>>> [et-juphub:699473] Set MCA parameter "orte_base_help_aggregate" to 0 > to see all help / error messages > >>>>> Simfactory Done at date: Thu 26 Jun 2025 09:19:08 AM UTC > >>>>> And I did not know how to move on from there. > >>>>> I am working on the ETK Tutorial server, in case that is important. > >>>>> Thanks in advance to anyone who took the time to read until here, I > will be looking forward to any piece of advice :) > >>>>> Best, > >>>>> Javiera H.M. > >>>>> _______________________________________________ > >>>>> Users mailing list > >>>>> Users at einsteintoolkit.org > >>>>> http://lists.einsteintoolkit.org/mailman/listinfo/users > >>>> > >>> > > > > Yours, > > Roland > > > _______________________________________________ > Users mailing list > Users at einsteintoolkit.org > http://lists.einsteintoolkit.org/mailman/listinfo/users > -------------- next part -------------- An HTML attachment was scrubbed... URL: From sbrandt at cct.lsu.edu Wed Jul 2 13:50:11 2025 From: sbrandt at cct.lsu.edu (Steven Brandt) Date: Wed, 2 Jul 2025 13:50:11 -0500 Subject: [Users] Problems with output in CarpetX In-Reply-To: References:

<40E50047-069B-4E89-B913-37D46EB9711D@gmail.com> <20250630102048.4b986a9b@fdea4908> <811e7662-3773-4980-8841-3ed340b8108e@cct.lsu.edu> Message-ID: <1dc0f5bb-f772-4220-b0d6-98266d78228f@cct.lsu.edu> Some of the tutorials on ETK1 use Kuibit to plot data from CarpetX runs, including the basic one on building and running with the ET. --Steve On 7/2/2025 9:41 AM, Gabriele Bozzola wrote: > Hi Javiera (and all), > > Last year, we added experimental support > > to reading OpenPMD files to kuibit . > > I haven't tried it for anything more than toy data, but the example we > were working with is precisely the output generated with WaveToyX, as > seen here > . > > You could give it a try. kuibit will read the data and represent it > with its HierarchicalGridData data structure, which can be sampled to > a uniform grid and visualized with matplotlib (or you can slice it a > given dimension to get a 1D/2D cut and visualize those). > > Best, > Gabriele > > > On Tue, Jul 1, 2025 at 8:49?AM Steven Brandt wrote: > > I could add Visit. I included it on etk1 for the workshop. The only > catch is it has to run headless. > > --Steve > > On 6/30/2025 12:20 PM, Roland Haas wrote: > > Hello all, > > > > There's no VisIt instance installed on the tutorial server (it only > > provides an interface using jupyter notebooks, no X11 server > access). > > > > I believe Steve has written some Python code to visualize > CarpetX data. > > > > Yours, > > Roland > > > >> [CAUTION: Non-UBC Email] > >> > >> Javiera > >> > >> Unfortunately I don't recall what software is installed on the > ETK tutorial server... I'm sure someone else can help us out here? > >> > >> -erik > >> > >>> On Jun 30, 2025, at 12:17, Javiera Hern?ndez Morales > wrote: > >>> > >>> Hi Erik, > >>> > >>> Thanks for your answer. Maybe for more context I should > mention that I am starting to learn how to use the ETK, and for > that I am using the notebooks provided in the tutorial server > (etk.cct.lsu.edu > ), together with some of the ones used > for the last ETK workshop. > >>> > >>> To answer your last question, I am not doing anything specific > with the output, I wanted to get 3D data to see if that could help > me visualise the output of the examples I did with the WaveToyX thorn. > >>> > >>> Unfortunately I am not familiar with SILO, so I tried to use > the 1D .tsv files, but they only provide the values in a slice > (for example the wave as a function of x, for y=z=0), but I wanted > to create a 3D plot similar to the one found in the > CreatingANewThorn-WaveEqn notebook from the tutorial server (I > leave a screenshot attached), that is why I thought of outputting 3D. > >>> > >>> To use SILO files I understand that one has to have VisIt > installed, however when I wanted to install it in the tutorial > server via terminal, I got the error that I am not allowed to > install software, which I suppose makes sense. > >>> > >>> In principle all I want to do is experiment with the WaveToyX > thorn (and the other examples, like SpacetimeWaveToyX or > FluxWaveToyX), changing parameters and see how things behave, but > I am struggling a lot with the visualisation part. Do you have any > recommendations that I could implement in the environment of the > ETK Tutorial server? > >>> > >>> Thanks in advance. > >>> > >>> Best regards, > >>> > >>> Javiera H.M. > >>> > >>> > >>> > >>>> El 30.06.2025, a las 16:45, Erik Schnetter > escribi?: > >>>> > >>>> Javiera > >>>> > >>>> I recommend using 3D output in the way others are. CarpetX > supports many parameters, but not all parameter combinations are > well tested, or are intended for production runs. > >>>> > >>>> (1, 2) Yes, there are two ways of 3D ASCII (tsv) output. This > is very slow and is thus only intended for debugging. Therefore it > doesn't have all the features needed for comfortable output in > production runs. > >>>> > >>>> (3) Either OpenPMD or Silo output are the preferred ways for > production runs. These formats are also supported by > post-processing and visualization tools. These are not file > formats, but they only describe how the metadata are arranged > (i.e. what names the attributes have) in the output files. The > actual file formats are ADIOS2 (preferred, faster) or HDF5. > OpenPMD uses ADIOS2 by default, Silo uses HDF5 by default. > >>>> > >>>> I do not know why the OpenPMD/HDF5 output is not working. The > error message doesn't show enough detail. > >>>> > >>>> If you want HDF5 output then I recommend using Silo. This > will produce an HDF5 file. I'm not sure that's what you want > because, whatever you are going to do with the file, you will need > to understand the metadata in the file, and thus that tool or > script will need to understand either the OpenMPD or Silo layout. > Otherwise your HDF5 will look like a bag of 3D arrays, each array > describing a Fortran array, and it will be very difficult to find > out which component of which variable and which time step on which > refinement level is stored in what array: That's what the metadata > describes. > >>>> > >>>> On the other hand, if you want to have an easy way to read > the file, then you could use e.g. the Julia or Python of C++ > bindings of the OpenMPD library, and you then wouldn't need to > care whether the low-level file format is ADIOS2 or HDF5. > >>>> > >>>> What do you want to do with the 3D output? > >>>> > >>>> -erik > >>>> > >>>>> On Jun 27, 2025, at 12:54, Javiera Hern?ndez Morales > wrote: > >>>>> > >>>>> Hi all, > >>>>> > >>>>> I am starting to learn the use of CarpetX and for this I > started using the WaveToyX thorn. I wanted to ask for guidance for > three problems that I have encountered so far, all of them related > to the output data: > >>>>> > >>>>> 1) I wanted to get 3D data, for example in .tsv format. Here > I realised that there is an ambiguous definition in the > documentation for the parameter ?out_tsv?, which in section 12.1 > it says to control TSV output and have default value ?yes?, but in > section 14 it says to control the 3D output in TSV and have > default value ?no?. This last behaviour is the one that I > encounter when running the parfiles. > >>>>> > >>>>> 2) When I set up ?out_tsv" to ?yes? and ?out_tsv_every? to a > number, for example = 16, I get 1D output every 16 iterations, but > 3D output in every iteration, therefore I suspect that > ?out_tsv_every? is only controlling the frequency of the 1D > output, and I cannot find a way to control the frequency of 3D output. > >>>>> > >>>>> 3) I attempted to output data in HDF5 format but I did not > succeed. I first set the parameter ?openpmd_format? to ?HDF5? and > also set ?out_openpmd_vars? to the variables that I wanted, but I > got an error in the first iteration: > >>>>> > >>>>> INFO (CarpetX): OutputGH: iteration 0, time 0.000000, run > time 1 s > >>>>> INFO (CarpetX): OutputOpenPMD... > >>>>> INFO (CarpetX): Creating openPMD object... > >>>>> INFO (CarpetX):? ?options: > >>>>>? ? { > >>>>>? ? ? "adios2": { > >>>>>? ? ? ? "dataset": { > >>>>>? ? ? ? ? "operators": [ > >>>>>? ? ? ? ? ] > >>>>>? ? ? ? } > >>>>>? ? ? } > >>>>>? ? } > >>>>> > >>>>> [et-juphub:699479] *** An error occurred in > MPI_Comm_create_keyval > >>>>> [et-juphub:699479] *** reported by process [211877889,2] > >>>>> [et-juphub:699479] *** on communicator MPI_COMM_WORLD > >>>>> [et-juphub:699479] *** MPI_ERR_ARG: invalid argument of some > other kind > >>>>> [et-juphub:699479] *** MPI_ERRORS_ARE_FATAL (processes in > this communicator will now abort, > >>>>> [et-juphub:699479] ***? ? and potentially your MPI job) > >>>>> [et-juphub:699473] PMIX ERROR: UNREACHABLE in file > ../../../src/server/pmix_server.c at line 2193 > >>>>> [et-juphub:699473] 3 more processes have sent help message > help-mpi-errors.txt / mpi_errors_are_fatal > >>>>> [et-juphub:699473] Set MCA parameter > "orte_base_help_aggregate" to 0 to see all help / error messages > >>>>> Simfactory Done at date: Thu 26 Jun 2025 09:19:08 AM UTC > >>>>> And I did not know how to move on from there. > >>>>> I am working on the ETK Tutorial server, in case that is > important. > >>>>> Thanks in advance to anyone who took the time to read until > here, I will be looking forward to any piece of advice :) > >>>>> Best, > >>>>> Javiera H.M. > >>>>> _______________________________________________ > >>>>> Users mailing list > >>>>> Users at einsteintoolkit.org > >>>>> http://lists.einsteintoolkit.org/mailman/listinfo/users > >>>> > >>> > > > > Yours, > > Roland > > > _______________________________________________ > Users mailing list > Users at einsteintoolkit.org > http://lists.einsteintoolkit.org/mailman/listinfo/users > -------------- next part -------------- An HTML attachment was scrubbed... URL: From users at einsteintoolkit.org Wed Jul 2 17:15:02 2025 From: users at einsteintoolkit.org (users at einsteintoolkit.org) Date: Wed, 02 Jul 2025 17:15:02 -0500 Subject: [Users] Einstein Toolkit Meeting Reminder Message-ID: <6865af66.UXzNK7SyEF0dlsmd%users@einsteintoolkit.org> Hello, Please consider joining the weekly Einstein Toolkit phone call at 9:00 am US central time on Thursdays. For details on how to connect and what agenda items are to be discussed, use the link below. https://docs.einsteintoolkit.org/et-docs/Main_Page#Weekly_Users_Call --The Maintainers From javihernandezm at posteo.net Thu Jul 3 03:36:38 2025 From: javihernandezm at posteo.net (=?utf-8?Q?Javiera_Hern=C3=A1ndez_Morales?=) Date: Thu, 3 Jul 2025 08:36:38 +0000 Subject: [Users] Problems with output in CarpetX In-Reply-To: <1dc0f5bb-f772-4220-b0d6-98266d78228f@cct.lsu.edu> References:

<40E50047-069B-4E89-B913-37D46EB9711D@gmail.com> <20250630102048.4b986a9b@fdea4908> <811e7662-3773-4980-8841-3ed340b8108e@cct.lsu.edu> <1dc0f5bb-f772-4220-b0d6-98266d78228f@cct.lsu.edu> Message-ID: Hi all, I used the code from the tutorials from the last workshop that use kuibit and now I can indeed visualise the results. Thanks a lot for all the help! :) Best, Javiera > El 02.07.2025, a las 20:50, Steven Brandt escribi?: > > Some of the tutorials on ETK1 use Kuibit to plot data from CarpetX runs, including the basic one on building and running with the ET. > > --Steve > > On 7/2/2025 9:41 AM, Gabriele Bozzola wrote: >> Hi Javiera (and all), >> >> Last year, we added experimental support to reading OpenPMD files to kuibit . >> >> I haven't tried it for anything more than toy data, but the example we were working with is precisely the output generated with WaveToyX, as seen here . >> >> You could give it a try. kuibit will read the data and represent it with its HierarchicalGridData data structure, which can be sampled to a uniform grid and visualized with matplotlib (or you can slice it a given dimension to get a 1D/2D cut and visualize those). >> >> Best, >> Gabriele >> >> >> On Tue, Jul 1, 2025 at 8:49?AM Steven Brandt > wrote: >>> I could add Visit. I included it on etk1 for the workshop. The only >>> catch is it has to run headless. >>> >>> --Steve >>> >>> On 6/30/2025 12:20 PM, Roland Haas wrote: >>> > Hello all, >>> > >>> > There's no VisIt instance installed on the tutorial server (it only >>> > provides an interface using jupyter notebooks, no X11 server access). >>> > >>> > I believe Steve has written some Python code to visualize CarpetX data. >>> > >>> > Yours, >>> > Roland >>> > >>> >> [CAUTION: Non-UBC Email] >>> >> >>> >> Javiera >>> >> >>> >> Unfortunately I don't recall what software is installed on the ETK tutorial server... I'm sure someone else can help us out here? >>> >> >>> >> -erik >>> >> >>> >>> On Jun 30, 2025, at 12:17, Javiera Hern?ndez Morales > wrote: >>> >>> >>> >>> Hi Erik, >>> >>> >>> >>> Thanks for your answer. Maybe for more context I should mention that I am starting to learn how to use the ETK, and for that I am using the notebooks provided in the tutorial server (etk.cct.lsu.edu ), together with some of the ones used for the last ETK workshop. >>> >>> >>> >>> To answer your last question, I am not doing anything specific with the output, I wanted to get 3D data to see if that could help me visualise the output of the examples I did with the WaveToyX thorn. >>> >>> >>> >>> Unfortunately I am not familiar with SILO, so I tried to use the 1D .tsv files, but they only provide the values in a slice (for example the wave as a function of x, for y=z=0), but I wanted to create a 3D plot similar to the one found in the CreatingANewThorn-WaveEqn notebook from the tutorial server (I leave a screenshot attached), that is why I thought of outputting 3D. >>> >>> >>> >>> To use SILO files I understand that one has to have VisIt installed, however when I wanted to install it in the tutorial server via terminal, I got the error that I am not allowed to install software, which I suppose makes sense. >>> >>> >>> >>> In principle all I want to do is experiment with the WaveToyX thorn (and the other examples, like SpacetimeWaveToyX or FluxWaveToyX), changing parameters and see how things behave, but I am struggling a lot with the visualisation part. Do you have any recommendations that I could implement in the environment of the ETK Tutorial server? >>> >>> >>> >>> Thanks in advance. >>> >>> >>> >>> Best regards, >>> >>> >>> >>> Javiera H.M. >>> >>> >>> >>> >>> >>> >>> >>>> El 30.06.2025, a las 16:45, Erik Schnetter > escribi?: >>> >>>> >>> >>>> Javiera >>> >>>> >>> >>>> I recommend using 3D output in the way others are. CarpetX supports many parameters, but not all parameter combinations are well tested, or are intended for production runs. >>> >>>> >>> >>>> (1, 2) Yes, there are two ways of 3D ASCII (tsv) output. This is very slow and is thus only intended for debugging. Therefore it doesn't have all the features needed for comfortable output in production runs. >>> >>>> >>> >>>> (3) Either OpenPMD or Silo output are the preferred ways for production runs. These formats are also supported by post-processing and visualization tools. These are not file formats, but they only describe how the metadata are arranged (i.e. what names the attributes have) in the output files. The actual file formats are ADIOS2 (preferred, faster) or HDF5. OpenPMD uses ADIOS2 by default, Silo uses HDF5 by default. >>> >>>> >>> >>>> I do not know why the OpenPMD/HDF5 output is not working. The error message doesn't show enough detail. >>> >>>> >>> >>>> If you want HDF5 output then I recommend using Silo. This will produce an HDF5 file. I'm not sure that's what you want because, whatever you are going to do with the file, you will need to understand the metadata in the file, and thus that tool or script will need to understand either the OpenMPD or Silo layout. Otherwise your HDF5 will look like a bag of 3D arrays, each array describing a Fortran array, and it will be very difficult to find out which component of which variable and which time step on which refinement level is stored in what array: That's what the metadata describes. >>> >>>> >>> >>>> On the other hand, if you want to have an easy way to read the file, then you could use e.g. the Julia or Python of C++ bindings of the OpenMPD library, and you then wouldn't need to care whether the low-level file format is ADIOS2 or HDF5. >>> >>>> >>> >>>> What do you want to do with the 3D output? >>> >>>> >>> >>>> -erik >>> >>>> >>> >>>>> On Jun 27, 2025, at 12:54, Javiera Hern?ndez Morales > wrote: >>> >>>>> >>> >>>>> Hi all, >>> >>>>> >>> >>>>> I am starting to learn the use of CarpetX and for this I started using the WaveToyX thorn. I wanted to ask for guidance for three problems that I have encountered so far, all of them related to the output data: >>> >>>>> >>> >>>>> 1) I wanted to get 3D data, for example in .tsv format. Here I realised that there is an ambiguous definition in the documentation for the parameter ?out_tsv?, which in section 12.1 it says to control TSV output and have default value ?yes?, but in section 14 it says to control the 3D output in TSV and have default value ?no?. This last behaviour is the one that I encounter when running the parfiles. >>> >>>>> >>> >>>>> 2) When I set up ?out_tsv" to ?yes? and ?out_tsv_every? to a number, for example = 16, I get 1D output every 16 iterations, but 3D output in every iteration, therefore I suspect that ?out_tsv_every? is only controlling the frequency of the 1D output, and I cannot find a way to control the frequency of 3D output. >>> >>>>> >>> >>>>> 3) I attempted to output data in HDF5 format but I did not succeed. I first set the parameter ?openpmd_format? to ?HDF5? and also set ?out_openpmd_vars? to the variables that I wanted, but I got an error in the first iteration: >>> >>>>> >>> >>>>> INFO (CarpetX): OutputGH: iteration 0, time 0.000000, run time 1 s >>> >>>>> INFO (CarpetX): OutputOpenPMD... >>> >>>>> INFO (CarpetX): Creating openPMD object... >>> >>>>> INFO (CarpetX): options: >>> >>>>> { >>> >>>>> "adios2": { >>> >>>>> "dataset": { >>> >>>>> "operators": [ >>> >>>>> ] >>> >>>>> } >>> >>>>> } >>> >>>>> } >>> >>>>> >>> >>>>> [et-juphub:699479] *** An error occurred in MPI_Comm_create_keyval >>> >>>>> [et-juphub:699479] *** reported by process [211877889,2] >>> >>>>> [et-juphub:699479] *** on communicator MPI_COMM_WORLD >>> >>>>> [et-juphub:699479] *** MPI_ERR_ARG: invalid argument of some other kind >>> >>>>> [et-juphub:699479] *** MPI_ERRORS_ARE_FATAL (processes in this communicator will now abort, >>> >>>>> [et-juphub:699479] *** and potentially your MPI job) >>> >>>>> [et-juphub:699473] PMIX ERROR: UNREACHABLE in file ../../../src/server/pmix_server.c at line 2193 >>> >>>>> [et-juphub:699473] 3 more processes have sent help message help-mpi-errors.txt / mpi_errors_are_fatal >>> >>>>> [et-juphub:699473] Set MCA parameter "orte_base_help_aggregate" to 0 to see all help / error messages >>> >>>>> Simfactory Done at date: Thu 26 Jun 2025 09:19:08 AM UTC >>> >>>>> And I did not know how to move on from there. >>> >>>>> I am working on the ETK Tutorial server, in case that is important. >>> >>>>> Thanks in advance to anyone who took the time to read until here, I will be looking forward to any piece of advice :) >>> >>>>> Best, >>> >>>>> Javiera H.M. >>> >>>>> _______________________________________________ >>> >>>>> Users mailing list >>> >>>>> Users at einsteintoolkit.org >>> >>>>> http://lists.einsteintoolkit.org/mailman/listinfo/users >>> >>>> >>> >>> >>> > >>> > Yours, >>> > Roland >>> > >>> _______________________________________________ >>> Users mailing list >>> Users at einsteintoolkit.org >>> http://lists.einsteintoolkit.org/mailman/listinfo/users -------------- next part -------------- An HTML attachment was scrubbed... URL: From rhaas at phas.ubc.ca Thu Jul 3 19:32:59 2025 From: rhaas at phas.ubc.ca (Roland Haas) Date: Thu, 3 Jul 2025 17:32:59 -0700 Subject: [Users] meeting minutes for 2025-07-03 Message-ID: <20250703173250.16a785f4@haengie2.phas.ubc.ca> Present: Leo, Roland, Peter, Beyhan, Keith, Wei Sun, Zach gallery examples ================ * still lacking updates for BBH gallery example for 2025_05 and 2024_11 gcc15 failures ============== * Zach suggested to compile ET with newest flavor of compilers regularly * Roland mentions that we could have a weekly build using the new compiler using github, Ubuntu 25.04 has gcc15, Zach suggests there may even be a gcc-latest or so image Benchmarking GRHayLHD simulations ================================= * Wei, Zach and Leo discussed results of benchmarking a GRHayLHD simulation for optimal speed varying ranks and threads * tests were done on NERSC Perlmutter (CPU only) * suggest to try both with and without IO enabled * suggest to check mpirun options to have correct affinity to cores / numa-nodes etc. Yours, Roland -- My email is as private as my paper mail. I therefore support encrypting and signing email messages. Get my PGP key from http://pgp.mit.edu . -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: