Dear Roland, Thank you for your reply. The attached file is .err file corresponding to the simulation of binary star with different central enthalpy. Can you have a look at it ?Thank you. For not able to track multiple stars, does it occur in binary blackhole case(http://einsteintoolkit.org/gallery/bbh/index.html) ? Best regards, Chia-Hui ________________________________ 寄件者: Roland Haas rhaas@illinois.edu 寄件日期: 2018年5月13日 下午 09:28:51 收件者: 林家暉 副本: Einstein Toolkit Users 主旨: Re: [Users] problems of reproducing the simulation of binary neutron star with different mass

Hello Chia-Hui Lin,

changing the central enthalpy seems like the right thing to do to change masses in LORENE since it prescribes the central enthalpy of each star.

The backtrace that you showed and the error message is unfortunately not detailed enough to make a guess as to what could have gone wrong. This output is produced by a call to CCTK_Error() however the CCTK_Error function takes an error string as an argument which is not present in the lines that you have copied. It is likely in the *.err file that the simulation produced and you should try and include that file as an attachment in your email.

If you have non-equal mass stars then you have to remove the 180 degree rotation symmetry from the parameter files. Ie you have to change the lines (these are scattered throughout throughout the par file)

ActiveThorns = "RotatingSymmetry180 ReflectionSymmetry" CoordBase::xmin = 0.00 CoordBase::boundary_shiftout_x_lower = 1 CarpetRegrid2::symmetry_rotating180 = "yes" Hydro_Analysis::Hydro_Analysis_rho_max_loc_only_positive_x = "true" Hydro_Analysis::Hydro_Analysis_average_multiple_maxima_locations = "yes"

to

ActiveThorns = "ReflectionSymmetry" CoordBase::xmin = 396.00 CoordBase::boundary_shiftout_x_lower = 0 CarpetRegrid2::symmetry_rotating180 = "no" Hydro_Analysis::Hydro_Analysis_rho_max_loc_only_positive_x = "false" Hydro_Analysis::Hydro_Analysis_average_multiple_maxima_locations = "no"

Please note though that currently the thorn NSTracker and HydroAnalysis *cannot* properly track multiple stars. It makes the hard-coded assumption that if one of the stars is at (x,y,z) the other one is at (-x,-y,z) which is only true for equal mass neutron stars. This will mean that one of the stars will eventually leave the box that is supposed to track it.

Yours, Roland

Dear whom it may concern,

I am trying to reproduce the simulation of binary neutron star (http://einsteintoolkit.org/gallery/bns/index.html) with different mass. I use Lorene code to produce the initial data . The parameter files I used is locate in /Lorene/Codes/Bin_star/Parameters/Polytrope_irrot/Test_GR . And I changed central enthalpy of each star in par_init.d to get different mass (Is there other way I can get different mass?). And then I simulate with the ET. However I met some problems:

1.I requested 48 cores with 10 hours to run the code , however the code terminated after running ~4 hours. And the end of .err file reads:

cactus_sim: /global/u1/c/chlin/Cactus/arrangements/Carpet/Carpet/src/helpers.cc:275: int Carpet::Abort(const _cGH *, int): Assertion `0' failed.

cactus_sim: /global/u1/c/chlin/Cactus/arrangements/Carpet/Carpet/src/helpers.cc:275: int Carpet::Abort(const _cGH *, int): Assertion `0' failed.

Rank 0 with PID 47930 received signal 6

Writing backtrace to nsnstohmns/backtrace.0.txt

Rank 1 with PID 47931 received signal 6

Writing backtrace to nsnstohmns/backtrace.1.txt

cactus_sim: /global/u1/c/chlin/Cactus/arrangements/Carpet/Carpet/src/helpers.cc:275: int Carpet::Abort(const _cGH *, int): Assertion `0' failed.

cactus_sim: /global/u1/c/chlin/Cactus/arrangements/Carpet/Carpet/src/helpers.cc:275: int Carpet::Abort(const _cGH *, int): Assertion `0' failed.

Rank 2 with PID 13239 received signal 6

Rank 3 with PID 13240 received signal 6

Writing backtrace to nsnstohmns/backtrace.2.txt

Writing backtrace to nsnstohmns/backtrace.3.txt

srun: error: nid05930: tasks 2-3: Aborted

srun: Terminating job step 9192104.0

srun: error: nid05920: tasks 0-1: Aborted

The code is aborted without completing the whole process until the merger. How can I perform the complete process?

2. When I use visit to do the visualization ,I found the file rho.xy.h5 only has half of the whole simulation as :

[cid:e66a53d5-d406-4599-997f-00e90a864dc0]

The tutorial(scripts.tar.gzhttp://einsteintoolkit.org/gallery/bns/scripts.tar.gz) tells that I should use symmetric reflection to perform the whole visualization. However I cannot use this method since I simulated binary star with different mass . So there is no symmetry. How can I get the whole picture instead of only half ?

Thank you.

Best regards,

Chia-Hui Lin

-- My email is as private as my paper mail. I therefore support encrypting and signing email messages. Get my PGP key from http://pgp.mit.edu .

Dear Roland,

Yes, I did use the parameter file from gallery page and use unequal mass initial data from Lorene.

I just check the initial location of the binary star is (-34.59 km , 45.41 km).However you said it makes the hard-coded assumption that if one of the stars is at (x,y,z) the other one is at (-x,-y,z) .How should I set CarpetRegrid2::position_x_1 and CarpetRegrid2::position_x_2 ?

Thank you.

Best regards,

Chia-Hui

________________________________ 寄件者: Roland Haas rhaas@illinois.edu 寄件日期: 2018年5月15日 上午 02:13:06 收件者: 林家暉 副本: Einstein Toolkit Users 主旨: Re: [Users] problems of reproducing the simulation of binary neutron star with different mass

Hello Chia-Hui,

thank you for the attached file.

Yes, the error is related to the grid structure:

WARNING level 0 from host nid05920 process 0 while executing schedule bin (none), routine (no thorn)::(no routine) in thorn CarpetLib, file /global/u1/c/chlin/Cactus/arrangements/Carpet/CarpetLib/src/gh.cc:178: -> The refinement hierarchy is not properly nested.

This is for the run where you used the parameter file from the gallery page but used unequal mass initial data from LORENE?

I am asking because, given that the parameter file specifies Rotating180 symmetry, the evolution will be as if equal mass NS were given in the initial data file (it will duplicate the right hand NS and ignore the left hand one). So the error that you saw should not be due to the unequal mass per-se. It could be if the location of the stars changed initially (eg for unequal mass the more massive one will be closer to the origin than the other one). This still should not give you this particular error (it would be wrong in other ways though) since the parameter file uses CarpetRegrid2::ensure_proper_nesting = "yes" which should avoid this particular error.

Looking at the plot from your initial email it may be that there was an error in the initial setups as it looks to me as if the neutron stars have actually drifted apart. At least the gallery examples has the NS initially at ~+/-15 units from the origin but in you plot it looks like much more.

Wen you added the new ID you must make sure to adjust the initial location in the parfile in the lines reading:

CarpetRegrid2::position_x_1 = 15.1875 CarpetRegrid2::position_x_2 = -15.1875

The binary black hole example does not have the restriction of assuming 180 degree symmetry, no.

Yours, Roland

Dear Roland, Thank you for your reply. The attached file is .err file corresponding to the simulation of binary star with different central enthalpy. Can you have a look at it ?Thank you. For not able to track multiple stars, does it occur in binary blackhole case(http://einsteintoolkit.org/gallery/bbh/index.html) ? Best regards, Chia-Hui ________________________________ 寄件者: Roland Haas rhaas@illinois.edu 寄件日期: 2018年5月13日 下午 09:28:51 收件者: 林家暉 副本: Einstein Toolkit Users 主旨: Re: [Users] problems of reproducing the simulation of binary neutron star with different mass

Hello Chia-Hui Lin,

changing the central enthalpy seems like the right thing to do to change masses in LORENE since it prescribes the central enthalpy of each star.

The backtrace that you showed and the error message is unfortunately not detailed enough to make a guess as to what could have gone wrong. This output is produced by a call to CCTK_Error() however the CCTK_Error function takes an error string as an argument which is not present in the lines that you have copied. It is likely in the *.err file that the simulation produced and you should try and include that file as an attachment in your email.

If you have non-equal mass stars then you have to remove the 180 degree rotation symmetry from the parameter files. Ie you have to change the lines (these are scattered throughout throughout the par file)

ActiveThorns = "RotatingSymmetry180 ReflectionSymmetry" CoordBase::xmin = 0.00 CoordBase::boundary_shiftout_x_lower = 1 CarpetRegrid2::symmetry_rotating180 = "yes" Hydro_Analysis::Hydro_Analysis_rho_max_loc_only_positive_x = "true" Hydro_Analysis::Hydro_Analysis_average_multiple_maxima_locations = "yes"

to

ActiveThorns = "ReflectionSymmetry" CoordBase::xmin = 396.00 CoordBase::boundary_shiftout_x_lower = 0 CarpetRegrid2::symmetry_rotating180 = "no" Hydro_Analysis::Hydro_Analysis_rho_max_loc_only_positive_x = "false" Hydro_Analysis::Hydro_Analysis_average_multiple_maxima_locations = "no"

Please note though that currently the thorn NSTracker and HydroAnalysis *cannot* properly track multiple stars. It makes the hard-coded assumption that if one of the stars is at (x,y,z) the other one is at (-x,-y,z) which is only true for equal mass neutron stars. This will mean that one of the stars will eventually leave the box that is supposed to track it.

Yours, Roland

...

Dear whom it may concern,

I am trying to reproduce the simulation of binary neutron star (http://einsteintoolkit.org/gallery/bns/index.html) with different mass. I use Lorene code to produce the initial data . The parameter files I used is locate in /Lorene/Codes/Bin_star/Parameters/Polytrope_irrot/Test_GR . And I changed central enthalpy of each star in par_init.d to get different mass (Is there other way I can get different mass?). And then I simulate with the ET. However I met some problems:

1.I requested 48 cores with 10 hours to run the code , however the code terminated after running ~4 hours. And the end of .err file reads:

cactus_sim: /global/u1/c/chlin/Cactus/arrangements/Carpet/Carpet/src/helpers.cc:275: int Carpet::Abort(const _cGH *, int): Assertion `0' failed.

cactus_sim: /global/u1/c/chlin/Cactus/arrangements/Carpet/Carpet/src/helpers.cc:275: int Carpet::Abort(const _cGH *, int): Assertion `0' failed.

Rank 0 with PID 47930 received signal 6

Writing backtrace to nsnstohmns/backtrace.0.txt

Rank 1 with PID 47931 received signal 6

Writing backtrace to nsnstohmns/backtrace.1.txt

cactus_sim: /global/u1/c/chlin/Cactus/arrangements/Carpet/Carpet/src/helpers.cc:275: int Carpet::Abort(const _cGH *, int): Assertion `0' failed.

cactus_sim: /global/u1/c/chlin/Cactus/arrangements/Carpet/Carpet/src/helpers.cc:275: int Carpet::Abort(const _cGH *, int): Assertion `0' failed.

Rank 2 with PID 13239 received signal 6

Rank 3 with PID 13240 received signal 6

Writing backtrace to nsnstohmns/backtrace.2.txt

Writing backtrace to nsnstohmns/backtrace.3.txt

srun: error: nid05930: tasks 2-3: Aborted

srun: Terminating job step 9192104.0

srun: error: nid05920: tasks 0-1: Aborted

The code is aborted without completing the whole process until the merger. How can I perform the complete process?

2. When I use visit to do the visualization ,I found the file rho.xy.h5 only has half of the whole simulation as :

[cid:e66a53d5-d406-4599-997f-00e90a864dc0]

The tutorial(scripts.tar.gzhttp://einsteintoolkit.org/gallery/bns/scripts.tar.gz) tells that I should use symmetric reflection to perform the whole visualization. However I cannot use this method since I simulated binary star with different mass . So there is no symmetry. How can I get the whole picture instead of only half ?

Thank you.

Best regards,

Chia-Hui Lin

-- My email is as private as my paper mail. I therefore support encrypting and signing email messages. Get my PGP key from http://pgp.mit.edu .

-- My email is as private as my paper mail. I therefore support encrypting and signing email messages. Get my PGP key from http://pgp.mit.edu .

Dear Roland,

I do not quite understand what causes ,in your example,

CarpetRegrid2::position_x_1 = 42. CarpetRegrid2::position_x_2 = -17

to become fail. In my case, if the initial location of the binary star is (-34.59 km , 45.41 km) ,I should set

CarpetRegrid2::position_x_1 = -34.59/1.48=-23.37 CarpetRegrid2::position_x_2 = 45.41/1.48=30.68

right? Would this soon be set to (-23.37,23.37)? Because the mass difference is not so much, I am wondering whether there exists some maximum value of mass ratio which can tolerate small mass difference.

By the way ,the link https://trac.einsteintoolkit.org/ticket/1974 seems not work. I would like to check it up.

Thanks,

Best regards,

Chia-Hui

________________________________ 寄件者: Roland Haas rhaas@illinois.edu 寄件日期: 2018年5月15日 上午 02:46:15 收件者: 林家暉 副本: Einstein Toolkit Users 主旨: Re: [Users] problems of reproducing the simulation of binary neutron star with different mass

Hello Chia-Hui,

ok that would make the distance from the origin ~38 units (Msun) which is certainly not what the parameter file is designed to handle.

The code only flips the x and y coordinates, not z as well so (x,y,z) and (-x,-y,z) thought this will not make a difference for you as the center of the stars is at z=0 anyway.

You have to edit the .par file and there are two lines in there that read:

CarpetRegrid2::position_x_1 = 15.1875

and

CarpetRegrid2::position_x_2 = -15.1875

You have to change 15.1875 to the initial x position of one of the stars (measured in solar masses ie about 1.48 km) and the -15.1875 to that of the other one.

You then have to restart your simulation from the beginning. Note that I expect this to still fail since even if you initially give the code:

CarpetRegrid2::position_x_1 = 42. CarpetRegrid2::position_x_2 = -17

it will very soon set the centers to be

x_1 = 17, x_2 = -17

since the star at x_2 = -17 is the heavier one (and thus has larger central density).

There is a thorn that Zach Etienne is proposing for inclusion in the Einstein Toolkit: https://trac.einsteintoolkit.org/ticket/1974 that he says can track two unequal mass stars. The thorn in question would seem to be VolumeIntegrals_GRMHD .

Yours, Roland

Dear Roland,

Yes, I did use the parameter file from gallery page and use unequal mass initial data from Lorene.

I just check the initial location of the binary star is (-34.59 km , 45.41 km).However you said it makes the hard-coded assumption that if one of the stars is at (x,y,z) the other one is at (-x,-y,z) .How should I set CarpetRegrid2::position_x_1 and CarpetRegrid2::position_x_2 ?

Thank you.

Best regards,

Chia-Hui

---

寄件者: Roland Haas rhaas@illinois.edu 寄件日期: 2018年5月15日 上午 02:13:06 收件者: 林家暉 副本: Einstein Toolkit Users 主旨: Re: [Users] problems of reproducing the simulation of binary neutron star with different mass

Hello Chia-Hui,

thank you for the attached file.

Yes, the error is related to the grid structure:

WARNING level 0 from host nid05920 process 0 while executing schedule bin (none), routine (no thorn)::(no routine) in thorn CarpetLib, file /global/u1/c/chlin/Cactus/arrangements/Carpet/CarpetLib/src/gh.cc:178: -> The refinement hierarchy is not properly nested.

This is for the run where you used the parameter file from the gallery page but used unequal mass initial data from LORENE?

I am asking because, given that the parameter file specifies Rotating180 symmetry, the evolution will be as if equal mass NS were given in the initial data file (it will duplicate the right hand NS and ignore the left hand one). So the error that you saw should not be due to the unequal mass per-se. It could be if the location of the stars changed initially (eg for unequal mass the more massive one will be closer to the origin than the other one). This still should not give you this particular error (it would be wrong in other ways though) since the parameter file uses CarpetRegrid2::ensure_proper_nesting = "yes" which should avoid this particular error.

Looking at the plot from your initial email it may be that there was an error in the initial setups as it looks to me as if the neutron stars have actually drifted apart. At least the gallery examples has the NS initially at ~+/-15 units from the origin but in you plot it looks like much more.

Wen you added the new ID you must make sure to adjust the initial location in the parfile in the lines reading:

CarpetRegrid2::position_x_1 = 15.1875 CarpetRegrid2::position_x_2 = -15.1875

The binary black hole example does not have the restriction of assuming 180 degree symmetry, no.

Yours, Roland

...

Dear Roland, Thank you for your reply. The attached file is .err file corresponding to the simulation of binary star with different central enthalpy. Can you have a look at it ?Thank you. For not able to track multiple stars, does it occur in binary blackhole case(http://einsteintoolkit.org/gallery/bbh/index.html) ? Best regards, Chia-Hui ________________________________ 寄件者: Roland Haas rhaas@illinois.edu 寄件日期: 2018年5月13日 下午 09:28:51 收件者: 林家暉 副本: Einstein Toolkit Users 主旨: Re: [Users] problems of reproducing the simulation of binary neutron star with different mass

Hello Chia-Hui Lin,

changing the central enthalpy seems like the right thing to do to change masses in LORENE since it prescribes the central enthalpy of each star.

The backtrace that you showed and the error message is unfortunately not detailed enough to make a guess as to what could have gone wrong. This output is produced by a call to CCTK_Error() however the CCTK_Error function takes an error string as an argument which is not present in the lines that you have copied. It is likely in the *.err file that the simulation produced and you should try and include that file as an attachment in your email.

If you have non-equal mass stars then you have to remove the 180 degree rotation symmetry from the parameter files. Ie you have to change the lines (these are scattered throughout throughout the par file)

ActiveThorns = "RotatingSymmetry180 ReflectionSymmetry" CoordBase::xmin = 0.00 CoordBase::boundary_shiftout_x_lower = 1 CarpetRegrid2::symmetry_rotating180 = "yes" Hydro_Analysis::Hydro_Analysis_rho_max_loc_only_positive_x = "true" Hydro_Analysis::Hydro_Analysis_average_multiple_maxima_locations = "yes"

to

ActiveThorns = "ReflectionSymmetry" CoordBase::xmin = 396.00 CoordBase::boundary_shiftout_x_lower = 0 CarpetRegrid2::symmetry_rotating180 = "no" Hydro_Analysis::Hydro_Analysis_rho_max_loc_only_positive_x = "false" Hydro_Analysis::Hydro_Analysis_average_multiple_maxima_locations = "no"

Please note though that currently the thorn NSTracker and HydroAnalysis *cannot* properly track multiple stars. It makes the hard-coded assumption that if one of the stars is at (x,y,z) the other one is at (-x,-y,z) which is only true for equal mass neutron stars. This will mean that one of the stars will eventually leave the box that is supposed to track it.

Yours, Roland

...

Dear whom it may concern,

I am trying to reproduce the simulation of binary neutron star (http://einsteintoolkit.org/gallery/bns/index.html) with different mass. I use Lorene code to produce the initial data . The parameter files I used is locate in /Lorene/Codes/Bin_star/Parameters/Polytrope_irrot/Test_GR . And I changed central enthalpy of each star in par_init.d to get different mass (Is there other way I can get different mass?). And then I simulate with the ET. However I met some problems:

1.I requested 48 cores with 10 hours to run the code , however the code terminated after running ~4 hours. And the end of .err file reads:

cactus_sim: /global/u1/c/chlin/Cactus/arrangements/Carpet/Carpet/src/helpers.cc:275: int Carpet::Abort(const _cGH *, int): Assertion `0' failed.

cactus_sim: /global/u1/c/chlin/Cactus/arrangements/Carpet/Carpet/src/helpers.cc:275: int Carpet::Abort(const _cGH *, int): Assertion `0' failed.

Rank 0 with PID 47930 received signal 6

Writing backtrace to nsnstohmns/backtrace.0.txt

Rank 1 with PID 47931 received signal 6

Writing backtrace to nsnstohmns/backtrace.1.txt

cactus_sim: /global/u1/c/chlin/Cactus/arrangements/Carpet/Carpet/src/helpers.cc:275: int Carpet::Abort(const _cGH *, int): Assertion `0' failed.

cactus_sim: /global/u1/c/chlin/Cactus/arrangements/Carpet/Carpet/src/helpers.cc:275: int Carpet::Abort(const _cGH *, int): Assertion `0' failed.

Rank 2 with PID 13239 received signal 6

Rank 3 with PID 13240 received signal 6

Writing backtrace to nsnstohmns/backtrace.2.txt

Writing backtrace to nsnstohmns/backtrace.3.txt

srun: error: nid05930: tasks 2-3: Aborted

srun: Terminating job step 9192104.0

srun: error: nid05920: tasks 0-1: Aborted

The code is aborted without completing the whole process until the merger. How can I perform the complete process?

2. When I use visit to do the visualization ,I found the file rho.xy.h5 only has half of the whole simulation as :

[cid:e66a53d5-d406-4599-997f-00e90a864dc0]

The tutorial(scripts.tar.gzhttp://einsteintoolkit.org/gallery/bns/scripts.tar.gz) tells that I should use symmetric reflection to perform the whole visualization. However I cannot use this method since I simulated binary star with different mass . So there is no symmetry. How can I get the whole picture instead of only half ?

Thank you.

Best regards,

Chia-Hui Lin

-- My email is as private as my paper mail. I therefore support encrypting and signing email messages. Get my PGP key from http://pgp.mit.edu .

-- My email is as private as my paper mail. I therefore support encrypting and signing email messages. Get my PGP key from http://pgp.mit.edu .

-- My email is as private as my paper mail. I therefore support encrypting and signing email messages. Get my PGP key from http://pgp.mit.edu .

Dear Roland,

Thank you for the detailed information.

I have tried to simulate unequal mass binary neutron star with new setting of initial position of the two stars.It seems fine at least in first several orbits .I will try the thorn you mentioned to check the difference. Thank you for your help.

Best regards,

Chia-Hui

________________________________ 寄件者: Roland Haas rhaas@illinois.edu 寄件日期: 2018年5月15日 下午 08:16:12 收件者: 林家暉 副本: Einstein Toolkit Users 主旨: Re: [Users] problems of reproducing the simulation of binary neutron star with different mass

Hello Chia-Hui,

I do not quite understand what causes ,in your example,

CarpetRegrid2::position_x_1 = 42. CarpetRegrid2::position_x_2 = -17

to become fail.

This has to do with the way that the thorn GRHydro_Analysis that looks for the location of the star works. It first determines the location on the grid with the largest density, then it computes a center of mass integral in a region around that point. This computed center of mass if used as the location of the star and to move boxes. There is code in thorn NSTracker that uses this single found location and moves two refined regions to (x,y,z) and (-x,-y,z) where (x,y,z) is the found location of the star.

There are two reasons why this does not work well with unequal masses:

1. it only looks for a single star and bases that star's location on the highest density 2. NSTracker makes the hard-coded assumption that the two regions that cover the star are mirror immages of each other

In your case since the star at -17 (always assuming you actually have set up a grid that has no symmetries) is the heavier one (since it is closer to the center of rotation) and thus has the higher central density, therefore GRHydro_Analysis will use it as the star to track and thus will find the star location to be (-17,0,0) when it looks for the star.

In my case, if the initial location of the binary star is (-34.59 km , 45.41 km) , I should set

CarpetRegrid2::position_x_1 = -34.59/1.48=-23.37 CarpetRegrid2::position_x_2 = 45.41/1.48=30.68

right?

Yes that is correct.

Would this soon be set to (-23.37,23.37)? Because the mass difference is not so much, I am wondering whether there exists some maximum value of mass ratio which can tolerate small mass difference.

By the way ,the link https://trac.einsteintoolkit.org/ticket/1974 seems not work. I would like to check it up.

Thank you for pointing this out. It seems that there is an endless redirection loop in the trac website already for https://trac.einsteintoolkit.org

Yours, Roland

-- My email is as private as my paper mail. I therefore support encrypting and signing email messages. Get my PGP key from http://pgp.mit.edu .