Users July 2018

users@lists.einsteintoolkit.org

15 participants
24 discussions

Einstein Toolkit Meeting Reminder
by Steven R. Brandt 31 Oct '19

31 Oct '19

Hi Please consider joining the weekly Einstein Toolkit phone call at 9:30 am US central time on Mondays. For details on how to connect and what agenda items are to be discussed, use the link below. https://docs.einsteintoolkit.org/et-docs/Main_Page#Weekly_Users_Call --Steve

8 94

Agenda for Monday's Meeting
by Steven R. Brandt 30 Oct '19

30 Oct '19

Please update the Wiki with agenda items for Monday's meeting. Thanks! https://docs.einsteintoolkit.org/et-docs/meeting_agenda --Steve

3 46

Re: [Users] [ET Trac] [Einstein Toolkit] #1326: running loopcontrol on strange number of threads fails
by Erik Schnetter 11 Apr '19

11 Apr '19

Do you have a backtrace? -erik On Sunday, March 22, 2015, Einstein Toolkit < trac-noreply(a)einsteintoolkit.org> wrote: > #1326: running loopcontrol on strange number of threads fails > ------------------------------------+--------------------------------------- > Reporter: rhaas | Owner: > Type: defect | Status: new > Priority: minor | Milestone: > Component: EinsteinToolkit thorn | Version: > Resolution: | Keywords: LoopControl > ------------------------------------+--------------------------------------- > > Comment (by rhaas): > > This still happens even with current (Sun Mar 22 18:46:21 CET 2015) trunk, > though failure looks a bit different now: > {{{ > actus_sim: > /data/rhaas/postdoc/gr/ET_trunk/configs/sim/build/LoopControl/loopcontrol.cc:312: > T {anonymous}::divexact(T, T) [with T = int]: Assertion `i % j == 0' > failed > }}} > > -- > Ticket URL: <https://trac.einsteintoolkit.org/ticket/1326#comment:1> > Einstein Toolkit <http://einsteintoolkit.org> > The Einstein Toolkit > _______________________________________________ > Trac mailing list > Trac(a)einsteintoolkit.org > http://lists.einsteintoolkit.org/mailman/listinfo/trac > -- Erik Schnetter <schnetter(a)cct.lsu.edu> http://www.perimeterinstitute.ca/personal/eschnetter/

3 2

How to set adaptive refinement.
by Yuhua Li 24 Sep '18

24 Sep '18

Hello all, When I doing the simulation Hydro_RNSID in Einstein Toolkit. I met some problems: WARNING[L1,P0] (GRHydro): Con2Prim failed; stopping the code. WARNING[L1,P0] (GRHydro): Even with mesh refinement, this point is not restricted from a finer level, so this is really an error WARNING[L1,P0] (GRHydro): on carpet reflevel: 1 WARNING[L1,P0] (GRHydro): xyz location: 0.000000 -6.000000 0.000000 WARNING[L1,P0] (GRHydro): radius: 6.000000 WARNING[L1,P0] (GRHydro): hydro_excision_mask, C2Pfailed: 0 1.000000 WARNING[L1,P0] (GRHydro): rho eps press w_lorentz: 0.1057347E-05 0.1857035 0.1963532E-06 1.047789 WARNING[L1,P0] (GRHydro): velocities: 0.2045605 -0.4772050E-01 0.9116232E-16 WARNING[L1,P0] (GRHydro): dens tau: 0.3259242E-05 0.8464639E-06 WARNING[L1,P0] (GRHydro): scon: 0.1949336E-05 -0.3921220E-06 0.8515884E-21 WARNING[L1,P0] (GRHydro): 3-metric: 2.054383 0.8422013E-01 0.3666352E-20 2.115545 -0.1509363E-20 1.994599 WARNING[L1,P0] (GRHydro): lapse, shift: 0.6504332 -0.9400928E-01 -0.2831831E-01 -0.3942654E-21 WARNING level 0 from host lyh-Lenovo process 0 while executing schedule bin CCTK_POSTSTEP, routine GRHydro::check_GRHydro_C2P_failed in thorn GRHydro, file GRHydro_Con2Prim.F90:1847: -> Aborting. I don't know how to mesh refinement. Yours, Yuhua

2 1

Issues with Downloading/Installing the tool kit
by Joe Hellmers 04 Sep '18

04 Sep '18

I’m loading the toolkit on the Comet Supercomputer at the San Diego Super Computing facility (SDSC). When running the command ./GetComponents --parallel https://bitbucket.org/einsteintoolkit/manifest/raw/ET_2018_02/einsteintoolk… I’m getting the error below at the end. I’m not sure if these errors are important or not. If someone would be so kind as to let me know, it would be greatly appreciated. Thank, Joe Hellmers Unable to process EinsteinExact/doc Unable to process EinsteinExact/m Unable to process EinsteinExact/tests Unable to process EinsteinExact/EinsteinExact_Test Unable to process EinsteinExact/GaugeWave Unable to process EinsteinExact/KerrSchild Unable to process EinsteinExact/Minkowski Unable to process KrancNumericalTools/GenericFD Unable to process EinsteinExact/ModifiedSchwarzschildBL Unable to process EinsteinExact/ShiftedGaugeWave Unable to process EinsteinExact/Vaidya2 Unable to process EinsteinInitialData/DistortedBHIVP Unable to process GetComponents Summary of Warnings: Could not checkout module EinsteinExact/doc fatal: unable to access 'https://github.com/barrywardell/EinsteinExact.git/': SSL connect error Could not checkout module EinsteinExact/m fatal: unable to access 'https://github.com/barrywardell/EinsteinExact.git/': SSL connect error Could not checkout module EinsteinExact/tests fatal: unable to access 'https://github.com/barrywardell/EinsteinExact.git/': SSL connect error Could not checkout module EinsteinExact/EinsteinExact_Test fatal: unable to access 'https://github.com/barrywardell/EinsteinExact.git/': SSL connect error Could not checkout module EinsteinExact/GaugeWave fatal: unable to access 'https://github.com/barrywardell/EinsteinExact.git/': SSL connect error Could not checkout module EinsteinExact/KerrSchild fatal: unable to access 'https://github.com/barrywardell/EinsteinExact.git/': SSL connect error Could not checkout module EinsteinExact/Minkowski fatal: unable to access 'https://github.com/barrywardell/EinsteinExact.git/': SSL connect error Could not checkout module KrancNumericalTools/GenericFD fatal: unable to access 'https://github.com/ianhinder/Kranc.git/': SSL connect error Could not checkout module EinsteinExact/ModifiedSchwarzschildBL fatal: unable to access 'https://github.com/barrywardell/EinsteinExact.git/': SSL connect error Could not checkout module EinsteinExact/ShiftedGaugeWave fatal: unable to access 'https://github.com/barrywardell/EinsteinExact.git/': SSL connect error Could not checkout module EinsteinExact/Vaidya2 fatal: unable to access 'https://github.com/barrywardell/EinsteinExact.git/': SSL connect error Could not checkout EinsteinInitialData/DistortedBHIVP, unable to switch to tag . Any existing symlinks to EinsteinInitialData/DistortedBHIVP will be broken Could not checkout module GetComponents fatal: unable to access 'https://github.com/gridaphobe/CRL.git/': SSL connect error Time Elapsed: 1 minutes, 48 seconds

2 11

memory leak in Carpet?
by Miguel Zilhão 10 Aug '18

10 Aug '18

hi all, i've noticed that my runs (using latest ET release) with CarpetRegrid2 exhibit a significant increase in memory during runtime. this seems to happen immediately after some non-trivial regridding operation is done. the increase is steady, and at some point i run out of memory and the simulation crashes. this is happening both on my workstation (running Ubuntu 18.04) as well as our local cluster (running Debian 9). i was wondering if someone has seen something like this? i have not seen this happen for simulations without CarpetRegrid2. i show below some relevant portions of the stdout file for a standard inspiral BH run (note the last column--maxrss_mb): ------------------------------------------------------------------------------------ Iteration Time | *me_per_hour | LEANBSSNMOL::conf_fac | *TISTICS::maxrss_mb | | minimum maximum | minimum maximum ------------------------------------------------------------------------------------ 0 0.000 | 0.0000000 | 0.2213400 0.9977828 | 1057 1359 4 0.025 | 1.8911444 | 0.2213388 0.9977828 | 1060 1361 8 0.050 | 2.8049414 | 0.2213330 0.9977828 | 1060 1361 12 0.075 | 3.2859229 | 0.2213195 0.9977828 | 1060 1361 16 0.100 | 3.6219375 | 0.2212959 0.9977828 | 1061 1361 20 0.125 | 3.7521230 | 0.2212596 0.9977828 | 1064 1361 24 0.150 | 3.9448186 | 0.2212081 0.9977828 | 1064 1361 28 0.175 | 4.0652624 | 0.2211358 0.9977828 | 1064 1361 32 0.200 | 4.1492619 | 0.2210378 0.9977828 | 1064 1361 36 0.225 | 4.1272708 | 0.2209119 0.9977828 | 1064 1361 40 0.250 | 4.2206476 | 0.2207447 0.9977828 | 1064 1361 44 0.275 | 4.2801401 | 0.2205343 0.9977828 | 1064 1361 48 0.300 | 4.3460198 | 0.2202766 0.9977828 | 1064 1361 52 0.325 | 4.3485616 | 0.2199651 0.9977828 | 1064 1361 56 0.350 | 4.4054911 | 0.2195938 0.9977828 | 1065 1361 60 0.375 | 4.4398502 | 0.2191577 0.9977828 | 1065 1361 64 0.400 | 4.3724216 | 0.2186524 0.9977828 | 1065 1361 (...) INFO (QuasiLocalMeasures): Weinberg angular momentum z: 0.405200 384 2.400 | 4.2719649 | 0.3525426 0.9977828 | 1068 1379 INFO (CarpetRegrid2): Enforcing grid structure properties, iteration 0 INFO (CarpetRegrid2): Enforcing grid structure properties, iteration 1 INFO (CarpetRegrid2): Enforcing grid structure properties, iteration 0 INFO (CarpetRegrid2): Enforcing grid structure properties, iteration 1 INFO (CarpetRegrid2): Enforcing grid structure properties, iteration 0 INFO (CarpetRegrid2): Enforcing grid structure properties, iteration 1 INFO (CarpetRegrid2): Enforcing grid structure properties, iteration 0 INFO (CarpetRegrid2): Enforcing grid structure properties, iteration 1 INFO (CarpetRegrid2): Enforcing grid structure properties, iteration 0 INFO (CarpetRegrid2): Enforcing grid structure properties, iteration 1 INFO (CarpetRegrid2): Enforcing grid structure properties, iteration 0 INFO (CarpetRegrid2): Enforcing grid structure properties, iteration 1 INFO (CarpetRegrid2): Enforcing grid structure properties, iteration 0 INFO (CarpetRegrid2): Enforcing grid structure properties, iteration 1 INFO (Carpet): Grid structure (superregions, grid points): [0][0][0] exterior: [0,0,0] : [165,324,165] ([166,325,166] + PADDING) 8955700 [1][0][0] exterior: [3,155,3] : [179,493,175] ([177,339,173] + PADDING) 10380519 [2][0][0] exterior: [9,558,9] : [108,738,101] ([100,181,93] + PADDING) 1683300 [3][0][0] exterior: [21,1206,21] : [128,1386,113] ([108,181,93] + PADDING) 1817964 [4][0][0] exterior: [45,2521,45] : [147,2663,117] ([103,143,73] + PADDING) 1075217 [5][0][0] exterior: [93,5111,93] : [225,5257,165] ([133,147,73] + PADDING) 1427223 [6][0][0] exterior: [242,10307,189] : [380,10445,261] ([139,139,73] + PADDING) 1410433 [7][0][0] exterior: [554,20684,381] : [692,20822,453] ([139,139,73] + PADDING) 1410433 INFO (Carpet): Grid structure (superregions, coordinates): [0][0][0] exterior: [-4.800000000000001,-259.199999999999989,-4.800000000000001] : [259.199999999999989,259.199999999999989,259.199999999999989] : [1.600000000000000,1.600000000000000,1.600000000000000] [1][0][0] exterior: [-2.400000000000000,-135.199999999999989,-2.400000000000000] : [138.400000000000006,135.199999999999989,135.199999999999989] : [0.800000000000000,0.800000000000000,0.800000000000000] [2][0][0] exterior: [-1.200000000000001,-36.000000000000000,-1.200000000000001] : [38.400000000000006,36.000000000000000,35.600000000000009] : [0.400000000000000,0.400000000000000,0.400000000000000] [3][0][0] exterior: [-0.600000000000001,-18.000000000000000,-0.600000000000001] : [20.800000000000001,18.000000000000000,17.800000000000001] : [0.200000000000000,0.200000000000000,0.200000000000000] [4][0][0] exterior: [-0.300000000000001,-7.100000000000023,-0.300000000000001] : [9.900000000000000,7.099999999999966,6.900000000000000] : [0.100000000000000,0.100000000000000,0.100000000000000] [5][0][0] exterior: [-0.150000000000000,-3.650000000000006,-0.150000000000000] : [6.449999999999999,3.649999999999977,3.449999999999999] : [0.050000000000000,0.050000000000000,0.050000000000000] [6][0][0] exterior: [1.250000000000000,-1.525000000000034,-0.075000000000000] : [4.699999999999999,1.925000000000011,1.725000000000000] : [0.025000000000000,0.025000000000000,0.025000000000000] [7][0][0] exterior: [2.125000000000000,-0.650000000000034,-0.037500000000001] : [3.850000000000000,1.074999999999989,0.862500000000000] : [0.012500000000000,0.012500000000000,0.012500000000000] INFO (Carpet): Global grid structure statistics: INFO (Carpet): GF: rhs: 2551k active, 3661k owned (+44%), 6190k total (+69%), 320 steps/time INFO (Carpet): GF: vars: 161, pts: 3631M active, 4271M owned (+18%), 6261M total (+47%), 1.0 comp/proc INFO (Carpet): GA: vars: 1044, pts: 82M active, 82M total (+0%) INFO (Carpet): Total required memory: 50.821 GByte (for GAs and currently active GFs) INFO (Carpet): Load balance: min avg max sdv max/avg-1 INFO (Carpet): Level 0: 25M 27M 31M 1M owned 12% INFO (Carpet): Level 1: 31M 34M 36M 1M owned 6% INFO (Carpet): Level 2: 5M 5M 6M 0M owned 8% INFO (Carpet): Level 3: 5M 6M 6M 0M owned 8% INFO (Carpet): Level 4: 3M 3M 4M 0M owned 8% INFO (Carpet): Level 5: 4M 4M 5M 0M owned 9% INFO (Carpet): Level 6: 4M 5M 5M 0M owned 4% INFO (Carpet): Level 7: 4M 5M 5M 0M owned 4% 388 2.425 | 4.1930426 | 0.3508349 0.9977828 | 1149 1450 392 2.450 | 4.2022566 | 0.3491189 0.9977828 | 1149 1450 396 2.475 | 4.2091891 | 0.3471720 0.9977828 | 1149 1450 ------------------------------------------------------------------------------------ Iteration Time | *me_per_hour | LEANBSSNMOL::conf_fac | *TISTICS::maxrss_mb | | minimum maximum | minimum maximum ------------------------------------------------------------------------------------ 400 2.500 | 4.2177082 | 0.3450791 0.9977828 | 1149 1450 404 2.525 | 4.2195973 | 0.3429667 0.9977828 | 1149 1450 (...) 1344 8.400 | 4.4943523 | 0.3648129 0.9977828 | 1156 1455 INFO (CarpetRegrid2): Enforcing grid structure properties, iteration 0 INFO (CarpetRegrid2): Enforcing grid structure properties, iteration 1 INFO (CarpetRegrid2): Enforcing grid structure properties, iteration 0 INFO (CarpetRegrid2): Enforcing grid structure properties, iteration 1 INFO (CarpetRegrid2): Enforcing grid structure properties, iteration 0 INFO (CarpetRegrid2): Enforcing grid structure properties, iteration 1 INFO (CarpetRegrid2): Enforcing grid structure properties, iteration 0 INFO (CarpetRegrid2): Enforcing grid structure properties, iteration 1 INFO (CarpetRegrid2): Enforcing grid structure properties, iteration 0 INFO (CarpetRegrid2): Enforcing grid structure properties, iteration 1 INFO (CarpetRegrid2): Enforcing grid structure properties, iteration 0 INFO (CarpetRegrid2): Enforcing grid structure properties, iteration 1 INFO (CarpetRegrid2): Enforcing grid structure properties, iteration 0 INFO (CarpetRegrid2): Enforcing grid structure properties, iteration 1 INFO (Carpet): Grid structure (superregions, grid points): [0][0][0] exterior: [0,0,0] : [165,324,165] ([166,325,166] + PADDING) 8955700 [1][0][0] exterior: [3,154,3] : [178,494,175] ([176,341,173] + PADDING) 10382768 [2][0][0] exterior: [9,556,9] : [108,740,101] ([100,185,93] + PADDING) 1720500 [3][0][0] exterior: [21,1202,21] : [127,1390,113] ([107,189,93] + PADDING) 1880739 [4][0][0] exterior: [45,2512,45] : [145,2672,117] ([101,161,73] + PADDING) 1187053 [5][0][0] exterior: [93,5094,93] : [110,5135,165] ([18,42,73] + PADDING) 55188 [5][0][1] exterior: [93,5136,93] : [220,5274,165] ([128,139,73] + PADDING) 1298816 [6][0][0] exterior: [234,10342,189] : [372,10480,261] ([139,139,73] + PADDING) 1410433 [7][0][0] exterior: [537,20752,381] : [675,20890,453] ([139,139,73] + PADDING) 1410433 INFO (Carpet): Grid structure (superregions, coordinates): [0][0][0] exterior: [-4.800000000000001,-259.199999999999989,-4.800000000000001] : [259.199999999999989,259.199999999999989,259.199999999999989] : [1.600000000000000,1.600000000000000,1.600000000000000] [1][0][0] exterior: [-2.400000000000000,-136.000000000000000,-2.400000000000000] : [137.599999999999994,136.000000000000000,135.199999999999989] : [0.800000000000000,0.800000000000000,0.800000000000000] [2][0][0] exterior: [-1.200000000000001,-36.800000000000011,-1.200000000000001] : [38.400000000000006,36.800000000000011,35.600000000000009] : [0.400000000000000,0.400000000000000,0.400000000000000] [3][0][0] exterior: [-0.600000000000001,-18.800000000000011,-0.600000000000001] : [20.600000000000001,18.800000000000011,17.800000000000001] : [0.200000000000000,0.200000000000000,0.200000000000000] [4][0][0] exterior: [-0.300000000000001,-8.000000000000000,-0.300000000000001] : [9.699999999999999,8.000000000000000,6.900000000000000] : [0.100000000000000,0.100000000000000,0.100000000000000] [5][0][0] exterior: [-0.150000000000000,-4.500000000000000,-0.150000000000000] : [0.699999999999999,-2.449999999999989,3.449999999999999] : [0.050000000000000,0.050000000000000,0.050000000000000] [5][0][1] exterior: [-0.150000000000000,-2.400000000000034,-0.150000000000000] : [6.199999999999999,4.500000000000000,3.449999999999999] : [0.050000000000000,0.050000000000000,0.050000000000000] [6][0][0] exterior: [1.050000000000000,-0.650000000000034,-0.075000000000000] : [4.500000000000000,2.800000000000011,1.725000000000000] : [0.025000000000000,0.025000000000000,0.025000000000000] [7][0][0] exterior: [1.912500000000000,0.199999999999989,-0.037500000000001] : [3.637499999999999,1.925000000000011,0.862500000000000] : [0.012500000000000,0.012500000000000,0.012500000000000] INFO (Carpet): Global grid structure statistics: INFO (Carpet): GF: rhs: 2544k active, 3659k owned (+44%), 6187k total (+69%), 320 steps/time INFO (Carpet): GF: vars: 161, pts: 3644M active, 4290M owned (+18%), 6288M total (+47%), 1.0 comp/proc INFO (Carpet): GA: vars: 1044, pts: 82M active, 82M total (+0%) INFO (Carpet): Total required memory: 51.040 GByte (for GAs and currently active GFs) INFO (Carpet): Load balance: min avg max sdv max/avg-1 INFO (Carpet): Level 0: 25M 27M 31M 1M owned 12% INFO (Carpet): Level 1: 31M 34M 35M 1M owned 4% INFO (Carpet): Level 2: 5M 5M 6M 0M owned 6% INFO (Carpet): Level 3: 5M 6M 6M 0M owned 8% INFO (Carpet): Level 4: 3M 4M 4M 0M owned 10% INFO (Carpet): Level 5: 3M 4M 5M 0M owned 11% INFO (Carpet): Level 6: 4M 5M 5M 0M owned 4% INFO (Carpet): Level 7: 4M 5M 5M 0M owned 4% 1348 8.425 | 4.4749366 | 0.3620783 0.9977828 | 1417 1726 1352 8.450 | 4.4793418 | 0.3593239 0.9977828 | 1417 1727 1356 8.475 | 4.4831132 | 0.3565493 0.9977828 | 1417 1727 ------------------------------------------------------------------------------------ Iteration Time | *me_per_hour | LEANBSSNMOL::conf_fac | *TISTICS::maxrss_mb | | minimum maximum | minimum maximum ------------------------------------------------------------------------------------ 1360 8.500 | 4.4873776 | 0.3537546 0.9977828 | 1417 1727 1364 8.525 | 4.4896835 | 0.3509394 0.9977828 | 1417 1727 and the pattern continues such that at iteration 9000 we're at maximum(maxrss_mb) = 2722... is this normal, or expected? it becomes very inconvenient since, for some high-resolutions runs, i inevitably run out of memory. thanks, Miguel

5 23

meeting minutes for 2018-07-30
by Roland Haas 30 Jul '18

30 Jul '18

Present: Peter, Bhavesh, Roland, Sam, Andreas Boden, Zach wiki editing without accounts: * Peter will talk with admins * Peter will also poke the amins about SSL certificates for docs.einsteintoolkit.org memory consumption in BBH simulations * email thread: http://lists.einsteintoolkit.org/pipermail/users/2018-July/006396.html * Roland summarized the thread: not enough information yet, Miguel has checked that RSS increases but not yet that Carpet's reported memory for grid functions does not increase by the same amount ET release: Gallery examples: * BBH: Sam was away at PEARC, will do it later this week * poisson: Roland to poke Shawn * scalar fields, TOV: Roland will do today if call ends early * BNS: Peter ran with old code over the weekend, will check once in the office GiRaFFE review: * no progress on actual report * Zach is finishing up the tests himself * Intel compiler <19 does not work with newest Ubuntu, Intel 19 fails right now to compile PAPI. Zach will compile without. WVUDiagnostics: * no updates Lean: * no updates Proca: * Peter downloaded and compiled the code, read through paper. Milestones: * simfactory on OSX: Roland to test Yours, Roland -- My email is as private as my paper mail. I therefore support encrypting and signing email messages. Get my PGP key from http://pgp.mit.edu .

1 0

non vacuum binary neutron star
by 黃世杰 30 Jul '18

30 Jul '18

If I want to simulate the situation of the binary neutron star with the gas surrounding, and the gas can be in different density and different velocity distribution, how could I do ?(Should I insert the thorns "GRHydro_InitData"and "EOS_Polytrope" into the parameter file of binary neutron stars--"NsNsToHMNS.par" ?) Best, Shih-Jie

1 0

Weekly Einstein Toolkit Users Call.
by Peter Diener 29 Jul '18

29 Jul '18

Hi All, Sorry about the late e-mail but this is just a reminder that next week's Users Call will take place at the usual time (Monday at 10:00am US Eastern time) and the usual place (https://bluejeans.com/194244555). If you have any items for the agenda please add them to the wiki (https://docs.einsteintoolkit.org/et-docs/meeting_agenda#Meeting_Agenda) or send them to me (diener(a)cct.lsu.edu). Cheers, Peter

1 0

non vacuum binary neutron star
by 黃世杰 27 Jul '18

27 Jul '18

If I want to simulate the situation of the binary neutron star with the gas surrounding, and the gas can be in different density and different velocity distribution, how could I do (how to modify the parameter file🤔) sorry, I am a new hand😥, please give me some guides, thank you~~

1 0

2026

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

Users July 2018