[Users] problem with first example simulation with ETK

Ian Hinder ian.hinder at aei.mpg.de
Wed Feb 8 14:37:39 CST 2012 

On 8 Feb 2012, at 21:05, Jose Fiestas Iquira wrote:

> Hi,
> 
> The first problem was solved as I reported, by disabling the QuasiLocalMeasures. Some MPI problem. Btw, which information am I missing without QuasiLocalMeasures? (the output file was the wrong one, of a successful simulation without WeyScal4)
> 
> About the .par file. I found out (please correct me if necessary) that I need to change the name of the run by the simfactory submission (here static_tov):
> ./simfactory/bin/sim submit static_tov --parfile=par/static_tov.par --procs=32 --walltime=8:0:0
> 
> Otherwise the run works with the original .par (first used with this run) , and not to the newly specified .par file. So, it worked by doing
> ./simfactory/bin/sim submit static_tov01 --parfile=par/static_tov_new.par --procs=32 --walltime=8:0:0

Simulation output goes to a directory of the same name as the simulation, so if you want a new simulation, you need to give it a different name.  SimFactory interprets a second "submit" command on a simulation as an attempt to "resubmit" the simulation, which means to start a second Cactus run which continues the simulation from checkpoint files written by the first one.  So if you have forgotten to change the name or to delete the first simulation directory, simfactory doesn't do what you wanted it to.  I don't usually use the submit command for starting simulations, because of this problem.  There is a command called "create-submit" which gives an error message if the simulation already exists.  Maybe we should change simfactory to only accept explicit "submit" and "resubmit" commands to help this.


> 
> Best,
> Jose
> 
> On Wed, Feb 8, 2012 at 10:05 AM, Ian Hinder <ian.hinder at aei.mpg.de> wrote:
> 
> On 23 Jan 2012, at 22:36, Jose Fiestas Iquira wrote:
> 
>> Hi,
>> 
>> In order to use the wave generator, I am probably not submitting the
>> job properly.
>> 
>> I had modified the .par file located at:
>> /home/ettest24/Cactus/par/qc0-mclachlan.par
>> since I run the example like this:
>> ./simfactory/bin/sim submit qc0-mclachlan
>> --parfile=par/qc0-mclachlan.par --procs=32 --walltime=8:0:0
>> 
>> Here the run stopped with errors, as previously reported.
> 
> Hi Jose,
> 
> Could you update me on your status?  Did you solve this problem?  In the standard output file that you sent to the ET mailing list on 16-Jan-2012, I do not see any problem.  The run reached 100 M of evolution time.  The only problem is a non-fatal warning message which is caused by some problem with the way that Carpet keeps track of points which should or should not be included in norm operations.  You can ignore this warning.
> 
>> Could be possible that the proper .par is located somewhere else? I
>> found it also here:
>> 
>> /home/ettest24/Cactus/simfactory/etc/par/qc0-mclachlan.par
>> /home/ettest24/Cactus/arrangements/McLachlan/par/qc0-mclachlan.par
>> 
>> Btw, in the last one WeylScal4 is activated, in a similar way as you told me.
> 
> SimFactory takes the parameter file that you specify on the command line - it does not look anywhere else.  Can you be more specific about the problem you are having?
> 
> PS: if you ask these questions on the mailing list, other people can help and benefit from the discussion.
> 
> 
>> 
>> Best,
>> Jose
>> 
>> 
>> 
>> On Mon, Jan 23, 2012 at 1:11 PM, Ian Hinder <ian.hinder at aei.mpg.de> wrote:
>>> 
>>> On 23 Jan 2012, at 14:57, Jose Fiestas Iquira wrote:
>>> 
>>> Hello,
>>> 
>>> Regarding the Binary Black Hole coalescence example from the ETK.
>>> 
>>> In the BH diagnostics files are columns with quadrupole terms, which
>>> are not constant. Do you know of some tool in ETK to calculate the GW
>>> emission from these terms?
>>> 
>>> 
>>> No, not that I am aware of.  These are the quadrupole moments of the
>>> individual apparent horizons, rather than the quadrupole moment of the
>>> system as a whole.  The gravitational wave emission can be computed by using
>>> the WeylScal4 and Multipole thorns.  These give you the spherical harmonic
>>> multipoles of Psi4 (e.g. l = 2, m = 2 is the dominant one), which can then
>>> be converted to strain by "two integrations in time" [this is actually quite
>>> subtle and hard to get right - see  Reisswig and Pollney, arXiv:1006.1632,
>>> for an overview and solution).
>>> 
>>> 
>>> 
>>> Thanks,
>>> Jose
>>> 
>>> On Tue, Jan 17, 2012 at 5:18 AM, Ian Hinder <ian.hinder at aei.mpg.de> wrote:
>>> 
>>> 
>>> On 17 Jan 2012, at 13:37, Jose Fiestas Iquira wrote:
>>> 
>>> 
>>> The Einstein Analysis guide is very useful. I found here
>>> 
>>> http://einsteintoolkit.org/documentation/ThornGuide/ThornGuidech142.html#x159-2520000P8.4
>>> 
>>> the units used in 'Hydrobase'.
>>> 
>>> 
>>> If I assume these are the units for all thorns, I obtain from data in
>>> 
>>> BH_diagnostics.ah1.gp a unit of distance of about 1 Km. To estimate
>>> 
>>> the time I am using a mass of ~0.5 M_sun for both BHs (m_irreducible
>>> 
>>> ?) , and I read a final time (cctk_time) ~ 1microsec. Are these
>>> 
>>> quantities in the usual range?
>>> 
>>> 
>>> 
>>> The length unit is G M / c^2 and the time unit is G M / c^3.  For M = M_sun,
>>> 
>>> I get the length unit as 1.5 km and the time unit as 4.9 us.  The final time
>>> 
>>> in the simulation is 100 M (i.e. 100 G M / c^3), which would be 0.5 ms.
>>> 
>>> 
>>> 
>>> 
>>> 
>>> 
>>> 
>>> Jose
>>> 
>>> 
>>> 
>>> On Tue, Jan 17, 2012 at 4:04 AM, Jose Fiestas Iquira <jafiestas at lbl.gov>
>>> 
>>> wrote:
>>> 
>>> 
>>> I sent the wrong file, here is the right one.
>>> 
>>> 
>>> 
>>> $ hg tip
>>> 
>>> 
>>>  gives me:
>>> 
>>> 
>>> -----------------------------------------------------
>>> 
>>> 
>>> changeset:   3421:dff4df2b3687
>>> 
>>> 
>>> tag:         tip
>>> 
>>> 
>>> user:        Erik Schnetter <schnetter at cct.lsu.edu>
>>> 
>>> 
>>> date:        Thu Jan 12 14:11:27 2012 -0500
>>> 
>>> 
>>> summary:     CarpetLib: Allocate space for dstbox before
>>> 
>>> 
>>> restricting/prolongating
>>> 
>>> 
>>> 
>>> [40514 refs]
>>> 
>>> 
>>> -----------------------------------------------------
>>> 
>>> 
>>> 
>>> I copied this version from:
>>> 
>>> 
>>> 
>>> ./GetComponents -a
>>> 
>>> 
>>> http://svn.einsteintoolkit.org/manifest/trunk/einsteintoolkit.th
>>> 
>>> 
>>> 
>>> Thanks,
>>> 
>>> 
>>> Jose
>>> 
>>> 
>>> 
>>> 
>>> 
>>> 
>>> On Tue, Jan 17, 2012 at 1:13 AM, Ian Hinder <ian.hinder at aei.mpg.de> wrote:
>>> 
>>> 
>>> 
>>> On 16 Jan 2012, at 23:21, Jose Fiestas Iquira wrote:
>>> 
>>> 
>>> 
>>> Thanks, I am attaching the output file.
>>> 
>>> 
>>> 
>>> 
>>> The output file looks fine - I can't see any problem with the simulation.
>>> 
>>> 
>>>  According to the output file, it reached t = 100 M, and in the parameter
>>> 
>>> 
>>> file that is listed as cctk_final_time, so all looks well!
>>> 
>>> 
>>> 
>>> I did copy the development version. As I can see, it was already
>>> 
>>> 
>>> fixed, so I should not get this warning, since I copied the
>>> 
>>> 
>>> development version.
>>> 
>>> 
>>> 
>>> 
>>> Yes, if you are using the current development version, then I don't know why
>>> 
>>> 
>>> you get the warning.  Can you check which version of Carpet you have by
>>> 
>>> 
>>> doing
>>> 
>>> 
>>> 
>>> cd repos/carpet
>>> 
>>> 
>>> hg tip
>>> 
>>> 
>>> 
>>> to be sure you are using the development version?
>>> 
>>> 
>>> 
>>> Is  the puncture tracker not a better way to plot BH trajectories.
>>> 
>>> 
>>> 
>>> 
>>> It's cheaper to use the puncture tracker, because computing the horizon
>>> 
>>> 
>>> centroid is more expensive, but it has to be done every iteration, whereas
>>> 
>>> 
>>> the horizon can be found less frequently.  For comparison with
>>> 
>>> 
>>> point-particle results, I don't think it makes much difference; the results
>>> 
>>> 
>>> are gauge dependent anyway so differences on the order of the horizon size
>>> 
>>> 
>>> cannot be interpreted physically.  Since the parameter file already has
>>> 
>>> 
>>> horizon-finding built in, it is easier for you to use that.
>>> 
>>> 
>>> 
>>> If you want to figure out how to add the puncture tracker to the simulation,
>>> 
>>> 
>>> you can start with these parameters:
>>> 
>>> 
>>> 
>>> ActiveThorns = "PunctureTracker"
>>> 
>>> 
>>> 
>>> PunctureTracker::track                      [0] = yes
>>> 
>>> 
>>> PunctureTracker::initial_x                  [0] = $xp
>>> 
>>> 
>>> PunctureTracker::which_surface_to_store_info[0] = 0
>>> 
>>> 
>>> PunctureTracker::track                      [1] = yes
>>> 
>>> 
>>> PunctureTracker::initial_x                  [1] = $xm
>>> 
>>> 
>>> PunctureTracker::which_surface_to_store_info[1] = 1
>>> 
>>> 
>>> 
>>> IOASCII::out0D_vars = "PunctureTracker::pt_loc"
>>> 
>>> 
>>> 
>>> Replace $xp and $xm with the initial x coordinates of the +x and -x
>>> 
>>> 
>>> punctures (this will be \pm TwoPunctures::par_b).  You then need to choose
>>> 
>>> 
>>> two spherical surfaces
>>> 
>>> 
>>> (https://docs.einsteintoolkit.org/et-docs/Einstein_Toolkit_standards) which
>>> 
>>> 
>>> are not being used currently to store the puncture locations in.  If there
>>> 
>>> 
>>> is already 0D output, just add the puncturetracker variable separated with a
>>> 
>>> 
>>> space.
>>> 
>>> 
>>> 
>>> I
>>> 
>>> 
>>> did some plots, what are really the combinations xx, xy, xz, yy, zz ?
>>> 
>>> 
>>> Are they related to the trajectories of one BH (projections)? What
>>> 
>>> 
>>> about the second BH?
>>> 
>>> 
>>> 
>>> 
>>> You should have BH_diagnostics.ah1.gp and BH_diagnostics.ah2.gp.  These
>>> 
>>> 
>>> correspond to the first and second BHs.  If you look at the top of these
>>> 
>>> 
>>> files, you will see an explanation of the columns.  More information about
>>> 
>>> 
>>> each column is available in the AHFinderDirect documentation
>>> 
>>> 
>>> (http://einsteintoolkit.org/documentation/ThornGuide/ThornGuidech128.html#x144-2264000O4).
>>> 
>>> 
>>>  Specifically, information about the BH_diagnostics output files is
>>> 
>>> 
>>> at http://einsteintoolkit.org/documentation/ThornGuide/ThornGuidech128.html#x144-2274000O4.5.5
>>> 
>>> 
>>> 
>>> To plot the trajectory of one of the BHs, you want to plot column 3 and
>>> 
>>> 
>>> column 4 in a parametric plot.
>>> 
>>> 
>>> 
>>> 
>>> Cheers,
>>> 
>>> 
>>> Jose
>>> 
>>> 
>>> 
>>> On Mon, Jan 16, 2012 at 4:39 AM, Ian Hinder <ian.hinder at aei.mpg.de> wrote:
>>> 
>>> 
>>> 
>>> 
>>> On 14 Jan 2012, at 00:41, Jose Fiestas Iquira wrote:
>>> 
>>> 
>>> 
>>> 
>>> Hi Ian,
>>> 
>>> 
>>> 
>>> I did the changes but it started and finished early. I am attaching
>>> 
>>> 
>>> 
>>> the qc0-mclachlan.err file. At the end there is a warning in one of
>>> 
>>> 
>>> 
>>> the Carpet files (mask_test.c)
>>> 
>>> 
>>> 
>>> 
>>> 
>>> The warning you see,
>>> 
>>> 
>>> 
>>> 
>>> WARNING level 1 in thorn CarpetReduce processor 0 host qb517
>>> 
>>> 
>>> 
>>>   (line 120 of
>>> 
>>> 
>>> 
>>> /home/ettest24/Cactus/arrangements/Carpet/CarpetReduce/src/mask_test.c):
>>> 
>>> 
>>> 
>>>   -> Simulation domain volume and reduction weight sum differ
>>> 
>>> 
>>> 
>>> 
>>> is a known issue (https://trac.einsteintoolkit.org/ticket/434) which was
>>> 
>>> 
>>> 
>>> fixed after the last release and the fix is only available in the
>>> 
>>> 
>>> 
>>> development version.  I believe that the warning is not fatal, and it should
>>> 
>>> 
>>> 
>>> not have terminated the simulation.  Can you also post the qc0-mclachlan.out
>>> 
>>> 
>>> 
>>> file?
>>> 
>>> 
>>> 
>>> 
>>> Where can I find the puncture tracker?
>>> 
>>> 
>>> 
>>> 
>>> 
>>> The PunctureTracker thorn is part of the Einstein Toolkit thornlist, so you
>>> 
>>> 
>>> 
>>> should have it already.  However, I think the best thing for you to do
>>> 
>>> 
>>> 
>>> initially would be to look at the BH_diagnostics files for the apparent
>>> 
>>> 
>>> 
>>> horizon centroid locations for tracking the locations of the BHs.
>>> 
>>> 
>>> 
>>> 
>>> 
>>> 
>>> 
>>> Thanks,
>>> 
>>> 
>>> 
>>> Jose
>>> 
>>> 
>>> 
>>> 
>>> On Fri, Jan 13, 2012 at 4:38 AM, Ian Hinder <ian.hinder at aei.mpg.de> wrote:
>>> 
>>> 
>>> 
>>> 
>>> 
>>> On 13 Jan 2012, at 12:32, Jose Fiestas Iquira wrote:
>>> 
>>> 
>>> 
>>> 
>>> 
>>> Yes, trajectories and wave forms would be enough for now. Which are
>>> 
>>> 
>>> 
>>> 
>>> the names of this files? Many of the .asc files have data like kxx,
>>> 
>>> 
>>> 
>>> 
>>> kxy, kxz, etc. Is there any description of them?
>>> 
>>> 
>>> 
>>> 
>>> 
>>> Not that I am aware of.  Files with extension .x.asc are 1D ascii output of
>>> 
>>> 
>>> 
>>> grid functions from CarpetIOASCII.  kxy is the variable name for the x,y
>>> 
>>> 
>>> 
>>> component of the extrinsic curvature tensor K.  The waveform file should be
>>> 
>>> 
>>> 
>>> in mp_psi4_l2_m2_r*.asc where * corresponds to the radius of the extraction
>>> 
>>> 
>>> 
>>> sphere.  However, looking at the qc0-mclachlan.par file, I see that there is
>>> 
>>> 
>>> 
>>> no wave extraction switched on.  You should be able to enable wave
>>> 
>>> 
>>> 
>>> extraction by adding something like the following to the parameter file:
>>> 
>>> 
>>> 
>>> 
>>> 
>>> ActiveThorns = "WeylScal4 CarpetInterp AEILocalInterp Multipole"
>>> 
>>> 
>>> 
>>> 
>>> 
>>> Multipole::nradii    = 3
>>> 
>>> 
>>> 
>>> 
>>> Multipole::radius[0] = 30
>>> 
>>> 
>>> 
>>> 
>>> Multipole::radius[1] = 40
>>> 
>>> 
>>> 
>>> 
>>> Multipole::radius[2] = 50
>>> 
>>> 
>>> 
>>> 
>>> Multipole::variables = "WeylScal4::Psi4r{sw=-2 cmplx=’WeylScal4::Psi4i’}"
>>> 
>>> 
>>> 
>>> 
>>> Multipole::l_max     = 4
>>> 
>>> 
>>> 
>>> 
>>> WeylScal4::fdOrder   = 4
>>> 
>>> 
>>> 
>>> 
>>> 
>>> (I haven't tested this, so let me know if it doesn't work)
>>> 
>>> 
>>> 
>>> 
>>> 
>>> I've just found that the usual PunctureTracker thorn is not enabled in this
>>> 
>>> 
>>> 
>>> example parameter file.  So to look at trajectories you will have to use the
>>> 
>>> 
>>> 
>>> apparent horizon centroids.  These are in the files BH_diagnostics.ah*.gp.
>>> 
>>> 
>>> 
>>>  The meanings of the columns are at the top of the files.  You want the
>>> 
>>> 
>>> 
>>> centroid x and y columns to plot in GnuPlot.
>>> 
>>> 
>>> 
>>> 
>>> 
>>> 
>>> 
>>> 
>>> 
>>> Thanks,
>>> 
>>> 
>>> 
>>> 
>>> Jose
>>> 
>>> 
>>> 
>>> 
>>> 
>>> On Fri, Jan 13, 2012 at 12:41 AM, Ian Hinder <ian.hinder at aei.mpg.de> wrote:
>>> 
>>> 
>>> 
>>> 
>>> 
>>> On 12 Jan 2012, at 23:46, Jose Fiestas Iquira wrote:
>>> 
>>> 
>>> 
>>> 
>>> 
>>> Dear all,
>>> 
>>> 
>>> 
>>> 
>>> 
>>> I run the tutorial and found other interesting examples like the
>>> 
>>> 
>>> 
>>> 
>>> McLachlan tool for Binary Black Holes.
>>> 
>>> 
>>> 
>>> 
>>> 
>>> Regarding the McLachlan tool,
>>> 
>>> 
>>> 
>>> 
>>> Could somebody help me finding a description of the output? Is all
>>> 
>>> 
>>> 
>>> 
>>> contained in the .asc files? I would like to visualize it.
>>> 
>>> 
>>> 
>>> 
>>> 
>>> 
>>> Hi Jose,
>>> 
>>> 
>>> 
>>> 
>>> 
>>> What sort of output do you want to visualise?  Cactus thorns typically
>>> 
>>> 
>>> 
>>> 
>>> output "lightweight" information such as BH trajectories and waveforms as
>>> 
>>> 
>>> 
>>> 
>>> ASCII, and more heavyweight data -- like gridfunctions -- as HDF5.  For the
>>> 
>>> 
>>> 
>>> 
>>> trajectories (from thorn Puncture Tracker) and waveforms (from Multipole)
>>> 
>>> 
>>> 
>>> 
>>> you can use gnuplot or anything that can read standard ASCII output to
>>> 
>>> 
>>> 
>>> 
>>> visualise them.  For gridfunctions in HDF5 format, I think the easiest way
>>> 
>>> 
>>> 
>>> 
>>> to get started is to use VisIt, for which there is a Carpet plugin (though
>>> 
>>> 
>>> 
>>> 
>>> installing VisIt is sometimes not so easy!).
>>> 
>>> 
>>> 
>>> 
>>> 
>>> Does anyone know if the Carpet plugin is distributed with VisIt yet?  I
>>> 
>>> 
>>> 
>>> 
>>> believe that was the intention.
>>> 
>>> 
>>> 
>>> 
>>> 
>>> 
>>> 
>>> Cheers,
>>> 
>>> 
>>> 
>>> 
>>> Jose
>>> 
>>> 
>>> 
>>> 
>>> 
>>> 
>>> On Wed, Jan 11, 2012 at 7:06 PM, Jose Fiestas Iquira <jafiestas at lbl.gov>
>>> 
>>> 
>>> 
>>> 
>>> wrote:
>>> 
>>> 
>>> 
>>> 
>>> 
>>> Thanks to Erik for advise. ETK example simulation run successfully.
>>> 
>>> 
>>> 
>>> 
>>> 
>>> 
>>> 
>>> On Wed, Jan 11, 2012 at 11:49 AM, Jose Fiestas Iquira <jafiestas at lbl.gov>
>>> 
>>> 
>>> 
>>> 
>>> wrote:
>>> 
>>> 
>>> 
>>> 
>>> 
>>> Right, it looks good now:
>>> 
>>> 
>>> 
>>> 
>>> 
>>> 
>>> ./simfactory/bin/sim list-simulations
>>> 
>>> 
>>> 
>>> 
>>> 
>>>   static_tov              [ACTIVE (RUNNING), restart 0000, job id 595490]
>>> 
>>> 
>>> 
>>> 
>>> 
>>> 
>>> Let see how it looks when it finishes. The example was set to 8 hours.
>>> 
>>> 
>>> 
>>> 
>>> 
>>> 
>>> Thanks,
>>> 
>>> 
>>> 
>>> 
>>> 
>>> Jose
>>> 
>>> 
>>> 
>>> 
>>> 
>>> 
>>> 
>>> On Wed, Jan 11, 2012 at 11:45 AM, Erik Schnetter <schnetter at cct.lsu.edu>
>>> 
>>> 
>>> 
>>> 
>>> wrote:
>>> 
>>> 
>>> 
>>> 
>>> 
>>> On Wed, Jan 11, 2012 at 2:42 PM, Jose Fiestas Iquira <jafiestas at lbl.gov>
>>> 
>>> 
>>> 
>>> 
>>> wrote:
>>> 
>>> 
>>> 
>>> 
>>> 
>>> 
>>> The only thing is that now
>>> 
>>> 
>>> 
>>> 
>>> 
>>> $ ./simfactory/bin/sim list-simulations
>>> 
>>> 
>>> 
>>> 
>>> 
>>> returns
>>> 
>>> 
>>> 
>>> 
>>> 
>>> 
>>> Error: unknown command list-simulation
>>> 
>>> 
>>> 
>>> 
>>> 
>>> Aborting Simfactory.
>>> 
>>> 
>>> 
>>> 
>>> 
>>> 
>>> Did you really type "list-simulations", and the error message says
>>> 
>>> 
>>> 
>>> 
>>> 
>>> "list-simulation" (without the final "s")?
>>> 
>>> 
>>> 
>>> 
>>> 
>>> 
>>> -erik
>>> 
>>> 
>>> 
>>> 
>>> 
>>> 
>>> --
>>> 
>>> 
>>> 
>>> 
>>> 
>>> Erik Schnetter <schnetter at cct.lsu.edu>   http://www.cct.lsu.edu/~eschnett/
>>> 
>>> 
>>> 
>>> 
>>> 
>>> _______________________________________________
>>> 
>>> 
>>> 
>>> 
>>> 
>>> Users mailing list
>>> 
>>> 
>>> 
>>> 
>>> Users at einsteintoolkit.org
>>> 
>>> 
>>> 
>>> 
>>> http://lists.einsteintoolkit.org/mailman/listinfo/users
>>> 
>>> 
>>> 
>>> 
>>> 
>>> 
>>> --
>>> 
>>> 
>>> 
>>> 
>>> Ian Hinder
>>> 
>>> 
>>> 
>>> 
>>> http://numrel.aei.mpg.de/people/hinder
>>> 
>>> 
>>> 
>>> 
>>> 
>>> 
>>> --
>>> 
>>> 
>>> 
>>> 
>>> Ian Hinder
>>> 
>>> 
>>> 
>>> 
>>> http://numrel.aei.mpg.de/people/hinder
>>> 
>>> 
>>> 
>>> 
>>> 
>>> <qc0-mclachlan.err>
>>> 
>>> 
>>> 
>>> 
>>> 
>>> --
>>> 
>>> 
>>> 
>>> Ian Hinder
>>> 
>>> 
>>> 
>>> http://numrel.aei.mpg.de/people/hinder
>>> 
>>> 
>>> 
>>> 
>>> <qc0-mclachlan.out>
>>> 
>>> 
>>> 
>>> 
>>> --
>>> 
>>> 
>>> Ian Hinder
>>> 
>>> 
>>> http://numrel.aei.mpg.de/people/hinder
>>> 
>>> 
>>> 
>>> 
>>> --
>>> 
>>> Ian Hinder
>>> 
>>> http://numrel.aei.mpg.de/people/hinder
>>> 
>>> 
>>> 
>>> --
>>> Ian Hinder
>>> http://numrel.aei.mpg.de/people/hinder
>>> 
> 
> -- 
> Ian Hinder
> http://numrel.aei.mpg.de/people/hinder
> 
> 

-- 
Ian Hinder
http://numrel.aei.mpg.de/people/hinder

More information about the Users mailing list