#970: CarpetLib::barriers fails with multipatch
--------------------+-------------------------------------------------------
Reporter: rhaas | Owner: eschnett
Type: defect | Status: new
Priority: minor | Milestone:
Component: Carpet | Version:
Keywords: |
--------------------+-------------------------------------------------------
This happens during the initial storage allocation where there are
mismatching barriers in dh::add and gdata::gdata. The underlying reason
seems to be that Carpet/StorageCrease has a loop (schematically) around
line 93 of Storage.cc.
{{
for(m=0;m<maps;++m)
new gf<T> // (which calls dh::add)
arrdata.AT(group).AT(m).data.AT(var)->set_timelevels // which eventually
call gdata::gdata
}}
this causes the a barrier error when on process owns a component on map 0
but another does only onwn a component on map 1, since in this case the
first one will encounter the barriers as:
dhd::add (map 0)
gdata::gdata (component on map 0)
dh::add (map 1)
while the other process sees:
dhd::add (map 0)
gdata::gdata (component on map 0)
dh::add (map 1)
The actual error is then (where there are some extra printf() lines that I
added):
{{{
INFO (Carpet): [tl=0] Starting initialisation
INFO (Carpet): [tl=0] GroupStorageIncrease
INFO (Carpet): [tl=0] ADMBASE::SHIFT_STATE: increase to 1
dh::add added varindex 0: shift_state
CHECKPOINT: processor 16, file
/work/00945/rhaas/Zelmani/arrangements/Carpet/CarpetLib/src/dh.cc, line
2176
Adding varindex 0: shift_state
INFO (Carpet): [tl=0] ADMBASE::DTLAPSE_STATE: increase to 1
dh::add added varindex 1: dtlapse_state
CHECKPOINT: processor 16, file
/work/00945/rhaas/Zelmani/arrangements/Carpet/CarpetLib/src/dh.cc, line
2176
Adding varindex 1: dtlapse_state
INFO (Carpet): [tl=0] ADMBASE::DTSHIFT_STATE: increase to 1
dh::add added varindex 2: dtshift_state
CHECKPOINT: processor 16, file
/work/00945/rhaas/Zelmani/arrangements/Carpet/CarpetLib/src/dh.cc, line
2176
Adding varindex 2: dtshift_state
INFO (Carpet): [tl=0] ADMBASE::LAPSE: increase to 1
dh::add added varindex 15: alp
CHECKPOINT: processor 16, file
/work/00945/rhaas/Zelmani/arrangements/Carpet/CarpetLib/src/dh.cc, line
2176
dh::add added varindex 15: alp
CHECKPOINT: processor 16, file
/work/00945/rhaas/Zelmani/arrangements/Carpet/CarpetLib/src/dh.cc, line
2176
WARNING level 0 in thorn CarpetLib processor 16 host
c305-212.ls4.tacc.utexas.edu
(line 251 of
/work/00945/rhaas/Zelmani/arrangements/Carpet/CarpetLib/src/dist.cc):
-> Wrong id for Barrier "CarpetLib::dist::checkpoint": expected
506880075d, found 783988953d
}}}
This like something that is rather hard to fix generally for little
benefit (ie. it affects only debugging runs with multipatch). Should this
even be reported (if only so that there is official notice that this is
known behaviour)? Should the fix be just a warning if Carpet encounters
this situation?
--
Ticket URL: <https://trac.einsteintoolkit.org/ticket/970>
Einstein Toolkit <http://einsteintoolkit.org>
The Einstein Toolkit
#967: GRHydro uses EOS_Omni routines without DECLARE_CCTK_FUNCTIONS
-----------------------------------+----------------------------------------
Reporter: rhaas | Owner:
Type: defect | Status: new
Priority: minor | Milestone:
Component: EinsteinToolkit thorn | Version:
Keywords: GRHydro |
-----------------------------------+----------------------------------------
and adding DECLARE_CCTK_FUNCTIONS eg. in ConservativeToPrimitive reveals
that we pass scalars to EOS_Omni routines that would expect (1 element)
arrays.
Making them (pmin, epsmin, rhomin are affected) all arrays makes the code
very ugly (even for GRHydro) :-).
--
Ticket URL: <https://trac.einsteintoolkit.org/ticket/967>
Einstein Toolkit <http://einsteintoolkit.org>
The Einstein Toolkit
#965: Carpet does not call global-early routines in POSTREGRID
--------------------+-------------------------------------------------------
Reporter: rhaas | Owner: eschnett
Type: defect | Status: new
Priority: major | Milestone:
Component: Carpet | Version:
Keywords: |
--------------------+-------------------------------------------------------
Carpet only calls routines on refinement levels that changed
(did_recompose == true). However global-early is hard-wired to rl==0,
which means it is never executed since reflevel 0 is never recomposed.
Unfortunately this affects HydroBase_InitExcisionMask which ends up not
being called (since #958). The error mode is not a fatal abort but was a
more subtle change in data for Christian Reisswig.
The attached patch attempts to fix this for EVOL and INITIAL.
Ok to apply?
While looking at this Christian Reisswig and I noticed that Carpet seems
to call the routines on the finest level whenever any recompose happened.
This is ok to ensure that global (also global-late) routines are called
but also means that local routines are called. In cases where the finest
level did not actually change (happens in core collapses we believe), this
causes unnecessary calls to eg. MoL_PostStep with its attending SYNC
calls. This might be candidate for optimization.
--
Ticket URL: <https://trac.einsteintoolkit.org/ticket/965>
Einstein Toolkit <http://einsteintoolkit.org>
The Einstein Toolkit
#942: GetComponents should not use shallow clones for Git checkouts
---------------------------+------------------------------------------------
Reporter: hinder | Owner: eric9
Type: enhancement | Status: new
Priority: major | Milestone:
Component: GetComponents | Version:
Keywords: |
---------------------------+------------------------------------------------
GetComponents currently uses a "shallow clone" for Git repositories by
default. This checks out only the repository information needed for the
current version.
Problems with shallow clones:
* It is impossible to switch branches to another version after the initial
clone. Since GetComponents checks out release branches using "git clone
...; git checkout ...", this means that release branches cannot be checked
out using the default GetComponents options (see #934). This has to be
fixed.
* You cannot push or pull from/to a shallow clone. In fact, you can do
very little with a shallow clone that you couldn't also do with source
obtained from a tarball.
Benefits of shallow clones:
* You save a small amount of space and checkout time due to not including
the (compressed) version history, which was the original rationale for
using them (see #148). The checkout size of Carpet was measured to
increase from 73 MB to 110 MB.
There was discussion in #148 amounting to the idea of providing a two-tier
Einstein Toolkit. The "developer tier" would be interested in full clones
and authenticated repositories, and the "user tier" would be interested in
shallow clones and non-authenticated (incorrectly conflated with "public")
repositories. I strongly dislike this idea, and agree with the comments
in that ticket which said that nearly all users of Cactus are also
developers, and should be treated in the same way. Let's keep things
simple and egalitarian.
I do not consider the space-saving to be significant, even if this was
representative of the ET as a whole, which it is not.
Shallow clones are nonstandard and lead to problems and confusion. I
propose removing support for shallow clones from GetComponents. If
there are truly users of the ET who do not want to interact with version
control systems at all, then we can accommodate these users by providing
release tarballs, which will be much smaller, easier and faster to
download.
--
Ticket URL: <https://trac.einsteintoolkit.org/ticket/942>
Einstein Toolkit <http://einsteintoolkit.org>
The Einstein Toolkit
#964: make "-a" default option for GetComponents
---------------------------+------------------------------------------------
Reporter: rhaas | Owner: eric9
Type: enhancement | Status: new
Priority: minor | Milestone:
Component: GetComponents | Version:
Keywords: |
---------------------------+------------------------------------------------
We seem to regularly receive support requests from users where
GetComponents fails because they forgot to add the "-a" option. Since
GetComponents is supposed to make checking out Cactus easier for novice
users, I suggest changing its default options to reflect what is most
likely required in this case. Having already the download of Cactus fail
would turn me away from using it.
For the developer that actually require write access, we should all be
computer and version-control savy enough to either change the checkout
afterwards or modify the thorn list. Its just a 'svn switch --relocate',
'$EDITOR .git/config' and '$EDITOR repos/carpet_hg/.hg/hgrc' after all.
So in summary I would like to suggest changing the default to "-a"
possibly at the same time we change the defaults to --no-shallow
--
Ticket URL: <https://trac.einsteintoolkit.org/ticket/964>
Einstein Toolkit <http://einsteintoolkit.org>
The Einstein Toolkit
#732: qc0-mclachlan example parameter file should be more functional
-------------------------+--------------------------------------------------
Reporter: hinder | Owner:
Type: enhancement | Status: new
Priority: minor | Milestone:
Component: Other | Version:
Keywords: |
-------------------------+--------------------------------------------------
I propose adding wave extraction (using WeylScal4 and Multipole), as well
as puncture tracking using PunctureTracker, to qc0-mclachlan.par.
--
Ticket URL: <https://trac.einsteintoolkit.org/ticket/732>
Einstein Toolkit <http://einsteintoolkit.org>
The Einstein Toolkit
#933: add test suite to EOS_Omni table reader
-------------------+--------------------------------------------------------
Reporter: rhaas | Type: enhancement
Status: new | Priority: minor
Milestone: | Component: EinsteinToolkit thorn
Version: | Keywords: EOS_Omni
-------------------+--------------------------------------------------------
the attached patch to EOS Omni (plus attached sample hdf5 table) adds a
small routine EOS_OMNI_dumptable to output the read in data as ASCII into
a user selected file.
It can be used to test the low level table reader facility (adding options
to make an EOS call would also be possible but is not implemented right
now).
Code and table kindly provided by Evan O'Connor.
--
Ticket URL: <https://trac.einsteintoolkit.org/ticket/933>
Einstein Toolkit <http://einsteintoolkit.org>
The Einstein Toolkit
#962: parallelize Multipole using OpenMP and MPI
-----------------------------------+----------------------------------------
Reporter: rhaas | Owner:
Type: enhancement | Status: new
Priority: minor | Milestone:
Component: EinsteinToolkit thorn | Version:
Keywords: Multipole |
-----------------------------------+----------------------------------------
The attached patch parallelizes each extraction sphere in multipole using
OpenMP and distributes the spheres across processors using MPI.
It contains threadprivate OMP pragmas for some memory that is allocated by
simpson integration rule which will mean that more memory is allocated
than would be for the non-openmp case. The alternative would be to
allocate and free the memory each time the function is called (this would
be much cleaner).
--
Ticket URL: <https://trac.einsteintoolkit.org/ticket/962>
Einstein Toolkit <http://einsteintoolkit.org>
The Einstein Toolkit
#941: CCTK_ActiveTimeLevels returns "wrong" number
----------------------------------------+-----------------------------------
Reporter: reisswig@… | Type: enhancement
Status: new | Priority: optional
Milestone: | Component: Cactus
Version: | Keywords: CCTK_ActiveTimeLevels
----------------------------------------+-----------------------------------
CCTK_ActiveTimeLevels is supposed to return the current number of active
timelevels for a given grid function.
Currently, however, with multiple reflevels (and also maps),
CCTK_ActiveTimeLevels returns "number of active time levels"*"number of
reflevels"*"number of maps".
Unfortunately, e.g. CarpetInterp can get confused when asked to time
interpolate. A grid function, which technically only has one level per
reflevel active, will have 3 levels active according to
CCTK_ActiveTimeLevels when there are 3 reflevels. CarpetInterp may then
think that indeed there are enough active time levels to time interpolate
even though there are not. This previously led to segfaults in
CarpetInterp in certain circumstances (now fixed by avoiding to call
CCTK_ActiveTimeLevels).
CCTK_ActiveTimeLevels really calls GroupStorageCrease in
Carpet/src/Storage.cc. At line, 207, the total number of timelevels is
computed. Instead of making this a product between number of reflevels,
timelevels, and maps, would it be possible to instead compute the minimum
over reflevels, maps? According to Erik, different reflevels/maps can have
different number of active timelevels. If we return the minimum, then we
are on the safe side and CCTK_ActiveTimeLevels would return a correct
number.
--
Ticket URL: <https://trac.einsteintoolkit.org/ticket/941>
Einstein Toolkit <http://einsteintoolkit.org>
The Einstein Toolkit
#904: Compiling with -DDEBUG causes problems for CactusBase/Boundary
---------------------+------------------------------------------------------
Reporter: sbrandt | Owner:
Type: defect | Status: new
Priority: minor | Milestone: ET_2012_11
Component: Cactus | Version:
Keywords: |
---------------------+------------------------------------------------------
Variable names were changed, but code inside #ifdef DEBUG's was not
updated.
--
Ticket URL: <https://trac.einsteintoolkit.org/ticket/904>
Einstein Toolkit <http://einsteintoolkit.org>
The Einstein Toolkit