#1900: Formaline circumvents git's file change caching ------------------------------------+--------------------------------------- Reporter: rhaas | Owner: Type: defect | Status: new Priority: minor | Milestone: Component: EinsteinToolkit thorn | Version: development version Resolution: | Keywords: Formaline ------------------------------------+---------------------------------------

Comment (by rhaas):

Yes, that patch that broke this is mine. The idea behind the patch was to avoid having to make hard links when adding files to the repo (both for hoped for speed and because some FS do not allow hard links). I actually looked at the git source code (complicated) to try and find out if there is a way to use git's cache of metadata when using the plumbing commands. So far I have not found anything that would let me do that.

#1900: Formaline circumvents git's file change caching ------------------------------------+--------------------------------------- Reporter: rhaas | Owner: Type: defect | Status: new Priority: minor | Milestone: Component: EinsteinToolkit thorn | Version: development version Resolution: | Keywords: Formaline ------------------------------------+---------------------------------------

Comment (by rhaas):

I have no suggestion. The problem with the previous one is that it relies in hard links between directories which not all file systems support (well ok: 1 file system does not support, but it is on a cluster that I use).

If you want to revert the patch in master then that is fine with me since I have no real better solution so far. However is this breaks (rhather than slows down) formaline on eg loewe then we need a workaround for the failure on loewe I think.

#1900: Formaline circumvents git's file change caching ------------------------------------+--------------------------------------- Reporter: rhaas | Owner: Type: defect | Status: new Priority: minor | Milestone: Component: EinsteinToolkit thorn | Version: development version Resolution: | Keywords: Formaline ------------------------------------+---------------------------------------

Comment (by rhaas):

A workaround is probably simple (since the change itself is simple). I am not sure how large the slowdown is because each time a file changes (and thus all files are currently re-hashed) the tarball is also recreated causing all files to be read and tar-gzipped. It is possible that the slowdown is more noticeable in cases where updating the git repo and creating the tarballs happens at different times since the OS caches have been flushed. For cases where updating the git repo happens just after creating the tarball then all files should still be in cache and any slowdown would be due to increased CPU load which should be much less than due to IO.