#1772: Simfactory: potentially serious problem with CACHE directory in the simulations directory -------------------------+-------------------------------------------------- Reporter: bmundim | Owner: Type: defect | Status: new Priority: critical | Milestone: ET_2015_05 Component: SimFactory | Version: development version Resolution: | Keywords: CACHE -------------------------+--------------------------------------------------
Comment (by eschnett):
If there is a machine where creating a hard link leads to a soft link being created, then someone needs to get a stern talking-to. They have very different behaviour, and this leads to exactly the problem you describe. I assume that Loewe may have a file system that doesn't support hard links, and that someone decided to use file system options that replaces hard links by soft links. I'm not happy.
That's exactly the reason why Simfactory is so complex: It's not that things are difficult, it's that each HPC system is its own fiefdom that does things every so slightly differently than others. To provide a uniform interface, one needs to check all the corner cases.
What machine and what file system is Loewe?
Regarding file size vs. number of files: No, it is really the file size that makes the difference, not the number of files.
The fallback when a hard link cannot be created (e.g. on a Blue Gene/Q, which does not support hard links) is to copy files instead. In this case the cache becomes meaningless.
In my steps above, the cache is not updated at step 1. It is only updated (or created if need be) in step 6, after the simulation has the correct executable.
And with "right executable" I mean that it is the same executable, byte for byte, as determined by an actual comparison, as the one in the source tree.