[trac-noreply@einsteintoolkit.org: [Einstein Toolkit] #69: Formaline might stop build because of errors reported by tar]
Hi,
I forward the following trac message to the users mailing list to start a discussion on this issue, but to have the issue recorded in trac at the same time.
Frank
----- Forwarded message from Einstein Toolkit trac-noreply@einsteintoolkit.org -----
#69: Formaline might stop build because of errors reported by tar --------------------+------------------------------------------------------- Reporter: knarf | Owner: Type: defect | Status: new Priority: major | Milestone: ET_2010_11 Component: Cactus | Version: Keywords: | --------------------+------------------------------------------------------- Beginning with version 1.16 of gnu tar: <quote> After creating an archive, tar exits with code 1 if some files were changed while being read. Previous versions exited with code 2 (fatal error), and only if some files were truncated while being archived. </quote>
Even reading a file (as is done when compiling it in parallel) will change it in tar's view, because this changes the atime of the file. There does not seem to be an option which turns this new behavior off. Instead it is recommended to check the exit code of tar which is 0: if everything went ok 1: if files changed while being read/written 2: if some other error occurs.
This means that Formaline should treat an exit code of 1 in tar as 'ok', while currently it aborts the build. However, this is safely true only for gnu tar. For example the version of tar on the 'pelican' system states: <quote> 0 Successful completion.
0 An error occurred.
</quote> This is too vague to decide if ignoring an exit code of 1 can also here be ignored. File changes are a kind of error - of a kind one might be able to ignore. The documentation of this old version of tar doesn't mention which values are actually used for which errors though.
Somewhere else I found [http://aplawrence.com/Bofcusm/2073.html this]: <quote> 0 - success 1 - bad directory tree, failed to extract a requested file, input file same as output file, failed to open input file, could not create link, link table malloc failure 2 - internationalization error that should never occur, checksum error 5 - checksum error 9 (EBADF) - error reading /etc/default/tar, misplaced end of volume </quote>
So, it is probably not safe to assume that an exit value of 1 always means 'success' for any version of tar, in the case of Formaline. How should we proceed with Formaline? The current state is not desirable. As far as I can see we have two options:
1) Assume that an exit value of 1 to be a 'success'. I do have a patch for this. 2) Test for gnu tar and only then assume 1)
Opinions?
----- End forwarded message -----
On Tue, Oct 26, 2010 at 4:31 PM, Frank Loeffler knarf@cct.lsu.edu wrote:
Hi,
I forward the following trac message to the users mailing list to start a discussion on this issue, but to have the issue recorded in trac at the same time.
Frank
----- Forwarded message from Einstein Toolkit < trac-noreply@einsteintoolkit.org> -----
#69: Formaline might stop build because of errors reported by tar
--------------------+------------------------------------------------------- Reporter: knarf | Owner: Type: defect | Status: new Priority: major | Milestone: ET_2010_11 Component: Cactus | Version: Keywords: |
--------------------+------------------------------------------------------- Beginning with version 1.16 of gnu tar:
<quote> After creating an archive, tar exits with code 1 if some files were changed while being read. Previous versions exited with code 2 (fatal error), and only if some files were truncated while being archived. </quote>
Even reading a file (as is done when compiling it in parallel) will change it in tar's view, because this changes the atime of the file. There does not seem to be an option which turns this new behavior off. Instead it is recommended to check the exit code of tar which is 0: if everything went ok 1: if files changed while being read/written 2: if some other error occurs.
This means that Formaline should treat an exit code of 1 in tar as 'ok', while currently it aborts the build. However, this is safely true only for gnu tar. For example the version of tar on the 'pelican' system states:
<quote> 0 Successful completion. >0 An error occurred. </quote> This is too vague to decide if ignoring an exit code of 1 can also here be ignored. File changes are a kind of error - of a kind one might be able to ignore. The documentation of this old version of tar doesn't mention which values are actually used for which errors though.
Somewhere else I found [http://aplawrence.com/Bofcusm/2073.html this]:
<quote> 0 - success 1 - bad directory tree, failed to extract a requested file, input file same as output file, failed to open input file, could not create link, link table malloc failure 2 - internationalization error that should never occur, checksum error 5 - checksum error 9 (EBADF) - error reading /etc/default/tar, misplaced end of volume </quote>
So, it is probably not safe to assume that an exit value of 1 always means 'success' for any version of tar, in the case of Formaline. How should we proceed with Formaline? The current state is not desirable. As far as I can see we have two options:
- Assume that an exit value of 1 to be a 'success'. I do have a patch for
this. 2) Test for gnu tar and only then assume 1)
Opinions?
I would check whether it is gnu tar, then check its version, and then possibly accept an exit value of 1 as correct.
Alternatively, we could build the tar files only after the thorn has been built, and the source files won't be read any more. (This may already be the case.) We would also need to build the tar files either before or after git is run.
-erik
Hi,
On Sat, Oct 30, 2010 at 11:54:35PM -0400, Erik Schnetter wrote:
Alternatively, we could build the tar files only after the thorn has been built, and the source files won't be read any more.
That is not correct for all thorns. Some thorns do (I know this is bad form, but reality) look for header files in other thorns (inside the arrangements directory), not using the Cactus header file mechanism.
We would also need to build the tar files either before or after git is run.
True. For some reason this didn't seem to be a problem. Maybe this is already the case?
Frank
On Sun, Oct 31, 2010 at 12:00 AM, Frank Loeffler knarf@cct.lsu.edu wrote:
Hi,
On Sat, Oct 30, 2010 at 11:54:35PM -0400, Erik Schnetter wrote:
Alternatively, we could build the tar files only after the thorn has been built, and the source files won't be read any more.
That is not correct for all thorns. Some thorns do (I know this is bad form, but reality) look for header files in other thorns (inside the arrangements directory), not using the Cactus header file mechanism.
We could still build the tarballs after all thorns have been built.
-erik
We would also need to build the tar files either before or after git
is run.
True. For some reason this didn't seem to be a problem. Maybe this is already the case?
Frank
On Sun, Oct 31, 2010 at 12:28:50AM -0400, Erik Schnetter wrote:
We could still build the tarballs after all thorns have been built.
Do you have a patch for that ready? I would like to have this issue fixed as soon as possible to have it in the new ET release. That means it should be fixed within the next 1-2 days.
If we don't come up with a better solution than the currently proposed patch I suggest to apply it now, and think about something else later.
Frank
I suggest to apply your GNUTAR patch.
-erik
On Mon, Nov 1, 2010 at 12:29 PM, Frank Loeffler knarf@cct.lsu.edu wrote:
On Sun, Oct 31, 2010 at 12:28:50AM -0400, Erik Schnetter wrote:
We could still build the tarballs after all thorns have been built.
Do you have a patch for that ready? I would like to have this issue fixed as soon as possible to have it in the new ET release. That means it should be fixed within the next 1-2 days.
If we don't come up with a better solution than the currently proposed patch I suggest to apply it now, and think about something else later.
Frank
users@lists.einsteintoolkit.org
-
Erik Schnetter -
Frank Loeffler