From: Thomas Rast <hidden> Date: 2016-06-15 22:55:51
Eric Chamberland [off-list ref] writes:
So, hum, do we have some sort of conclusion?
Shall it be a fix for git to get around that lustre "behavior"?
If something can be done in git it would be great: it is a *lot*
easier to change git than the lustre filesystem software for a cluster
in running in production mode... (words from cluster team) :-/
I thought you already established that simply disabling the progress
display is a sufficient workaround? If that doesn't help, you can try
patching out all use of SIGALRM within git.
Other than that I agree with Junio, from what we've seen so far, Lustre
returns EINTR on all sorts of calls that simply aren't allowed to do so.
--
Thomas Rast
trast@{inf,student}.ethz.ch
From: Eric Chamberland <hidden> Date: 2016-06-15 22:55:51
On 01/22/2013 05:14 PM, Thomas Rast wrote:
Eric Chamberland [off-list ref] writes:
quoted
So, hum, do we have some sort of conclusion?
Shall it be a fix for git to get around that lustre "behavior"?
If something can be done in git it would be great: it is a *lot*
easier to change git than the lustre filesystem software for a cluster
in running in production mode... (words from cluster team) :-/
I thought you already established that simply disabling the progress
display is a sufficient workaround? If that doesn't help, you can try
patching out all use of SIGALRM within git.
I tried that solution after Brian told me to try it, but it didn't
solved the problem for me! :-(
Other than that I agree with Junio, from what we've seen so far, Lustre
returns EINTR on all sorts of calls that simply aren't allowed to do so.
Ok, so now the "good" move would be to have all this reported to lustre
development team? Brian, have you seen anything new from what you have
already reported? I have to admit that I am not a fs expert...
And I also agree with Junio point of view: The problem may impact
mission critical applications....
Eric
So, hum, do we have some sort of conclusion?
Shall it be a fix for git to get around that lustre "behavior"?
If something can be done in git it would be great: it is a *lot*
easier to change git than the lustre filesystem software for a cluster
in running in production mode... (words from cluster team) :-/
I thought you already established that simply disabling the progress
display is a sufficient workaround? If that doesn't help, you can try
patching out all use of SIGALRM within git.
In git (9591fcc6d66), I have found these SIGALRM signal handling:
builtin/log.c:268: sigaction(SIGALRM, &sa, NULL);
builtin/log.c:285: signal(SIGALRM, SIG_IGN);
compat/mingw.c:1590: mingw_raise(SIGALRM);
compat/mingw.c:1666: if (sig != SIGALRM)
compat/mingw.c:1668: error("sigaction only implemented for SIGALRM");
compat/mingw.c:1683: case SIGALRM:
compat/mingw.c:1702: case SIGALRM:
compat/mingw.c:1706: exit(128 + SIGALRM);
compat/mingw.c:1708: timer_fn(SIGALRM);
compat/mingw.h:42:#define SIGALRM 14
perl/Git/SVN.pm:2121: SIGALRM, SIGUSR1, SIGUSR2);
progress.c:56: sigaction(SIGALRM, &sa, NULL);
progress.c:68: signal(SIGALRM, SIG_IGN);
I suppose that compat/mingw.{h,c} and SVN.pm can be ignored as our patch to work
around this problem won't be pushed upstream because the real problem is not in git, right ?
If I understand correctly, some VFS system calls get interrupted by SIGALRM, but when
they resume (via SA_RESTART) they return EINTR. Thomas said that these failed calls may need to be retried,
but that open(O_CREAT|O_EXCL) is still tricky around this case.
progress.c SIGALRM code paths are for progress and therefore are required, right ?
builtin/log.c SIGALRM code paths are for early output, and the comments in the code say that
"If we can get the whole output in less than a tenth of a second, don't even bother doing the
early-output thing."
So where do I start for the patch ?
Other than that I agree with Junio, from what we've seen so far, Lustre
returns EINTR on all sorts of calls that simply aren't allowed to do so.
[I forgot to subscribe to the git mailing list, sorry for that]
On 01/22/2013 05:14 PM, Thomas Rast wrote:
Eric Chamberland [off-list ref] writes:
quoted
So, hum, do we have some sort of conclusion?
Shall it be a fix for git to get around that lustre "behavior"?
If something can be done in git it would be great: it is a *lot*
easier to change git than the lustre filesystem software for a cluster
in running in production mode... (words from cluster team) :-/
I thought you already established that simply disabling the progress
display is a sufficient workaround? If that doesn't help, you can try
patching out all use of SIGALRM within git.
In git (9591fcc6d66), I have found these SIGALRM signal handling:
builtin/log.c:268: sigaction(SIGALRM, &sa, NULL);
builtin/log.c:285: signal(SIGALRM, SIG_IGN);
compat/mingw.c:1590: mingw_raise(SIGALRM);
compat/mingw.c:1666: if (sig != SIGALRM)
compat/mingw.c:1668: error("sigaction only implemented for SIGALRM");
compat/mingw.c:1683: case SIGALRM:
compat/mingw.c:1702: case SIGALRM:
compat/mingw.c:1706: exit(128 + SIGALRM);
compat/mingw.c:1708: timer_fn(SIGALRM);
compat/mingw.h:42:#define SIGALRM 14
perl/Git/SVN.pm:2121: SIGALRM, SIGUSR1, SIGUSR2);
progress.c:56: sigaction(SIGALRM, &sa, NULL);
progress.c:68: signal(SIGALRM, SIG_IGN);
I suppose that compat/mingw.{h,c} and SVN.pm can be ignored as our patch to work
around this problem won't be pushed upstream because the real problem is not in git, right ?
If I understand correctly, some VFS system calls get interrupted by SIGALRM, but when
they resume (via SA_RESTART) they return EINTR. Thomas said that these failed calls may need to be retried,
but that open(O_CREAT|O_EXCL) is still tricky around this case.
progress.c SIGALRM code paths are for progress and therefore are required, right ?
builtin/log.c SIGALRM code paths are for early output, and the comments in the code say that
"If we can get the whole output in less than a tenth of a second, don't even bother doing the
early-output thing."
So where do I start for the patch ?
Other than that I agree with Junio, from what we've seen so far, Lustre
returns EINTR on all sorts of calls that simply aren't allowed to do so.
From: Erik Faye-Lund <hidden> Date: 2016-06-15 22:55:52
On Tue, Jan 22, 2013 at 11:14 PM, Thomas Rast [off-list ref] wrote:
Eric Chamberland [off-list ref] writes:
Other than that I agree with Junio, from what we've seen so far, Lustre
returns EINTR on all sorts of calls that simply aren't allowed to do so.
I don't think this analysis is 100% accurate, POSIX allows error codes
to be generated other than those defined. From
http://pubs.opengroup.org/onlinepubs/009695399/functions/xsh_chap02_03.html:
"Implementations may support additional errors not included in this
list, *may generate errors included in this list under circumstances
other than those described here*, or may contain extensions or
limitations that prevent some errors from occurring."
So I don't think Lustre violates POSIX by erroring with errno=EINTR,
but I also think guarding every single function call for EINTR just to
be safe to be insane :)
However, looking at Eric's log, I can't see that being what has
happened here - grepping it for EINTR does not produce a single match.
Hello,
Here is a patch (with git format-patch) that removes any timer if NO_SETITIMER is set.
Éric:
To test it with your workflow:
$ module load apps/git/1.8.1.1.348.g78eb407-NO_SETITIMER-patch
$ git clone ...
Sébastien
On 01/22/2013 05:14 PM, Thomas Rast wrote:
Eric Chamberland [off-list ref] writes:
quoted
So, hum, do we have some sort of conclusion?
Shall it be a fix for git to get around that lustre "behavior"?
If something can be done in git it would be great: it is a *lot*
easier to change git than the lustre filesystem software for a cluster
in running in production mode... (words from cluster team) :-/
I thought you already established that simply disabling the progress
display is a sufficient workaround? If that doesn't help, you can try
patching out all use of SIGALRM within git.
Other than that I agree with Junio, from what we've seen so far, Lustre
returns EINTR on all sorts of calls that simply aren't allowed to do so.