From: Junio C Hamano <hidden> Date: 2016-06-15 22:52:44
Clemens Buchacher [off-list ref] writes:
I have rebased Junio's cb/git-daemon-tests onto your
jk/child-cleanup and replaced the call to pkill with a regular kill
command.
On top of that, I have added two commits to fix the discussed race
condition. I also verified that the race condition actually happens
by adding an artificial delay in the daemon (this change is
obviously not included).
I pushed the new cb/git-daemon-tests to
https://github.com/drizzd/git . If you have no objections I will
post the entire series including your run-command and send-pack
patches to the list.
Looked fine except that some patches seem to lack enough justification
(justification in Peff's reply was good enough).
I actually was thinking that the previous round was good enough (perhaps
dropping the "pkill" bit altogether and replacing it with "kill" on the
daemon process itself, if OSX folks complain loudly), so it is in "next"
already, but it seems that the best course of action would be to drop it
and queue your re-roll afresh, aiming for the next cycle.
Thanks.
From: Clemens Buchacher <hidden> Date: 2016-06-15 22:52:44
It feels natural for a user to view git commands as monolithic
commands with a single thread of execution. If the parent git
command dies, it should therefore clean up its child processes as
well. So enable the cleanup mechanism by default.
For dashed externals, this means that killing the git wrapper will
kill the command itself, just like what would happen in case of an
internal command. A notable exception is the credentials cache
daemon, which must stay alive after the store command has
completed.
Signed-off-by: Clemens Buchacher <redacted>
---
I considered squashing this into the previous commit. But it's a fairly
small change and may help with bisecting in case of problems.
credential-cache.c | 1 +
run-command.c | 4 ++--
run-command.h | 2 +-
3 files changed, 4 insertions(+), 3 deletions(-)
From: Clemens Buchacher <hidden> Date: 2016-06-15 22:52:44
If a client tries to connect after git-daemon starts, but before it
opens a listening socket, the connection will fail. Output "[PID]
Ready to rumble]" after opening the socket successfully in order to
inform the user that the daemon is now ready to receive
connections.
Signed-off-by: Clemens Buchacher <redacted>
---
daemon.c | 6 +++---
1 files changed, 3 insertions(+), 3 deletions(-)
@@ -1086,6 +1086,8 @@ static int serve(struct string_list *listen_addr, int listen_port,drop_privileges(cred);+loginfo("Ready to rumble");+returnservice_loop(&socklist);}
@@ -1270,10 +1272,8 @@ int main(int argc, char **argv)if(inetd_mode||serve_mode)returnexecute();-if(detach){+if(detach)daemonize();-loginfo("Ready to rumble");-}elsesanitize_stdfds();
From: Clemens Buchacher <hidden> Date: 2016-06-15 22:52:44
On Fri, Jan 06, 2012 at 02:49:05PM -0800, Junio C Hamano wrote:
but it seems that the best course of action would be to drop it
and queue your re-roll afresh, aiming for the next cycle.
Here's the re-rolled series, also available as cb/git-daemon-tests based
on current master at https://github.com/drizzd/git .
[PATCH 1/5] run-command: optionally kill children on exit
[PATCH 2/5] run-command: kill children on exit by default
[PATCH 3/5] git-daemon: add tests
[PATCH 4/5] git-daemon: produce output when ready
[PATCH 5/5] git-daemon tests: wait until daemon is ready
On Fri, Jan 06, 2012 at 05:32:15PM -0500, Jeff King wrote:
On Fri, Jan 06, 2012 at 08:48:00PM +0100, Clemens Buchacher wrote:
quoted
I have rebased Junio's cb/git-daemon-tests onto your
jk/child-cleanup and replaced the call to pkill with a regular kill
command.
Looks pretty good from my cursory examination. I think you should fill
out the rationale for "kill dashed externals on exit" a bit. My
reasoning is that whether a git command is an internal or external
process is purely an implementation detail, and killing the git wrapper
should behave identically in both cases.
The previous version of this patch only changed the behavior for users
of run_command_v_opt, but not for those who filled out the child_process
structure by themselves. I could have manually enabled all of those, but
that felt unnatural. Instead, I have now reversed the meaning of
clean_on_exit to stay_alive_on_exit in [PATCH 2/5] run-command: kill
children on exit by default. Cleanup is on by default and callers of
run_command must disable it if children should stay alive.
From: Clemens Buchacher <hidden> Date: 2016-06-15 22:52:44
In start_daemon, git-daemon is started as a background process. In
theory, the tests may try to connect before the daemon had a chance
to open a listening socket. Avoid this race condition by waiting
for it to output "Ready to rumble". Any other output is considered
an error and the test is aborted.
Should git-daemon produce no output at all, lib-git-daemon would
block forever. This could be fixed by introducing a timeout. On
the other hand, we have no timeout for other git commands which
could suffer from the same problem. Since such a mechanism adds
some complexity, I have decided against it.
Signed-off-by: Clemens Buchacher <redacted>
---
t/lib-git-daemon.sh | 18 +++++++++++++++++-
1 files changed, 17 insertions(+), 1 deletions(-)
From: Clemens Buchacher <hidden> Date: 2016-06-15 22:52:44
From: Jeff King <redacted>
When we spawn a helper process, it should generally be done
and finish_command called before we exit. However, if we
exit abnormally due to an early return or a signal, the
helper may continue to run in our absence.
In the best case, this may simply be wasted CPU cycles or a
few stray messages on a terminal. But it could also mean a
process that the user thought was aborted continues to run
to completion (e.g., a push's pack-objects helper will
complete the push, even though you killed the push process).
This patch provides infrastructure for run-command to keep
track of PIDs to be killed, and clean them on signal
reception or input, just as we do with tempfiles. PIDs can
be added in two ways:
1. If NO_PTHREADS is defined, async helper processes are
automatically marked. By definition this code must be
ready to die when the parent dies, since it may be
implemented as a thread of the parent process.
2. If the run-command caller specifies the "clean_on_exit"
option. This is not the default, as there are cases
where it is OK for the child to outlive us (e.g., when
spawning a pager).
PIDs are cleared from the kill-list automatically during
wait_or_whine, which is called from finish_command and
finish_async.
Signed-off-by: Clemens Buchacher <redacted>
---
Not sure if I can sign off without your sign-off. Should I have
replaced this with Acked-by?
run-command.c | 68 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++
run-command.h | 1 +
2 files changed, 69 insertions(+), 0 deletions(-)
From: Clemens Buchacher <hidden> Date: 2016-06-15 22:52:44
The semantics of the git daemon tests are similar to the http transport
tests. In fact, they are only a slightly modified copy of t5550, plus the
newly added remote error tests.
All git-daemon tests will be skipped unless the environment variable
GIT_TEST_GIT_DAEMON is set.
Signed-off-by: Clemens Buchacher <redacted>
Helped-by: Jeff King [off-list ref]
Signed-off-by: Junio C Hamano <redacted>
---
t/lib-git-daemon.sh | 53 +++++++++++++++++
t/t5570-git-daemon.sh | 148 +++++++++++++++++++++++++++++++++++++++++++++++++
2 files changed, 201 insertions(+), 0 deletions(-)
create mode 100644 t/lib-git-daemon.sh
create mode 100755 t/t5570-git-daemon.sh
Our Windows implementation of kill (mingw_kill in compat/mingw.c) only
supports SIGKILL, so propagating other signals to child-processes will
fail with EINVAL. That being said, Windows' support for signals is
severely limited, but I'm not entirely sure which ones can be
generated in this case.
From: Jeff King <hidden> Date: 2016-06-15 22:52:44
On Sat, Jan 07, 2012 at 12:42:43PM +0100, Clemens Buchacher wrote:
Signed-off-by: Clemens Buchacher <redacted>
---
Not sure if I can sign off without your sign-off. Should I have
replaced this with Acked-by?
Sorry, I usually sign-off when I sent to the list. But:
Signed-off-by: Jeff King <redacted>
for this and the other patch in this series.
As for whether you can sign-off, I think it is OK in this case. You are
basically signing off on the "Certificate of Origin" found in
SubmittingPatches. I think you are covered under (b), which is that to
the best of your knowledge it is based on open source work (i.e., even
though I didn't sign off explicitly, it is pretty obvious that this is
meant to be open source). But it's nicer to be explicit.
-Peff
From: Jeff King <hidden> Date: 2016-06-15 22:52:44
On Sat, Jan 07, 2012 at 12:42:44PM +0100, Clemens Buchacher wrote:
It feels natural for a user to view git commands as monolithic
commands with a single thread of execution. If the parent git
command dies, it should therefore clean up its child processes as
well. So enable the cleanup mechanism by default.
I'm not sure this is a good idea. run_command is used in ~70 places in
git, and I'm sure at least one of them is going to be unhappy (I see you
found one in credential-cache, but how many others are there). I'd
rather be conservative and leave the default the same, and then switch
over callsites that make sense.
-Peff
PS I thought this would certainly break the pager, since it should
outlast us after we finish producing output. But I think at one point
I switched the pager invocation so that the git wrapper lives and
waits until the pager dies.
From: Clemens Buchacher <hidden> Date: 2016-06-15 22:52:44
On Sat, Jan 07, 2012 at 01:45:03PM +0100, Erik Faye-Lund wrote:
Our Windows implementation of kill (mingw_kill in compat/mingw.c) only
supports SIGKILL, so propagating other signals to child-processes will
fail with EINVAL. That being said, Windows' support for signals is
severely limited, but I'm not entirely sure which ones can be
generated in this case.
On Linux at least, SIGKILL is not a viable alternative for SIGTERM,
since it does not give the child process to do any cleanup of its own
(such as signaling its own children, for example).
In any case, due this whole experience, and recently another one with
overzealous virus scanners, I have added a "get rid of dashed externals"
work item to my TODO list.