Re: [PATCH 0/2] fix http deadlock on giant ref negotiations

On vr, 2015-05-15 at 04:38 -0400, Jeff King wrote:

On Fri, May 15, 2015 at 09:41:20AM +0200, Dennis Kaarsemaker wrote:

quoted

quoted

I have no clue how to write a test that would trigger this reliably
without requiring a gigantic test fixture. However, I did confirm that
it fixes the problem on the chromium case you provided (which otherwise
deadlocks reliably for me).

This looks similar to the failure I posted about alst year in
http://thread.gmane.org/gmane.comp.version-control.git/258514

Though the issue is different, it has the same 'hanging git fetch'
symptom due to the deadlock between upload-pack and http-backend.

Thanks, I think it is the same issue (in the end I was replicating not
with `--reference`, but just by doing a fetch from the other
repository). And our solutions are essentially the same. I do prefer
mine because:

  1. It keeps the buffering logic in http-backend; the half-duplex
     nature is an http detail.

  2. I think it's better to buffer the request rather than the response,
     for the reasons I stated in the commit message.

quoted

The patch I sent back then is suboptimal, as it can cause larger packs
than necessary (we still use it though, as the alternative is a
non-working git), but it does include a test you may be able to use to
verify your fix, if this is indeed the same issue.

I applied the test from your patch, but couldn't get it to fail even
with stock git.  The test above it shrunk a bit, but I was able to tweak
yours to generate tags from 2001..100000, which I thought would have
worked.  I suspect it's something silly like the size not being quite
big enough for the pipe buffer on my system, or something like that.
Though I couldn't get it to fail even with 200,000 tags, so perhaps it's
something else.

The shrinkage in the test above it will actually work around the issue,
as there are now fewer already-fetched tags to negotiate. Either
reverting that shrinkage or executing the new test twice should do the
trick.
-- 
Dennis Kaarsemaker
http://www.kaarsemaker.net