Re: [PATCH 0/2] fix http deadlock on giant ref negotiations

On vr, 2015-05-15 at 02:29 -0400, Jeff King wrote:

On Wed, May 13, 2015 at 08:47:24PM -0400, Jeff King wrote:

quoted

The fundamental problem is the deadlock on the server side, which is
producing bogus protocol output. And that's a mismatch between what
Apache expects (that the CGI will read all of the input request and then
generate an output request) and what the CGI wants to do (stream output
as it reads the input).

At first I was irritated with Apache for this. But thinking on it more,
it's really due to our shoe-horning of a full-duplex protocol into the
half-duplex HTTP protocol. Even if we could convince Apache to work in a
full-duplex way here, and even if our client is full-duplex (since
otherwise we are just trading pipe buffers for TCP buffers), we still
may face arbitrary HTTP proxies or other infrastructure in the middle.

So here's a series to try to address the issue. The first patch is a
fixed version of the die-recursion fixup I posted earlier. The second is
the interesting one.

  [1/2]: http-backend: fix die recursion with custom handler
  [2/2]: http-backend: spool ref negotiation requests to buffer

I have no clue how to write a test that would trigger this reliably
without requiring a gigantic test fixture. However, I did confirm that
it fixes the problem on the chromium case you provided (which otherwise
deadlocks reliably for me).

This looks similar to the failure I posted about alst year in
http://thread.gmane.org/gmane.comp.version-control.git/258514 

Though the issue is different, it has the same 'hanging git fetch'
symptom due to the deadlock between upload-pack and http-backend.

The patch I sent back then is suboptimal, as it can cause larger packs
than necessary (we still use it though, as the alternative is a
non-working git), but it does include a test you may be able to use to
verify your fix, if this is indeed the same issue.

-- 
Dennis Kaarsemaker
http://www.kaarsemaker.net