Re: kernel.org mirroring (Re: [GIT PULL] MMC update)

On 12/9/06, H. Peter Anvin [off-list ref] wrote:

Martin Langhoff wrote:

quoted

On 12/9/06, Jakub Narebski [off-list ref] wrote:

quoted

Perhaps gitweb should generate it's own ETag instead of messing with
'expires' header?

That'll be the winning solution.

Doesn't solve the thundering herd problem or the timeout problem at all,
though.

I posted separately about those. And I've been mulling about whether
the thundering herd is really such a big problem that we need to
address it head-on. If we doHTTP  caching headers right (that is, a
bit better than now) then the fact that web caches are distributed
means that even a cache restart or cache invalidation won't trigger a
thundering herd.

And gitweb rarely has a "new" URL that gets a ton of hits immediately.
Our real problem is the summary page, and the fact that we aren't
setting an effecting ETag there. If we do, a front-end cache plus the
ability to revalidate the ETag cheaply will get us through.

We get 99% of the benefit from ETags and cheap revalidations,
specially if they are coupled with a reverse caching proxy,. The
remaining 1% of dealing with the highly infrequent thundering herd can
be addressed with the scheme I've posted 5 minutes ago.

cheers