Re: [RFC PATCH 10/10] gitweb: Show appropriate "Generating..." page when regenerating cache (WIP)

On 01/25/2010 05:48 AM, Jakub Narebski wrote:

On Mon, Jan 25, 2010, Petr Baudis wrote:

quoted

On Mon, Jan 25, 2010 at 12:46:39PM +0100, Jakub Narebski wrote:

quoted

On Sun, 24 Jan 2010, Petr Baudis wrote:

quoted

quoted

quoted

  I have stupid question, common to both the original patch and this
RFC.

quoted

[RFC PATCH 10/10] gitweb: Show appropriate "Generating..." page when
regenerating cache (WIP)

  Just why is a "Generating..." page appropriate?

  I have to admit I hate it; can you please at least make it
configurable? Why is it needed at all? It [...] confuses
non-interactive HTTP clients [...]

quoted

quoted

Second, gitweb can always check User-Agent header, and serve 
"Generating..." page only to web browsers:

  unless (defined $cgi->user_agent() &&
          $cgi->user_agent() =~ /\b(Mozilla|Opera)\b/i) {
  	return;
  }

or something like that.

I'm not too happy with this. What about Safari? Opera? ELinks? There's a
lot of web browsers.

The "Mozilla" part would catch all "Mozilla compatibile" web browsers,
including Firefox (and other Gecko-based web browsers), Internet Explorer,
WebKit based browsers including Safari and Chrome and Konqueror.
The "Opera" part would catch Opera.
http://www.nczonline.net/blog/2010/01/12/history-of-the-user-agent-string/

As to other web browsers like Elinks, Lynx, w3m, Dillo, etc.: the issue
is whether they honor '<meta http-equiv="refresh" content="0" />'.  
I think it is better to stay on the safe side; it is not disaster if web
browser is not shown "Generating..." page where it could (but see 
explanation below).

Most of them do, that particular tag has been around for a long time and
since it doesn't require Javascript to do the page refresh it's pretty
much universal.

The problem is going to be with things like wget when someone wants to
snag a binary file.  This works fine if the file is already cached, but
the user doesn't get what they are expecting if they get a blob that
isn't the final file, but the html contents of the page.  I don't know
of any hint that things like wget would send to the server that you
could switch based on, but it would be more or less the non-background
caching state.

quoted

Most of the issues can be worked around, but I'm not sure why to go
through all the trouble. I just personally don't see the value in having
the placeholder in there at all, to me it is distracting UI even if all
the technicalities are put aside.

The issue that "Generating..." page tries to solve is, I think, the 
following.  

Some actions, like 'blame' view or pickaxe search, or grep search,
can take quite a long time to generate, with times counted in
tens of seconds.  It is not that visible for non-caching, because
gitweb streams output so we have at least _some_ output upfront quite
fast.

Now those patches (mine and J.H. both) make gitweb use locking
(it is IIRC configurable in J.H. patch) to make only one process
generate the page if it is missing from cache, or is stale.  Now
if it is missing, we have to wait until it is generated in full
before being able to show it to client.  While it is possible to
"tee" output (using PerlIO::tee, or Capture::Tiny, or tie like
CGI::Cache) writing it simultaneously to browser and to cache for 
the process that is generating data, it is as far as I understand
it impossible for processes which are waiting for data.  Therefore
the need for "Generating..." page, so the user does not think that
web server hung or something, and is not generating output.
 
We can try to reduce occurrences of cache miss stampedes by using
'expires_variance' feature[1] from CHI - Unified caching interface.
We can also turn off locking and tee output to have some output upfront
as an activity indicator instead of this "Generating..." page.

[1]: http://search.cpan.org/~jswartz/CHI-0.33/lib/CHI.pm#set

quoted

But if it will be possible to turn this off eventually, it's all your
call whether to bother implementing it. :-)

In my implementation it is (or rather would be) as simple as just
not passing 'generating_info' => \&git_generating_data_html in the
GitwebCache::SimpleFileCache constructor.

At least in mine it was don't allow background caching.  It would force
everyone to wait on the one process that was actually generating
content.  But it means a few blank pages with a spinning working icon
until the cache releases it's exclusive lock.

- John 'Warthog9' Hawley