Re: [RFC] On watchman support

On Tue, Nov 18, 2014 at 7:25 AM, David Turner [off-list ref] wrote:

quoted

So we got a few options:

1) Convince watchman devs to add something to make it work

Based on the thread on the watchman github it looks like this won't
happen.

Yeah. I came to the conclusion that I needed an extra daemon. And
because I would need an extra daemon anyway to speed up index read
time, that one could be used for caching something else like watchman.
It works out quite nice: because watchman is not tied to the main
'git' binary, people don't need libwatchman and libjansson by default.
When people want watchman, they can install an extra package that
includes the index-helper and its dependencies. This only matters for
binary-based distros of course.

quoted

Comments?

I don't think it would be impossible to add Windows support to watchman;
the necessary functions exist, although I don't know how well they work.
My experience with watchman is that it is something of a stress test of
a filesystem's notification layer.  It has exposed bugs in inotify, and
caused system instability on OS X.

The way i'm adding watchman to index-helper should work on Windows as
well, IPC is really simple. But let's wait and see.

My patches are not the world's most beautiful, but they do work.  I
think some improvement might be possible by keeping info about tracked
files in the index, and only storing the tree of ignored and untracked
files separately.  But I have not thought this through fully.  In any
case, making use of shared memory for the fs_cache (as some of your
other patches do for the index) would definitely save time.

By the way, what happened to your sse optimization in refs.c? I see
it's reverted but I didn't follow closely to know why. Or will you go
with cityhash now.. I ask because you have another sse optimization
for hashmap on your watchman branch and that could reduce init time
for name-hash. Name-hash is used often on case-insensitive fs (less
often on case-sensitive fs).

I did a simple test and your optimization could init name-hash (on
webkit) in 35ms, while unmodified hashmap took 88ms. Loading index on
this machine took 360ms for reference (probably down too 100ms with
index-helper running, when that 88ms starts to become significant).
-- 
Duy