Re: [PATCH v3 03/16] index-helper: new daemon for caching index and related stuff

On Tue, Apr 12, 2016 at 6:27 AM, David Turner [off-list ref] wrote:

On Fri, 2016-04-08 at 18:16 -0400, David Turner wrote:

quoted

And SHM on Macs works a bit differently than on Linux in at least two
irritating ways.

So, uh, new version to come once I actually make it work on Mac.
Probably Monday.

I was chatting with a friend about this and he mentioned that SHM does
not really fit well into the Unix "everything is a file" model.  It
lives in a separate namespace, and still requires most of the file-like
operations just with funny names and a separate namespace: shm_open,
shm_unlink.  This weirdness is something I noticed in my porting work:
on OS X, a shm name can only be 32 bytes long, requiring weird hacks.
And on OSX, fstat on a shm fd is rounded up to the page size (!).
 There may also be other portability issues that I have not yet
discovered.

Instead, my friend suggests that we should just use files.  For
instance, we could do $TMPDIR/$index_helper_pid/shm-index.$sha.

(I'm proposing $TMPDIR because it's cleaned up on reboot so we don't
need any manual intervention or complicated gc schemes)

What do folks think of this?

I avoided actual files for two reasons

 - disk error rate is higher than memory one, and we might need
trailing SHA-1 back
 - access is slow (unless cached, but we can't be sure)

If we can keep index-helper stuff on tmpfs or similar, then it would
address both, but that's even more OS-specific than shm. If we have a
good abstraction layer then people can put stuff on $TMPDIR on Mac,
for example. But then it's not full POSIX file interface anymore...
And you forgot Windows which does not strictly follow UNIX design.
-- 
Duy