On Thu, 17 Jan 2008, Shawn O. Pearce wrote:
fast-import was relying on the fact that on most systems mmap() and
write() are synchronized by the filesystem's buffer cache. We were
relying on the ability to mmap() 20 bytes beyond the current end
of the file, then later fill in those bytes with a future write()
call, then read them through the previously obtained mmap() address.
This isn't always true with some implementations of NFS, but it is
especially not true with our NO_MMAP=YesPlease build time option used
on some platforms.
In fact, even with mmap(), it's not guaranteed. There are really crappy
mmap implementations out there, partly due to bad CPU design (virtual CPU
caches without coherency), but more often due to total crap OS.
(Yeah, Linux did count in that area at some point. Long ago. Early 90's.
Maybe)
I think HP-UX used to have non-coherent mmap for the longest time, due to
carrying around some totally crap memory management based on some ancient
BSD version that everybody else (including the BSD's) had long since
jettisoned.
That said, I suspect any unix you can run today (without calling it a
retro setup) probably has coherent-enough mmap. The possible virtual cache
coherency issue is unlikely to be able to trigger this (and not relevant
on any sane hardware anyway).
Linus
On Thu, Jan 17, 2008 at 08:27:08PM -0800, Linus Torvalds wrote:
On Thu, 17 Jan 2008, Shawn O. Pearce wrote:
quoted
fast-import was relying on the fact that on most systems mmap() and
write() are synchronized by the filesystem's buffer cache. We were
relying on the ability to mmap() 20 bytes beyond the current end
of the file, then later fill in those bytes with a future write()
call, then read them through the previously obtained mmap() address.
This isn't always true with some implementations of NFS, but it is
especially not true with our NO_MMAP=YesPlease build time option used
on some platforms.
In fact, even with mmap(), it's not guaranteed. There are really crappy
mmap implementations out there, partly due to bad CPU design (virtual CPU
caches without coherency), but more often due to total crap OS.
(Yeah, Linux did count in that area at some point. Long ago. Early 90's.
Maybe)
I think HP-UX used to have non-coherent mmap for the longest time, due to
carrying around some totally crap memory management based on some ancient
BSD version that everybody else (including the BSD's) had long since
jettisoned.
That said, I suspect any unix you can run today (without calling it a
retro setup) probably has coherent-enough mmap. The possible virtual cache
coherency issue is unlikely to be able to trigger this (and not relevant
on any sane hardware anyway).
Linus
I've just checked the Mac OS X build and it looks like there is a mmap
and git is indeed using it, so this is obviously an example of a
"really crappy" mmap implementation.
This adds more ammunition to the fight against the whole Mac OS X is
powered/built/based on UNIX myth.
Charles.
On Fri, 18 Jan 2008, Charles Bailey wrote:
I've just checked the Mac OS X build and it looks like there is a mmap
and git is indeed using it, so this is obviously an example of a
"really crappy" mmap implementation.
Looking closer, this is not necessarily the case here.
Git uses MAP_PRIVATE, because that whole pack-file mapping was really
*meant* to map an existing read-only pack-file, and fast-import seems to
really be screwing with it.
It so happens that Linux has a particularly clean and streamlined VM, and
if you do only reads to a MAP_PRIVATE mapping on a normal filesystem,
you'll always be as coherent as with MAP_SHARED because Linux will simply
map in the page cache pages directly.
But this is definitely not portable, and the git fast-import mmap window
usage before Shawn's patch it was simply wrong.
So in this case, it really was git that was crap.
It just happened to work because the Linux mmap handling is just generally
pretty sane. It probably also worked fine on pretty much any other modern
UNIX (ie Solaris).
I'm not quite sure what OS X does to MAP_PRIVATE mappings, but if OS X is
still based on Mach (with FreeBSD just as a single-server on top), I
suspect that may be why it broke on OS X. The Mach VM is insanely complex
and does really odd things.
But the fact is, without MAP_SHARED, you shouldn't expect things to be
coherent, even if they often will be (especially for PROT_READ).
Btw, even with Shawn's patch, I wonder if the index_data usage is correct.
Linus