Re: Cygwin can't handle huge packfiles?
From: <hidden>
Date: 2016-06-15 22:42:23
Right now we LRU the pack files and evict older ones when we mmap too many, but the unit of eviction is the whole file, so it would not help the case like yours at all. It might be possible to mmap only part of a packfile, but it would involve fairly major surgery to sha1_file.c.
The simplest solution seems to be to limit pack file size to a reasonable fraction of a 32-bit address space. Say, 0.5 G. That should be a fairly straightforward hack to git-pack-objects. It already emits two files; just make it emit more. You can tweak the heurisitics to try to find a good break point: start thinking about splitting the pack when you get to one size, but don't force a break until you hit a harder limit as long as the deltas are working well. This can all be adjustable with a command line and/or config file option to allow for the eventual demise of 32-bit systems.