Re: Problem with large files on different OSes

On 27 May 2009, at 12:37, Andreas Ericsson wrote:

Christopher Jefferson wrote:

quoted

I recently came across a very annoying problem, characterised by  
the following example:
On a recent ubuntu install:
dd if=/dev/zero of=file bs=1300k count=1k
git commit file -m "Add huge file"
The repository can be pulled and pushed successfully to other  
ubuntu installs, but on Mac OS X, 10.5.7 machine with 4GB ram git  
pull produces:
remote: Counting objects: 6, done.
remote: git(1533,0xb0081000) malloc: *** mmap(size=1363152896)  
failed (error code=12)
remote: *** error: can't allocate region
remote: *** set a breakpoint in malloc_error_break to debug
remote: git(1533,0xb0081000) malloc: *** mmap(size=1363152896)  
failed (error code=12)
remote: *** error: can't allocate region
remote: *** set a breakpoint in malloc_error_break to debug
remote: fatal: Out of memory, malloc failed
error: git upload-pack: git-pack-objects died with error.
fatal: git upload-pack: aborting due to possible repository  
corruption on the remote side.
remote: aborting due to possible repository corruption on the  
remote side.
fatal: protocol error: bad pack header
The problem appears to be the different maximum mmap sizes  
available on different OSes. Whic I don't really mind the maximum  
file size restriction git imposes, this restriction varying from OS  
to OS is very annoying, fixing this required rewriting history to  
remove the commit, which caused problems as the commit had already  
been pulled, and built on, by a number of developers.
If the requirement that all files can be mmapped cannot be easily  
removed, would be it perhaps be acceptable to impose a (soft?)  
1GB(ish) file size limit?

Most definitely not. Why should we limit a cross-platform system for
the benefit of one particular developer's lacking hardware?

Out of curiosity, why do you say lacking hardware? I am running  
ubuntu, windows and Mac OS X on exactly the same machine, which is not  
running out of physical memory, never mind swap, when using git on any  
OS. The problem is purely a software (and OS) problem.


Chris