Re: Problem with large files on different OSes

Christopher Jefferson wrote:

I recently came across a very annoying problem, characterised by the 
following example:

On a recent ubuntu install:

dd if=/dev/zero of=file bs=1300k count=1k
git commit file -m "Add huge file"


The repository can be pulled and pushed successfully to other ubuntu 
installs, but on Mac OS X, 10.5.7 machine with 4GB ram git pull produces:

remote: Counting objects: 6, done.
remote: git(1533,0xb0081000) malloc: *** mmap(size=1363152896) failed 
(error code=12)
remote: *** error: can't allocate region
remote: *** set a breakpoint in malloc_error_break to debug
remote: git(1533,0xb0081000) malloc: *** mmap(size=1363152896) failed 
(error code=12)
remote: *** error: can't allocate region
remote: *** set a breakpoint in malloc_error_break to debug
remote: fatal: Out of memory, malloc failed
error: git upload-pack: git-pack-objects died with error.
fatal: git upload-pack: aborting due to possible repository corruption 
on the remote side.
remote: aborting due to possible repository corruption on the remote side.
fatal: protocol error: bad pack header


The problem appears to be the different maximum mmap sizes available on 
different OSes. Whic I don't really mind the maximum file size 
restriction git imposes, this restriction varying from OS to OS is very 
annoying, fixing this required rewriting history to remove the commit, 
which caused problems as the commit had already been pulled, and built 
on, by a number of developers.

If the requirement that all files can be mmapped cannot be easily 
removed, would be it perhaps be acceptable to impose a (soft?) 1GB(ish) 
file size limit?

Most definitely not. Why should we limit a cross-platform system for
the benefit of one particular developer's lacking hardware?

Such a convention should, if anything, be enforced by social policy,
but not by the tool itself.

Otherwise, why not just restrict the tool that created the huge file
so that it makes smaller files that fit into git on all platforms
instead? (No, that wasn't a real suggestion. It was just to make the
point that your suggestion for git to impose artificial limits is
equally ludicrous)

-- 
Andreas Ericsson                   andreas.ericsson@op5.se
OP5 AB                             www.op5.se
Tel: +46 8-230225                  Fax: +46 8-230231

Register now for Nordic Meet on Nagios, June 3-4 in Stockholm
 http://nordicmeetonnagios.op5.org/

Considering the successes of the wars on alcohol, poverty, drugs and
terror, I think we should give some serious thought to declaring war
on peace.