Re: Something is broken in repack

On Tue, Dec 11, 2007 at 11:08:47AM +0000, David Kastrup wrote:

Nicolas Pitre [off-list ref] writes:

quoted

On Mon, 10 Dec 2007, Junio C Hamano wrote:

quoted

"Jon Smirl" [off-list ref] writes:

quoted

95%   530  2.8G - 1,420 total to here, previous was 1,983
100% 1390 2.85G
During the writing phase RAM fell to 1.6G
What is being freed in the writing phase??

entry->delta_data is the only thing I can think of that are freed
in the function that have been allocated much earlier before entering
the function.

Yet all ->delta-data instances are limited to 256MB according to Jon's 
config.

Maybe address space fragmentation is involved here?  malloc/free for
large areas works using mmap in glibc.  There must be enough
_contiguous_ space for a new allocation to succeed.

  Well, that's interesting, but there is a way to know for sure instead
of taking bets. Just use valgrind --tool=massif and look at the pretty
picture, it'll tell what was going on very accurately.

  Note that I find your explanation unlikely: glibc uses mmap for sizes
over 128k by default (IIRC), and as soon as you use mmaps, that's the
kernel that deals with the address space, and it's not necessarily
contiguous, that's only true for the heap.
-- 
·O·  Pierre Habouzit
··O                                                madcoder@debian.org
OOO                                                http://www.madism.org