Re: low memory system to clone larger repo

A more "final" version of the tuning exercise I did is here:

http://mail-index.netbsd.org/tech-repository/2015/01/08/msg000520.html

I did try some of these setting on the server and it made the repo much much larger so I guess I am looking for ways to just reduce client memory usage/the best balance of disk, memory, and bandwidth. 

If there is a way to turn off some memory hogging speed ups I m interested in trying them. 

I will try to get you the output later since I have since started working on the server side.

Let me know if you want to try some server side stuff and I can give you a git:// or http:// off list. 

Thanks for looking, 
Matt

On Feb 9, 2015, at 5:40 AM, Duy Nguyen [off-list ref] wrote:

quoted

On Thu, Jan 8, 2015 at 11:10 PM, matthew sporleder [off-list ref] wrote:
I am attempting to clone this repo: https://github.com/jsonn/src/

and have been successful on some lower memory systems, but i'm
interested in continuing to push down the limit.

I am getting more success running clone via https:// than git:// or
ssh (which is confusing to me) and the smallest system that works is a
raspberry pi with 256 RAM + 256 swap.

I seem to run out of memory consistently around 16% into Resolving
deltas phase but I don't notice an RSS jump so that's another
confusing spot.

Sorry for a really late reply. The command that's running when you run
out of memory is index-pack. I guess it's verifying the delta chain. I
think it needs enough memory for two uncompressed objects (or files)
in a delta chain. I haven't finished cloning this repo yet so I don't
know what these delta chains look like.

What does it say when it runs out of memory? I'm thinking maybe we
could force a core dump, then look at how memory is used.

What if you "git init", then do "git fetch https://..." manually?
There's an optimization for git-clone that may make index-pack use a
bit more memory (and push it over the edge)

quoted

My config is below and I'd appreciate any more suggestions of getting
that down to working on a 128MB box (or smaller).

I suppose it's ~/.gitconfig or /etc/gitconfig, it's not added after
the clone is complete, correct? Sounds interesting, let me profile its
memory usage..

quoted

[pack]
       windowMemory = 1m
       packSizeLimit = 1m
       deltaCacheSize = 1m
       deltaCacheLimit = 10
       packSizeLimit = 1m

I think many of these only affect the server side. If you clone from
github, then they are useless. You may want to provide your own server
side with these settings and see if things change. Also play with
pack.depth (affecting server side)

quoted

       threads = 1

-- 
Duy