Re: pack operation is thrashing my server
From: Ken Pratt <hidden>
Date: 2016-06-15 22:45:08
What is the largest object in that repository? Do you have a rough guess? You said earlier:quoted
The remote repository is bare, and is 180MB in size (says du), with 1824 objects.That implies there is at least one really large object in that repository. The average of 101KB per object is not going to be a correct figure here as most commits and trees are _very_ tiny. It must be a large object. Those big objects are going to consume a lot of memory if they get inflated in memory.
Largest object is ~150MB, and there are a couple 5-10MB objects as well.
You said earlier this was Git 1.5.6.4. I recently fixed a bug in the code that reads data from packs to prevent it from blowing out memory usage, but that bug fix was included in 1.5.6.4.
I tried upgrading to 1.5.6.5 as well, but that didn't help.
On the up side, packing should only be consuming huge memory like this when it needs to move loose objects into a pack file. I think Martin Langhoff suggested packing this on your laptop then using rsync over SSH to copy the pack file and .idx file to the server, so the server didn't have to spend time figuring out the deltas itself.
Unfortunately, that will only work as a band-aid solution for my workflow. I think I'll have to limit the file size in the repository to something that the server can handle.