Re: QGit: Shrink used memory with custom git log format

Marco Costalba [off-list ref] wrote:

Now instead of --pretty=raw a custom made --pretty=format is given,
this shrinks loaded data of 30% (17MB less on Linux tree) and gives a
good speed up when you are low on memory (especially on big repos)

Next step _would_ be to load log message body on demand (another 50%
reduction) but this has two drawbacks:

(1) Text search/filter on log message would be broken

(2) Slower to browse through revisions because for each revision an
additional git-rev-list /git-log command should be executed to read
the body

The second point is worsted by the fact that it is not possible to
keep a command running and "open" like as example git-diff-tree
--stdin and feed with additional revision's sha when needed. Avoiding
the burden to startup a new process each time to read a new log
message given an sha would let the answer much more quick especially
on lesser OS's

Indeed there is a git-rev-list --stdin option but with different
behaviour from git-diff-tree --stdin and not suitable for this.

There was a proposed patch for git-cat-file that would let you run
it in a --stdin mode; the git-svn folks wanted this to speed up
fetching raw objects from the repository.  That may help as you
could get commit bodies (in raw format - not reencoded format!)
quite efficiently.

Otherwise I think what you really want here is a libgit that you can
link into your process and that can efficiently inflate an object
on demand for you.  Like the work Luiz was working on this past
summer for GSOC.  Lots of downsides to that current tree though...
like die() kills the GUI...

-- 
Shawn.