Re: git-fast-import out of memory

On 2020-06-05 at 05:15:04, Billes Tibor wrote:

Can you help me fix this issue? I hope the information I gathered is
enough to
help you find the cause of this behavior. I'd be happy to provide more
information if needed or test patches.  Unfortunately the source code I was
fetching is proprietary, I cannot post it.

Hey, thanks for this report.  I can confirm this with git.git and the
following commands:

  git fast-export --all --reencode=yes --signed-tags=verbatim --tag-of-filtered-object=drop >$TMP/git.fe
  cd $TMP && rm -fr test-repo && git init test-repo &&
    (cd test-repo && /usr/bin/time git fast-import --export-marks=$TMP/git.marks < $TMP/git.fe &&
    /usr/bin/time git fast-import --import-marks=$TMP/git.marks < $TMP/git.fe)

The latter fast-import command gets killed with SIGKILL due to using 16
GiB of memory, which I would agree is unreasonable.

It looks like with Junio's patch at
[ref], the memory usage is down to
1.4 GiB, which considering that a pack must be written, is much more
reasonable.
-- 
brian m. carlson: Houston, Texas, US
OpenPGP: https://keybase.io/bk2204