Re: [PATCH 0/9] Make git-svn fetch ~1.7x faster

Mike Hommey wrote:

On Mon, Oct 22, 2007 at 10:46:28PM -0700, Adam Roben wrote:

quoted

This patch series makes git-svn fetch about 1.7x faster by reducing the number
of forks/execs that occur for each file retrieved from Subversion. To do so, a
few new options are added to git-cat-file and git-hash-object to allow
continuous input on stdin and continuous output on stdout, so that one instance
of each of these commands can be kept running for the duration of the fetch.

You don't need to do this to avoid forks. Just use git-fast-import
instead.

git-fast-import only covers the hash-object side of things, not cat-file.

git-fast-import does not currently suit 'gradual deployment' for
converters such as git-svn, because it;

  - returns object IDs at the end, when you checkpoint.

    This could be 'fixed' by allowing a marks log file instead of or in
    addition to the current behaviour, though if the exporter is
    continually waiting for the tokens rather than using marks, it will
    slow it down.

  - you can't use plumbing commands, such as rev-parse, cat-file, etc on
    objects which have not been checkpointed yet.

  - can't just stream a file of unknown length to it as you can to
    hash-object

These are the design trade-offs of using fast-import.  Using
fast-import, you are creating a 'transaction' area which uses user
sequences instead of (git)database-issued identifiers.  And this
transaction is isolated from the other concurrent users of the object
database.  However the interface does not have the full git CLI
available to it, so unlike a regular database transaction, you end up
having to care.

Rewriting the importer so as to correctly deal with these problems is
quite challenging, and for slow import sources such as Subversion, of
limited merit.

Sam.