I have lately added new Git speed benchmark, from Bryan Murdock blog.
The repository is bit untypical:
<quote>
By performance, I mean that I used the UNIX time command to see how
long various basic operations took. Performing the various basic
operations gave me some insight into the usability of each as well.
For this test I used a directory with 266 MB of files, 258 KB of which
were text files, with the rest being image files. I know, kind of
weird to version all those binary files, but that was the project I
was interested in testing this out on. Your mileage may vary and all
that. Here’s a table summarizing the real times reported by time(1):
</quote>
If I remember correctly there were some patches to git which tried to
better deal with large blobs. In this simple benchmark git was
outperformed by Mercurial and even Bazaar-NG a bit.
http://git.or.cz/gitwiki/GitBenchmarks#head-5657b8361895b5a02c0de39337c410e4d8dcdbce
http://bryan-murdock.blogspot.com/2007/03/cutting-edge-revision-control.html
--
Jakub Narebski
Poland
On Wed, 1 Aug 2007, Jakub Narebski wrote:
If I remember correctly there were some patches to git which tried to
better deal with large blobs. In this simple benchmark git was
outperformed by Mercurial and even Bazaar-NG a bit.
It's almost certainly not the binary blobs.
I think almost all the difference is from the cloning, without repacking
the souce or using a local clone.
The default action for a git clone is to create a pack-file, and do a
local clone as if you did it over the network. That is obviously much
slower than using the "-l" flag for the _clone_ action, but it tends to be
better for the end result - since you get a nice packed starting point,
and none of the confusion with hardlinks etc.
[ Maybe I'm just a worry-wart, but hardlinking two repos still makes me
worried. Even though we never modify the object files.
Quite frankly, I almost wish we hadn't ever done "-l" at all, and I
cannot really suggest using it. Either use "-s" for the truly shared
repository, or use the default pack-generating one. The hardlinking one
was simple and made sense, but it's really not very nice.
But that aversion to "git clone -l" is really totally illogical. The way
we do the object handling, hardlinking object files in git is just about
the most safe operation you can think of - and I *still* shudder at it ]
Now, I think the "always act as if you were network transparent" by
default is great, but especially if you have never run "git gc" to
generate a pack to begin with, it's going to be a very costly thing. And I
think that's what the numbers show. That's the only op we do a *lot* worse
on than we should.
(The "nonconflicting merge" is probably - once more - the diffstat
generation that bites us. That's generally the most costly thing of the
whole merge, but I *love* the diffstat).
That said, even if he had done a "git gc", to be fair he would have had to
include the cost of that first garbage collect in the "initial import", so
the end result would have been exactly the same. Git _does_ end up having
a very odd performance profile, and while it's optimized for certain
thing, the "initial import" is not one of them.
(Which admittedly is a bit odd. The reason I didn't ever seriously even
consider monotone was that the initial import was so *incredibly* sucky,
and took hours for the kernel. So use "-l" for benchmarks, and damn my
"I hate hardlinking repos" idiocy).
So the only way to truly do a fast initial import *and* get a reasonably
good initial clone is likely one of:
- take full advantage of git, and use local branches, instead of
bothering with lots of clones.
I think that this is often the right thing to do, but it's obviously
not fair for comparisons, since it's really something different from
what's likely available in the other SCM's. But it's the "git way".
- use "git clone -s" (or "-l").
I think the hg numbers are the result of hg defaulting to "-l"
behaviour. Which makes sense for hg, since people need to clone more
(in git, you'd generally work with local branches instead).
- or the initial import would be done with some "git fast-import" thing,
rather than "git add ." We don't do it now, and the resulting pack-file
wouldn't be optimal, but it would be reasonable. It would at least cut
down a _bit_ on the clone cost.
The other reaction I took away from that (quite reasonable, I think)
comparison is that I think Murdock would have been much happier if git
diff defaulted to "-C". We don't do that (for the best of reasons:
interoperability), but maybe we should document the "-M/-C" options more.
The options do show up in the man-page, but apparently not
obviously enough, since he hadn't noticed.
Linus
Jakub Narebski [off-list ref] wrote:
I have lately added new Git speed benchmark, from Bryan Murdock blog.
The repository is bit untypical:
<quote>
By performance, I mean that I used the UNIX time command to see how
long various basic operations took. Performing the various basic
operations gave me some insight into the usability of each as well.
For this test I used a directory with 266 MB of files, 258 KB of which
were text files, with the rest being image files. I know, kind of
weird to version all those binary files, but that was the project I
was interested in testing this out on. Your mileage may vary and all
that. Here’s a table summarizing the real times reported by time(1):
</quote>
If I remember correctly there were some patches to git which tried to
better deal with large blobs. In this simple benchmark git was
outperformed by Mercurial and even Bazaar-NG a bit.
Yes. And we backed them out more recently. :-(
A while ago someone had issues with large binary blobs being added to
the repository as loose objects (e.g. by git-add/git-update-index).
Repacking that repository (for just git-gc or for transport/clone)
was ugly as the large binary blob had to be deflated then
reinflated to encode it in the packfile. The solution was the
core.legacyheaders = false configuration setting, which used
packfile encoding for loose objects, thereby allowing the packer
to just copy the already compressed data into the output packfile.
Unfortunately we backed that out recently to "simplify the code".
We can still read that loose object format, but we cannot create
it and during packing we don't copy the data (we deflate/inflate
anyway). So we're back to the horrible deflate/inflate problem.
That probably explains the large clone time seen by the author.
I wonder if hg realizes that the two repositories are on the
same filesystem and automatically uses hardlinks if possible (aka
git clone -l). That would easily explain how they can clone so
dang fast. Maybe we should do the same in git-clone, its a pretty
simple thing to do.
I do have to question the author's timing method. I don't know if
this was hot-cache or not, and he doesn't say. I don't know if the
system was 100% idle when running these times, or the times were
averaged over a few runs. Usually the first run of anything can
give inaccurate timings, as for example the executable code may
not be paged in from disk. One of the tools may have had a bias
as maybe he poked around with that tool first, before starting the
timings, so its executables were still hot in cache. Etc.
However assuming everything was actually done in a way that the
timings can be accurately relied upon...
Regarding the initial file import it looks like we about broke even
with bzr if you add the "initial file import" and "initial commit"
times together. Remember we have to hash and compress the data
during git-add; bzr probably delayed their equivilant operation(s)
until the commit operation. Summing these two times is probably
needed to really compare them.
We were also rather close to hg if you again sum the times up.
But we do appear to be slower, by about 27s. I guess I find that
hard to believe, but sure, maybe hg somehow has a faster codepath
for their file revision disk IO than we do. Maybe its because hg
is streaming data and we're loading it all in-core first; maybe the
author's system had to swap get enough virtual memory for git-add.
Maybe it is just because the author's testing methodology was not
very good and one or more of these numbers are just bunk.
Our merge time is pretty respectible giving the competition.
Its probably within the margin of error of the author's testing
methodology.
--
Shawn.
Linus Torvalds wrote:
(The "nonconflicting merge" is probably - once more - the diffstat
generation that bites us. That's generally the most costly thing of the
whole merge, but I *love* the diffstat).
http://bryan-murdock.blogspot.com/2007/03/cutting-edge-revision-control.html
doesn't tell what is the directory structure of imported files.
If it is flat, then git does not use advantage of hierarchical tree
structure.
By the way, I guess that "nonconflicting merge" is trivial tree-level
merge, as "no changes" merge should be faster (or fast-forward).
About clone: there was "pack loose, copy existing packs" idea. I don't
remember what happened with it. At least for local clone it would be
nice.
--
Jakub Narebski
Poland