Thread (52 messages) 52 messages, 14 authors, 2016-06-15

Re: [PATCH] write-tree performance problems

From: Chris Mason <hidden>
Date: 2016-06-15 22:41:53

On Wednesday 20 April 2005 13:06, Linus Torvalds wrote:
On Wed, 20 Apr 2005, Chris Mason wrote:
quoted
At any rate, the time for a single write-tree is pretty consistent. 
Before it was around .5 seconds, and with this change it goes down to
.128s.
Oh, wow.

I bet your SHA1 implementation is done with hand-optimized and scheduled
x86 MMX code or something, while my poor G5 is probably using some slow
generic routine. As a result, it only improved by 33% for me since the
compression was just part of the picture, but with your cheap SHA1 the
compression costs really dominated, and so it's almost four times faster
for you.
Aha, I was wondering why your write-tree speeds sounded so bad...this athlon 
machine is ~2years old now.

Your comments about costs for writing the index file got me thinking, so I 
benchmarked how long the update-cache takes if we don't do the sha1 of the 
index file.  There was almost no difference at all.  update-cache currently 
takes about .152 seconds

The code to write the cache calls write() for every cache entry, writing just 
a few bytes at a time.  I changed it to collect these into a 16k buffer, 
which brings me down to .044s.  This might not help as much on ext23, since 
they are faster than reiser for tiny writes.

The patch below with your current tree brings my 100 patch test down to 22 
seconds again.

-chris

Attachments

Keyboard shortcuts
hback out one level
jnext message in thread
kprevious message in thread
ldrill in
Escclose help / fold thread tree
?toggle this help