Re: [PATCH 0/5] Using index-pack in place of verify-pack

Sverre Rabbelier [off-list ref] writes:

On Mon, Feb 28, 2011 at 10:49, Junio C Hamano [off-list ref] wrote:

quoted

The basic idea, which comes from Shawn Pearce, is to run index-pack on a
packfile being verified, and compare the resulting .idx file with the
existing one, and make sure they match.

Do you have any timings, on say, the kernel repo?

Not yet; the code has just become in good enough shape to be built and
measured by anybody interested --- didn't I say WIP somewhere?

Heya,

On Mon, Feb 28, 2011 at 17:16, Junio C Hamano [off-list ref] wrote:

Not yet; the code has just become in good enough shape to be built and
measured by anybody interested --- didn't I say WIP somewhere?

Yes, but I figured that we'd want to see if it's worth pursuing at
all, e.g., whether we should try and get this in decent shape or not?

-- 
Cheers,

Sverre Rabbelier

On Mon, Feb 28, 2011 at 05:46:19PM +0100, Sverre Rabbelier wrote:

On Mon, Feb 28, 2011 at 17:16, Junio C Hamano [off-list ref] wrote:

quoted

Not yet; the code has just become in good enough shape to be built and
measured by anybody interested --- didn't I say WIP somewhere?

Yes, but I figured that we'd want to see if it's worth pursuing at
all, e.g., whether we should try and get this in decent shape or not?

I'm not at all interested in this topic, but in the time it took the two
of you to write your emails, I did this:

  $ cd linux-2.6

  $ time git verify-pack \
      .git/objects/pack/pack-36c50f6082df404c26a699f2702946e0cce6208f.idx
  real    2m37.338s
  user    2m35.874s
  sys     0m1.348s

  $ time git index-pack --verify \
      .git/objects/pack/pack-36c50f6082df404c26a699f2702946e0cce6208f.pack
  real    1m37.208s
  user    1m36.106s
  sys     0m1.048s


OK, with the CPU time it probably took longer than your emails. But I
ate some ice cream while it computed. :)

-Peff

Heya,

On Mon, Feb 28, 2011 at 17:58, Jeff King [off-list ref] wrote:

 $ time git verify-pack \
     .git/objects/pack/pack-36c50f6082df404c26a699f2702946e0cce6208f.idx
 real    2m37.338s
 user    2m35.874s
 sys     0m1.348s

 $ time git index-pack --verify \
     .git/objects/pack/pack-36c50f6082df404c26a699f2702946e0cce6208f.pack
 real    1m37.208s
 user    1m36.106s
 sys     0m1.048s

Wow, impressive results, a full minute shaved off.

OK, with the CPU time it probably took longer than your emails. But I
ate some ice cream while it computed. :)

You lucky man ;)

-- 
Cheers,

Sverre Rabbelier

Jeff King [off-list ref] wrote:

  $ cd linux-2.6

  $ time git verify-pack \
      .git/objects/pack/pack-36c50f6082df404c26a699f2702946e0cce6208f.idx
  real    2m37.338s
  user    2m35.874s
  sys     0m1.348s

  $ time git index-pack --verify \
      .git/objects/pack/pack-36c50f6082df404c26a699f2702946e0cce6208f.pack
  real    1m37.208s
  user    1m36.106s
  sys     0m1.048s

Not the improvement I had hoped. Your verify-pack ran much more
quickly than the fsck I was seeing that led me to suggest this to
Junio a few weeks ago. But saving a minute is still pretty good.

That index-pack run is basically what we get during clone, so
I don't really expect to improve on that. (However I have been
contemplating that the locality within the pack isn't optimal and
we can do better placement.)

-- 
Shawn.