Re: [PATCH v3 25/25] p5326: perf tests for MIDX bitmaps

On Tue, Jul 27, 2021 at 05:20:28PM -0400, Taylor Blau wrote:

These new performance tests demonstrate effectively the same behavior as
p5310, but use a multi-pack bitmap instead of a single-pack one.

Notably, p5326 does not create a MIDX bitmap with multiple packs. This
is so we can measure a direct comparison between it and p5310. Any
difference between the two is measuring just the overhead of using MIDX
bitmaps.

Here are the results of p5310 and p5326 together, measured at the same
time and on the same machine (using a Xenon W-2255 CPU):

Neat. I think having separate perf regression tests for regular and mix
bitmaps will be useful, but being able to compare the pack and mix
versions is a cherry on top.

There was one funny number:

    5310.2: repack to disk                                96.78(93.39+11.33)
    5326.2: setup multi-pack index                        78.99(75.29+11.58)

In p5310, that step is repacking and writing bitmaps. With the midx,
it's repacking, then writing a midx with bitmaps. I'd expect the latter
to be strictly slower than the former, but here it's faster.

Running the code locally, I got similar results (with p5310 just a tiny
bit faster). So it may have just been noise or some other timing issue.

  As an aside, I think that test is a little bit bogus due to
  GIT_PERF_REPEAT_COUNT; the first trial will generate bitmaps from
  scratch, and then subsequent runs will reuse partial results. It
  probably should "rm -f .git/objects/*.bitmap" within the test. We can
  deal with that separately, though.

-Peff