Re: [PATCH 0/3] speeding up on-demand fetch for blobs in partial clone

On 9/6/2022 6:58 PM, Jeff King wrote:

On Tue, Sep 06, 2022 at 02:38:41PM -0400, Jeff King wrote:

quoted

On Mon, Sep 05, 2022 at 11:17:21AM +0000, 程洋 wrote:

quoted

Sorry, I told you the wrong branch. It should be "android-t-preview-1"
git clone --filter=blob:none --no-local -b android-t-preview-1 grade-plugin

Can you try this one?

Yes, I see more slow-down there. There are many more blobs there, but I
don't think it's really the number of them, but their sizes.

The problem is that both upload-pack and pack-objects are keen to call
parse_object() on their inputs. For commits, etc, that is usually
sensible; we have to parse the object to see what it points to. But for
blobs, the only thing we do is inflate a ton of bytes in order to check
the sha1. That's not really productive here; if there is a bit
corruption, the client will notice it on the receiving side.

Thanks for finding this very subtle issue!
 

So here's a cleaned-up series which makes this a lot faster.

The special sauce is in patch 2, along with timings. The first one is
just preparing, and the final one is a small cleanup it enables.

I carefully read these patches as well as applied them on my machine
and did some extra digging and performance tests to understand the
change.

LGTM.

Thanks,
-Stolee