Re: [PATCH v3 1/5] speed up alt_odb_usable() with many alternates
From: Junio C Hamano <hidden>
Date: 2021-07-08 00:20:56
Eric Wong [off-list ref] writes:
With many alternates, the duplicate check in alt_odb_usable()
wastes many cycles doing repeated fspathcmp() on every existing
alternate. Use a khash to speed up lookups by odb->path.
Since the kh_put_* API uses the supplied key without
duplicating it, we also take advantage of it to replace both
xstrdup() and strbuf_release() in link_alt_odb_entry() with
strbuf_detach() to avoid the allocation and copy.
In a test repository with 50K alternates and each of those 50K
alternates having one alternate each (for a total of 100K total
alternates); this speeds up lookup of a non-existent blob from
over 16 minutes to roughly 2.7 seconds on my busy workstation.
Note: all underlying git object directories were small and
unpacked with only loose objects and no packs. Having to load
packs increases times significantly.
v3: Introduce and use fspatheq and fspathhash functions;
avoid unnecessary checks for allocation failures already
handled by our own *alloc wrappers.The last one does not belong to the commit log message, as "git log" readers do not care about and will not have access to v2 and earlier.