Re: [PATCH v3 1/5] speed up alt_odb_usable() with many alternates

2 messages, 2 authors, 2021-07-08 · open the first message on its own page

Re: [PATCH v3 1/5] speed up alt_odb_usable() with many alternates

From: Junio C Hamano <hidden>
Date: 2021-07-08 00:20:56

Eric Wong [off-list ref] writes:
With many alternates, the duplicate check in alt_odb_usable()
wastes many cycles doing repeated fspathcmp() on every existing
alternate.  Use a khash to speed up lookups by odb->path.

Since the kh_put_* API uses the supplied key without
duplicating it, we also take advantage of it to replace both
xstrdup() and strbuf_release() in link_alt_odb_entry() with
strbuf_detach() to avoid the allocation and copy.

In a test repository with 50K alternates and each of those 50K
alternates having one alternate each (for a total of 100K total
alternates); this speeds up lookup of a non-existent blob from
over 16 minutes to roughly 2.7 seconds on my busy workstation.

Note: all underlying git object directories were small and
unpacked with only loose objects and no packs.  Having to load
packs increases times significantly.

v3: Introduce and use fspatheq and fspathhash functions;
    avoid unnecessary checks for allocation failures already
    handled by our own *alloc wrappers.
The last one does not belong to the commit log message, as "git log"
readers do not care about and will not have access to v2 and earlier.

Re: [PATCH v3 1/5] speed up alt_odb_usable() with many alternates

From: Eric Wong <hidden>
Date: 2021-07-08 01:14:47

Junio C Hamano [off-list ref] wrote:
Eric Wong [off-list ref] writes:
quoted
v3: Introduce and use fspatheq and fspathhash functions;
    avoid unnecessary checks for allocation failures already
    handled by our own *alloc wrappers.
The last one does not belong to the commit log message, as "git log"
readers do not care about and will not have access to v2 and earlier.
Oops :x  Are you going to remove that on your end or do you want a resend?
Looking at "git log", it looks like I've done it a bit over the years :x
Keyboard shortcuts
hback out one level
jnext message in thread
kprevious message in thread
ldrill in
Escclose help / fold thread tree
?toggle this help