Thanks for all the feedback.
The biggest change since the last version is how this patch series interacts
with the split-index feature. With a split index, most of the cache entries
are stored in the shared index so would benefit from multi-threaded parsing.
To enable that, the EOIE and IEOT extensions are now written into the shared
index (rather than being stripped out like the other extensions).
Because of this, I can now update the tests in t1700-split-index.sh to have
updated SHA values that include the EOIE extension instead of disabling the
extension.
Using p0002-read-cache.sh to generate some performance numbers shows how
each of the various patches contribute to the overall performance win on a
particularly large repo.
Repo w/3M files Baseline Optimize V4 Extensions Entries
--------------------------------------------------------------------------
0002.1: read_cache 693.29 655.65 -5.4% 470.71 -32.1% 399.62 -42.4%
Note how this cuts nearly 300ms off the index load time!
Base Ref: master
Web-Diff: https://github.com/benpeart/git/commit/c1125a5d9a
Checkout: git fetch https://github.com/benpeart/git read-index-multithread-v7 && git checkout c1125a5d9a
### Patches
Ben Peart (6):
read-cache: clean up casting and byte decoding
eoie: add End of Index Entry (EOIE) extension
config: add new index.threads config setting
read-cache: load cache extensions on a worker thread
ieot: add Index Entry Offset Table (IEOT) extension
read-cache: load cache entries on worker threads
Nguyễn Thái Ngọc Duy (1):
read-cache.c: optimize reading index format v4
Documentation/config.txt | 7 +
Documentation/technical/index-format.txt | 41 ++
config.c | 18 +
config.h | 1 +
read-cache.c | 749 +++++++++++++++++++----
t/README | 5 +
t/t1700-split-index.sh | 13 +-
7 files changed, 715 insertions(+), 119 deletions(-)
base-commit: fe8321ec057f9231c26c29b364721568e58040f7
--
2.18.0.windows.1