Hi everyone,
I've been working on a revision caching system (with gsoc), and would be curious to get some feedback on it and it's (potential) installation into git. I believe the core mechanics of it are largely finished, but integration is still rather rough. I've tried to divide it into 4 self-contained patches -- each representing a working implemenation with incremental featuresets. The patchset is seperated as:
* basic - the simplest working revision cache implementation, with full docs and relavant tests
* objects - non-commit object caching + tests
* misc - extra (maintenance) features and refactor of object caching (requiring slight modification of some object structures)
* integration - (proof of concept) integration of rev-cache into rev-list, along with extensive tests
I'm afraid the first patch is rather big, as the topo-data encoding scheme is a smidgen complicated, but its effect on git's internals is very small -- it's less of a patch and more an addition of a few files. I know you guys prefer inline patches, but in view of the patchfile size I decided it'd be more prudent to include that seperately as an attachment. The only other caveat I can think of is that the tests currently require python, although that could change in future.
So yeah, I hope you find it interesting. Apologies if I'm breaking protocol...
- Nick
hi all,
understandably, someone dumping a load of source without much comment
on you guys didn't exactly enthrall you. this project feels pretty
finished, so I'm not in desperate need of feedback, but I figured you
guys might like some high-level info on my part. I've modified the
integration bit slightly, and performance is now reasonably
satisfactory. here are some stats for the linux repo (~270MB) on my
laptop:
( note that I've edited the last patch (integration) a bit, so I've
attached the new patch. I've also, in a seperate patch, tweaked
builtin-pack-objects to reduce a bottleneck in object checking/writing
(also attached). )
git-rev-list --all --objects >/dev/null
default : ~55s
w/ rev-cache: ~8s
git-pack-objects --revs --all --stdout >/dev/null
default : 4m 48s (cold); 3m 41s (warm)
w/ rev-cache : ~58s
w/ rev-cache + tweak: ~36s
I tested these on cygwin on a dual-core laptop with 1gb ram, with a
few apps running in the background. times could vary a bit, depending
on what's been run how many times with what in the background, etc.,
but I tried to capture the general trend.
- Nick
On Fri, Jul 3, 2009 at 5:13 PM, Nick Edelen[off-list ref] wrote:
Hi everyone,
I've been working on a revision caching system (with gsoc), and would be curious to get some feedback on it and it's (potential) installation into git. I believe the core mechanics of it are largely finished, but integration is still rather rough. I've tried to divide it into 4 self-contained patches -- each representing a working implemenation with incremental featuresets. The patchset is seperated as:
* basic - the simplest working revision cache implementation, with full docs and relavant tests
* objects - non-commit object caching + tests
* misc - extra (maintenance) features and refactor of object caching (requiring slight modification of some object structures)
* integration - (proof of concept) integration of rev-cache into rev-list, along with extensive tests
I'm afraid the first patch is rather big, as the topo-data encoding scheme is a smidgen complicated, but its effect on git's internals is very small -- it's less of a patch and more an addition of a few files. I know you guys prefer inline patches, but in view of the patchfile size I decided it'd be more prudent to include that seperately as an attachment. The only other caveat I can think of is that the tests currently require python, although that could change in future.
So yeah, I hope you find it interesting. Apologies if I'm breaking protocol...
- Nick