git-svn: persistent memoization

From: Andrew Myrick <hidden>
Date: 2016-06-15 22:48:03

git-svn uses the Memoize perl module to cache the return values of a few functions.  This speeds up svn:mergeinfo processing considerably, but as currently implemented, this memoization table must be reconstructed on every run of git-svn.  This isn't a problem on small projects, but it introduces a delay of several minutes when fetching from large repositories with a lot of merge info.

The Memoize module has support for storing its cache persistently (more below, and see http://perldoc.perl.org/Memoize.html for details), and I would love to take advantage of this support in git-svn.  Unfortunately, it's not entirely obvious to me how to get this working.  Here are the questions I have so far, moving from high level design questions to implementation details:

1) Is there any reason not to store these caches on disk?

2) Are there situations where the caches would need to be invalidated?  Perhaps when git-svn rebuilds its metadata?

3) Where should I put these caches?  I was thinking something like ".git/svn/caches/<cachefile>.db" would be appropriate.

4) What's the correct way to reference the path to those caches?  I tried using "$ENV{GIT_DIR}/svn/caches/has_no_changes.db", but the memoize calls are in a BEGIN block, and it seems that $ENV{GIT_DIR} hadn't been initialized at that point.  This has taken me way past my limited knowledge of Perl.

5) What backend should I pick to store the cache?  Memoize supports storing to any tied hash that supports TIEHASH, FETCH, STORE, and EXISTS, like DB_File.  It also supports storing to SDBM_File and NDBM_File through glue modules (they lack EXISTS support), and Storable, which isn't a tied hash at all, but just a serializer for hashes.  I'm leaning towards using Storable since it seems simple and would fit the workload well, but I would appreciate any insight from someone with more domain knowledge here.

Regards,
Andrew
Keyboard shortcuts
hback out one level
jnext message in thread
kprevious message in thread
ldrill in
Escclose help / fold thread tree
?toggle this help