cherry-pick is slow

Hello.

I have a very big git repository (the .git directory is about 5.3 Gb),
which is a copy of an svn repository fetched via git-svn. In fact
there are a few repositories ("working copies") that share the same
.git directory (via symlinks), in which I have different svn branches
checked out. Now I want to merge a commit from one svn branch to
another via git cherry-pick. The commit contains diff in only one
file. So I do

git cherry-pick <commit>

And the operation takes tens of seconds to finish. In "top" output I
see that git process uses almost no CPU, but has hundreds of page
faults, so I assume that it is reading a lot of files from disk. I
also tried running git in gdb and interrupting it in random places,
the stacktrace I get is usually like this:

#0  0x00000000004fb70c in experimental_loose_object (
    map=0x3268e000
"x\001+)JMU043g040031QpöMÌNõÉ,.)Ö+©(aøt*äãú½\vÍ4\236Yñ\230¤&z¯ß<{º\211\001\020(¤eV0\024\177ò\177á$\\$\034\224\036ö*÷vÂ\216#3ºâ!²\231)@Ù*íü»Ü7\233BmÜ\005^·\034ñÏOä¨\205Èæf\227\024e¦2¬fÐ\tY¾}ѳ\212,û\027î<ê\236[³w\234\204*(Í)Éd0ø\024\030n\233øìP\210\214üDSÆ?ó\216½¹>\a")
at sha1_file.c:1259
#1  0x00000000004fb8df in unpack_sha1_header (stream=0x7fffffffc9f0,
    map=0x3268e000
"x\001+)JMU043g040031QpöMÌNõÉ,.)Ö+©(aøt*äãú½\vÍ4\236Yñ\230¤&z¯ß<{º\211\001\020(¤eV0\024\177ò\177á$\\$\034\224\036ö*÷vÂ\216#3ºâ!²\231)@Ù*íü»Ü7\233BmÜ\005^·\034ñÏOä¨\205Èæf\227\024e¦2¬fÐ\tY¾}ѳ\212,û\027î<ê\236[³w\234\204*(Í)Éd0ø\024\030n\233øìP\210\214üDSÆ?ó\216½¹>\a",
mapsize=173,
    buffer=0x7fffffffa9f0, bufsiz=8192) at sha1_file.c:1308
#2  0x00000000004fbc85 in unpack_sha1_file (map=0x3268e000,
mapsize=173, type=0x7fffffffcbf0, size=0x7fffffffcbe8,
    sha1=0x7fffffffcbd0 "\001/Ç4&箺\036© wK`\214\"Ë\035H\203") at
sha1_file.c:1435
#3  0x00000000004fd96c in read_object (sha1=0x7fffffffcbd0
"\001/Ç4&箺\036© wK`\214\"Ë\035H\203", type=0x7fffffffcbf0,
size=0x7fffffffcbe8)
    at sha1_file.c:2233
#4  0x00000000004fda0d in read_sha1_file_extended (sha1=0x7fffffffcbd0
"\001/Ç4&箺\036© wK`\214\"Ë\035H\203", type=0x7fffffffcbf0,
size=0x7fffffffcbe8,
    flag=1) at sha1_file.c:2258
#5  0x00000000004fdcda in read_sha1_file (sha1=0x7fffffffcbd0
"\001/Ç4&箺\036© wK`\214\"Ë\035H\203", type=0x7fffffffcbf0,
size=0x7fffffffcbe8)
    at cache.h:761
#6  0x00000000004fdbb1 in read_object_with_reference (sha1=0x334a8130
"\001/Ç4&箺\036© wK`\214\"Ë\035H\203", required_type_name=0x55a1a0
"tree",
    size=0x7fffffffcc30, actual_sha1_return=0x0) at sha1_file.c:2299
#7  0x0000000000510f50 in fill_tree_descriptor (desc=0x7fffffffcd10,
sha1=0x334a8130 "\001/Ç4&箺\036© wK`\214\"Ë\035H\203") at
tree-walk.c:57
#8  0x00000000005133dd in traverse_trees_recursive (n=1, dirmask=1,
df_conflicts=0, names=0x349d68a0, info=0x7fffffffd020) at
unpack-trees.c:456
#9  0x0000000000514239 in unpack_callback (n=1, mask=1, dirmask=1,
names=0x349d68a0, info=0x7fffffffd020) at unpack-trees.c:809
#10 0x00000000005119c9 in traverse_trees (n=1, t=0x7fffffffd0b0,
info=0x7fffffffd020) at tree-walk.c:407
#11 0x000000000051342d in traverse_trees_recursive (n=1, dirmask=0,
df_conflicts=0, names=0x349d6860, info=0x7fffffffd3c0) at
unpack-trees.c:460
#12 0x0000000000514239 in unpack_callback (n=1, mask=1, dirmask=1,
names=0x349d6860, info=0x7fffffffd3c0) at unpack-trees.c:809
#13 0x00000000005119c9 in traverse_trees (n=1, t=0x7fffffffd450,
info=0x7fffffffd3c0) at tree-walk.c:407
#14 0x000000000051342d in traverse_trees_recursive (n=1, dirmask=0,
df_conflicts=0, names=0x349d6840, info=0x7fffffffd760) at
unpack-trees.c:460
#15 0x0000000000514239 in unpack_callback (n=1, mask=1, dirmask=1,
names=0x349d6840, info=0x7fffffffd760) at unpack-trees.c:809
#16 0x00000000005119c9 in traverse_trees (n=1, t=0x7fffffffda40,
info=0x7fffffffd760) at tree-walk.c:407
#17 0x0000000000514af5 in unpack_trees (len=1, t=0x7fffffffda40,
o=0x7fffffffd830) at unpack-trees.c:1063
#18 0x000000000049f140 in diff_cache (revs=0x7fffffffdaf0,
tree_sha1=0x32f3b094 "ò\032'\023\220U", tree_name=0x546766 "HEAD",
cached=1) at diff-lib.c:476
#19 0x000000000049f18a in run_diff_index (revs=0x7fffffffdaf0,
cached=1) at diff-lib.c:484
#20 0x000000000049f34d in index_differs_from (def=0x546766 "HEAD",
diff_flags=0) at diff-lib.c:519
#21 0x0000000000470288 in do_pick_commit (commit=0x32f3b000,
opts=0x7fffffffe270) at builtin/revert.c:502
#22 0x0000000000471d38 in single_pick (cmit=0x32f3b000,
opts=0x7fffffffe270) at builtin/revert.c:1069
#23 0x0000000000471ea9 in pick_revisions (opts=0x7fffffffe270) at
builtin/revert.c:1113
#24 0x0000000000472045 in cmd_cherry_pick (argc=2,
argv=0x7fffffffe4c0, prefix=0x6a81c1 ) at builtin/revert.c:1161
#25 0x0000000000405093 in run_builtin (p=0x65d190, argc=2,
argv=0x7fffffffe4c0) at git.c:308
#26 0x0000000000405288 in handle_internal_command (argc=2,
argv=0x7fffffffe4c0) at git.c:467

It always interrupts inside experimental_loose_object, when reading
memory-mapped data from disk(?).

git diff <commit>^ <commit>

works blazingly fast, so I assume that cherry-picking should also be,
but it is not. What can I do to make the cherry-picking go quicker?

I am using git 1.7.10 on FreeBSD 7.2.

-- 
Dmitry Risenberg

On Sat, May 12, 2012 at 3:39 PM, Dmitry Risenberg
[off-list ref] wrote:

Hello.

I have a very big git repository (the .git directory is about 5.3 Gb),
which is a copy of an svn repository fetched via git-svn. In fact
there are a few repositories ("working copies") that share the same
.git directory (via symlinks), in which I have different svn branches
checked out. Now I want to merge a commit from one svn branch to
another via git cherry-pick. The commit contains diff in only one
file. So I do

git cherry-pick <commit>

And the operation takes tens of seconds to finish. In "top" output I
see that git process uses almost no CPU, but has hundreds of page
faults, so I assume that it is reading a lot of files from disk.

Wild guess: poorly (or worse yet, never) packed repository?

2012/5/13 Junio C Hamano [off-list ref]:

On Sat, May 12, 2012 at 3:39 PM, Dmitry Risenberg
[off-list ref] wrote:

quoted

Hello.

I have a very big git repository (the .git directory is about 5.3 Gb),
which is a copy of an svn repository fetched via git-svn. In fact
there are a few repositories ("working copies") that share the same
.git directory (via symlinks), in which I have different svn branches
checked out. Now I want to merge a commit from one svn branch to
another via git cherry-pick. The commit contains diff in only one
file. So I do

git cherry-pick <commit>

And the operation takes tens of seconds to finish. In "top" output I
see that git process uses almost no CPU, but has hundreds of page
faults, so I assume that it is reading a lot of files from disk.

Wild guess: poorly (or worse yet, never) packed repository?

You were absolutely right.
I set "gc.auto = 0" during the initial checkout of svn and forgot to
turn it on afterwards. After running "git gc", my repo became two
times smaller, and git operations are now running much faster.

However, cherry-picking is still not as fast as I expected it to be -
cherry-picking a single-file commit takes about 14-15 seconds, fully
using one CPU core. Anything else I can improve?

-- 
Dmitry Risenberg

On Sun, May 13, 2012 at 07:39:49PM +0400, Dmitry Risenberg wrote:

However, cherry-picking is still not as fast as I expected it to be -
cherry-picking a single-file commit takes about 14-15 seconds, fully
using one CPU core. Anything else I can improve?

It's probably detecting renames as part of the merge, which can be
expensive if the thing you are cherry-picking is far away from HEAD. You
can try setting the merge.renamelimit config variable to something small
(like 1; setting it to 0 means "no limit").

-Peff