Improve git-rev-list memory usage further
From: Linus Torvalds <torvalds@osdl.org>
Date: 2016-06-15 22:42:06
This avoids keeping tree entries around, and free's them as it traverses the list. This avoids building up a huge memory footprint just for these small but very common allocations. Before: [torvalds@g5 linux]$ /usr/bin/time git-rev-list --objects v2.6.12..HEAD | wc -l 11.65user 0.38system 0:12.65elapsed 95%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+42934minor)pagefaults 0swaps 59124 After: [torvalds@g5 linux]$ /usr/bin/time git-rev-list --objects v2.6.12..HEAD | wc -l 12.28user 0.29system 0:12.57elapsed 99%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+26718minor)pagefaults 0swaps 59124 note how the minor fault numbers - which ends up being how many pages we needed to map - go down from 42934 (167 MB) to 26718 (104 MB). That's still a honking big memory footprint, but it's about half of what it was just a day or two ago (and this is the object list for a pretty big update - almost 60,000 objects. Smaller updates need less memory). Signed-off-by: Linus Torvalds <torvalds@osdl.org> ---
diff --git a/rev-list.c b/rev-list.c
--- a/rev-list.c
+++ b/rev-list.c@@ -147,11 +147,16 @@ static struct object_list **process_tree die("bad tree object %s", sha1_to_hex(obj->sha1)); obj->flags |= SEEN; p = add_object(obj, p, name); - for (entry = tree->entries ; entry ; entry = entry->next) { + entry = tree->entries; + tree->entries = NULL; + while (entry) { + struct tree_entry_list *next = entry->next; if (entry->directory) p = process_tree(entry->item.tree, p, entry->name); else p = process_blob(entry->item.blob, p, entry->name); + free(entry); + entry = next; } return p; }
@@ -218,12 +223,15 @@ static void mark_tree_uninteresting(stru if (parse_tree(tree) < 0) die("bad tree %s", sha1_to_hex(obj->sha1)); entry = tree->entries; + tree->entries = NULL; while (entry) { + struct tree_entry_list *next = entry->next; if (entry->directory) mark_tree_uninteresting(entry->item.tree); else mark_blob_uninteresting(entry->item.blob); - entry = entry->next; + free(entry); + entry = next; } }