Re: [PATCH 3/3] refs: use strings directly in find_containing_dir()

René Scharfe [off-list ref] writes:

Why allocate a NUL-terminated copy at all when we can teach the code to
stop after a given number of characters just as easily?  Alas, this
will still trigger an allocation in search_ref_dir() (see first patch).

Yeah, but it is only because search_ref_dir() tries to use ref_entry_cmp(),
whose signature is geared more towards being used as a qsort(3) callback,
as the comparison function for bsearch(3).

A bsearch() callback takes two pointers, one is for the key and the other
for an array element, and there is no reason to require the two types be
the same.

In other words, something like this patch and we won't need an allocation
of the ref_entry that did not have to be a full ref_entry in the first
place (it only had to be something that supplies the "key" into a sorted
array).

 refs.c | 32 +++++++++++++++++++++++---------
 1 file changed, 23 insertions(+), 9 deletions(-)

diff --git a/refs.c b/refs.c
index 96e943c..52709ab 100644
--- a/refs.c
+++ b/refs.c

@@ -315,6 +315,23 @@ static int ref_entry_cmp(const void *a, const void *b)
 
 static void sort_ref_dir(struct ref_dir *dir);
 
+struct string_slice {
+	size_t len;
+	const char *str;
+};
+
+static int ref_entry_cmp_sslice(const void *key_, const void *ent_)
+{
+	struct string_slice *key = (struct string_slice *)key_;
+	struct ref_entry *ent = *(struct ref_entry **)ent_;
+	int entlen = strlen(ent->name);
+	int cmplen = key->len < entlen ? key->len : entlen;
+	int cmp = memcmp(key->str, ent->name, cmplen);
+	if (cmp)
+		return cmp;
+	return key->len - entlen;
+}
+
 /*
  * Return the entry with the given refname from the ref_dir
  * (non-recursively), sorting dir if necessary.  Return NULL if no

@@ -323,20 +340,17 @@ static void sort_ref_dir(struct ref_dir *dir);
 static struct ref_entry *search_ref_dir(struct ref_dir *dir,
 					const char *refname, size_t len)
 {
-	struct ref_entry *e, **r;
+	struct ref_entry **r;
+	struct string_slice key;
 
 	if (refname == NULL || !dir->nr)
 		return NULL;
 
 	sort_ref_dir(dir);
-
-	e = xmalloc(sizeof(struct ref_entry) + len + 1);
-	memcpy(e->name, refname, len);
-	e->name[len] = '\0';
-
-	r = bsearch(&e, dir->entries, dir->nr, sizeof(*dir->entries), ref_entry_cmp);
-
-	free(e);
+	key.len = len;
+	key.str = refname;
+	r = bsearch(&key, dir->entries, dir->nr, sizeof(*dir->entries),
+		    ref_entry_cmp_sslice);
 
 	if (r == NULL)
 		return NULL;

Am 22.05.2012 23:27, schrieb Junio C Hamano:

René Scharfe[off-list ref]  writes:

quoted

Why allocate a NUL-terminated copy at all when we can teach the code to
stop after a given number of characters just as easily?  Alas, this
will still trigger an allocation in search_ref_dir() (see first patch).

Yeah, but it is only because search_ref_dir() tries to use ref_entry_cmp(),
whose signature is geared more towards being used as a qsort(3) callback,
as the comparison function for bsearch(3).

A bsearch() callback takes two pointers, one is for the key and the other
for an array element, and there is no reason to require the two types be
the same.

In other words, something like this patch and we won't need an allocation
of the ref_entry that did not have to be a full ref_entry in the first
place (it only had to be something that supplies the "key" into a sorted
array).

Right, and this order (key-first) is documented for Linux, *BSD and by 
Microsoft, so we can probably rely on it.  The proposed asymmetry 
between sorting and lookup is a bit ... untidy, nevertheless.  But it's 
certainly worth it to avoid that ugly allocation.

Here's a random observation that led me to write the three patches: When 
running the following command under valgrind, it reports a few 
interesting numbers for total heap usage:

	$ git grep guess xdiff/xutils.c

   v1.7.8       591 allocs,    96 frees,   383,565 bytes allocated
   v1.7.9     2,940 allocs,   121 frees,   361,001 bytes allocated
   v1.7.10    3,002 allocs,   129 frees,   366,487 bytes allocated
   master     4,555 allocs, 1,586 frees, 2,380,265 bytes allocated
   3 patches  4,079 allocs, 1,110 frees,   430,093 bytes allocated
   4 patches  3,093 allocs,   124 frees,   377,749 bytes allocated

With your last patch, I think we're doing fine again, as the total
allocated size is within a few KB of the smallest one in this
arbitrary list of versions.

What has git grep to do with refs?  It checks if the path in the command
above is a ref, which makes it iterate over all of them..

René