Re: [PATCH 1/5] refs: store submodule ref stores in a hashmap

3 messages, 3 authors, 2017-02-10 · open the first message on its own page

Re: [PATCH 1/5] refs: store submodule ref stores in a hashmap

From: Junio C Hamano <hidden>
Date: 2017-02-09 22:24:42

Michael Haggerty [off-list ref] writes:
Aside from scaling better, this means that the submodule name needn't be
stored in the ref_store instance anymore (which will be changed in a
moment).
Nice.  I like the latter reason very much (this is not a suggestion
to change the description).
+struct submodule_hash_entry
+{
+	struct hashmap_entry ent; /* must be the first member! */
+
+	struct ref_store *refs;
+
+	/* NUL-terminated name of submodule: */
+	char submodule[FLEX_ARRAY];
+};
+
+static int submodule_hash_cmp(const void *entry, const void *entry_or_key,
+			      const void *keydata)
+{
+	const struct submodule_hash_entry *e1 = entry, *e2 = entry_or_key;
+	const char *submodule = keydata;
+
+	return strcmp(e1->submodule, submodule ? submodule : e2->submodule);
I would have found it more readable if it were like so:

	const char *submodule = keydata ? keydata : e2->submodule;

	return strcmp(e1->submodule, submodule);

but I suspect the difference is not that huge.
+}
+
+static struct submodule_hash_entry *alloc_submodule_hash_entry(
+		const char *submodule, struct ref_store *refs)
+{
+	size_t len = strlen(submodule);
+	struct submodule_hash_entry *entry = malloc(sizeof(*entry) + len + 1);
I think this (and the later memcpy) is what FLEX_ALLOC_MEM() was
invented for.
quoted hunk
+	hashmap_entry_init(entry, strhash(submodule));
+	entry->refs = refs;
+	memcpy(entry->submodule, submodule, len + 1);
+	return entry;
+}
...
@@ -1373,16 +1405,17 @@ void base_ref_store_init(struct ref_store *refs,
 			die("BUG: main_ref_store initialized twice");
 
 		refs->submodule = "";
-		refs->next = NULL;
 		main_ref_store = refs;
 	} else {
-		if (lookup_ref_store(submodule))
+		refs->submodule = xstrdup(submodule);
+
+		if (!submodule_ref_stores.tablesize)
+			hashmap_init(&submodule_ref_stores, submodule_hash_cmp, 20);
Makes me wonder what "20" stands for.  Perhaps the caller should be
allowed to say "I do not quite care what initial size is" by passing
0 or some equally but more clealy meaningless value (which of course
would be outside the scope of this series).

Re: [PATCH 1/5] refs: store submodule ref stores in a hashmap

From: Jeff King <hidden>
Date: 2017-02-10 04:42:03

On Thu, Feb 09, 2017 at 12:34:04PM -0800, Junio C Hamano wrote:
quoted
+static struct submodule_hash_entry *alloc_submodule_hash_entry(
+		const char *submodule, struct ref_store *refs)
+{
+	size_t len = strlen(submodule);
+	struct submodule_hash_entry *entry = malloc(sizeof(*entry) + len + 1);
I think this (and the later memcpy) is what FLEX_ALLOC_MEM() was
invented for.
Yes, it was. Though since the length comes from a strlen() call, it can
actually use the _STR variant, like:

  FLEX_ALLOC_STR(entry, submodule, submodule);

Besides being shorter, this does integer-overflow checks on the final
length.
quoted
@@ -1373,16 +1405,17 @@ void base_ref_store_init(struct ref_store *refs,
 			die("BUG: main_ref_store initialized twice");
 
 		refs->submodule = "";
-		refs->next = NULL;
 		main_ref_store = refs;
 	} else {
-		if (lookup_ref_store(submodule))
+		refs->submodule = xstrdup(submodule);
+
+		if (!submodule_ref_stores.tablesize)
+			hashmap_init(&submodule_ref_stores, submodule_hash_cmp, 20);
Makes me wonder what "20" stands for.  Perhaps the caller should be
allowed to say "I do not quite care what initial size is" by passing
0 or some equally but more clealy meaningless value (which of course
would be outside the scope of this series).
I think this is what "0" already does (grep for HASHMAP_INITIAL_SIZE).
In fact, that constant is 64. The 20 we pass in goes through some magic
load-factor computation and ends up as 25. That being smaller than the
INITIAL_SIZE constant, I believe that we end up allocating 64 entries
either way (that's just from reading the code, though; I didn't run it
to double check).

-Peff

Re: [PATCH 1/5] refs: store submodule ref stores in a hashmap

From: Michael Haggerty <hidden>
Date: 2017-02-10 10:24:26

On 02/09/2017 09:34 PM, Junio C Hamano wrote:
Michael Haggerty [off-list ref] writes:
quoted
[...]
+static int submodule_hash_cmp(const void *entry, const void *entry_or_key,
+			      const void *keydata)
+{
+	const struct submodule_hash_entry *e1 = entry, *e2 = entry_or_key;
+	const char *submodule = keydata;
+
+	return strcmp(e1->submodule, submodule ? submodule : e2->submodule);
I would have found it more readable if it were like so:

	const char *submodule = keydata ? keydata : e2->submodule;

	return strcmp(e1->submodule, submodule);

but I suspect the difference is not that huge.
Yes, that's better. I'll change it.

On 02/10/2017 05:04 AM, Jeff King wrote:
On Thu, Feb 09, 2017 at 12:34:04PM -0800, Junio C Hamano wrote:
quoted
quoted
+static struct submodule_hash_entry *alloc_submodule_hash_entry(
+		const char *submodule, struct ref_store *refs)
+{
+	size_t len = strlen(submodule);
+	struct submodule_hash_entry *entry = malloc(sizeof(*entry) + len + 1);
I think this (and the later memcpy) is what FLEX_ALLOC_MEM() was
invented for.
Yes, it was. Though since the length comes from a strlen() call, it can
actually use the _STR variant, like:

  FLEX_ALLOC_STR(entry, submodule, submodule);

Besides being shorter, this does integer-overflow checks on the final
length.
Nice. TIL. Will fix.
quoted
quoted
@@ -1373,16 +1405,17 @@ void base_ref_store_init(struct ref_store *refs,
 			die("BUG: main_ref_store initialized twice");

 		refs->submodule = "";
-		refs->next = NULL;
 		main_ref_store = refs;
 	} else {
-		if (lookup_ref_store(submodule))
+		refs->submodule = xstrdup(submodule);
+
+		if (!submodule_ref_stores.tablesize)
+			hashmap_init(&submodule_ref_stores, submodule_hash_cmp, 20);
Makes me wonder what "20" stands for.  Perhaps the caller should be
allowed to say "I do not quite care what initial size is" by passing
0 or some equally but more clealy meaningless value (which of course
would be outside the scope of this series).
I think this is what "0" already does (grep for HASHMAP_INITIAL_SIZE).
In fact, that constant is 64. The 20 we pass in goes through some magic
load-factor computation and ends up as 25. That being smaller than the
INITIAL_SIZE constant, I believe that we end up allocating 64 entries
either way (that's just from reading the code, though; I didn't run it
to double check).
I guess I might as well change it to zero, then.

Thanks for the feedback!

Michael
Keyboard shortcuts
hback out one level
jnext message in thread
kprevious message in thread
ldrill in
Escclose help / fold thread tree
?toggle this help