[PATCH 0/7] Rewrite `git_config()` using config-set API

STALE3735d

Revision v1 of 9 in this series.

9 messages, 2 authors, 2016-06-15 · open the first message on its own page

[PATCH 0/7] Rewrite `git_config()` using config-set API

From: Tanay Abhra <hidden>
Date: 2016-06-15 23:02:00

This series builds on the top of 5def4132 (ta/config-set) in pu or topic[1]
in the mailing list with name "git config cache & special querying API utilizing
the cache".

This series aims to do these three things,

* Use the config-set API to rewrite git_config().

* Solve any legacy bugs in the previous system while at it.

* To be feature complete compared to the previous git_config() implementation,
  which I think it is now. (added the line number and file name info just for
  completeness)

Also, I haven't yet checked the exact improvements but still as a teaser,
git status now only rereads the configuration files twice instead of four
times.

[1]: http://thread.gmane.org/gmane.comp.version-control.git/253862

Tanay Abhra (7):

 Documentation/technical/api-config.txt |  5 ++
 cache.h                                |  1 +
 config.c                               | 93 +++++++++++++++++++++++++++++++---
 t/t1308-config-set.sh                  | 17 +++++++
 test-config.c                          | 10 ++++
 userdiff.c                             | 14 ++++-
 6 files changed, 131 insertions(+), 9 deletions(-)

-- 
1.9.0.GIT

[PATCH 1/7] config.c: fix accuracy of line number in errors

From: Tanay Abhra <hidden>
Date: 2016-06-15 23:02:00

If a callback returns a negative value to `git_config*()` family,
they call `die()` while printing the line number and the file name.
Currently the printed line number is off by one, thus printing the
wrong line number.

Make `linenr` point to the line we just parsed during the call
to callback to get accurate line number in error messages.

Discovered-by: Tanay Abhra [off-list ref]
Signed-off-by: Matthieu Moy <redacted>
---
 config.c | 12 +++++++++++-
 1 file changed, 11 insertions(+), 1 deletion(-)
diff --git a/config.c b/config.c
index 22971e9..6db8f97 100644
--- a/config.c
+++ b/config.c
@@ -244,6 +244,7 @@ static int get_next_char(void)
 		cf->linenr++;
 	if (c == EOF) {
 		cf->eof = 1;
+		cf->linenr++;
 		c = '\n';
 	}
 	return c;
@@ -319,6 +320,7 @@ static int get_value(config_fn_t fn, void *data, struct strbuf *name)
 {
 	int c;
 	char *value;
+	int ret;
 
 	/* Get the full name */
 	for (;;) {
@@ -341,7 +343,15 @@ static int get_value(config_fn_t fn, void *data, struct strbuf *name)
 		if (!value)
 			return -1;
 	}
-	return fn(name->buf, value, data);
+	/*
+	 * We already consumed the \n, but we need linenr to point to
+	 * the line we just parsed during the call to fn to get
+	 * accurate line number in error messages.
+	 */
+	cf->linenr--;
+	ret = fn(name->buf, value, data);
+	cf->linenr++;
+	return ret;
 }
 
 static int get_extended_base_var(struct strbuf *name, int c)
-- 
1.9.0.GIT

[PATCH 2/7] rewrite git_config() to use the config-set API

From: Tanay Abhra <hidden>
Date: 2016-06-15 23:02:00

Of all the functions in `git_config*()` family, `git_config()` has the
most invocations in the whole code base. Each `git_config()` invocation
causes config file rereads which can be avoided using the config-set API.

Use the config-set API to rewrite `git_config()` to use the config caching
layer to avoid config file rereads on each invocation during a git process
lifetime. First invocation constructs the cache, and after that for each
successive invocation, `git_config()` feeds values from the config cache
instead of rereading the configuration files.

Signed-off-by: Tanay Abhra <redacted>
---
 config.c | 29 +++++++++++++++++++++++++++--
 1 file changed, 27 insertions(+), 2 deletions(-)
diff --git a/config.c b/config.c
index 6db8f97..a7fb9a4 100644
--- a/config.c
+++ b/config.c
@@ -1232,11 +1232,36 @@ int git_config_with_options(config_fn_t fn, void *data,
 	return ret;
 }
 
-int git_config(config_fn_t fn, void *data)
+static int git_config_raw(config_fn_t fn, void *data)
 {
 	return git_config_with_options(fn, data, NULL, 1);
 }
 
+static int configset_iter(struct config_set *cs, config_fn_t fn, void *data)
+{
+	int i;
+	struct string_list *strptr;
+	struct config_set_element *entry;
+	struct hashmap_iter iter;
+	hashmap_iter_init(&cs->config_hash, &iter);
+	while ((entry = hashmap_iter_next(&iter))) {
+		strptr = &entry->value_list;
+		for (i = 0; i < strptr->nr; i++) {
+			if (fn(entry->key, strptr->items[i].string, data) < 0)
+				die("bad config file line in (TODO: file/line info)");
+		}
+	}
+	return 0;
+}
+
+static void git_config_check_init(void);
+
+int git_config(config_fn_t fn, void *data)
+{
+	git_config_check_init();
+	return configset_iter(&the_config_set, fn, data);
+}
+
 static struct config_set_element *configset_find_element(struct config_set *cs, const char *key)
 {
 	struct config_set_element k;
@@ -1418,7 +1443,7 @@ static void git_config_check_init(void)
 	if (the_config_set.hash_initialized)
 		return;
 	git_configset_init(&the_config_set);
-	git_config(config_set_callback, &the_config_set);
+	git_config_raw(config_set_callback, &the_config_set);
 }
 
 void git_config_clear(void)
-- 
1.9.0.GIT

[PATCH 3/7] add a test for semantic errors in config files

From: Tanay Abhra <hidden>
Date: 2016-06-15 23:02:00

Semantic errors (for example, for alias.* variables NULL values are
not allowed) in configuration files cause a die printing the line
number and file name of the offending value.

Add a test documenting that such errors cause a die printing the
accurate line number and file name.

Signed-off-by: Tanay Abhra <redacted>
---
 t/t1308-config-set.sh | 8 ++++++++
 1 file changed, 8 insertions(+)
diff --git a/t/t1308-config-set.sh b/t/t1308-config-set.sh
index 7fdf840..bd033df 100755
--- a/t/t1308-config-set.sh
+++ b/t/t1308-config-set.sh
@@ -197,4 +197,12 @@ test_expect_success 'proper error on error in custom config files' '
 	test_cmp expect actual
 '
 
+test_expect_success 'check line errors for malformed values' '
+	cp .git/config .git/config.old &&
+	test_when_finished "mv .git/config.old .git/config" &&
+	echo "[alias]\n br" >.git/config &&
+	test_expect_code 128 git br 2>result &&
+	grep "fatal: bad config file line 2 in .git/config" result
+'
+
 test_done
-- 
1.9.0.GIT

[PATCH 4/7] add line number and file name info to `config_set`

From: Tanay Abhra <hidden>
Date: 2016-06-15 23:02:00

Store file name and line number for each key-value pair in the cache.
Use the information to print line number and file name in errors raised
by `git_config()` which now uses the configuration files caching layer
internally.

Signed-off-by: Tanay Abhra <redacted>
---
 config.c | 32 ++++++++++++++++++++++++++++----
 1 file changed, 28 insertions(+), 4 deletions(-)
diff --git a/config.c b/config.c
index a7fb9a4..43a951f 100644
--- a/config.c
+++ b/config.c
@@ -1237,9 +1237,15 @@ static int git_config_raw(config_fn_t fn, void *data)
 	return git_config_with_options(fn, data, NULL, 1);
 }
 
+struct key_value_info {
+	const char *filename;
+	int linenr;
+};
+
 static int configset_iter(struct config_set *cs, config_fn_t fn, void *data)
 {
 	int i;
+	struct key_value_info *kv_info;
 	struct string_list *strptr;
 	struct config_set_element *entry;
 	struct hashmap_iter iter;
@@ -1247,8 +1253,15 @@ static int configset_iter(struct config_set *cs, config_fn_t fn, void *data)
 	while ((entry = hashmap_iter_next(&iter))) {
 		strptr = &entry->value_list;
 		for (i = 0; i < strptr->nr; i++) {
-			if (fn(entry->key, strptr->items[i].string, data) < 0)
-				die("bad config file line in (TODO: file/line info)");
+			if (fn(entry->key, strptr->items[i].string, data) < 0) {
+				kv_info = strptr->items[i].util;
+				if (!kv_info->linenr)
+					die("unable to parse command-line config");
+				else
+					die("bad config file line %d in %s",
+						kv_info->linenr,
+						kv_info->filename);
+			}
 		}
 	}
 	return 0;
@@ -1287,6 +1300,8 @@ static struct config_set_element *configset_find_element(struct config_set *cs,
 static int configset_add_value(struct config_set *cs, const char *key, const char *value)
 {
 	struct config_set_element *e;
+	struct key_value_info *kv_info = xmalloc(sizeof(*kv_info));
+	struct string_list_item *si;
 	e = configset_find_element(cs, key);
 	/*
 	 * Since the keys are being fed by git_config*() callback mechanism, they
@@ -1299,7 +1314,16 @@ static int configset_add_value(struct config_set *cs, const char *key, const cha
 		string_list_init(&e->value_list, 1);
 		hashmap_add(&cs->config_hash, e);
 	}
-	string_list_append_nodup(&e->value_list, value ? xstrdup(value) : NULL);
+	si = string_list_append_nodup(&e->value_list, value ? xstrdup(value) : NULL);
+	if (cf) {
+		kv_info->filename = strintern(cf->name);
+		kv_info->linenr = cf->linenr;
+	} else {
+		/* for values read from `git_config_from_parameters()` */
+		kv_info->filename = strintern("parameter");
+		kv_info->linenr = 0;
+	}
+	si->util = kv_info;
 
 	return 0;
 }
@@ -1326,7 +1350,7 @@ void git_configset_clear(struct config_set *cs)
 	hashmap_iter_init(&cs->config_hash, &iter);
 	while ((entry = hashmap_iter_next(&iter))) {
 		free(entry->key);
-		string_list_clear(&entry->value_list, 0);
+		string_list_clear(&entry->value_list, 1);
 	}
 	hashmap_free(&cs->config_hash, 1);
 	cs->hash_initialized = 0;
-- 
1.9.0.GIT

[PATCH 7/7] Add tests for `git_config_get_string()`

From: Tanay Abhra <hidden>
Date: 2016-06-15 23:02:00

Add tests for `git_config_get_string()`, check whether it
dies printing the line number and the file name if an NULL
value is retrieved for the given key.

Signed-off-by: Tanay Abhra <redacted>
---
 t/t1308-config-set.sh |  9 +++++++++
 test-config.c         | 10 ++++++++++
 2 files changed, 19 insertions(+)
diff --git a/t/t1308-config-set.sh b/t/t1308-config-set.sh
index bd033df..1cb453b 100755
--- a/t/t1308-config-set.sh
+++ b/t/t1308-config-set.sh
@@ -119,6 +119,15 @@ test_expect_success 'find integer value for a key' '
 	check_config get_int lamb.chop 65
 '
 
+test_expect_success 'find string value for a key' '
+	check_config get_string case.baz hask
+'
+
+test_expect_success 'check line error when NULL string is queried' '
+	test_expect_code 128 test-config get_string case.foo 2>result &&
+	grep "fatal: bad config file line 7 in .git/config" result
+'
+
 test_expect_success 'find integer if value is non parse-able' '
 	check_config expect_code 128 get_int lamb.head
 '
diff --git a/test-config.c b/test-config.c
index 9dd1b22..994741a 100644
--- a/test-config.c
+++ b/test-config.c
@@ -16,6 +16,8 @@
  *
  * get_bool -> print bool value for the entered key or die
  *
+ * get_string -> print string value for the entered key or die
+ *
  * configset_get_value -> returns value with the highest priority for the entered key
  * 			from a config_set constructed from files entered as arguments.
  *
@@ -84,6 +86,14 @@ int main(int argc, char **argv)
 			printf("Value not found for \"%s\"\n", argv[2]);
 			goto exit1;
 		}
+	} else if (argc == 3 && !strcmp(argv[1], "get_string")) {
+		if (!git_config_get_string(argv[2], &v)) {
+			printf("%s\n", v);
+			goto exit0;
+		} else {
+			printf("Value not found for \"%s\"\n", argv[2]);
+			goto exit1;
+		}
 	} else if (!strcmp(argv[1], "configset_get_value")) {
 		for (i = 3; i < argc; i++) {
 			int err;
-- 
1.9.0.GIT

[PATCH 6/7] config: add `git_die_config()` to the config-set API

From: Tanay Abhra <hidden>
Date: 2016-06-15 23:02:01

Add `git_die_config` that dies printing the line number and the file name
of the highest priority value for the configuration variable `key`.

It has usage in non-callback based config value retrieval where we can
raise an error and die if there is a semantic error.
For example,

	if (!git_config_get_value(key, &value)) {
		/* NULL values not allowed */
		if (!value)
			git_config_die(key);
		else
			/* do work */
	}

Signed-off-by: Tanay Abhra <redacted>
---
 Documentation/technical/api-config.txt |  5 +++++
 cache.h                                |  1 +
 config.c                               | 24 ++++++++++++++++++++++--
 3 files changed, 28 insertions(+), 2 deletions(-)
diff --git a/Documentation/technical/api-config.txt b/Documentation/technical/api-config.txt
index 8a86e45..14571e7 100644
--- a/Documentation/technical/api-config.txt
+++ b/Documentation/technical/api-config.txt
@@ -150,6 +150,11 @@ as well as retrieval for the queried variable, including:
 	Similar to `git_config_get_string`, but expands `~` or `~user` into
 	the user's home directory when found at the beginning of the path.
 
+`void git_die_config(const char *key)`::
+
+	Dies printing the line number and the file name of the highest
+	priority value for the configuration variable `key`.
+
 See test-config.c for usage examples.
 
 Value Parsing Helpers
diff --git a/cache.h b/cache.h
index 2f63fd1..fc886c3 100644
--- a/cache.h
+++ b/cache.h
@@ -1380,6 +1380,7 @@ extern int git_config_get_bool(const char *key, int *dest);
 extern int git_config_get_bool_or_int(const char *key, int *is_bool, int *dest);
 extern int git_config_get_maybe_bool(const char *key, int *dest);
 extern int git_config_get_pathname(const char *key, const char **dest);
+extern void git_die_config(const char *key);
 
 extern int committer_ident_sufficiently_given(void);
 extern int author_ident_sufficiently_given(void);
diff --git a/config.c b/config.c
index 43a951f..f0c9805 100644
--- a/config.c
+++ b/config.c
@@ -1491,8 +1491,12 @@ const struct string_list *git_config_get_value_multi(const char *key)
 
 int git_config_get_string(const char *key, const char **dest)
 {
+	int ret;
 	git_config_check_init();
-	return git_configset_get_string(&the_config_set, key, dest);
+	ret = git_configset_get_string(&the_config_set, key, dest);
+	if (ret < 0)
+		git_die_config(key);
+	return ret;
 }
 
 int git_config_get_int(const char *key, int *dest)
@@ -1527,8 +1531,24 @@ int git_config_get_maybe_bool(const char *key, int *dest)
 
 int git_config_get_pathname(const char *key, const char **dest)
 {
+	int ret;
 	git_config_check_init();
-	return git_configset_get_pathname(&the_config_set, key, dest);
+	ret = git_configset_get_pathname(&the_config_set, key, dest);
+	if (ret < 0)
+		git_die_config(key);
+	return ret;
+}
+
+void git_die_config(const char *key)
+{
+	const struct string_list *strptr;
+	struct key_value_info *kv_info;
+	strptr = git_config_get_value_multi(key);
+	kv_info = strptr->items[strptr->nr - 1].util;
+	if (!kv_info->linenr)
+		die("unable to parse command-line config");
+	else
+		die("bad config file line %d in %s",kv_info->linenr, kv_info->filename);
 }
 
 /*
-- 
1.9.0.GIT

[PATCH 5/7] enforce `xfuncname` precedence over `funcname`

From: Tanay Abhra <hidden>
Date: 2016-06-15 23:02:01

t4018-diff-funcname.sh fails for the new `git_config()` which uses the
configuration files caching layer internally.
The test introduced in commit d64d6cdc checks that whether `xfuncname` takes
precedence over `funcname` variable which was not guaranteed by config API
previously and worked only because values were parsed and fed in order. The
new  `git_config()` only guarantees precedence order for variables with the
same name.

Also `funcname` variable is deprecated and not documented properly.
`xfuncname` is mentioned in the docs and takes precedence over `funcname`.
Instead of removing `funcname` variable, enforce `xfuncname` precedence over
`funcname` when the variables have the same subsection. Remove dependency
that required values to be fed to userdiff_config() in parsing order for the
test to succeed.

Signed-off-by: Tanay Abhra <redacted>
---
Note: this the only test that failed for the new git_config() rewrite.

 userdiff.c | 14 ++++++++++++--
 1 file changed, 12 insertions(+), 2 deletions(-)
diff --git a/userdiff.c b/userdiff.c
index fad52d6..a51bc89 100644
--- a/userdiff.c
+++ b/userdiff.c
@@ -2,6 +2,7 @@
 #include "userdiff.h"
 #include "cache.h"
 #include "attr.h"
+#include "string-list.h"
 
 static struct userdiff_driver *drivers;
 static int ndrivers;
@@ -211,9 +212,12 @@ int userdiff_config(const char *k, const char *v)
 	struct userdiff_driver *drv;
 	const char *name, *type;
 	int namelen;
+	char *subsection = NULL;
+	static struct string_list xflag = STRING_LIST_INIT_DUP;
 
 	if (parse_config_key(k, "diff", &name, &namelen, &type) || !name)
 		return 0;
+	subsection = xstrndup(name, namelen);
 
 	drv = userdiff_find_by_namelen(name, namelen);
 	if (!drv) {
@@ -224,10 +228,16 @@ int userdiff_config(const char *k, const char *v)
 		drv->binary = -1;
 	}
 
-	if (!strcmp(type, "funcname"))
+	if (!strcmp(type, "funcname") && !unsorted_string_list_has_string(&xflag, subsection)) {
+		free (subsection);
 		return parse_funcname(&drv->funcname, k, v, 0);
-	if (!strcmp(type, "xfuncname"))
+	}
+	if (!strcmp(type, "xfuncname")) {
+		string_list_append(&xflag, subsection);
+		free (subsection);
 		return parse_funcname(&drv->funcname, k, v, REG_EXTENDED);
+	}
+	free(subsection);
 	if (!strcmp(type, "binary"))
 		return parse_tristate(&drv->binary, k, v);
 	if (!strcmp(type, "command"))
-- 
1.9.0.GIT

Re: [PATCH 5/7] enforce `xfuncname` precedence over `funcname`

From: Eric Sunshine <hidden>
Date: 2016-06-15 23:02:01

On Wed, Jul 23, 2014 at 2:42 PM, Tanay Abhra [off-list ref] wrote:
t4018-diff-funcname.sh fails for the new `git_config()` which uses the
configuration files caching layer internally.
The test introduced in commit d64d6cdc checks that whether `xfuncname` takes
s/that//
precedence over `funcname` variable which was not guaranteed by config API
previously and worked only because values were parsed and fed in order. The
new  `git_config()` only guarantees precedence order for variables with the
s/\s+/ /
quoted hunk
same name.

Also `funcname` variable is deprecated and not documented properly.
`xfuncname` is mentioned in the docs and takes precedence over `funcname`.
Instead of removing `funcname` variable, enforce `xfuncname` precedence over
`funcname` when the variables have the same subsection. Remove dependency
that required values to be fed to userdiff_config() in parsing order for the
test to succeed.

Signed-off-by: Tanay Abhra <redacted>
---
Note: this the only test that failed for the new git_config() rewrite.

 userdiff.c | 14 ++++++++++++--
 1 file changed, 12 insertions(+), 2 deletions(-)
diff --git a/userdiff.c b/userdiff.c
index fad52d6..a51bc89 100644
--- a/userdiff.c
+++ b/userdiff.c
@@ -2,6 +2,7 @@
 #include "userdiff.h"
 #include "cache.h"
 #include "attr.h"
+#include "string-list.h"

 static struct userdiff_driver *drivers;
 static int ndrivers;
@@ -211,9 +212,12 @@ int userdiff_config(const char *k, const char *v)
        struct userdiff_driver *drv;
        const char *name, *type;
        int namelen;
+       char *subsection = NULL;
+       static struct string_list xflag = STRING_LIST_INIT_DUP;

        if (parse_config_key(k, "diff", &name, &namelen, &type) || !name)
                return 0;
+       subsection = xstrndup(name, namelen);

        drv = userdiff_find_by_namelen(name, namelen);
        if (!drv) {
@@ -224,10 +228,16 @@ int userdiff_config(const char *k, const char *v)
                drv->binary = -1;
        }

-       if (!strcmp(type, "funcname"))
+       if (!strcmp(type, "funcname") && !unsorted_string_list_has_string(&xflag, subsection)) {
+               free (subsection);
                return parse_funcname(&drv->funcname, k, v, 0);
-       if (!strcmp(type, "xfuncname"))
+       }
+       if (!strcmp(type, "xfuncname")) {
+               string_list_append(&xflag, subsection);
+               free (subsection);
                return parse_funcname(&drv->funcname, k, v, REG_EXTENDED);
+       }
+       free(subsection);
        if (!strcmp(type, "binary"))
                return parse_tristate(&drv->binary, k, v);
        if (!strcmp(type, "command"))
--
1.9.0.GIT
Keyboard shortcuts
hback out one level
jnext message in thread
kprevious message in thread
ldrill in
Escclose help / fold thread tree
?toggle this help