[PATCH 0/2] Show Git Mailing List: a builtin difftool

STALE3482d

Revision v1 of 4 in this series.

44 messages, 6 authors, 2017-01-19 · open the first message on its own page

[PATCH 0/2] Show Git Mailing List: a builtin difftool

From: Johannes Schindelin <hidden>
Date: 2016-11-22 17:01:42

I have been working on the builtin difftool for a little over a week,
for two reasons:

1. Perl is really not native on Windows. Not only is there a performance
   penalty to be paid just for running Perl scripts, we also have to deal
   with the fact that users may have different Perl installations, with
   different options, and some other Perl installation may decide to set
   PERL5LIB globally, wreaking havoc with Git for Windows' Perl (which we
   have to use because almost all other Perl distributions lack the
   Subversion bindings we need for `git svn`).

2. Perl makes for a rather large reason that Git for Windows' installer
   weighs in with >30MB. While one Perl script less does not relieve us
   of that burden, it is one step in the right direction.

This pair of patches serves two purposes: to ask for reviews, and to
show what I plan to release as part of Git for Windows v2.11.0 (which is
due this Thursday, if Git v2.11.0 is released tomorrow, as planned).

The second patch really only explains how I will make sure that the
builtin difftool will only affect users who want to opt in to testing.


Johannes Schindelin (2):
  difftool: add the builtin
  difftool: add a feature flag for the builtin vs scripted version

 .gitignore                 |   2 +
 Makefile                   |   1 +
 builtin.h                  |   1 +
 builtin/builtin-difftool.c | 680 +++++++++++++++++++++++++++++++++++++++++++++
 git-difftool.perl          |   7 +
 git.c                      |  21 ++
 6 files changed, 712 insertions(+)
 create mode 100644 builtin/builtin-difftool.c


base-commit: 1310affe024fba407bff55dbe65cd6d670c8a32d
Published-As: https://github.com/dscho/git/releases/tag/builtin-difftool-v1
Fetch-It-Via: git fetch https://github.com/dscho/git builtin-difftool-v1

-- 
2.10.1.583.g721a9e0

[PATCH 1/2] difftool: add the builtin

From: Johannes Schindelin <hidden>
Date: 2016-11-22 17:01:46

This adds a builtin difftool that represents a conversion of the current
Perl script version of the difftool.

The motivation is that Perl scripts are not at all native on Windows,
and that `git difftool` therefore is pretty slow on that platform, when
there is no good reason for it to be slow.

In addition, Perl does not really have access to Git's internals. That
means that any script will always have to jump through unnecessary
hoops.

The current version of the builtin difftool does not, however, make full
use of the internals but instead chooses to spawn a couple of Git
processes, still, to make for an easier conversion. There remains a lot
of room for improvement, left for a later date.

Note: the original difftool is still called by `git difftool`. To get the
new, experimental version, call `git builtin-difftool`. The reason: this
new, experimental, builtin difftool will be shipped as part of Git for
Windows v2.11.0, to allow for easier large-scale testing.

Signed-off-by: Johannes Schindelin <redacted>
---
 .gitignore                 |   1 +
 Makefile                   |   1 +
 builtin.h                  |   1 +
 builtin/builtin-difftool.c | 680 +++++++++++++++++++++++++++++++++++++++++++++
 git.c                      |   1 +
 5 files changed, 684 insertions(+)
 create mode 100644 builtin/builtin-difftool.c
diff --git a/.gitignore b/.gitignore
index 05cb58a..4f54531 100644
--- a/.gitignore
+++ b/.gitignore
@@ -51,6 +51,7 @@
 /git-diff-tree
 /git-difftool
 /git-difftool--helper
+/git-builtin-difftool
 /git-describe
 /git-fast-export
 /git-fast-import
diff --git a/Makefile b/Makefile
index f53fcc9..f764174 100644
--- a/Makefile
+++ b/Makefile
@@ -888,6 +888,7 @@ BUILTIN_OBJS += builtin/diff-files.o
 BUILTIN_OBJS += builtin/diff-index.o
 BUILTIN_OBJS += builtin/diff-tree.o
 BUILTIN_OBJS += builtin/diff.o
+BUILTIN_OBJS += builtin/builtin-difftool.o
 BUILTIN_OBJS += builtin/fast-export.o
 BUILTIN_OBJS += builtin/fetch-pack.o
 BUILTIN_OBJS += builtin/fetch.o
diff --git a/builtin.h b/builtin.h
index b9122bc..409a61e 100644
--- a/builtin.h
+++ b/builtin.h
@@ -60,6 +60,7 @@ extern int cmd_diff_files(int argc, const char **argv, const char *prefix);
 extern int cmd_diff_index(int argc, const char **argv, const char *prefix);
 extern int cmd_diff(int argc, const char **argv, const char *prefix);
 extern int cmd_diff_tree(int argc, const char **argv, const char *prefix);
+extern int cmd_builtin_difftool(int argc, const char **argv, const char *prefix);
 extern int cmd_fast_export(int argc, const char **argv, const char *prefix);
 extern int cmd_fetch(int argc, const char **argv, const char *prefix);
 extern int cmd_fetch_pack(int argc, const char **argv, const char *prefix);
diff --git a/builtin/builtin-difftool.c b/builtin/builtin-difftool.c
new file mode 100644
index 0000000..9feefcd
--- /dev/null
+++ b/builtin/builtin-difftool.c
@@ -0,0 +1,680 @@
+/*
+ * "git difftool" builtin command
+ *
+ * This is a wrapper around the GIT_EXTERNAL_DIFF-compatible
+ * git-difftool--helper script.
+ *
+ * This script exports GIT_EXTERNAL_DIFF and GIT_PAGER for use by git.
+ * The GIT_DIFF* variables are exported for use by git-difftool--helper.
+ *
+ * Any arguments that are unknown to this script are forwarded to 'git diff'.
+ *
+ * Copyright (C) 2016 Johannes Schindelin
+ */
+#include "cache.h"
+#include "builtin.h"
+#include "parse-options.h"
+#include "run-command.h"
+#include "argv-array.h"
+#include "strbuf.h"
+#include "lockfile.h"
+
+static char *diff_gui_tool;
+static int trust_exit_code;
+
+static const char * const builtin_difftool_usage[] = {
+	N_("git add [<options>] [--] <pathspec>..."),
+	NULL
+};
+
+static int difftool_config(const char *var, const char *value, void *cb)
+{
+	if (!strcmp(var, "diff.guitool")) {
+		diff_gui_tool = xstrdup(value);
+		return 0;
+	}
+
+	if (!strcmp(var, "difftool.trustexitcode")) {
+		trust_exit_code = git_config_bool(var, value);
+		return 0;
+	}
+
+	return git_default_config(var, value, cb);
+}
+
+static int print_tool_help(void)
+{
+	const char *argv[] = { "mergetool", "--tool-help=diff", NULL };
+	return run_command_v_opt(argv, RUN_GIT_CMD);
+}
+
+static int parse_index_info(char *p, int *mode1, int *mode2,
+			    struct object_id *oid1, struct object_id *oid2,
+			    char *status)
+{
+	if (*p != ':')
+		return error("expected ':', got '%c'", *p);
+	*mode1 = (int)strtol(p + 1, &p, 8);
+	if (*p != ' ')
+		return error("expected ' ', got '%c'", *p);
+	*mode2 = (int)strtol(p + 1, &p, 8);
+	if (*p != ' ')
+		return error("expected ' ', got '%c'", *p);
+	if (get_oid_hex(++p, oid1))
+		return error("expected object ID, got '%s'", p + 1);
+	p += GIT_SHA1_HEXSZ;
+	if (*p != ' ')
+		return error("expected ' ', got '%c'", *p);
+	if (get_oid_hex(++p, oid2))
+		return error("expected object ID, got '%s'", p + 1);
+	p += GIT_SHA1_HEXSZ;
+	if (*p != ' ')
+		return error("expected ' ', got '%c'", *p);
+	*status = *++p;
+	if (!status || p[1])
+		return error("unexpected trailer: '%s'", p);
+	return 0;
+}
+
+/*
+ * Remove any trailing slash from $workdir
+ * before starting to avoid double slashes in symlink targets.
+ */
+static void add_path(struct strbuf *buf, size_t base_len, const char *path)
+{
+	strbuf_setlen(buf, base_len);
+	if (buf->len && buf->buf[buf->len - 1] != '/')
+		strbuf_addch(buf, '/');
+	strbuf_addstr(buf, path);
+}
+
+/*
+ * Determine whether we can simply reuse the file in the worktree.
+ */
+static int use_wt_file(const char *workdir, const char *name,
+		       struct object_id *oid)
+{
+	struct strbuf buf = STRBUF_INIT;
+	struct stat st;
+	int use = 0;
+
+	strbuf_addstr(&buf, workdir);
+	add_path(&buf, buf.len, name);
+
+	if (!lstat(buf.buf, &st) && !S_ISLNK(st.st_mode)) {
+		struct object_id wt_oid;
+		int fd = open(buf.buf, O_RDONLY);
+
+		if (!index_fd(wt_oid.hash, fd, &st, OBJ_BLOB, name, 0)) {
+			if (is_null_oid(oid)) {
+				oidcpy(oid, &wt_oid);
+				use = 1;
+			} else if (!oidcmp(oid, &wt_oid))
+				use = 1;
+		}
+	}
+
+	strbuf_release(&buf);
+
+	return use;
+}
+
+struct working_tree_entry {
+	struct hashmap_entry entry;
+	char path[FLEX_ARRAY];
+};
+
+static int working_tree_entry_cmp(struct working_tree_entry *a,
+				  struct working_tree_entry *b, void *keydata)
+{
+	return strcmp(a->path, b->path);
+}
+
+/*
+ * The `left` and `right` entries hold paths for the symlinks hashmap,
+ * and a SHA-1 surrounded by brief text for submodules.
+ */
+struct pair_entry {
+	struct hashmap_entry entry;
+	char left[PATH_MAX], right[PATH_MAX];
+	const char path[FLEX_ARRAY];
+};
+
+static int pair_cmp(struct pair_entry *a, struct pair_entry *b, void *keydata)
+{
+	return strcmp(a->path, b->path);
+}
+
+static void add_left_or_right(struct hashmap *map, const char *path,
+			      const char *content, int is_right)
+{
+	struct pair_entry *e, *existing;
+
+	FLEX_ALLOC_STR(e, path, path);
+	hashmap_entry_init(e, strhash(path));
+	existing = hashmap_get(map, e, NULL);
+	if (existing) {
+		free(e);
+		e = existing;
+	} else {
+		e->left[0] = e->right[0] = '\0';
+		hashmap_add(map, e);
+	}
+	strcpy(is_right ? e->right : e->left, content);
+}
+
+struct path_entry {
+	struct hashmap_entry entry;
+	char path[FLEX_ARRAY];
+};
+
+int path_entry_cmp(struct path_entry *a, struct path_entry *b, void *key)
+{
+	return strcmp(a->path, key ? key : b->path);
+}
+
+static void changed_files(struct hashmap *result, const char *index_path,
+			  const char *workdir)
+{
+	struct child_process update_index = CHILD_PROCESS_INIT;
+	struct child_process diff_files = CHILD_PROCESS_INIT;
+	struct strbuf index_env = STRBUF_INIT, buf = STRBUF_INIT;
+	const char *git_dir = absolute_path(get_git_dir()), *env[] = {
+		NULL, NULL
+	};
+	FILE *fp;
+
+	strbuf_addf(&index_env, "GIT_INDEX_FILE=%s", index_path);
+	env[0] = index_env.buf;
+
+	argv_array_pushl(&update_index.args,
+			 "--git-dir", git_dir, "--work-tree", workdir,
+			 "update-index", "--really-refresh", "-q",
+			 "--unmerged", NULL);
+	update_index.no_stdin = 1;
+	update_index.no_stdout = 1;
+	update_index.no_stderr = 1;
+	update_index.git_cmd = 1;
+	update_index.use_shell = 0;
+	update_index.clean_on_exit = 1;
+	update_index.dir = workdir;
+	update_index.env = env;
+	/* Ignore any errors of update-index */
+	run_command(&update_index);
+
+	argv_array_pushl(&diff_files.args,
+			 "--git-dir", git_dir, "--work-tree", workdir,
+			 "diff-files", "--name-only", "-z", NULL);
+	diff_files.no_stdin = 1;
+	diff_files.git_cmd = 1;
+	diff_files.use_shell = 0;
+	diff_files.clean_on_exit = 1;
+	diff_files.out = -1;
+	diff_files.dir = workdir;
+	diff_files.env = env;
+	if (start_command(&diff_files))
+		die("could not obtain raw diff");
+	fp = xfdopen(diff_files.out, "r");
+	while (!strbuf_getline_nul(&buf, fp)) {
+		struct path_entry *entry;
+		FLEX_ALLOC_STR(entry, path, buf.buf);
+		hashmap_entry_init(entry, strhash(buf.buf));
+		hashmap_add(result, entry);
+	}
+	if (finish_command(&diff_files))
+		die("diff-files did not exit properly");
+	strbuf_release(&index_env);
+	strbuf_release(&buf);
+}
+
+#include "dir.h"
+
+static NORETURN void exit_cleanup(const char *tmpdir, int exit_code)
+{
+	struct strbuf buf = STRBUF_INIT;
+	strbuf_addstr(&buf, tmpdir);
+	remove_dir_recursively(&buf, 0);
+	if (exit_code)
+		warning(_("failed: %d"), exit_code);
+	exit(exit_code);
+}
+
+static int ensure_leading_directories(char *path)
+{
+	switch (safe_create_leading_directories(path)) {
+		case SCLD_OK:
+		case SCLD_EXISTS:
+			return 0;
+		default:
+			return error(_("could not create leading directories "
+				       "of '%s'"), path);
+	}
+}
+
+static int run_dir_diff(const char *extcmd, int symlinks,
+			int argc, const char **argv)
+{
+	char tmpdir[PATH_MAX];
+	struct strbuf info = STRBUF_INIT, lpath = STRBUF_INIT;
+	struct strbuf rpath = STRBUF_INIT, buf = STRBUF_INIT;
+	struct strbuf ldir = STRBUF_INIT, rdir = STRBUF_INIT;
+	struct strbuf wtdir = STRBUF_INIT;
+	size_t ldir_len, rdir_len, wtdir_len;
+	struct cache_entry *ce = xcalloc(1, sizeof(ce) + PATH_MAX + 1);
+	const char *workdir, *tmp;
+	int ret = 0, i;
+	FILE *fp;
+	struct hashmap working_tree_dups, submodules, symlinks2;
+	struct hashmap_iter iter;
+	struct pair_entry *entry;
+	enum object_type type;
+	unsigned long size;
+	struct index_state wtindex;
+	struct checkout lstate, rstate;
+	int rc, flags = RUN_GIT_CMD, err = 0;
+	struct child_process child = CHILD_PROCESS_INIT;
+	const char *helper_argv[] = { "difftool--helper", NULL, NULL, NULL };
+	struct hashmap wt_modified, tmp_modified;
+	int indices_loaded = 0;
+
+	setup_work_tree();
+	workdir = get_git_work_tree();
+
+	/* Setup temp directories */
+	tmp = getenv("TMPDIR");
+	sprintf(tmpdir, "%s/git-difftool.XXXXXX", tmp ? tmp : "/tmp");
+	if (!mkdtemp(tmpdir))
+		return error("could not create temporary directory");
+	strbuf_addf(&ldir, "%s/left/", tmpdir);
+	strbuf_addf(&rdir, "%s/right/", tmpdir);
+	strbuf_addstr(&wtdir, workdir);
+	if (!wtdir.len || !is_dir_sep(wtdir.buf[wtdir.len - 1]))
+		strbuf_addch(&wtdir, '/');
+	mkdir(ldir.buf, 0777);
+	mkdir(rdir.buf, 0777);
+
+	memset(&wtindex, 0, sizeof(wtindex));
+
+	memset(&lstate, 0, sizeof(lstate));
+	lstate.base_dir = ldir.buf;
+	lstate.base_dir_len = ldir.len;
+	lstate.force = 1;
+	memset(&rstate, 0, sizeof(rstate));
+	rstate.base_dir = rdir.buf;
+	rstate.base_dir_len = rdir.len;
+	rstate.force = 1;
+
+	ldir_len = ldir.len;
+	rdir_len = rdir.len;
+	wtdir_len = wtdir.len;
+
+	hashmap_init(&working_tree_dups,
+		     (hashmap_cmp_fn)working_tree_entry_cmp, 0);
+	hashmap_init(&submodules, (hashmap_cmp_fn)pair_cmp, 0);
+	hashmap_init(&symlinks2, (hashmap_cmp_fn)pair_cmp, 0);
+
+	child.no_stdin = 1;
+	child.git_cmd = 1;
+	child.use_shell = 0;
+	child.clean_on_exit = 1;
+	child.out = -1;
+	argv_array_pushl(&child.args, "diff", "--raw", "--no-abbrev", "-z",
+			 NULL);
+	for (i = 0; i < argc; i++)
+		argv_array_push(&child.args, argv[i]);
+	if (start_command(&child))
+		die("could not obtain raw diff");
+	fp = xfdopen(child.out, "r");
+
+	/* Build index info for left and right sides of the diff */
+	while (!strbuf_getline_nul(&info, fp)) {
+		int lmode, rmode;
+		struct object_id loid, roid;
+		char status;
+		const char *src_path, *dst_path;
+		size_t src_path_len, dst_path_len;
+
+		if (starts_with(info.buf, "::"))
+			die(N_("combined diff formats('-c' and '--cc') are "
+			       "not supported in\n"
+			       "directory diff mode('-d' and '--dir-diff')."));
+
+		if (parse_index_info(info.buf, &lmode, &rmode, &loid, &roid,
+				     &status))
+			break;
+		if (strbuf_getline_nul(&lpath, fp))
+			break;
+		src_path = lpath.buf;
+		src_path_len = lpath.len;
+
+		if (status != 'C' && status != 'R') {
+			dst_path = src_path;
+			dst_path_len = src_path_len;
+		} else {
+			if (strbuf_getline_nul(&rpath, fp))
+				break;
+			dst_path = rpath.buf;
+			dst_path_len = rpath.len;
+		}
+
+		if (S_ISGITLINK(lmode) || S_ISGITLINK(rmode)) {
+			strbuf_reset(&buf);
+			strbuf_addf(&buf, "Subproject commit %s",
+				    oid_to_hex(&loid));
+			add_left_or_right(&submodules, src_path, buf.buf, 0);
+			strbuf_reset(&buf);
+			strbuf_addf(&buf, "Subproject commit %s",
+				    oid_to_hex(&roid));
+			if (!oidcmp(&loid, &roid))
+				strbuf_addstr(&buf, "-dirty");
+			add_left_or_right(&submodules, dst_path, buf.buf, 1);
+			continue;
+		}
+
+		if (S_ISLNK(lmode)) {
+			char *content = read_sha1_file(loid.hash, &type, &size);
+			add_left_or_right(&symlinks2, src_path, content, 0);
+			free(content);
+		}
+
+		if (S_ISLNK(rmode)) {
+			char *content = read_sha1_file(roid.hash, &type, &size);
+			add_left_or_right(&symlinks2, dst_path, content, 1);
+			free(content);
+		}
+
+		if (lmode && status != 'C') {
+			ce->ce_mode = lmode;
+			oidcpy(&ce->oid, &loid);
+			strcpy(ce->name, src_path);
+			ce->ce_namelen = src_path_len;
+			if (checkout_entry(ce, &lstate, NULL))
+				return error("could not write '%s'", src_path);
+		}
+
+		if (rmode) {
+			struct working_tree_entry *entry;
+
+			/* Avoid duplicate working_tree entries */
+			FLEX_ALLOC_STR(entry, path, dst_path);
+			hashmap_entry_init(entry, strhash(dst_path));
+			if (hashmap_get(&working_tree_dups, entry, NULL)) {
+				free(entry);
+				continue;
+			}
+			hashmap_add(&working_tree_dups, entry);
+
+			if (!use_wt_file(workdir, dst_path, &roid)) {
+				ce->ce_mode = rmode;
+				oidcpy(&ce->oid, &roid);
+				strcpy(ce->name, dst_path);
+				ce->ce_namelen = dst_path_len;
+				if (checkout_entry(ce, &rstate, NULL))
+					return error("could not write '%s'",
+						     dst_path);
+			} else if (!is_null_oid(&roid)) {
+				/*
+				 * Changes in the working tree need special
+				 * treatment since they are not part of the
+				 * index.
+				 */
+				struct cache_entry *ce2 =
+					make_cache_entry(rmode, roid.hash,
+							 dst_path, 0, 0);
+				ce_mode_from_stat(ce2, rmode);
+
+				add_index_entry(&wtindex, ce2,
+						ADD_CACHE_JUST_APPEND);
+
+				add_path(&wtdir, wtdir_len, dst_path);
+				add_path(&rdir, rdir_len, dst_path);
+				if (ensure_leading_directories(rdir.buf))
+					return error("could not create "
+						     "directory for '%s'",
+						     dst_path);
+				if (symlinks) {
+					if (symlink(wtdir.buf, rdir.buf)) {
+						ret = error_errno("could not symlink '%s' to '%s'", wtdir.buf, rdir.buf);
+						goto finish;
+					}
+				} else {
+					struct stat st;
+					if (stat(wtdir.buf, &st))
+						st.st_mode = 0644;
+					if (copy_file(rdir.buf, wtdir.buf,
+						      st.st_mode)) {
+						ret = error("could not copy '%s' to '%s'", wtdir.buf, rdir.buf);
+						goto finish;
+					}
+				}
+			}
+		}
+	}
+	if (finish_command(&child)) {
+		ret = error("error occurred running diff --raw");
+		goto finish;
+	}
+
+	/*
+	 * Changes to submodules require special treatment.This loop writes a
+	 * temporary file to both the left and right directories to show the
+	 * change in the recorded SHA1 for the submodule.
+	 */
+	hashmap_iter_init(&submodules, &iter);
+	while ((entry = hashmap_iter_next(&iter))) {
+		if (*entry->left) {
+			add_path(&ldir, ldir_len, entry->path);
+			ensure_leading_directories(ldir.buf);
+			write_file(ldir.buf, "%s", entry->left);
+		}
+		if (*entry->right) {
+			add_path(&rdir, rdir_len, entry->path);
+			ensure_leading_directories(rdir.buf);
+			write_file(rdir.buf, "%s", entry->right);
+		}
+	}
+
+	/*
+	 * Symbolic links require special treatment.The standard "git diff"
+	 * shows only the link itself, not the contents of the link target.
+	 * This loop replicates that behavior.
+	 */
+	hashmap_iter_init(&symlinks2, &iter);
+	while ((entry = hashmap_iter_next(&iter))) {
+		if (*entry->left) {
+			add_path(&ldir, ldir_len, entry->path);
+			ensure_leading_directories(ldir.buf);
+			write_file(ldir.buf, "%s", entry->left);
+		}
+		if (*entry->right) {
+			add_path(&rdir, rdir_len, entry->path);
+			ensure_leading_directories(rdir.buf);
+			write_file(rdir.buf, "%s", entry->right);
+		}
+	}
+
+	strbuf_release(&buf);
+
+	strbuf_setlen(&ldir, ldir_len);
+	helper_argv[1] = ldir.buf;
+	strbuf_setlen(&rdir, rdir_len);
+	helper_argv[2] = rdir.buf;
+
+	if (extcmd) {
+		helper_argv[0] = extcmd;
+		flags = 0;
+	} else
+		setenv("GIT_DIFFTOOL_DIRDIFF", "true", 1);
+	rc = run_command_v_opt(helper_argv, flags);
+
+	/*
+	 * If the diff includes working copy files and those
+	 * files were modified during the diff, then the changes
+	 * should be copied back to the working tree.
+	 * Do not copy back files when symlinks are used and the
+	 * external tool did not replace the original link with a file.
+	 *
+	 * These hashes are loaded lazily since they aren't needed
+	 * in the common case of --symlinks and the difftool updating
+	 * files through the symlink.
+	 */
+	hashmap_init(&wt_modified, (hashmap_cmp_fn)path_entry_cmp,
+		     wtindex.cache_nr);
+	hashmap_init(&tmp_modified, (hashmap_cmp_fn)path_entry_cmp,
+		     wtindex.cache_nr);
+
+	for (i = 0; i < wtindex.cache_nr; i++) {
+		struct hashmap_entry dummy;
+		const char *name = wtindex.cache[i]->name;
+		struct stat st;
+
+		add_path(&rdir, rdir_len, name);
+		if (lstat(rdir.buf, &st))
+			continue;
+
+		if ((symlinks && S_ISLNK(st.st_mode)) || !S_ISREG(st.st_mode))
+			continue;
+
+		if (!indices_loaded) {
+			static struct lock_file lock;
+			strbuf_reset(&buf);
+			strbuf_addf(&buf, "%s/wtindex", tmpdir);
+			if (hold_lock_file_for_update(&lock, buf.buf, 0) < 0 ||
+			    write_locked_index(&wtindex, &lock, COMMIT_LOCK)) {
+				ret = error("could not write %s", buf.buf);
+				rollback_lock_file(&lock);
+				goto finish;
+			}
+			changed_files(&wt_modified, buf.buf, workdir);
+			strbuf_setlen(&rdir, rdir_len);
+			changed_files(&tmp_modified, buf.buf, rdir.buf);
+			add_path(&rdir, rdir_len, name);
+			indices_loaded = 1;
+		}
+
+		hashmap_entry_init(&dummy, strhash(name));
+		if (hashmap_get(&tmp_modified, &dummy, name)) {
+			add_path(&wtdir, wtdir_len, name);
+			if (hashmap_get(&wt_modified, &dummy, name)) {
+				warning(_("both files modified: '%s' and '%s'."),
+					wtdir.buf, rdir.buf);
+				warning(_("working tree file has been left."));
+				warning("");
+				err = 1;
+			} else if (unlink(wtdir.buf) ||
+				   copy_file(wtdir.buf, rdir.buf, st.st_mode))
+				warning_errno(_("could not copy '%s' to '%s'"),
+					      rdir.buf, wtdir.buf);
+		}
+	}
+
+	if (err) {
+		warning(_("temporary files exist in '%s'."), tmpdir);
+		warning(_("you may want to cleanup or recover these."));
+		exit(1);
+	} else
+		exit_cleanup(tmpdir, rc);
+
+finish:
+	free(ce);
+	strbuf_release(&ldir);
+	strbuf_release(&rdir);
+	strbuf_release(&wtdir);
+	strbuf_release(&buf);
+
+	return ret;
+}
+
+static int run_file_diff(int prompt, int argc, const char **argv)
+{
+	struct argv_array args = ARGV_ARRAY_INIT;
+	const char *env[] = {
+		"GIT_PAGER=", "GIT_EXTERNAL_DIFF=git-difftool--helper", NULL,
+		NULL
+	};
+	int ret = 0, i;
+
+	if (prompt > 0)
+		env[2] = "GIT_DIFFTOOL_PROMPT=true";
+	else if (!prompt)
+		env[2] = "GIT_DIFFTOOL_NO_PROMPT=true";
+
+	argv_array_push(&args, "diff");
+	for (i = 0; i < argc; i++)
+		argv_array_push(&args, argv[i]);
+	ret = run_command_v_opt_cd_env(args.argv, RUN_GIT_CMD, NULL, env);
+	exit(ret);
+}
+
+int cmd_builtin_difftool(int argc, const char ** argv, const char * prefix)
+{
+	int use_gui_tool = 0, dir_diff = 0, prompt = -1, symlinks = 0,
+	    tool_help = 0;
+	static char *difftool_cmd = NULL, *extcmd = NULL;
+
+	struct option builtin_difftool_options[] = {
+		OPT_BOOL('g', "gui", &use_gui_tool,
+			 N_("use `diff.guitool` instead of `diff.tool`")),
+		OPT_BOOL('d', "dir-diff", &dir_diff,
+			 N_("perform a full-directory diff")),
+		{ OPTION_SET_INT, 'y', "no-prompt", &prompt, NULL,
+			N_("do not prompt before launching a diff tool"),
+			PARSE_OPT_NOARG | PARSE_OPT_NONEG, NULL, 0},
+		{ OPTION_SET_INT, 0, "prompt", &prompt, NULL, NULL,
+			PARSE_OPT_NOARG | PARSE_OPT_NONEG | PARSE_OPT_HIDDEN,
+			NULL, 1 },
+		OPT_BOOL(0, "symlinks", &symlinks,
+			 N_("use symlinks in dir-diff mode")),
+		OPT_STRING('t', "tool", &difftool_cmd, N_("<tool>"),
+			   N_("use the specified diff tool")),
+		OPT_BOOL(0, "tool-help", &tool_help,
+			 N_("print a list of diff tools that may be used with "
+			    "`--tool`")),
+		OPT_BOOL(0, "trust-exit-code", &trust_exit_code,
+			 N_("make 'git-difftool' exit when an invoked diff "
+			    "tool returns a non - zero exit code")),
+		OPT_STRING('x', "extcmd", &extcmd, N_("<command>"),
+			   N_("specify a custom command for viewing diffs")),
+		OPT_END()
+	};
+
+	symlinks = has_symlinks;
+
+	git_config(difftool_config, NULL);
+
+	argc = parse_options(argc, argv, prefix, builtin_difftool_options,
+			     builtin_difftool_usage, PARSE_OPT_KEEP_UNKNOWN |
+			     PARSE_OPT_KEEP_DASHDASH);
+
+	if (tool_help)
+		return print_tool_help();
+
+	if (use_gui_tool && diff_gui_tool && *diff_gui_tool)
+		setenv("GIT_DIFF_TOOL", diff_gui_tool, 1);
+	else if (difftool_cmd) {
+		if (*difftool_cmd)
+			setenv("GIT_DIFF_TOOL", difftool_cmd, 1);
+		else
+			die(_("no <tool> given for --tool=<tool>"));
+	}
+
+	if (extcmd) {
+		if (*extcmd)
+			setenv("GIT_DIFFTOOL_EXTCMD", extcmd, 1);
+		else
+			die(_("no <cmd> given for --extcmd=<cmd>"));
+	}
+
+	setenv("GIT_DIFFTOOL_TRUST_EXIT_CODE",
+	       trust_exit_code ? "true" : "false", 1);
+
+	/*
+	 * In directory diff mode, 'git-difftool--helper' is called once
+	 * to compare the a / b directories.In file diff mode, 'git diff'
+	 * will invoke a separate instance of 'git-difftool--helper' for
+	 * each file that changed.
+	 */
+	if (dir_diff)
+		return run_dir_diff(extcmd, symlinks, argc, argv);
+	return run_file_diff(prompt, argc, argv);
+}
diff --git a/git.c b/git.c
index efa1059..eaa0f67 100644
--- a/git.c
+++ b/git.c
@@ -424,6 +424,7 @@ static struct cmd_struct commands[] = {
 	{ "diff-files", cmd_diff_files, RUN_SETUP | NEED_WORK_TREE },
 	{ "diff-index", cmd_diff_index, RUN_SETUP },
 	{ "diff-tree", cmd_diff_tree, RUN_SETUP },
+	{ "builtin-difftool", cmd_builtin_difftool, RUN_SETUP | NEED_WORK_TREE },
 	{ "fast-export", cmd_fast_export, RUN_SETUP },
 	{ "fetch", cmd_fetch, RUN_SETUP },
 	{ "fetch-pack", cmd_fetch_pack, RUN_SETUP },
-- 
2.10.1.583.g721a9e0

Re: [PATCH 1/2] difftool: add the builtin

From: David Aguilar <hidden>
Date: 2016-11-23 08:22:56

On Tue, Nov 22, 2016 at 06:01:23PM +0100, Johannes Schindelin wrote:
This adds a builtin difftool that represents a conversion of the current
Perl script version of the difftool.

The motivation is that Perl scripts are not at all native on Windows,
and that `git difftool` therefore is pretty slow on that platform, when
there is no good reason for it to be slow.

In addition, Perl does not really have access to Git's internals. That
means that any script will always have to jump through unnecessary
hoops.
Nice!
The current version of the builtin difftool does not, however, make full
use of the internals but instead chooses to spawn a couple of Git
processes, still, to make for an easier conversion. There remains a lot
of room for improvement, left for a later date.

Note: the original difftool is still called by `git difftool`. To get the
new, experimental version, call `git builtin-difftool`. The reason: this
new, experimental, builtin difftool will be shipped as part of Git for
Windows v2.11.0, to allow for easier large-scale testing.
I like this plan.  I was going to ask for an environment
variable (to preset in git-cola) but since Git for Windows is
handling it then everyone benefits.
quoted hunk
diff --git a/builtin/builtin-difftool.c b/builtin/builtin-difftool.c
new file mode 100644
index 0000000..9feefcd
--- /dev/null
+++ b/builtin/builtin-difftool.c
@@ -0,0 +1,680 @@
+/*
+ * "git difftool" builtin command
+ *
+ * This is a wrapper around the GIT_EXTERNAL_DIFF-compatible
+ * git-difftool--helper script.
+ *
+ * This script exports GIT_EXTERNAL_DIFF and GIT_PAGER for use by git.
+ * The GIT_DIFF* variables are exported for use by git-difftool--helper.
+ *
+ * Any arguments that are unknown to this script are forwarded to 'git diff'.
+ *
+ * Copyright (C) 2016 Johannes Schindelin
+ */
+#include "cache.h"
+#include "builtin.h"
+#include "parse-options.h"
+#include "run-command.h"
+#include "argv-array.h"
+#include "strbuf.h"
+#include "lockfile.h"
+
+static char *diff_gui_tool;
+static int trust_exit_code;
+
+static const char * const builtin_difftool_usage[] = {
+	N_("git add [<options>] [--] <pathspec>..."),
+	NULL
+};
The usage should probably say "difftool" (or "builtin-difftool").
[...]
+static void changed_files(struct hashmap *result, const char *index_path,
+			  const char *workdir)
+{
+[...]
+}
+
+#include "dir.h"
Can this mid-file #include go to the top of the file?
+static int run_dir_diff(const char *extcmd, int symlinks,
+			int argc, const char **argv)
+{
+	char tmpdir[PATH_MAX];
+	struct strbuf info = STRBUF_INIT, lpath = STRBUF_INIT;
+	struct strbuf rpath = STRBUF_INIT, buf = STRBUF_INIT;
+	struct strbuf ldir = STRBUF_INIT, rdir = STRBUF_INIT;
+	struct strbuf wtdir = STRBUF_INIT;
+	size_t ldir_len, rdir_len, wtdir_len;
+	struct cache_entry *ce = xcalloc(1, sizeof(ce) + PATH_MAX + 1);
+	const char *workdir, *tmp;
+	int ret = 0, i;
+	FILE *fp;
+	struct hashmap working_tree_dups, submodules, symlinks2;
+	struct hashmap_iter iter;
+	struct pair_entry *entry;
+	enum object_type type;
+	unsigned long size;
+	struct index_state wtindex;
+	struct checkout lstate, rstate;
+	int rc, flags = RUN_GIT_CMD, err = 0;
+	struct child_process child = CHILD_PROCESS_INIT;
+	const char *helper_argv[] = { "difftool--helper", NULL, NULL, NULL };
+	struct hashmap wt_modified, tmp_modified;
+	int indices_loaded = 0;
+
+	setup_work_tree();
+	workdir = get_git_work_tree();
+
+	/* Setup temp directories */
+	tmp = getenv("TMPDIR");
+	sprintf(tmpdir, "%s/git-difftool.XXXXXX", tmp ? tmp : "/tmp");
Maybe snprintf instead?

getenv() won't return anything longer than PATH_MAX for most
users, but users are weird.
+	if (!mkdtemp(tmpdir))
+		return error("could not create temporary directory");
Mention the tmpdir here?
+	strbuf_addf(&ldir, "%s/left/", tmpdir);
+	strbuf_addf(&rdir, "%s/right/", tmpdir);
+	strbuf_addstr(&wtdir, workdir);
+	if (!wtdir.len || !is_dir_sep(wtdir.buf[wtdir.len - 1]))
+		strbuf_addch(&wtdir, '/');
+	mkdir(ldir.buf, 0777);
+	mkdir(rdir.buf, 0777);
Seeing the perl mkpath() default 0777 spelled out this way
makes me wonder whether 0700 would be safer.

The mkdtemp() above is already using 0700 so it's ok, but it
might be worth making it consistent (later, perhaps).
[...]
+int cmd_builtin_difftool(int argc, const char ** argv, const char * prefix)
+{
+	int use_gui_tool = 0, dir_diff = 0, prompt = -1, symlinks = 0,
+	    tool_help = 0;
+	static char *difftool_cmd = NULL, *extcmd = NULL;
+
+	struct option builtin_difftool_options[] = {
+		OPT_BOOL('g', "gui", &use_gui_tool,
+			 N_("use `diff.guitool` instead of `diff.tool`")),
+		OPT_BOOL('d', "dir-diff", &dir_diff,
+			 N_("perform a full-directory diff")),
+		{ OPTION_SET_INT, 'y', "no-prompt", &prompt, NULL,
+			N_("do not prompt before launching a diff tool"),
+			PARSE_OPT_NOARG | PARSE_OPT_NONEG, NULL, 0},
+		{ OPTION_SET_INT, 0, "prompt", &prompt, NULL, NULL,
+			PARSE_OPT_NOARG | PARSE_OPT_NONEG | PARSE_OPT_HIDDEN,
+			NULL, 1 },
+		OPT_BOOL(0, "symlinks", &symlinks,
+			 N_("use symlinks in dir-diff mode")),
+		OPT_STRING('t', "tool", &difftool_cmd, N_("<tool>"),
+			   N_("use the specified diff tool")),
+		OPT_BOOL(0, "tool-help", &tool_help,
+			 N_("print a list of diff tools that may be used with "
+			    "`--tool`")),
+		OPT_BOOL(0, "trust-exit-code", &trust_exit_code,
+			 N_("make 'git-difftool' exit when an invoked diff "
+			    "tool returns a non - zero exit code")),
+		OPT_STRING('x', "extcmd", &extcmd, N_("<command>"),
+			   N_("specify a custom command for viewing diffs")),
+		OPT_END()
+	};
+
+	symlinks = has_symlinks;
+
+	git_config(difftool_config, NULL);
+
+	argc = parse_options(argc, argv, prefix, builtin_difftool_options,
+			     builtin_difftool_usage, PARSE_OPT_KEEP_UNKNOWN |
+			     PARSE_OPT_KEEP_DASHDASH);
+
+	if (tool_help)
+		return print_tool_help();
+
+	if (use_gui_tool && diff_gui_tool && *diff_gui_tool)
+		setenv("GIT_DIFF_TOOL", diff_gui_tool, 1);
+	else if (difftool_cmd) {
+		if (*difftool_cmd)
+			setenv("GIT_DIFF_TOOL", difftool_cmd, 1);
+		else
+			die(_("no <tool> given for --tool=<tool>"));
+	}
+
+	if (extcmd) {
+		if (*extcmd)
+			setenv("GIT_DIFFTOOL_EXTCMD", extcmd, 1);
+		else
+			die(_("no <cmd> given for --extcmd=<cmd>"));
+	}
+
+	setenv("GIT_DIFFTOOL_TRUST_EXIT_CODE",
+	       trust_exit_code ? "true" : "false", 1);
+
+	/*
+	 * In directory diff mode, 'git-difftool--helper' is called once
+	 * to compare the a / b directories.In file diff mode, 'git diff'
+	 * will invoke a separate instance of 'git-difftool--helper' for
+	 * each file that changed.
+	 */
Missing space after "." in the comment above.
quoted hunk
+	if (dir_diff)
+		return run_dir_diff(extcmd, symlinks, argc, argv);
+	return run_file_diff(prompt, argc, argv);
+}
diff --git a/git.c b/git.c
index efa1059..eaa0f67 100644
--- a/git.c
+++ b/git.c
@@ -424,6 +424,7 @@ static struct cmd_struct commands[] = {
 	{ "diff-files", cmd_diff_files, RUN_SETUP | NEED_WORK_TREE },
 	{ "diff-index", cmd_diff_index, RUN_SETUP },
 	{ "diff-tree", cmd_diff_tree, RUN_SETUP },
+	{ "builtin-difftool", cmd_builtin_difftool, RUN_SETUP | NEED_WORK_TREE },
 	{ "fast-export", cmd_fast_export, RUN_SETUP },
 	{ "fetch", cmd_fetch, RUN_SETUP },
 	{ "fetch-pack", cmd_fetch_pack, RUN_SETUP },
This isn't alphabetical anymore, but it actually is if you
consider that the final plan is to change "builtin-difftool" to
"difftool".

If we want to minimize that future diff we could name
cmd_builtin_difftool() as cmd_difftool() for consistency now so
that the future commit only needs to tweak the string here.
-- 
David

Re: [PATCH 1/2] difftool: add the builtin

From: Johannes Schindelin <hidden>
Date: 2016-11-23 11:41:14

Hi David,

On Wed, 23 Nov 2016, David Aguilar wrote:
On Tue, Nov 22, 2016 at 06:01:23PM +0100, Johannes Schindelin wrote:
quoted
+static const char * const builtin_difftool_usage[] = {
+	N_("git add [<options>] [--] <pathspec>..."),
+	NULL
+};
The usage should probably say "difftool" (or "builtin-difftool").
Ah, my dirty secret was spilled. I copy-edited this. *pours ashes over his
head*
quoted
[...]
+static void changed_files(struct hashmap *result, const char *index_path,
+			  const char *workdir)
+{
+[...]
+}
+
+#include "dir.h"
Can this mid-file #include go to the top of the file?
Yep, thanks.

In case you are interested: You probably guessed it, it was left for a
later clean-up. I worked a bit over the last weeks on getting Git to build
in Visual Studio, to be able to benefit from its quite nice features (I
was always a fan of Visual Studio, long before I started working at
Microsoft). I used the conversion of the difftool as an excuse to make use
of this myself: I did the entire conversion in Visual Studio, reverting to
the old, tedious command-line driven workflow to fix the bugs identified
by t7800-difftool.sh.
quoted
+static int run_dir_diff(const char *extcmd, int symlinks,
+			int argc, const char **argv)
+{
+	char tmpdir[PATH_MAX];
+	struct strbuf info = STRBUF_INIT, lpath = STRBUF_INIT;
+	struct strbuf rpath = STRBUF_INIT, buf = STRBUF_INIT;
+	struct strbuf ldir = STRBUF_INIT, rdir = STRBUF_INIT;
+	struct strbuf wtdir = STRBUF_INIT;
+	size_t ldir_len, rdir_len, wtdir_len;
+	struct cache_entry *ce = xcalloc(1, sizeof(ce) + PATH_MAX + 1);
+	const char *workdir, *tmp;
+	int ret = 0, i;
+	FILE *fp;
+	struct hashmap working_tree_dups, submodules, symlinks2;
+	struct hashmap_iter iter;
+	struct pair_entry *entry;
+	enum object_type type;
+	unsigned long size;
+	struct index_state wtindex;
+	struct checkout lstate, rstate;
+	int rc, flags = RUN_GIT_CMD, err = 0;
+	struct child_process child = CHILD_PROCESS_INIT;
+	const char *helper_argv[] = { "difftool--helper", NULL, NULL, NULL };
+	struct hashmap wt_modified, tmp_modified;
+	int indices_loaded = 0;
+
+	setup_work_tree();
+	workdir = get_git_work_tree();
+
+	/* Setup temp directories */
+	tmp = getenv("TMPDIR");
+	sprintf(tmpdir, "%s/git-difftool.XXXXXX", tmp ? tmp : "/tmp");
Maybe snprintf instead?

getenv() won't return anything longer than PATH_MAX for most
users, but users are weird.
True.
quoted
+	if (!mkdtemp(tmpdir))
+		return error("could not create temporary directory");
Mention the tmpdir here?
Sure thing.
quoted
+	strbuf_addf(&ldir, "%s/left/", tmpdir);
+	strbuf_addf(&rdir, "%s/right/", tmpdir);
+	strbuf_addstr(&wtdir, workdir);
+	if (!wtdir.len || !is_dir_sep(wtdir.buf[wtdir.len - 1]))
+		strbuf_addch(&wtdir, '/');
+	mkdir(ldir.buf, 0777);
+	mkdir(rdir.buf, 0777);
Seeing the perl mkpath() default 0777 spelled out this way
makes me wonder whether 0700 would be safer.

The mkdtemp() above is already using 0700 so it's ok, but it
might be worth making it consistent (later, perhaps).
Ah, of course! I stupidly imitated other `mkdir()` calls elsewhere, but
they refer to directories within the Git worktree...
quoted
+	/*
+	 * In directory diff mode, 'git-difftool--helper' is called once
+	 * to compare the a / b directories.In file diff mode, 'git diff'
+	 * will invoke a separate instance of 'git-difftool--helper' for
+	 * each file that changed.
+	 */
Missing space after "." in the comment above.
Yep. It was two spaces and I deleted one too many (we are so way past
actual print, where the two spaces may have made sense...).
quoted
+	if (dir_diff)
+		return run_dir_diff(extcmd, symlinks, argc, argv);
+	return run_file_diff(prompt, argc, argv);
+}
diff --git a/git.c b/git.c
index efa1059..eaa0f67 100644
--- a/git.c
+++ b/git.c
@@ -424,6 +424,7 @@ static struct cmd_struct commands[] = {
 	{ "diff-files", cmd_diff_files, RUN_SETUP | NEED_WORK_TREE },
 	{ "diff-index", cmd_diff_index, RUN_SETUP },
 	{ "diff-tree", cmd_diff_tree, RUN_SETUP },
+	{ "builtin-difftool", cmd_builtin_difftool, RUN_SETUP | NEED_WORK_TREE },
 	{ "fast-export", cmd_fast_export, RUN_SETUP },
 	{ "fetch", cmd_fetch, RUN_SETUP },
 	{ "fetch-pack", cmd_fetch_pack, RUN_SETUP },
This isn't alphabetical anymore, but it actually is if you
consider that the final plan is to change "builtin-difftool" to
"difftool".
Exactly, that was my thinking.
If we want to minimize that future diff we could name
cmd_builtin_difftool() as cmd_difftool() for consistency now so
that the future commit only needs to tweak the string here.
Yes!

For the record, this is a left-over from an impatient attempt at avoiding
problems with `make` overwriting the Perl version of `git difftool` by the
builtin version; I had originally assumed that a list of builtins was
generated from parsing git.c or builtin.h, but it turns out that the
BUILTIN_OBJS are actually responsible, i.e. the file name.

Fixed.

Thank you for your review!
Dscho

[PATCH 2/2] difftool: add a feature flag for the builtin vs scripted version

From: Johannes Schindelin <hidden>
Date: 2016-11-22 17:02:10

The popular difftool command was just converted into a builtin, for
better performance on Windows as well as to reduce the number of Perl
scripts (so that we may, in the very long run, be able to ship Git for
Windows without any Perl interpreter at all).

However, it would be sloppy practice to simply switch over from the
tried-and-tested (albeit slow) scripted version to the spanking new
builtin version that has not seen a whole lot of real-world testing.

So let's add a feature flag.

If the file `use-builtin-difftool` exists in Git's exec path, Git will
now automagically use the builtin version of the difftool, without
requiring the user to call `git builtin-difftool <args>`. This comes in
particularly handy when the difftool command is used from within
scripts.

If the file `use-builtin-difftool` is absent from Git's exec path, which
is the default, Git will use the scripted version as before.

The original idea was to use an environment variable
GIT_USE_BUILTIN_DIFFTOOL, but the test suite resets those variables, and
we do want to use that feature flag to run the tests with, and without,
the feature flag.

Besides, the plan is to add an opt-in flag in Git for Windows'
installer. If we implemented the feature flag as an environment
variable, we would have to modify the user's environment, in order to
make the builtin difftool the default when called from Git Bash, Git CMD
or third-party tools.

Signed-off-by: Johannes Schindelin <redacted>
---
 .gitignore        |  1 +
 git-difftool.perl |  7 +++++++
 git.c             | 20 ++++++++++++++++++++
 3 files changed, 28 insertions(+)
diff --git a/.gitignore b/.gitignore
index 4f54531..91bfd09 100644
--- a/.gitignore
+++ b/.gitignore
@@ -1,3 +1,4 @@
+/use-builtin-difftool
 /GIT-BUILD-OPTIONS
 /GIT-CFLAGS
 /GIT-LDFLAGS
diff --git a/git-difftool.perl b/git-difftool.perl
index a5790d0..28e47d8 100755
--- a/git-difftool.perl
+++ b/git-difftool.perl
@@ -23,6 +23,13 @@ use File::Temp qw(tempdir);
 use Getopt::Long qw(:config pass_through);
 use Git;
 
+if (-e Git::exec_path() . '/use-builtin-difftool') {
+	unshift(@ARGV, "builtin-difftool");
+	unshift(@ARGV, "git");
+	exec(@ARGV);
+	die("Could not execute builtin difftool");
+}
+
 sub usage
 {
 	my $exitcode = shift;
diff --git a/git.c b/git.c
index eaa0f67..7a0df7a 100644
--- a/git.c
+++ b/git.c
@@ -2,6 +2,7 @@
 #include "exec_cmd.h"
 #include "help.h"
 #include "run-command.h"
+#include "dir.h"
 
 const char git_usage_string[] =
 	"git [--version] [--help] [-C <path>] [-c name=value]\n"
@@ -542,6 +543,22 @@ static void strip_extension(const char **argv)
 #define strip_extension(cmd)
 #endif
 
+static int use_builtin_difftool(void)
+{
+	static int initialized, use;
+
+	if (!initialized) {
+		struct strbuf buf = STRBUF_INIT;
+		strbuf_addf(&buf, "%s/%s", git_exec_path(),
+			    "use-builtin-difftool");
+		use = file_exists(buf.buf);
+		strbuf_release(&buf);
+		initialized = 1;
+	}
+
+	return use;
+}
+
 static void handle_builtin(int argc, const char **argv)
 {
 	struct argv_array args = ARGV_ARRAY_INIT;
@@ -551,6 +568,9 @@ static void handle_builtin(int argc, const char **argv)
 	strip_extension(argv);
 	cmd = argv[0];
 
+	if (!strcmp("difftool", cmd) && use_builtin_difftool())
+		cmd = "builtin-difftool";
+
 	/* Turn "git cmd --help" into "git help --exclude-guides cmd" */
 	if (argc > 1 && !strcmp(argv[1], "--help")) {
 		int i;
-- 
2.10.1.583.g721a9e0

Re: [PATCH 2/2] difftool: add a feature flag for the builtin vs scripted version

From: Dennis Kaarsemaker <hidden>
Date: 2016-11-23 14:52:01

On Tue, 2016-11-22 at 18:01 +0100, Johannes Schindelin wrote:
The original idea was to use an environment variable
GIT_USE_BUILTIN_DIFFTOOL, but the test suite resets those variables, and
we do want to use that feature flag to run the tests with, and without,
the feature flag.

Besides, the plan is to add an opt-in flag in Git for Windows'
installer. If we implemented the feature flag as an environment
variable, we would have to modify the user's environment, in order to
make the builtin difftool the default when called from Git Bash, Git CMD
or third-party tools.
Hi Johannes,

Why is this not a normal configuration variable (as in git config
difftool.builtin true or something)? It doesn't make much sense to me
to introduce a way of configuring git by introducing magic files, when
a normal configuration variable would do just fine, and the GfW
installer can also set such variables, like it does for the crlf config
I believe.

-- 
Dennis Kaarsemaker
http://www.kaarsemaker.net

Re: [PATCH 2/2] difftool: add a feature flag for the builtin vs scripted version

From: Johannes Schindelin <hidden>
Date: 2016-11-23 17:30:24

Hi Dennis,

On Wed, 23 Nov 2016, Dennis Kaarsemaker wrote:
On Tue, 2016-11-22 at 18:01 +0100, Johannes Schindelin wrote:
quoted
The original idea was to use an environment variable
GIT_USE_BUILTIN_DIFFTOOL, but the test suite resets those variables, and
we do want to use that feature flag to run the tests with, and without,
the feature flag.

Besides, the plan is to add an opt-in flag in Git for Windows'
installer. If we implemented the feature flag as an environment
variable, we would have to modify the user's environment, in order to
make the builtin difftool the default when called from Git Bash, Git CMD
or third-party tools.
Why is this not a normal configuration variable (as in git config
difftool.builtin true or something)? It doesn't make much sense to me
to introduce a way of configuring git by introducing magic files, when
a normal configuration variable would do just fine, and the GfW
installer can also set such variables, like it does for the crlf config
I believe.
I considered that. Adding a config setting would mean we simply test for
it in git-difftool.perl and call the builtin if the setting is active,
right?

The downside is that we actually *do* go through Perl to do that. Only to
go back to a builtin. Which is exactly the thing I intended to avoid.

If we do not go through Perl, we have to set up the git directory and
parse the config in git.c *just* to figure out whether we want to
magically forward difftool to builtin-difftool. That is not only ugly, but
has potential side effects I was not willing to risk.

In any case, this feature flag will be there only for one or two Git for
Windows releases, to give early adopters a chance to send me bug reports
about any regressions.

To be crystal-clear: I never expected this patch to enter git.git.

In that light, I am okay with taking the heat for introducing a temporary,
Git for Windows-only feature flag that is implemented as a "does the file
<xyz> exist?" test.

Ciao,
Dscho

Re: [PATCH 2/2] difftool: add a feature flag for the builtin vs scripted version

From: Johannes Schindelin <hidden>
Date: 2016-11-23 22:01:58

Hi Dennis,

On Wed, 23 Nov 2016, Johannes Schindelin wrote:
On Wed, 23 Nov 2016, Dennis Kaarsemaker wrote:
quoted
On Tue, 2016-11-22 at 18:01 +0100, Johannes Schindelin wrote:
quoted
The original idea was to use an environment variable
GIT_USE_BUILTIN_DIFFTOOL, but the test suite resets those variables, and
we do want to use that feature flag to run the tests with, and without,
the feature flag.

Besides, the plan is to add an opt-in flag in Git for Windows'
installer. If we implemented the feature flag as an environment
variable, we would have to modify the user's environment, in order to
make the builtin difftool the default when called from Git Bash, Git CMD
or third-party tools.
Why is this not a normal configuration variable (as in git config
difftool.builtin true or something)? It doesn't make much sense to me
to introduce a way of configuring git by introducing magic files, when
a normal configuration variable would do just fine, and the GfW
installer can also set such variables, like it does for the crlf config
I believe.
I considered that. Adding a config setting would mean we simply test for
it in git-difftool.perl and call the builtin if the setting is active,
right?

The downside is that we actually *do* go through Perl to do that. Only to
go back to a builtin. Which is exactly the thing I intended to avoid.
Okay, I reconsidered. Junio's comment about how git-am did it made me
rethink the issue: I need not keep the name "difftool" for the script. So
what I do now is rename the Perl script to git-legacy-difftool and always
read the config in the builtin difftool, handing off to the legacy
difftool unless core.useBuiltinDifftool=true.

This is an easy way to do it, and a portable and clean blueprint for
similar feature-flags in the future.

Ciao,
Johannes

[PATCH v2 0/1] Show Git Mailing List: a builtin difftool

From: Johannes Schindelin <hidden>
Date: 2016-11-23 22:03:22

I have been working on the builtin difftool for almost two weeks,
for two reasons:

1. Perl is really not native on Windows. Not only is there a performance
   penalty to be paid just for running Perl scripts, we also have to deal
   with the fact that users may have different Perl installations, with
   different options, and some other Perl installation may decide to set
   PERL5LIB globally, wreaking havoc with Git for Windows' Perl (which we
   have to use because almost all other Perl distributions lack the
   Subversion bindings we need for `git svn`).

2. Perl makes for a rather large reason that Git for Windows' installer
   weighs in with >30MB. While one Perl script less does not relieve us
   of that burden, it is one step in the right direction.

This patch serves two purposes: to ask for reviews, and to show what I
plan to release as part of Git for Windows v2.11.0 (which is due this
coming Wednesday, if Git v2.11.0 is released on Tuesday, as planned).

Changes since v1:

- fixed the usage (pointed out by David Aguilar)

- moved the stray #include "dir.h" to cuddle with the other #include's
  (also pointed out by David Aguilar)

- changed the `sprintf()` call to a much safer `xsnprintf()` call
  (again, pointed out by David Aguilar)

- changed an error message to include the offending path (need I say,
  pointed out by David Aguilar?)

- used more restrictive permissions for the temporary directories (once
  again, David Aguilar's suggestion)

- fixed a comment that lacked a space after a period (another fix thanks
  to David Aguilar).

- switched the opt-in feature flag triggering the use of the builtin
  difftool to a config variable (this suggestion came from Junio
  Hamano).

- made difftool respect core.symlinks by moving the usage of
  has_symlinks after the config was parsed.


Johannes Schindelin (1):
  difftool: add the builtin

 .gitignore                                    |   1 +
 Makefile                                      |   3 +-
 builtin.h                                     |   1 +
 builtin/difftool.c                            | 692 ++++++++++++++++++++++++++
 git-difftool.perl => git-legacy-difftool.perl |   0
 git.c                                         |   1 +
 6 files changed, 697 insertions(+), 1 deletion(-)
 create mode 100644 builtin/difftool.c
 rename git-difftool.perl => git-legacy-difftool.perl (100%)


base-commit: 1e37181391e305a7ab0c382ca3c3b2de998d4138
Published-As: https://github.com/dscho/git/releases/tag/builtin-difftool-v2
Fetch-It-Via: git fetch https://github.com/dscho/git builtin-difftool-v2

Interdiff vs v1:

 diff --git a/.gitignore b/.gitignore
 index 91bfd09..f96e50e 100644
 --- a/.gitignore
 +++ b/.gitignore
 @@ -1,4 +1,3 @@
 -/use-builtin-difftool
  /GIT-BUILD-OPTIONS
  /GIT-CFLAGS
  /GIT-LDFLAGS
 @@ -52,7 +51,6 @@
  /git-diff-tree
  /git-difftool
  /git-difftool--helper
 -/git-builtin-difftool
  /git-describe
  /git-fast-export
  /git-fast-import
 @@ -78,6 +76,7 @@
  /git-init-db
  /git-interpret-trailers
  /git-instaweb
 +/git-legacy-difftool
  /git-log
  /git-ls-files
  /git-ls-remote
 diff --git a/Makefile b/Makefile
 index f764174..7863bc2 100644
 --- a/Makefile
 +++ b/Makefile
 @@ -527,7 +527,7 @@ SCRIPT_LIB += git-sh-setup
  SCRIPT_LIB += git-sh-i18n
  
  SCRIPT_PERL += git-add--interactive.perl
 -SCRIPT_PERL += git-difftool.perl
 +SCRIPT_PERL += git-legacy-difftool.perl
  SCRIPT_PERL += git-archimport.perl
  SCRIPT_PERL += git-cvsexportcommit.perl
  SCRIPT_PERL += git-cvsimport.perl
 @@ -888,7 +888,7 @@ BUILTIN_OBJS += builtin/diff-files.o
  BUILTIN_OBJS += builtin/diff-index.o
  BUILTIN_OBJS += builtin/diff-tree.o
  BUILTIN_OBJS += builtin/diff.o
 -BUILTIN_OBJS += builtin/builtin-difftool.o
 +BUILTIN_OBJS += builtin/difftool.o
  BUILTIN_OBJS += builtin/fast-export.o
  BUILTIN_OBJS += builtin/fetch-pack.o
  BUILTIN_OBJS += builtin/fetch.o
 diff --git a/builtin.h b/builtin.h
 index 409a61e..67f8051 100644
 --- a/builtin.h
 +++ b/builtin.h
 @@ -60,7 +60,7 @@ extern int cmd_diff_files(int argc, const char **argv, const char *prefix);
  extern int cmd_diff_index(int argc, const char **argv, const char *prefix);
  extern int cmd_diff(int argc, const char **argv, const char *prefix);
  extern int cmd_diff_tree(int argc, const char **argv, const char *prefix);
 -extern int cmd_builtin_difftool(int argc, const char **argv, const char *prefix);
 +extern int cmd_difftool(int argc, const char **argv, const char *prefix);
  extern int cmd_fast_export(int argc, const char **argv, const char *prefix);
  extern int cmd_fetch(int argc, const char **argv, const char *prefix);
  extern int cmd_fetch_pack(int argc, const char **argv, const char *prefix);
 diff --git a/builtin/builtin-difftool.c b/builtin/difftool.c
 similarity index 95%
 rename from builtin/builtin-difftool.c
 rename to builtin/difftool.c
 index 9feefcd..f845879 100644
 --- a/builtin/builtin-difftool.c
 +++ b/builtin/difftool.c
 @@ -18,12 +18,15 @@
  #include "argv-array.h"
  #include "strbuf.h"
  #include "lockfile.h"
 +#include "dir.h"
 +#include "exec_cmd.h"
  
  static char *diff_gui_tool;
  static int trust_exit_code;
 +static int use_builtin_difftool;
  
 -static const char * const builtin_difftool_usage[] = {
 -	N_("git add [<options>] [--] <pathspec>..."),
 +static const char *const builtin_difftool_usage[] = {
 +	N_("git difftool [<options>] [<commit> [<commit>]] [--] [<path>...]"),
  	NULL
  };
  
 @@ -39,6 +42,11 @@ static int difftool_config(const char *var, const char *value, void *cb)
  		return 0;
  	}
  
 +	if (!strcmp(var, "core.usebuiltindifftool")) {
 +		use_builtin_difftool = git_config_bool(var, value);
 +		return 0;
 +	}
 +
  	return git_default_config(var, value, cb);
  }
  
 @@ -227,8 +235,6 @@ static void changed_files(struct hashmap *result, const char *index_path,
  	strbuf_release(&buf);
  }
  
 -#include "dir.h"
 -
  static NORETURN void exit_cleanup(const char *tmpdir, int exit_code)
  {
  	struct strbuf buf = STRBUF_INIT;
 @@ -282,16 +288,16 @@ static int run_dir_diff(const char *extcmd, int symlinks,
  
  	/* Setup temp directories */
  	tmp = getenv("TMPDIR");
 -	sprintf(tmpdir, "%s/git-difftool.XXXXXX", tmp ? tmp : "/tmp");
 +	xsnprintf(tmpdir, sizeof(tmpdir), "%s/git-difftool.XXXXXX", tmp ? tmp : "/tmp");
  	if (!mkdtemp(tmpdir))
 -		return error("could not create temporary directory");
 +		return error("could not create '%s'", tmpdir);
  	strbuf_addf(&ldir, "%s/left/", tmpdir);
  	strbuf_addf(&rdir, "%s/right/", tmpdir);
  	strbuf_addstr(&wtdir, workdir);
  	if (!wtdir.len || !is_dir_sep(wtdir.buf[wtdir.len - 1]))
  		strbuf_addch(&wtdir, '/');
 -	mkdir(ldir.buf, 0777);
 -	mkdir(rdir.buf, 0777);
 +	mkdir(ldir.buf, 0700);
 +	mkdir(rdir.buf, 0700);
  
  	memset(&wtindex, 0, sizeof(wtindex));
  
 @@ -606,12 +612,11 @@ static int run_file_diff(int prompt, int argc, const char **argv)
  	exit(ret);
  }
  
 -int cmd_builtin_difftool(int argc, const char ** argv, const char * prefix)
 +int cmd_difftool(int argc, const char ** argv, const char * prefix)
  {
  	int use_gui_tool = 0, dir_diff = 0, prompt = -1, symlinks = 0,
  	    tool_help = 0;
  	static char *difftool_cmd = NULL, *extcmd = NULL;
 -
  	struct option builtin_difftool_options[] = {
  		OPT_BOOL('g', "gui", &use_gui_tool,
  			 N_("use `diff.guitool` instead of `diff.tool`")),
 @@ -638,9 +643,16 @@ int cmd_builtin_difftool(int argc, const char ** argv, const char * prefix)
  		OPT_END()
  	};
  
 +	git_config(difftool_config, NULL);
  	symlinks = has_symlinks;
 +	if (!use_builtin_difftool) {
 +		const char *path = mkpath("%s/git-legacy-difftool", git_exec_path());
  
 -	git_config(difftool_config, NULL);
 +		if (sane_execvp(path, (char **)argv) < 0)
 +			die_errno("could not exec %s", path);
 +
 +		return 0;
 +	}
  
  	argc = parse_options(argc, argv, prefix, builtin_difftool_options,
  			     builtin_difftool_usage, PARSE_OPT_KEEP_UNKNOWN |
 @@ -670,7 +682,7 @@ int cmd_builtin_difftool(int argc, const char ** argv, const char * prefix)
  
  	/*
  	 * In directory diff mode, 'git-difftool--helper' is called once
 -	 * to compare the a / b directories.In file diff mode, 'git diff'
 +	 * to compare the a / b directories. In file diff mode, 'git diff'
  	 * will invoke a separate instance of 'git-difftool--helper' for
  	 * each file that changed.
  	 */
 diff --git a/git-difftool.perl b/git-legacy-difftool.perl
 similarity index 98%
 rename from git-difftool.perl
 rename to git-legacy-difftool.perl
 index 28e47d8..a5790d0 100755
 --- a/git-difftool.perl
 +++ b/git-legacy-difftool.perl
 @@ -23,13 +23,6 @@ use File::Temp qw(tempdir);
  use Getopt::Long qw(:config pass_through);
  use Git;
  
 -if (-e Git::exec_path() . '/use-builtin-difftool') {
 -	unshift(@ARGV, "builtin-difftool");
 -	unshift(@ARGV, "git");
 -	exec(@ARGV);
 -	die("Could not execute builtin difftool");
 -}
 -
  sub usage
  {
  	my $exitcode = shift;
 diff --git a/git.c b/git.c
 index 7a0df7a..0e6bbee 100644
 --- a/git.c
 +++ b/git.c
 @@ -2,7 +2,6 @@
  #include "exec_cmd.h"
  #include "help.h"
  #include "run-command.h"
 -#include "dir.h"
  
  const char git_usage_string[] =
  	"git [--version] [--help] [-C <path>] [-c name=value]\n"
 @@ -425,7 +424,7 @@ static struct cmd_struct commands[] = {
  	{ "diff-files", cmd_diff_files, RUN_SETUP | NEED_WORK_TREE },
  	{ "diff-index", cmd_diff_index, RUN_SETUP },
  	{ "diff-tree", cmd_diff_tree, RUN_SETUP },
 -	{ "builtin-difftool", cmd_builtin_difftool, RUN_SETUP | NEED_WORK_TREE },
 +	{ "difftool", cmd_difftool, RUN_SETUP | NEED_WORK_TREE },
  	{ "fast-export", cmd_fast_export, RUN_SETUP },
  	{ "fetch", cmd_fetch, RUN_SETUP },
  	{ "fetch-pack", cmd_fetch_pack, RUN_SETUP },
 @@ -543,22 +542,6 @@ static void strip_extension(const char **argv)
  #define strip_extension(cmd)
  #endif
  
 -static int use_builtin_difftool(void)
 -{
 -	static int initialized, use;
 -
 -	if (!initialized) {
 -		struct strbuf buf = STRBUF_INIT;
 -		strbuf_addf(&buf, "%s/%s", git_exec_path(),
 -			    "use-builtin-difftool");
 -		use = file_exists(buf.buf);
 -		strbuf_release(&buf);
 -		initialized = 1;
 -	}
 -
 -	return use;
 -}
 -
  static void handle_builtin(int argc, const char **argv)
  {
  	struct argv_array args = ARGV_ARRAY_INIT;
 @@ -568,9 +551,6 @@ static void handle_builtin(int argc, const char **argv)
  	strip_extension(argv);
  	cmd = argv[0];
  
 -	if (!strcmp("difftool", cmd) && use_builtin_difftool())
 -		cmd = "builtin-difftool";
 -
  	/* Turn "git cmd --help" into "git help --exclude-guides cmd" */
  	if (argc > 1 && !strcmp(argv[1], "--help")) {
  		int i;

-- 
2.10.1.583.g721a9e0

[PATCH v2 1/1] difftool: add the builtin

From: Johannes Schindelin <hidden>
Date: 2016-11-23 22:03:46

This adds a builtin difftool that represents a conversion of the current
Perl script version of the difftool.

The motivation is that Perl scripts are not at all native on Windows,
and that `git difftool` therefore is pretty slow on that platform, when
there is no good reason for it to be slow.

In addition, Perl does not really have access to Git's internals. That
means that any script will always have to jump through unnecessary
hoops.

The current version of the builtin difftool does not, however, make full
use of the internals but instead chooses to spawn a couple of Git
processes, still, to make for an easier conversion. There remains a lot
of room for improvement, left for a later date.

Note: the original difftool is now called `git legacy-difftool`, but to
play it safe, it is still called by difftool unless the config setting
core.useBuiltinDifftool=true.

The reason: this new, experimental, builtin difftool will be shipped as
part of Git for Windows v2.11.0, to allow for easier large-scale
testing, but of course as an opt-in feature.

Sadly, the speedup is more noticable on Linux than on Windows: a quick
test shows that t7800-difftool.sh runs in (2.183s/0.052s/0.108s)
(real/user/sys) in a Linux VM, down from  (6.529s/3.112s/0.644s), while
on Windows, it is (36.064s/2.730s/7.194s), down from
(47.637s/2.407s/6.863s). The culprit is most likely the overhead
incurred from *still* having to shell out to mergetool-lib.sh and
difftool--helper.sh.

Signed-off-by: Johannes Schindelin <redacted>
---
 .gitignore                                    |   1 +
 Makefile                                      |   3 +-
 builtin.h                                     |   1 +
 builtin/difftool.c                            | 692 ++++++++++++++++++++++++++
 git-difftool.perl => git-legacy-difftool.perl |   0
 git.c                                         |   1 +
 6 files changed, 697 insertions(+), 1 deletion(-)
 create mode 100644 builtin/difftool.c
 rename git-difftool.perl => git-legacy-difftool.perl (100%)
diff --git a/.gitignore b/.gitignore
index 05cb58a..f96e50e 100644
--- a/.gitignore
+++ b/.gitignore
@@ -76,6 +76,7 @@
 /git-init-db
 /git-interpret-trailers
 /git-instaweb
+/git-legacy-difftool
 /git-log
 /git-ls-files
 /git-ls-remote
diff --git a/Makefile b/Makefile
index f53fcc9..7863bc2 100644
--- a/Makefile
+++ b/Makefile
@@ -527,7 +527,7 @@ SCRIPT_LIB += git-sh-setup
 SCRIPT_LIB += git-sh-i18n
 
 SCRIPT_PERL += git-add--interactive.perl
-SCRIPT_PERL += git-difftool.perl
+SCRIPT_PERL += git-legacy-difftool.perl
 SCRIPT_PERL += git-archimport.perl
 SCRIPT_PERL += git-cvsexportcommit.perl
 SCRIPT_PERL += git-cvsimport.perl
@@ -888,6 +888,7 @@ BUILTIN_OBJS += builtin/diff-files.o
 BUILTIN_OBJS += builtin/diff-index.o
 BUILTIN_OBJS += builtin/diff-tree.o
 BUILTIN_OBJS += builtin/diff.o
+BUILTIN_OBJS += builtin/difftool.o
 BUILTIN_OBJS += builtin/fast-export.o
 BUILTIN_OBJS += builtin/fetch-pack.o
 BUILTIN_OBJS += builtin/fetch.o
diff --git a/builtin.h b/builtin.h
index b9122bc..67f8051 100644
--- a/builtin.h
+++ b/builtin.h
@@ -60,6 +60,7 @@ extern int cmd_diff_files(int argc, const char **argv, const char *prefix);
 extern int cmd_diff_index(int argc, const char **argv, const char *prefix);
 extern int cmd_diff(int argc, const char **argv, const char *prefix);
 extern int cmd_diff_tree(int argc, const char **argv, const char *prefix);
+extern int cmd_difftool(int argc, const char **argv, const char *prefix);
 extern int cmd_fast_export(int argc, const char **argv, const char *prefix);
 extern int cmd_fetch(int argc, const char **argv, const char *prefix);
 extern int cmd_fetch_pack(int argc, const char **argv, const char *prefix);
diff --git a/builtin/difftool.c b/builtin/difftool.c
new file mode 100644
index 0000000..f845879
--- /dev/null
+++ b/builtin/difftool.c
@@ -0,0 +1,692 @@
+/*
+ * "git difftool" builtin command
+ *
+ * This is a wrapper around the GIT_EXTERNAL_DIFF-compatible
+ * git-difftool--helper script.
+ *
+ * This script exports GIT_EXTERNAL_DIFF and GIT_PAGER for use by git.
+ * The GIT_DIFF* variables are exported for use by git-difftool--helper.
+ *
+ * Any arguments that are unknown to this script are forwarded to 'git diff'.
+ *
+ * Copyright (C) 2016 Johannes Schindelin
+ */
+#include "cache.h"
+#include "builtin.h"
+#include "parse-options.h"
+#include "run-command.h"
+#include "argv-array.h"
+#include "strbuf.h"
+#include "lockfile.h"
+#include "dir.h"
+#include "exec_cmd.h"
+
+static char *diff_gui_tool;
+static int trust_exit_code;
+static int use_builtin_difftool;
+
+static const char *const builtin_difftool_usage[] = {
+	N_("git difftool [<options>] [<commit> [<commit>]] [--] [<path>...]"),
+	NULL
+};
+
+static int difftool_config(const char *var, const char *value, void *cb)
+{
+	if (!strcmp(var, "diff.guitool")) {
+		diff_gui_tool = xstrdup(value);
+		return 0;
+	}
+
+	if (!strcmp(var, "difftool.trustexitcode")) {
+		trust_exit_code = git_config_bool(var, value);
+		return 0;
+	}
+
+	if (!strcmp(var, "core.usebuiltindifftool")) {
+		use_builtin_difftool = git_config_bool(var, value);
+		return 0;
+	}
+
+	return git_default_config(var, value, cb);
+}
+
+static int print_tool_help(void)
+{
+	const char *argv[] = { "mergetool", "--tool-help=diff", NULL };
+	return run_command_v_opt(argv, RUN_GIT_CMD);
+}
+
+static int parse_index_info(char *p, int *mode1, int *mode2,
+			    struct object_id *oid1, struct object_id *oid2,
+			    char *status)
+{
+	if (*p != ':')
+		return error("expected ':', got '%c'", *p);
+	*mode1 = (int)strtol(p + 1, &p, 8);
+	if (*p != ' ')
+		return error("expected ' ', got '%c'", *p);
+	*mode2 = (int)strtol(p + 1, &p, 8);
+	if (*p != ' ')
+		return error("expected ' ', got '%c'", *p);
+	if (get_oid_hex(++p, oid1))
+		return error("expected object ID, got '%s'", p + 1);
+	p += GIT_SHA1_HEXSZ;
+	if (*p != ' ')
+		return error("expected ' ', got '%c'", *p);
+	if (get_oid_hex(++p, oid2))
+		return error("expected object ID, got '%s'", p + 1);
+	p += GIT_SHA1_HEXSZ;
+	if (*p != ' ')
+		return error("expected ' ', got '%c'", *p);
+	*status = *++p;
+	if (!status || p[1])
+		return error("unexpected trailer: '%s'", p);
+	return 0;
+}
+
+/*
+ * Remove any trailing slash from $workdir
+ * before starting to avoid double slashes in symlink targets.
+ */
+static void add_path(struct strbuf *buf, size_t base_len, const char *path)
+{
+	strbuf_setlen(buf, base_len);
+	if (buf->len && buf->buf[buf->len - 1] != '/')
+		strbuf_addch(buf, '/');
+	strbuf_addstr(buf, path);
+}
+
+/*
+ * Determine whether we can simply reuse the file in the worktree.
+ */
+static int use_wt_file(const char *workdir, const char *name,
+		       struct object_id *oid)
+{
+	struct strbuf buf = STRBUF_INIT;
+	struct stat st;
+	int use = 0;
+
+	strbuf_addstr(&buf, workdir);
+	add_path(&buf, buf.len, name);
+
+	if (!lstat(buf.buf, &st) && !S_ISLNK(st.st_mode)) {
+		struct object_id wt_oid;
+		int fd = open(buf.buf, O_RDONLY);
+
+		if (!index_fd(wt_oid.hash, fd, &st, OBJ_BLOB, name, 0)) {
+			if (is_null_oid(oid)) {
+				oidcpy(oid, &wt_oid);
+				use = 1;
+			} else if (!oidcmp(oid, &wt_oid))
+				use = 1;
+		}
+	}
+
+	strbuf_release(&buf);
+
+	return use;
+}
+
+struct working_tree_entry {
+	struct hashmap_entry entry;
+	char path[FLEX_ARRAY];
+};
+
+static int working_tree_entry_cmp(struct working_tree_entry *a,
+				  struct working_tree_entry *b, void *keydata)
+{
+	return strcmp(a->path, b->path);
+}
+
+/*
+ * The `left` and `right` entries hold paths for the symlinks hashmap,
+ * and a SHA-1 surrounded by brief text for submodules.
+ */
+struct pair_entry {
+	struct hashmap_entry entry;
+	char left[PATH_MAX], right[PATH_MAX];
+	const char path[FLEX_ARRAY];
+};
+
+static int pair_cmp(struct pair_entry *a, struct pair_entry *b, void *keydata)
+{
+	return strcmp(a->path, b->path);
+}
+
+static void add_left_or_right(struct hashmap *map, const char *path,
+			      const char *content, int is_right)
+{
+	struct pair_entry *e, *existing;
+
+	FLEX_ALLOC_STR(e, path, path);
+	hashmap_entry_init(e, strhash(path));
+	existing = hashmap_get(map, e, NULL);
+	if (existing) {
+		free(e);
+		e = existing;
+	} else {
+		e->left[0] = e->right[0] = '\0';
+		hashmap_add(map, e);
+	}
+	strcpy(is_right ? e->right : e->left, content);
+}
+
+struct path_entry {
+	struct hashmap_entry entry;
+	char path[FLEX_ARRAY];
+};
+
+int path_entry_cmp(struct path_entry *a, struct path_entry *b, void *key)
+{
+	return strcmp(a->path, key ? key : b->path);
+}
+
+static void changed_files(struct hashmap *result, const char *index_path,
+			  const char *workdir)
+{
+	struct child_process update_index = CHILD_PROCESS_INIT;
+	struct child_process diff_files = CHILD_PROCESS_INIT;
+	struct strbuf index_env = STRBUF_INIT, buf = STRBUF_INIT;
+	const char *git_dir = absolute_path(get_git_dir()), *env[] = {
+		NULL, NULL
+	};
+	FILE *fp;
+
+	strbuf_addf(&index_env, "GIT_INDEX_FILE=%s", index_path);
+	env[0] = index_env.buf;
+
+	argv_array_pushl(&update_index.args,
+			 "--git-dir", git_dir, "--work-tree", workdir,
+			 "update-index", "--really-refresh", "-q",
+			 "--unmerged", NULL);
+	update_index.no_stdin = 1;
+	update_index.no_stdout = 1;
+	update_index.no_stderr = 1;
+	update_index.git_cmd = 1;
+	update_index.use_shell = 0;
+	update_index.clean_on_exit = 1;
+	update_index.dir = workdir;
+	update_index.env = env;
+	/* Ignore any errors of update-index */
+	run_command(&update_index);
+
+	argv_array_pushl(&diff_files.args,
+			 "--git-dir", git_dir, "--work-tree", workdir,
+			 "diff-files", "--name-only", "-z", NULL);
+	diff_files.no_stdin = 1;
+	diff_files.git_cmd = 1;
+	diff_files.use_shell = 0;
+	diff_files.clean_on_exit = 1;
+	diff_files.out = -1;
+	diff_files.dir = workdir;
+	diff_files.env = env;
+	if (start_command(&diff_files))
+		die("could not obtain raw diff");
+	fp = xfdopen(diff_files.out, "r");
+	while (!strbuf_getline_nul(&buf, fp)) {
+		struct path_entry *entry;
+		FLEX_ALLOC_STR(entry, path, buf.buf);
+		hashmap_entry_init(entry, strhash(buf.buf));
+		hashmap_add(result, entry);
+	}
+	if (finish_command(&diff_files))
+		die("diff-files did not exit properly");
+	strbuf_release(&index_env);
+	strbuf_release(&buf);
+}
+
+static NORETURN void exit_cleanup(const char *tmpdir, int exit_code)
+{
+	struct strbuf buf = STRBUF_INIT;
+	strbuf_addstr(&buf, tmpdir);
+	remove_dir_recursively(&buf, 0);
+	if (exit_code)
+		warning(_("failed: %d"), exit_code);
+	exit(exit_code);
+}
+
+static int ensure_leading_directories(char *path)
+{
+	switch (safe_create_leading_directories(path)) {
+		case SCLD_OK:
+		case SCLD_EXISTS:
+			return 0;
+		default:
+			return error(_("could not create leading directories "
+				       "of '%s'"), path);
+	}
+}
+
+static int run_dir_diff(const char *extcmd, int symlinks,
+			int argc, const char **argv)
+{
+	char tmpdir[PATH_MAX];
+	struct strbuf info = STRBUF_INIT, lpath = STRBUF_INIT;
+	struct strbuf rpath = STRBUF_INIT, buf = STRBUF_INIT;
+	struct strbuf ldir = STRBUF_INIT, rdir = STRBUF_INIT;
+	struct strbuf wtdir = STRBUF_INIT;
+	size_t ldir_len, rdir_len, wtdir_len;
+	struct cache_entry *ce = xcalloc(1, sizeof(ce) + PATH_MAX + 1);
+	const char *workdir, *tmp;
+	int ret = 0, i;
+	FILE *fp;
+	struct hashmap working_tree_dups, submodules, symlinks2;
+	struct hashmap_iter iter;
+	struct pair_entry *entry;
+	enum object_type type;
+	unsigned long size;
+	struct index_state wtindex;
+	struct checkout lstate, rstate;
+	int rc, flags = RUN_GIT_CMD, err = 0;
+	struct child_process child = CHILD_PROCESS_INIT;
+	const char *helper_argv[] = { "difftool--helper", NULL, NULL, NULL };
+	struct hashmap wt_modified, tmp_modified;
+	int indices_loaded = 0;
+
+	setup_work_tree();
+	workdir = get_git_work_tree();
+
+	/* Setup temp directories */
+	tmp = getenv("TMPDIR");
+	xsnprintf(tmpdir, sizeof(tmpdir), "%s/git-difftool.XXXXXX", tmp ? tmp : "/tmp");
+	if (!mkdtemp(tmpdir))
+		return error("could not create '%s'", tmpdir);
+	strbuf_addf(&ldir, "%s/left/", tmpdir);
+	strbuf_addf(&rdir, "%s/right/", tmpdir);
+	strbuf_addstr(&wtdir, workdir);
+	if (!wtdir.len || !is_dir_sep(wtdir.buf[wtdir.len - 1]))
+		strbuf_addch(&wtdir, '/');
+	mkdir(ldir.buf, 0700);
+	mkdir(rdir.buf, 0700);
+
+	memset(&wtindex, 0, sizeof(wtindex));
+
+	memset(&lstate, 0, sizeof(lstate));
+	lstate.base_dir = ldir.buf;
+	lstate.base_dir_len = ldir.len;
+	lstate.force = 1;
+	memset(&rstate, 0, sizeof(rstate));
+	rstate.base_dir = rdir.buf;
+	rstate.base_dir_len = rdir.len;
+	rstate.force = 1;
+
+	ldir_len = ldir.len;
+	rdir_len = rdir.len;
+	wtdir_len = wtdir.len;
+
+	hashmap_init(&working_tree_dups,
+		     (hashmap_cmp_fn)working_tree_entry_cmp, 0);
+	hashmap_init(&submodules, (hashmap_cmp_fn)pair_cmp, 0);
+	hashmap_init(&symlinks2, (hashmap_cmp_fn)pair_cmp, 0);
+
+	child.no_stdin = 1;
+	child.git_cmd = 1;
+	child.use_shell = 0;
+	child.clean_on_exit = 1;
+	child.out = -1;
+	argv_array_pushl(&child.args, "diff", "--raw", "--no-abbrev", "-z",
+			 NULL);
+	for (i = 0; i < argc; i++)
+		argv_array_push(&child.args, argv[i]);
+	if (start_command(&child))
+		die("could not obtain raw diff");
+	fp = xfdopen(child.out, "r");
+
+	/* Build index info for left and right sides of the diff */
+	while (!strbuf_getline_nul(&info, fp)) {
+		int lmode, rmode;
+		struct object_id loid, roid;
+		char status;
+		const char *src_path, *dst_path;
+		size_t src_path_len, dst_path_len;
+
+		if (starts_with(info.buf, "::"))
+			die(N_("combined diff formats('-c' and '--cc') are "
+			       "not supported in\n"
+			       "directory diff mode('-d' and '--dir-diff')."));
+
+		if (parse_index_info(info.buf, &lmode, &rmode, &loid, &roid,
+				     &status))
+			break;
+		if (strbuf_getline_nul(&lpath, fp))
+			break;
+		src_path = lpath.buf;
+		src_path_len = lpath.len;
+
+		if (status != 'C' && status != 'R') {
+			dst_path = src_path;
+			dst_path_len = src_path_len;
+		} else {
+			if (strbuf_getline_nul(&rpath, fp))
+				break;
+			dst_path = rpath.buf;
+			dst_path_len = rpath.len;
+		}
+
+		if (S_ISGITLINK(lmode) || S_ISGITLINK(rmode)) {
+			strbuf_reset(&buf);
+			strbuf_addf(&buf, "Subproject commit %s",
+				    oid_to_hex(&loid));
+			add_left_or_right(&submodules, src_path, buf.buf, 0);
+			strbuf_reset(&buf);
+			strbuf_addf(&buf, "Subproject commit %s",
+				    oid_to_hex(&roid));
+			if (!oidcmp(&loid, &roid))
+				strbuf_addstr(&buf, "-dirty");
+			add_left_or_right(&submodules, dst_path, buf.buf, 1);
+			continue;
+		}
+
+		if (S_ISLNK(lmode)) {
+			char *content = read_sha1_file(loid.hash, &type, &size);
+			add_left_or_right(&symlinks2, src_path, content, 0);
+			free(content);
+		}
+
+		if (S_ISLNK(rmode)) {
+			char *content = read_sha1_file(roid.hash, &type, &size);
+			add_left_or_right(&symlinks2, dst_path, content, 1);
+			free(content);
+		}
+
+		if (lmode && status != 'C') {
+			ce->ce_mode = lmode;
+			oidcpy(&ce->oid, &loid);
+			strcpy(ce->name, src_path);
+			ce->ce_namelen = src_path_len;
+			if (checkout_entry(ce, &lstate, NULL))
+				return error("could not write '%s'", src_path);
+		}
+
+		if (rmode) {
+			struct working_tree_entry *entry;
+
+			/* Avoid duplicate working_tree entries */
+			FLEX_ALLOC_STR(entry, path, dst_path);
+			hashmap_entry_init(entry, strhash(dst_path));
+			if (hashmap_get(&working_tree_dups, entry, NULL)) {
+				free(entry);
+				continue;
+			}
+			hashmap_add(&working_tree_dups, entry);
+
+			if (!use_wt_file(workdir, dst_path, &roid)) {
+				ce->ce_mode = rmode;
+				oidcpy(&ce->oid, &roid);
+				strcpy(ce->name, dst_path);
+				ce->ce_namelen = dst_path_len;
+				if (checkout_entry(ce, &rstate, NULL))
+					return error("could not write '%s'",
+						     dst_path);
+			} else if (!is_null_oid(&roid)) {
+				/*
+				 * Changes in the working tree need special
+				 * treatment since they are not part of the
+				 * index.
+				 */
+				struct cache_entry *ce2 =
+					make_cache_entry(rmode, roid.hash,
+							 dst_path, 0, 0);
+				ce_mode_from_stat(ce2, rmode);
+
+				add_index_entry(&wtindex, ce2,
+						ADD_CACHE_JUST_APPEND);
+
+				add_path(&wtdir, wtdir_len, dst_path);
+				add_path(&rdir, rdir_len, dst_path);
+				if (ensure_leading_directories(rdir.buf))
+					return error("could not create "
+						     "directory for '%s'",
+						     dst_path);
+				if (symlinks) {
+					if (symlink(wtdir.buf, rdir.buf)) {
+						ret = error_errno("could not symlink '%s' to '%s'", wtdir.buf, rdir.buf);
+						goto finish;
+					}
+				} else {
+					struct stat st;
+					if (stat(wtdir.buf, &st))
+						st.st_mode = 0644;
+					if (copy_file(rdir.buf, wtdir.buf,
+						      st.st_mode)) {
+						ret = error("could not copy '%s' to '%s'", wtdir.buf, rdir.buf);
+						goto finish;
+					}
+				}
+			}
+		}
+	}
+	if (finish_command(&child)) {
+		ret = error("error occurred running diff --raw");
+		goto finish;
+	}
+
+	/*
+	 * Changes to submodules require special treatment.This loop writes a
+	 * temporary file to both the left and right directories to show the
+	 * change in the recorded SHA1 for the submodule.
+	 */
+	hashmap_iter_init(&submodules, &iter);
+	while ((entry = hashmap_iter_next(&iter))) {
+		if (*entry->left) {
+			add_path(&ldir, ldir_len, entry->path);
+			ensure_leading_directories(ldir.buf);
+			write_file(ldir.buf, "%s", entry->left);
+		}
+		if (*entry->right) {
+			add_path(&rdir, rdir_len, entry->path);
+			ensure_leading_directories(rdir.buf);
+			write_file(rdir.buf, "%s", entry->right);
+		}
+	}
+
+	/*
+	 * Symbolic links require special treatment.The standard "git diff"
+	 * shows only the link itself, not the contents of the link target.
+	 * This loop replicates that behavior.
+	 */
+	hashmap_iter_init(&symlinks2, &iter);
+	while ((entry = hashmap_iter_next(&iter))) {
+		if (*entry->left) {
+			add_path(&ldir, ldir_len, entry->path);
+			ensure_leading_directories(ldir.buf);
+			write_file(ldir.buf, "%s", entry->left);
+		}
+		if (*entry->right) {
+			add_path(&rdir, rdir_len, entry->path);
+			ensure_leading_directories(rdir.buf);
+			write_file(rdir.buf, "%s", entry->right);
+		}
+	}
+
+	strbuf_release(&buf);
+
+	strbuf_setlen(&ldir, ldir_len);
+	helper_argv[1] = ldir.buf;
+	strbuf_setlen(&rdir, rdir_len);
+	helper_argv[2] = rdir.buf;
+
+	if (extcmd) {
+		helper_argv[0] = extcmd;
+		flags = 0;
+	} else
+		setenv("GIT_DIFFTOOL_DIRDIFF", "true", 1);
+	rc = run_command_v_opt(helper_argv, flags);
+
+	/*
+	 * If the diff includes working copy files and those
+	 * files were modified during the diff, then the changes
+	 * should be copied back to the working tree.
+	 * Do not copy back files when symlinks are used and the
+	 * external tool did not replace the original link with a file.
+	 *
+	 * These hashes are loaded lazily since they aren't needed
+	 * in the common case of --symlinks and the difftool updating
+	 * files through the symlink.
+	 */
+	hashmap_init(&wt_modified, (hashmap_cmp_fn)path_entry_cmp,
+		     wtindex.cache_nr);
+	hashmap_init(&tmp_modified, (hashmap_cmp_fn)path_entry_cmp,
+		     wtindex.cache_nr);
+
+	for (i = 0; i < wtindex.cache_nr; i++) {
+		struct hashmap_entry dummy;
+		const char *name = wtindex.cache[i]->name;
+		struct stat st;
+
+		add_path(&rdir, rdir_len, name);
+		if (lstat(rdir.buf, &st))
+			continue;
+
+		if ((symlinks && S_ISLNK(st.st_mode)) || !S_ISREG(st.st_mode))
+			continue;
+
+		if (!indices_loaded) {
+			static struct lock_file lock;
+			strbuf_reset(&buf);
+			strbuf_addf(&buf, "%s/wtindex", tmpdir);
+			if (hold_lock_file_for_update(&lock, buf.buf, 0) < 0 ||
+			    write_locked_index(&wtindex, &lock, COMMIT_LOCK)) {
+				ret = error("could not write %s", buf.buf);
+				rollback_lock_file(&lock);
+				goto finish;
+			}
+			changed_files(&wt_modified, buf.buf, workdir);
+			strbuf_setlen(&rdir, rdir_len);
+			changed_files(&tmp_modified, buf.buf, rdir.buf);
+			add_path(&rdir, rdir_len, name);
+			indices_loaded = 1;
+		}
+
+		hashmap_entry_init(&dummy, strhash(name));
+		if (hashmap_get(&tmp_modified, &dummy, name)) {
+			add_path(&wtdir, wtdir_len, name);
+			if (hashmap_get(&wt_modified, &dummy, name)) {
+				warning(_("both files modified: '%s' and '%s'."),
+					wtdir.buf, rdir.buf);
+				warning(_("working tree file has been left."));
+				warning("");
+				err = 1;
+			} else if (unlink(wtdir.buf) ||
+				   copy_file(wtdir.buf, rdir.buf, st.st_mode))
+				warning_errno(_("could not copy '%s' to '%s'"),
+					      rdir.buf, wtdir.buf);
+		}
+	}
+
+	if (err) {
+		warning(_("temporary files exist in '%s'."), tmpdir);
+		warning(_("you may want to cleanup or recover these."));
+		exit(1);
+	} else
+		exit_cleanup(tmpdir, rc);
+
+finish:
+	free(ce);
+	strbuf_release(&ldir);
+	strbuf_release(&rdir);
+	strbuf_release(&wtdir);
+	strbuf_release(&buf);
+
+	return ret;
+}
+
+static int run_file_diff(int prompt, int argc, const char **argv)
+{
+	struct argv_array args = ARGV_ARRAY_INIT;
+	const char *env[] = {
+		"GIT_PAGER=", "GIT_EXTERNAL_DIFF=git-difftool--helper", NULL,
+		NULL
+	};
+	int ret = 0, i;
+
+	if (prompt > 0)
+		env[2] = "GIT_DIFFTOOL_PROMPT=true";
+	else if (!prompt)
+		env[2] = "GIT_DIFFTOOL_NO_PROMPT=true";
+
+	argv_array_push(&args, "diff");
+	for (i = 0; i < argc; i++)
+		argv_array_push(&args, argv[i]);
+	ret = run_command_v_opt_cd_env(args.argv, RUN_GIT_CMD, NULL, env);
+	exit(ret);
+}
+
+int cmd_difftool(int argc, const char ** argv, const char * prefix)
+{
+	int use_gui_tool = 0, dir_diff = 0, prompt = -1, symlinks = 0,
+	    tool_help = 0;
+	static char *difftool_cmd = NULL, *extcmd = NULL;
+	struct option builtin_difftool_options[] = {
+		OPT_BOOL('g', "gui", &use_gui_tool,
+			 N_("use `diff.guitool` instead of `diff.tool`")),
+		OPT_BOOL('d', "dir-diff", &dir_diff,
+			 N_("perform a full-directory diff")),
+		{ OPTION_SET_INT, 'y', "no-prompt", &prompt, NULL,
+			N_("do not prompt before launching a diff tool"),
+			PARSE_OPT_NOARG | PARSE_OPT_NONEG, NULL, 0},
+		{ OPTION_SET_INT, 0, "prompt", &prompt, NULL, NULL,
+			PARSE_OPT_NOARG | PARSE_OPT_NONEG | PARSE_OPT_HIDDEN,
+			NULL, 1 },
+		OPT_BOOL(0, "symlinks", &symlinks,
+			 N_("use symlinks in dir-diff mode")),
+		OPT_STRING('t', "tool", &difftool_cmd, N_("<tool>"),
+			   N_("use the specified diff tool")),
+		OPT_BOOL(0, "tool-help", &tool_help,
+			 N_("print a list of diff tools that may be used with "
+			    "`--tool`")),
+		OPT_BOOL(0, "trust-exit-code", &trust_exit_code,
+			 N_("make 'git-difftool' exit when an invoked diff "
+			    "tool returns a non - zero exit code")),
+		OPT_STRING('x', "extcmd", &extcmd, N_("<command>"),
+			   N_("specify a custom command for viewing diffs")),
+		OPT_END()
+	};
+
+	git_config(difftool_config, NULL);
+	symlinks = has_symlinks;
+	if (!use_builtin_difftool) {
+		const char *path = mkpath("%s/git-legacy-difftool", git_exec_path());
+
+		if (sane_execvp(path, (char **)argv) < 0)
+			die_errno("could not exec %s", path);
+
+		return 0;
+	}
+
+	argc = parse_options(argc, argv, prefix, builtin_difftool_options,
+			     builtin_difftool_usage, PARSE_OPT_KEEP_UNKNOWN |
+			     PARSE_OPT_KEEP_DASHDASH);
+
+	if (tool_help)
+		return print_tool_help();
+
+	if (use_gui_tool && diff_gui_tool && *diff_gui_tool)
+		setenv("GIT_DIFF_TOOL", diff_gui_tool, 1);
+	else if (difftool_cmd) {
+		if (*difftool_cmd)
+			setenv("GIT_DIFF_TOOL", difftool_cmd, 1);
+		else
+			die(_("no <tool> given for --tool=<tool>"));
+	}
+
+	if (extcmd) {
+		if (*extcmd)
+			setenv("GIT_DIFFTOOL_EXTCMD", extcmd, 1);
+		else
+			die(_("no <cmd> given for --extcmd=<cmd>"));
+	}
+
+	setenv("GIT_DIFFTOOL_TRUST_EXIT_CODE",
+	       trust_exit_code ? "true" : "false", 1);
+
+	/*
+	 * In directory diff mode, 'git-difftool--helper' is called once
+	 * to compare the a / b directories. In file diff mode, 'git diff'
+	 * will invoke a separate instance of 'git-difftool--helper' for
+	 * each file that changed.
+	 */
+	if (dir_diff)
+		return run_dir_diff(extcmd, symlinks, argc, argv);
+	return run_file_diff(prompt, argc, argv);
+}
diff --git a/git-difftool.perl b/git-legacy-difftool.perl
similarity index 100%
rename from git-difftool.perl
rename to git-legacy-difftool.perl
diff --git a/git.c b/git.c
index efa1059..0e6bbee 100644
--- a/git.c
+++ b/git.c
@@ -424,6 +424,7 @@ static struct cmd_struct commands[] = {
 	{ "diff-files", cmd_diff_files, RUN_SETUP | NEED_WORK_TREE },
 	{ "diff-index", cmd_diff_index, RUN_SETUP },
 	{ "diff-tree", cmd_diff_tree, RUN_SETUP },
+	{ "difftool", cmd_difftool, RUN_SETUP | NEED_WORK_TREE },
 	{ "fast-export", cmd_fast_export, RUN_SETUP },
 	{ "fetch", cmd_fetch, RUN_SETUP },
 	{ "fetch-pack", cmd_fetch_pack, RUN_SETUP },
-- 
2.10.1.583.g721a9e0

[PATCH v3 0/2] Show Git Mailing List: a builtin difftool

From: Johannes Schindelin <hidden>
Date: 2016-11-24 20:55:15

I have been working on the builtin difftool for almost two weeks,
for two reasons:

1. Perl is really not native on Windows. Not only is there a performance
   penalty to be paid just for running Perl scripts, we also have to deal
   with the fact that users may have different Perl installations, with
   different options, and some other Perl installation may decide to set
   PERL5LIB globally, wreaking havoc with Git for Windows' Perl (which we
   have to use because almost all other Perl distributions lack the
   Subversion bindings we need for `git svn`).

2. Perl makes for a rather large reason that Git for Windows' installer
   weighs in with >30MB. While one Perl script less does not relieve us
   of that burden, it is one step in the right direction.

This patch serves two purposes: to ask for reviews, and to show what I
plan to release as part of Git for Windows v2.11.0 (which is due this
coming Wednesday, if Git v2.11.0 is released on Tuesday, as planned).

Changes since v2:

- adjusted the config setting's name according to Junio's concerns

- fixed launching difftool in a subdirectory

- fixed dir-diff mode when there are no changes (it did not exit early
  but tried to diff two empty directories)


Johannes Schindelin (2):
  difftool: add a skeleton for the upcoming builtin
  difftool: implement the functionality in the builtin

 .gitignore                                    |   1 +
 Makefile                                      |   3 +-
 builtin.h                                     |   1 +
 builtin/difftool.c                            | 731 ++++++++++++++++++++++++++
 git-difftool.perl => git-legacy-difftool.perl |   0
 git.c                                         |   6 +
 t/t7800-difftool.sh                           |   2 +
 7 files changed, 743 insertions(+), 1 deletion(-)
 create mode 100644 builtin/difftool.c
 rename git-difftool.perl => git-legacy-difftool.perl (100%)


base-commit: e2b2d6a172b76d44cb7b1ddb12ea5bfac9613a44
Published-As: https://github.com/dscho/git/releases/tag/builtin-difftool-v3
Fetch-It-Via: git fetch https://github.com/dscho/git builtin-difftool-v3

Interdiff vs v2:

 diff --git a/builtin/difftool.c b/builtin/difftool.c
 index f845879..3480920 100644
 --- a/builtin/difftool.c
 +++ b/builtin/difftool.c
 @@ -13,17 +13,16 @@
   */
  #include "cache.h"
  #include "builtin.h"
 -#include "parse-options.h"
  #include "run-command.h"
 +#include "exec_cmd.h"
 +#include "parse-options.h"
  #include "argv-array.h"
  #include "strbuf.h"
  #include "lockfile.h"
  #include "dir.h"
 -#include "exec_cmd.h"
  
  static char *diff_gui_tool;
  static int trust_exit_code;
 -static int use_builtin_difftool;
  
  static const char *const builtin_difftool_usage[] = {
  	N_("git difftool [<options>] [<commit> [<commit>]] [--] [<path>...]"),
 @@ -42,11 +41,6 @@ static int difftool_config(const char *var, const char *value, void *cb)
  		return 0;
  	}
  
 -	if (!strcmp(var, "core.usebuiltindifftool")) {
 -		use_builtin_difftool = git_config_bool(var, value);
 -		return 0;
 -	}
 -
  	return git_default_config(var, value, cb);
  }
  
 @@ -257,7 +251,7 @@ static int ensure_leading_directories(char *path)
  	}
  }
  
 -static int run_dir_diff(const char *extcmd, int symlinks,
 +static int run_dir_diff(const char *extcmd, int symlinks, const char *prefix,
  			int argc, const char **argv)
  {
  	char tmpdir[PATH_MAX];
 @@ -283,7 +277,6 @@ static int run_dir_diff(const char *extcmd, int symlinks,
  	struct hashmap wt_modified, tmp_modified;
  	int indices_loaded = 0;
  
 -	setup_work_tree();
  	workdir = get_git_work_tree();
  
  	/* Setup temp directories */
 @@ -323,6 +316,7 @@ static int run_dir_diff(const char *extcmd, int symlinks,
  	child.git_cmd = 1;
  	child.use_shell = 0;
  	child.clean_on_exit = 1;
 +	child.dir = prefix;
  	child.out = -1;
  	argv_array_pushl(&child.args, "diff", "--raw", "--no-abbrev", "-z",
  			 NULL);
 @@ -333,6 +327,7 @@ static int run_dir_diff(const char *extcmd, int symlinks,
  	fp = xfdopen(child.out, "r");
  
  	/* Build index info for left and right sides of the diff */
 +	i = 0;
  	while (!strbuf_getline_nul(&info, fp)) {
  		int lmode, rmode;
  		struct object_id loid, roid;
 @@ -353,6 +348,7 @@ static int run_dir_diff(const char *extcmd, int symlinks,
  		src_path = lpath.buf;
  		src_path_len = lpath.len;
  
 +		i++;
  		if (status != 'C' && status != 'R') {
  			dst_path = src_path;
  			dst_path_len = src_path_len;
 @@ -456,11 +452,15 @@ static int run_dir_diff(const char *extcmd, int symlinks,
  			}
  		}
  	}
 +
  	if (finish_command(&child)) {
  		ret = error("error occurred running diff --raw");
  		goto finish;
  	}
  
 +	if (!i)
 +		return 0;
 +
  	/*
  	 * Changes to submodules require special treatment.This loop writes a
  	 * temporary file to both the left and right directories to show the
 @@ -591,7 +591,8 @@ static int run_dir_diff(const char *extcmd, int symlinks,
  	return ret;
  }
  
 -static int run_file_diff(int prompt, int argc, const char **argv)
 +static int run_file_diff(int prompt, const char *prefix,
 +			 int argc, const char **argv)
  {
  	struct argv_array args = ARGV_ARRAY_INIT;
  	const char *env[] = {
 @@ -605,14 +606,39 @@ static int run_file_diff(int prompt, int argc, const char **argv)
  	else if (!prompt)
  		env[2] = "GIT_DIFFTOOL_NO_PROMPT=true";
  
 +
  	argv_array_push(&args, "diff");
  	for (i = 0; i < argc; i++)
  		argv_array_push(&args, argv[i]);
 -	ret = run_command_v_opt_cd_env(args.argv, RUN_GIT_CMD, NULL, env);
 +	ret = run_command_v_opt_cd_env(args.argv, RUN_GIT_CMD, prefix, env);
  	exit(ret);
  }
  
 -int cmd_difftool(int argc, const char ** argv, const char * prefix)
 +/*
 + * NEEDSWORK: this function can go once the legacy-difftool Perl script is
 + * retired.
 + *
 + * We intentionally avoid reading the config directly here, to avoid messing up
 + * the GIT_* environment variables when we need to fall back to exec()ing the
 + * Perl script.
 + */
 +static int use_builtin_difftool(void) {
 +	struct child_process cp = CHILD_PROCESS_INIT;
 +	struct strbuf out = STRBUF_INIT;
 +	int ret;
 +
 +	argv_array_pushl(&cp.args,
 +			 "config", "--bool", "difftool.usebuiltin", NULL);
 +	cp.git_cmd = 1;
 +	if (capture_command(&cp, &out, 6))
 +		return 0;
 +	strbuf_trim(&out);
 +	ret = !strcmp("true", out.buf);
 +	strbuf_release(&out);
 +	return ret;
 +}
 +
 +int cmd_difftool(int argc, const char **argv, const char *prefix)
  {
  	int use_gui_tool = 0, dir_diff = 0, prompt = -1, symlinks = 0,
  	    tool_help = 0;
 @@ -643,16 +669,29 @@ int cmd_difftool(int argc, const char ** argv, const char * prefix)
  		OPT_END()
  	};
  
 -	git_config(difftool_config, NULL);
 -	symlinks = has_symlinks;
 -	if (!use_builtin_difftool) {
 -		const char *path = mkpath("%s/git-legacy-difftool", git_exec_path());
 +	/*
 +	 * NEEDSWORK: Once the builtin difftool has been tested enough
 +	 * and git-legacy-difftool.perl is retired to contrib/, this preamble
 +	 * can be removed.
 +	 */
 +	if (!use_builtin_difftool()) {
 +		const char *path = mkpath("%s/git-legacy-difftool",
 +					  git_exec_path());
  
  		if (sane_execvp(path, (char **)argv) < 0)
  			die_errno("could not exec %s", path);
  
  		return 0;
  	}
 +	prefix = setup_git_directory();
 +	trace_repo_setup(prefix);
 +	setup_work_tree();
 +	/* NEEDSWORK: once we no longer spawn anything, remove this */
 +	setenv(GIT_DIR_ENVIRONMENT, absolute_path(get_git_dir()), 1);
 +	setenv(GIT_WORK_TREE_ENVIRONMENT, absolute_path(get_git_work_tree()), 1);
 +
 +	git_config(difftool_config, NULL);
 +	symlinks = has_symlinks;
  
  	argc = parse_options(argc, argv, prefix, builtin_difftool_options,
  			     builtin_difftool_usage, PARSE_OPT_KEEP_UNKNOWN |
 @@ -687,6 +726,6 @@ int cmd_difftool(int argc, const char ** argv, const char * prefix)
  	 * each file that changed.
  	 */
  	if (dir_diff)
 -		return run_dir_diff(extcmd, symlinks, argc, argv);
 -	return run_file_diff(prompt, argc, argv);
 +		return run_dir_diff(extcmd, symlinks, prefix, argc, argv);
 +	return run_file_diff(prompt, prefix, argc, argv);
  }
 diff --git a/git.c b/git.c
 index e68b6eb..a8e6a15 100644
 --- a/git.c
 +++ b/git.c
 @@ -424,7 +424,12 @@ static struct cmd_struct commands[] = {
  	{ "diff-files", cmd_diff_files, RUN_SETUP | NEED_WORK_TREE },
  	{ "diff-index", cmd_diff_index, RUN_SETUP },
  	{ "diff-tree", cmd_diff_tree, RUN_SETUP },
 -	{ "difftool", cmd_difftool, RUN_SETUP | NEED_WORK_TREE },
 +	/*
 +	 * NEEDSWORK: Once the redirection to git-legacy-difftool.perl in
 +	 * builtin/difftool.c has been removed, this entry should be changed to
 +	 * RUN_SETUP | NEED_WORK_TREE
 +	 */
 +	{ "difftool", cmd_difftool },
  	{ "fast-export", cmd_fast_export, RUN_SETUP },
  	{ "fetch", cmd_fetch, RUN_SETUP },
  	{ "fetch-pack", cmd_fetch_pack, RUN_SETUP },
 diff --git a/t/t7800-difftool.sh b/t/t7800-difftool.sh
 index 70a2de4..b6a6c30 100755
 --- a/t/t7800-difftool.sh
 +++ b/t/t7800-difftool.sh
 @@ -23,6 +23,8 @@ prompt_given ()
  	test "$prompt" = "Launch 'test-tool' [Y/n]? branch"
  }
  
 +# NEEDSWORK: lose all the PERL prereqs once legacy-difftool is retired.
 +
  # Create a file on master and change it on branch
  test_expect_success PERL 'setup' '
  	echo master >file &&

-- 
2.10.1.583.g721a9e0

[PATCH v3 1/2] difftool: add a skeleton for the upcoming builtin

From: Johannes Schindelin <hidden>
Date: 2016-11-24 20:55:43

This adds a builtin difftool that still falls back to the legacy Perl
version, which has been renamed to `legacy-difftool`.

The idea is that the new, experimental, builtin difftool immediately hands
off to the legacy difftool for now, unless the config variable
difftool.useBuiltin is set to true.

This feature flag will be used in the upcoming Git for Windows v2.11.0
release, to allow early testers to opt-in to use the builtin difftool and
flesh out any bugs.

Signed-off-by: Johannes Schindelin <redacted>
---
 .gitignore                                    |  1 +
 Makefile                                      |  3 +-
 builtin.h                                     |  1 +
 builtin/difftool.c                            | 63 +++++++++++++++++++++++++++
 git-difftool.perl => git-legacy-difftool.perl |  0
 git.c                                         |  6 +++
 t/t7800-difftool.sh                           |  2 +
 7 files changed, 75 insertions(+), 1 deletion(-)
 create mode 100644 builtin/difftool.c
 rename git-difftool.perl => git-legacy-difftool.perl (100%)
diff --git a/.gitignore b/.gitignore
index 05cb58a..f96e50e 100644
--- a/.gitignore
+++ b/.gitignore
@@ -76,6 +76,7 @@
 /git-init-db
 /git-interpret-trailers
 /git-instaweb
+/git-legacy-difftool
 /git-log
 /git-ls-files
 /git-ls-remote
diff --git a/Makefile b/Makefile
index f53fcc9..7863bc2 100644
--- a/Makefile
+++ b/Makefile
@@ -527,7 +527,7 @@ SCRIPT_LIB += git-sh-setup
 SCRIPT_LIB += git-sh-i18n
 
 SCRIPT_PERL += git-add--interactive.perl
-SCRIPT_PERL += git-difftool.perl
+SCRIPT_PERL += git-legacy-difftool.perl
 SCRIPT_PERL += git-archimport.perl
 SCRIPT_PERL += git-cvsexportcommit.perl
 SCRIPT_PERL += git-cvsimport.perl
@@ -888,6 +888,7 @@ BUILTIN_OBJS += builtin/diff-files.o
 BUILTIN_OBJS += builtin/diff-index.o
 BUILTIN_OBJS += builtin/diff-tree.o
 BUILTIN_OBJS += builtin/diff.o
+BUILTIN_OBJS += builtin/difftool.o
 BUILTIN_OBJS += builtin/fast-export.o
 BUILTIN_OBJS += builtin/fetch-pack.o
 BUILTIN_OBJS += builtin/fetch.o
diff --git a/builtin.h b/builtin.h
index b9122bc..67f8051 100644
--- a/builtin.h
+++ b/builtin.h
@@ -60,6 +60,7 @@ extern int cmd_diff_files(int argc, const char **argv, const char *prefix);
 extern int cmd_diff_index(int argc, const char **argv, const char *prefix);
 extern int cmd_diff(int argc, const char **argv, const char *prefix);
 extern int cmd_diff_tree(int argc, const char **argv, const char *prefix);
+extern int cmd_difftool(int argc, const char **argv, const char *prefix);
 extern int cmd_fast_export(int argc, const char **argv, const char *prefix);
 extern int cmd_fetch(int argc, const char **argv, const char *prefix);
 extern int cmd_fetch_pack(int argc, const char **argv, const char *prefix);
diff --git a/builtin/difftool.c b/builtin/difftool.c
new file mode 100644
index 0000000..53870bb
--- /dev/null
+++ b/builtin/difftool.c
@@ -0,0 +1,63 @@
+/*
+ * "git difftool" builtin command
+ *
+ * This is a wrapper around the GIT_EXTERNAL_DIFF-compatible
+ * git-difftool--helper script.
+ *
+ * This script exports GIT_EXTERNAL_DIFF and GIT_PAGER for use by git.
+ * The GIT_DIFF* variables are exported for use by git-difftool--helper.
+ *
+ * Any arguments that are unknown to this script are forwarded to 'git diff'.
+ *
+ * Copyright (C) 2016 Johannes Schindelin
+ */
+#include "builtin.h"
+#include "run-command.h"
+#include "exec_cmd.h"
+
+/*
+ * NEEDSWORK: this function can go once the legacy-difftool Perl script is
+ * retired.
+ *
+ * We intentionally avoid reading the config directly here, to avoid messing up
+ * the GIT_* environment variables when we need to fall back to exec()ing the
+ * Perl script.
+ */
+static int use_builtin_difftool(void) {
+	struct child_process cp = CHILD_PROCESS_INIT;
+	struct strbuf out = STRBUF_INIT;
+	int ret;
+
+	argv_array_pushl(&cp.args,
+			 "config", "--bool", "difftool.usebuiltin", NULL);
+	cp.git_cmd = 1;
+	if (capture_command(&cp, &out, 6))
+		return 0;
+	strbuf_trim(&out);
+	ret = !strcmp("true", out.buf);
+	strbuf_release(&out);
+	return ret;
+}
+
+int cmd_difftool(int argc, const char **argv, const char *prefix)
+{
+	/*
+	 * NEEDSWORK: Once the builtin difftool has been tested enough
+	 * and git-legacy-difftool.perl is retired to contrib/, this preamble
+	 * can be removed.
+	 */
+	if (!use_builtin_difftool()) {
+		const char *path = mkpath("%s/git-legacy-difftool",
+					  git_exec_path());
+
+		if (sane_execvp(path, (char **)argv) < 0)
+			die_errno("could not exec %s", path);
+
+		return 0;
+	}
+	prefix = setup_git_directory();
+	trace_repo_setup(prefix);
+	setup_work_tree();
+
+	die("TODO");
+}
diff --git a/git-difftool.perl b/git-legacy-difftool.perl
similarity index 100%
rename from git-difftool.perl
rename to git-legacy-difftool.perl
diff --git a/git.c b/git.c
index e8b2baf..a8e6a15 100644
--- a/git.c
+++ b/git.c
@@ -424,6 +424,12 @@ static struct cmd_struct commands[] = {
 	{ "diff-files", cmd_diff_files, RUN_SETUP | NEED_WORK_TREE },
 	{ "diff-index", cmd_diff_index, RUN_SETUP },
 	{ "diff-tree", cmd_diff_tree, RUN_SETUP },
+	/*
+	 * NEEDSWORK: Once the redirection to git-legacy-difftool.perl in
+	 * builtin/difftool.c has been removed, this entry should be changed to
+	 * RUN_SETUP | NEED_WORK_TREE
+	 */
+	{ "difftool", cmd_difftool },
 	{ "fast-export", cmd_fast_export, RUN_SETUP },
 	{ "fetch", cmd_fetch, RUN_SETUP },
 	{ "fetch-pack", cmd_fetch_pack, RUN_SETUP },
diff --git a/t/t7800-difftool.sh b/t/t7800-difftool.sh
index 70a2de4..b6a6c30 100755
--- a/t/t7800-difftool.sh
+++ b/t/t7800-difftool.sh
@@ -23,6 +23,8 @@ prompt_given ()
 	test "$prompt" = "Launch 'test-tool' [Y/n]? branch"
 }
 
+# NEEDSWORK: lose all the PERL prereqs once legacy-difftool is retired.
+
 # Create a file on master and change it on branch
 test_expect_success PERL 'setup' '
 	echo master >file &&
-- 
2.10.1.583.g721a9e0

Re: [PATCH v3 1/2] difftool: add a skeleton for the upcoming builtin

From: Jeff King <hidden>
Date: 2016-11-24 21:08:57

On Thu, Nov 24, 2016 at 09:55:07PM +0100, Johannes Schindelin wrote:
+/*
+ * NEEDSWORK: this function can go once the legacy-difftool Perl script is
+ * retired.
+ *
+ * We intentionally avoid reading the config directly here, to avoid messing up
+ * the GIT_* environment variables when we need to fall back to exec()ing the
+ * Perl script.
+ */
+static int use_builtin_difftool(void) {
+	struct child_process cp = CHILD_PROCESS_INIT;
+	struct strbuf out = STRBUF_INIT;
+	int ret;
+
+	argv_array_pushl(&cp.args,
+			 "config", "--bool", "difftool.usebuiltin", NULL);
+	cp.git_cmd = 1;
+	if (capture_command(&cp, &out, 6))
+		return 0;
+	strbuf_trim(&out);
+	ret = !strcmp("true", out.buf);
+	strbuf_release(&out);
+	return ret;
+}
FWIW, it should mostly Just Work to use the internal config functions
these days, with the caveat that they will not read repo-level config if
you haven't done repo setup yet.

I think it would probably be OK to ship with that caveat (people would
probably use --global config, or "git -c" for a quick override), but if
you really wanted to address it, you can do something like what
pager.c:read_early_config() does.

Of course, your method here is fine, too; I just know you are sensitive
to forking extra processes.

Also, a minor nit: capture_command() might return data in "out" with a
non-zero exit if the command both generates stdout and exits non-zero
itself. I'm not sure that's possible with git-config, though, so it
might not be worth worrying about.

-Peff

Re: [PATCH v3 1/2] difftool: add a skeleton for the upcoming builtin

From: Johannes Schindelin <hidden>
Date: 2016-11-24 21:58:12

Hi Peff,

On Thu, 24 Nov 2016, Jeff King wrote:
On Thu, Nov 24, 2016 at 09:55:07PM +0100, Johannes Schindelin wrote:
quoted
+/*
+ * NEEDSWORK: this function can go once the legacy-difftool Perl script is
+ * retired.
+ *
+ * We intentionally avoid reading the config directly here, to avoid messing up
+ * the GIT_* environment variables when we need to fall back to exec()ing the
+ * Perl script.
+ */
+static int use_builtin_difftool(void) {
+	struct child_process cp = CHILD_PROCESS_INIT;
+	struct strbuf out = STRBUF_INIT;
+	int ret;
+
+	argv_array_pushl(&cp.args,
+			 "config", "--bool", "difftool.usebuiltin", NULL);
+	cp.git_cmd = 1;
+	if (capture_command(&cp, &out, 6))
+		return 0;
+	strbuf_trim(&out);
+	ret = !strcmp("true", out.buf);
+	strbuf_release(&out);
+	return ret;
+}
FWIW, it should mostly Just Work to use the internal config functions
these days, with the caveat that they will not read repo-level config if
you haven't done repo setup yet.

I think it would probably be OK to ship with that caveat (people would
probably use --global config, or "git -c" for a quick override), but if
you really wanted to address it, you can do something like what
pager.c:read_early_config() does.
The config setting is already overkill (and does even make something much
harder than before: running tests with the builtin difftool used to be as
simply as `touch use-builtin-difftool && make -C t t7800-difftool.sh, now
I have to edit t7800-difftool.sh to configure difftool.useBuiltin, and
without the repo-level config even that would not be working).

Imitating read_early_config() would be overkill deluxe.
Of course, your method here is fine, too; I just know you are sensitive
to forking extra processes.

Also, a minor nit: capture_command() might return data in "out" with a
non-zero exit if the command both generates stdout and exits non-zero
itself. I'm not sure that's possible with git-config, though, so it
might not be worth worrying about.
As it is, I spent way too much time on a feature flag *that will go away
real soon*. And not only I spent too much time on it: everybody who even
bothered to think about it spent too much time on it. It is a temporary
feature flag. It will go away. If it is inefficient, or inelegant, it
won't matter in a month from now.

In other words, it was not really necessary to spend all of that time and
all of that brain power to first discuss the shortcomings of having the
presence of a file in exec path as a feature flag, then converting it into
a config setting, only to find out that this *causes* problems that were
not there before.

Frankly, that was not what I was hoping for. I was hoping to get a decent
review of the *difftool* functionality. David did a fine job to provide
that review. All that discussion about the temporary feature flag was
really a side track for me, and I hate to admit that I let myself get
sucked into it, and it cost me quite some time that I would have rather
spent on release engineering the preview based on v2.11.0-rc3 (including
the difftool with *any* type of working opt-in feature flag).

Maybe I will write more explicitly in the next cover letter what I intend
to do, and what parts of the patch series is intended to be throw-away
material (hence "good enough" is good enough, and discussions about it are
really wasting all of our time). Of course, I cannot dictate what other
people find interesting to comment on, but at least I will have a good
excuse to ignore suggestions that only distract from the work that is
really needed.

My only solace is that I did some substantially more intensive testing in
the wake of this discussion, thanks to the test suite breakages incurred
by switching the feature flag to a soon-to-be-abandoned config setting.
That makes me much more confident about the builtin difftool, which is all
I wanted in the first place.

Ciao,
Dscho

Re: [PATCH v3 1/2] difftool: add a skeleton for the upcoming builtin

From: Jeff King <hidden>
Date: 2016-11-25 03:18:15

On Thu, Nov 24, 2016 at 10:56:23PM +0100, Johannes Schindelin wrote:
quoted
I think it would probably be OK to ship with that caveat (people would
probably use --global config, or "git -c" for a quick override), but if
you really wanted to address it, you can do something like what
pager.c:read_early_config() does.
The config setting is already overkill (and does even make something much
harder than before: running tests with the builtin difftool used to be as
simply as `touch use-builtin-difftool && make -C t t7800-difftool.sh, now
I have to edit t7800-difftool.sh to configure difftool.useBuiltin, and
without the repo-level config even that would not be working).

Imitating read_early_config() would be overkill deluxe.
I would have expected it to just be a build-time flag, like:

  make BUILTIN_DIFFTOOL=Yes test

but I did not closely follow the rest of the conversation, so I am
probably just repeating bits that were already said. So probably ignore
me.

I'm happy with pretty much anything under the reasoning of "this does not
matter much because it is going away soon".

-Peff

Re: [PATCH v3 1/2] difftool: add a skeleton for the upcoming builtin

From: Johannes Schindelin <hidden>
Date: 2016-11-25 11:05:38

Hi Peff,

On Thu, 24 Nov 2016, Jeff King wrote:
On Thu, Nov 24, 2016 at 10:56:23PM +0100, Johannes Schindelin wrote:
quoted
quoted
I think it would probably be OK to ship with that caveat (people would
probably use --global config, or "git -c" for a quick override), but if
you really wanted to address it, you can do something like what
pager.c:read_early_config() does.
The config setting is already overkill (and does even make something much
harder than before: running tests with the builtin difftool used to be as
simply as `touch use-builtin-difftool && make -C t t7800-difftool.sh, now
I have to edit t7800-difftool.sh to configure difftool.useBuiltin, and
without the repo-level config even that would not be working).

Imitating read_early_config() would be overkill deluxe.
I would have expected it to just be a build-time flag, like:

  make BUILTIN_DIFFTOOL=Yes test
That works for Git developers.

I want to let as many users as possible test the builtin difftool.
Hopefully a lot more users than there are Git developers.

Which means that I need a feature flag in production code, not a build
time flag.
I'm happy with pretty much anything under the reasoning of "this does not
matter much because it is going away soon".
Yeah, well, I am more happy with anything along the lines of David's
review, pointing out flaws in the current revision of the builtin difftool
before it bites users ;-)

Ciao,
Dscho

Re: [PATCH v3 1/2] difftool: add a skeleton for the upcoming builtin

From: Jeff King <hidden>
Date: 2016-11-25 17:19:54

On Fri, Nov 25, 2016 at 12:05:00PM +0100, Johannes Schindelin wrote:
quoted
I would have expected it to just be a build-time flag, like:

  make BUILTIN_DIFFTOOL=Yes test
That works for Git developers.

I want to let as many users as possible test the builtin difftool.
Hopefully a lot more users than there are Git developers.

Which means that I need a feature flag in production code, not a build
time flag.
Ah, I didn't realize that was a requirement. If this is going to be part
of a release and real end-users are going to see it, that does make me
think the config option is the better path (than the presence of some
file), as it's our standard way of tweaking run-time behavior.

The implementation can still remain slightly gross if it's eventually
going away.
quoted
I'm happy with pretty much anything under the reasoning of "this does not
matter much because it is going away soon".
Yeah, well, I am more happy with anything along the lines of David's
review, pointing out flaws in the current revision of the builtin difftool
before it bites users ;-)
Sorry, I can't really help much there, not having much knowledge of
difftool.

-Peff

Re: [PATCH v3 1/2] difftool: add a skeleton for the upcoming builtin

From: Johannes Schindelin <hidden>
Date: 2016-11-25 17:42:07

Hi Peff,

On Fri, 25 Nov 2016, Jeff King wrote:
On Fri, Nov 25, 2016 at 12:05:00PM +0100, Johannes Schindelin wrote:
quoted
quoted
I would have expected it to just be a build-time flag, like:

  make BUILTIN_DIFFTOOL=Yes test
That works for Git developers.

I want to let as many users as possible test the builtin difftool.
Hopefully a lot more users than there are Git developers.

Which means that I need a feature flag in production code, not a build
time flag.
Ah, I didn't realize that was a requirement. If this is going to be part
of a release and real end-users are going to see it, that does make me
think the config option is the better path (than the presence of some
file), as it's our standard way of tweaking run-time behavior.
So how do you easily switch back and forth between testing the old vs the
new difftool via the test suite?

Ciao,
Dscho

Re: [PATCH v3 1/2] difftool: add a skeleton for the upcoming builtin

From: Jeff King <hidden>
Date: 2016-11-25 17:47:39

On Fri, Nov 25, 2016 at 06:41:23PM +0100, Johannes Schindelin wrote:
quoted
Ah, I didn't realize that was a requirement. If this is going to be part
of a release and real end-users are going to see it, that does make me
think the config option is the better path (than the presence of some
file), as it's our standard way of tweaking run-time behavior.
So how do you easily switch back and forth between testing the old vs the
new difftool via the test suite?
If it's for a specific tool, I'd consider teaching the test suite to run
the whole script twice: once with the flag set and once without.

That is sometimes more complicated, though, if the script creates many
sub-repos. An environment variable might be more natural. If you already
support flipping the default via config, you can probably do:

  GIT_CONFIG_PARAMETERS="'difftool.usebuiltin=true'"
  export GIT_CONFIG_PARAMETERS

-Peff

Re: [PATCH v3 1/2] difftool: add a skeleton for the upcoming builtin

From: Johannes Schindelin <hidden>
Date: 2016-11-26 12:23:05

Hi Peff,

On Fri, 25 Nov 2016, Jeff King wrote:
On Fri, Nov 25, 2016 at 06:41:23PM +0100, Johannes Schindelin wrote:
quoted
quoted
Ah, I didn't realize that was a requirement. If this is going to be part
of a release and real end-users are going to see it, that does make me
think the config option is the better path (than the presence of some
file), as it's our standard way of tweaking run-time behavior.
So how do you easily switch back and forth between testing the old vs the
new difftool via the test suite?
If it's for a specific tool, I'd consider teaching the test suite to run
the whole script twice: once with the flag set and once without.

That is sometimes more complicated, though, if the script creates many
sub-repos. An environment variable might be more natural. If you already
support flipping the default via config, you can probably do:

  GIT_CONFIG_PARAMETERS="'difftool.usebuiltin=true'"
  export GIT_CONFIG_PARAMETERS
Except that that does not work, of course. To figure out why, apply this
diff:

-- snip --
diff --git a/t/t7800-difftool.sh b/t/t7800-difftool.sh
index 17f3008277..27159f65f3 100755
--- a/t/t7800-difftool.sh
+++ b/t/t7800-difftool.sh
@@ -10,6 +10,9 @@ Testing basic diff tool invocation
 
 . ./test-lib.sh
 
+echo "config $(git config difftool.usebuiltin)." >&2
+exit 1
+
 difftool_test_setup ()
 {
 	test_config diff.tool test-tool &&
diff --git a/t/test-lib.sh b/t/test-lib.sh
index 9980a46133..0ddeded92b 100644
--- a/t/test-lib.sh
+++ b/t/test-lib.sh
@@ -86,6 +86,7 @@ EDITOR=:
 # /usr/xpg4/bin/sh and /bin/ksh to bail out.  So keep the unsets
 # deriving from the command substitution clustered with the other
 # ones.
+echo "before $(git config difftool.usebuiltin)." >&2
 unset VISUAL EMAIL LANGUAGE COLUMNS $("$PERL_PATH" -e '
 	my @env = keys %ENV;
 	my $ok = join("|", qw(
@@ -104,6 +105,7 @@ unset VISUAL EMAIL LANGUAGE COLUMNS $("$PERL_PATH" -e
'
 	my @vars = grep(/^GIT_/ && !/^GIT_($ok)/o, @env);
 	print join("\n", @vars);
 ')
+echo "after $(git config difftool.usebuiltin)." >&2
 unset XDG_CONFIG_HOME
 unset GITPERLLIB
 GIT_AUTHOR_EMAIL=author@example.com
-- snap --

and then weep at this output:

GIT_CONFIG_PARAMETERS="'difftool.usebuiltin=true'"; export GIT_CONFIG_PARAMETERS; bash t7800-difftool.sh -i -v -x
before true.
after .
Initialized empty Git repository in
/home/virtualbox/git/git-for-windows/t/trash
directory.t7800-difftool/.git/
config .
FATAL: Unexpected exit with code 1

In other words, GIT_CONFIG_PARAMETERS is *explicitly scrubbed* from the
environment when we run our tests (by the code block between the "before"
and the "after" statements in the diff above).

Ciao,
Dscho

Re: [PATCH v3 1/2] difftool: add a skeleton for the upcoming builtin

From: Jeff King <hidden>
Date: 2016-11-26 16:19:15

On Sat, Nov 26, 2016 at 01:22:28PM +0100, Johannes Schindelin wrote:
In other words, GIT_CONFIG_PARAMETERS is *explicitly scrubbed* from the
environment when we run our tests (by the code block between the "before"
and the "after" statements in the diff above).
Sorry if I wasn't clear. I meant to modify t7800 to run the tests twice,
once with the existing script and once with the builtin. I.e., to set
the variable after test-lib.sh has done its scrubbing, and then use a
loop or similar to go through the tests twice.

If you want to control it from outside the test script, you'd need
something like:

  if test "$GIT_TEST_DIFFTOOL" = "builtin"
  then
	GIT_CONFIG_PARAMETERS=...
  fi

-Peff

Re: [PATCH v3 1/2] difftool: add a skeleton for the upcoming builtin

From: Johannes Schindelin <hidden>
Date: 2016-11-27 10:58:44

Hi Peff,

On Sat, 26 Nov 2016, Jeff King wrote:
On Sat, Nov 26, 2016 at 01:22:28PM +0100, Johannes Schindelin wrote:
quoted
In other words, GIT_CONFIG_PARAMETERS is *explicitly scrubbed* from the
environment when we run our tests (by the code block between the "before"
and the "after" statements in the diff above).
Sorry if I wasn't clear. I meant to modify t7800 to run the tests twice,
once with the existing script and once with the builtin. I.e., to set
the variable after test-lib.sh has done its scrubbing, and then use a
loop or similar to go through the tests twice.

If you want to control it from outside the test script, you'd need
something like:

  if test "$GIT_TEST_DIFFTOOL" = "builtin"
That is a bit magic. I first used "GIT_USE_BUILTIN_DIFFTOOL" and it did
not work. My name is arguably more correct (see also Jakub's note about
"naming is hard"), but yours works because there is a "TEST" substring in
it.

Ciao,
Dscho

Re: [PATCH v3 1/2] difftool: add a skeleton for the upcoming builtin

From: Jeff King <hidden>
Date: 2016-11-27 16:51:08

On Sat, Nov 26, 2016 at 02:01:36PM +0100, Johannes Schindelin wrote:
quoted
If you want to control it from outside the test script, you'd need
something like:

  if test "$GIT_TEST_DIFFTOOL" = "builtin"
That is a bit magic. I first used "GIT_USE_BUILTIN_DIFFTOOL" and it did
not work. My name is arguably more correct (see also Jakub's note about
"naming is hard"), but yours works because there is a "TEST" substring in
it.
Yes. You are free to add an exception to the env list in test-lib.sh,
but we usually use GIT_TEST_* to avoid having to do so.

-Peff

[PATCH v3 2/2] difftool: implement the functionality in the builtin

From: Johannes Schindelin <hidden>
Date: 2016-11-24 20:55:53

This patch gives life to the skeleton added in the previous patch.

The motivation for converting the difftool is that Perl scripts are not at
all native on Windows, and that `git difftool` therefore is pretty slow on
that platform, when there is no good reason for it to be slow.

In addition, Perl does not really have access to Git's internals. That
means that any script will always have to jump through unnecessary
hoops.

The current version of the builtin difftool does not, however, make full
use of the internals but instead chooses to spawn a couple of Git
processes, still, to make for an easier conversion. There remains a lot
of room for improvement, left for a later date.

Note: to play it safe, the original difftool is still called unless the
config setting difftool.useBuiltin is set to true.

The reason: this new, experimental, builtin difftool will be shipped as
part of Git for Windows v2.11.0, to allow for easier large-scale
testing, but of course as an opt-in feature.

Sadly, the speedup is more noticable on Linux than on Windows: a quick
test shows that t7800-difftool.sh runs in (2.183s/0.052s/0.108s)
(real/user/sys) in a Linux VM, down from  (6.529s/3.112s/0.644s), while
on Windows, it is (36.064s/2.730s/7.194s), down from
(47.637s/2.407s/6.863s). The culprit is most likely the overhead
incurred from *still* having to shell out to mergetool-lib.sh and
difftool--helper.sh.

Signed-off-by: Johannes Schindelin <redacted>
---
 builtin/difftool.c | 670 ++++++++++++++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 669 insertions(+), 1 deletion(-)
diff --git a/builtin/difftool.c b/builtin/difftool.c
index 53870bb..3480920 100644
--- a/builtin/difftool.c
+++ b/builtin/difftool.c
@@ -11,9 +11,608 @@
  *
  * Copyright (C) 2016 Johannes Schindelin
  */
+#include "cache.h"
 #include "builtin.h"
 #include "run-command.h"
 #include "exec_cmd.h"
+#include "parse-options.h"
+#include "argv-array.h"
+#include "strbuf.h"
+#include "lockfile.h"
+#include "dir.h"
+
+static char *diff_gui_tool;
+static int trust_exit_code;
+
+static const char *const builtin_difftool_usage[] = {
+	N_("git difftool [<options>] [<commit> [<commit>]] [--] [<path>...]"),
+	NULL
+};
+
+static int difftool_config(const char *var, const char *value, void *cb)
+{
+	if (!strcmp(var, "diff.guitool")) {
+		diff_gui_tool = xstrdup(value);
+		return 0;
+	}
+
+	if (!strcmp(var, "difftool.trustexitcode")) {
+		trust_exit_code = git_config_bool(var, value);
+		return 0;
+	}
+
+	return git_default_config(var, value, cb);
+}
+
+static int print_tool_help(void)
+{
+	const char *argv[] = { "mergetool", "--tool-help=diff", NULL };
+	return run_command_v_opt(argv, RUN_GIT_CMD);
+}
+
+static int parse_index_info(char *p, int *mode1, int *mode2,
+			    struct object_id *oid1, struct object_id *oid2,
+			    char *status)
+{
+	if (*p != ':')
+		return error("expected ':', got '%c'", *p);
+	*mode1 = (int)strtol(p + 1, &p, 8);
+	if (*p != ' ')
+		return error("expected ' ', got '%c'", *p);
+	*mode2 = (int)strtol(p + 1, &p, 8);
+	if (*p != ' ')
+		return error("expected ' ', got '%c'", *p);
+	if (get_oid_hex(++p, oid1))
+		return error("expected object ID, got '%s'", p + 1);
+	p += GIT_SHA1_HEXSZ;
+	if (*p != ' ')
+		return error("expected ' ', got '%c'", *p);
+	if (get_oid_hex(++p, oid2))
+		return error("expected object ID, got '%s'", p + 1);
+	p += GIT_SHA1_HEXSZ;
+	if (*p != ' ')
+		return error("expected ' ', got '%c'", *p);
+	*status = *++p;
+	if (!status || p[1])
+		return error("unexpected trailer: '%s'", p);
+	return 0;
+}
+
+/*
+ * Remove any trailing slash from $workdir
+ * before starting to avoid double slashes in symlink targets.
+ */
+static void add_path(struct strbuf *buf, size_t base_len, const char *path)
+{
+	strbuf_setlen(buf, base_len);
+	if (buf->len && buf->buf[buf->len - 1] != '/')
+		strbuf_addch(buf, '/');
+	strbuf_addstr(buf, path);
+}
+
+/*
+ * Determine whether we can simply reuse the file in the worktree.
+ */
+static int use_wt_file(const char *workdir, const char *name,
+		       struct object_id *oid)
+{
+	struct strbuf buf = STRBUF_INIT;
+	struct stat st;
+	int use = 0;
+
+	strbuf_addstr(&buf, workdir);
+	add_path(&buf, buf.len, name);
+
+	if (!lstat(buf.buf, &st) && !S_ISLNK(st.st_mode)) {
+		struct object_id wt_oid;
+		int fd = open(buf.buf, O_RDONLY);
+
+		if (!index_fd(wt_oid.hash, fd, &st, OBJ_BLOB, name, 0)) {
+			if (is_null_oid(oid)) {
+				oidcpy(oid, &wt_oid);
+				use = 1;
+			} else if (!oidcmp(oid, &wt_oid))
+				use = 1;
+		}
+	}
+
+	strbuf_release(&buf);
+
+	return use;
+}
+
+struct working_tree_entry {
+	struct hashmap_entry entry;
+	char path[FLEX_ARRAY];
+};
+
+static int working_tree_entry_cmp(struct working_tree_entry *a,
+				  struct working_tree_entry *b, void *keydata)
+{
+	return strcmp(a->path, b->path);
+}
+
+/*
+ * The `left` and `right` entries hold paths for the symlinks hashmap,
+ * and a SHA-1 surrounded by brief text for submodules.
+ */
+struct pair_entry {
+	struct hashmap_entry entry;
+	char left[PATH_MAX], right[PATH_MAX];
+	const char path[FLEX_ARRAY];
+};
+
+static int pair_cmp(struct pair_entry *a, struct pair_entry *b, void *keydata)
+{
+	return strcmp(a->path, b->path);
+}
+
+static void add_left_or_right(struct hashmap *map, const char *path,
+			      const char *content, int is_right)
+{
+	struct pair_entry *e, *existing;
+
+	FLEX_ALLOC_STR(e, path, path);
+	hashmap_entry_init(e, strhash(path));
+	existing = hashmap_get(map, e, NULL);
+	if (existing) {
+		free(e);
+		e = existing;
+	} else {
+		e->left[0] = e->right[0] = '\0';
+		hashmap_add(map, e);
+	}
+	strcpy(is_right ? e->right : e->left, content);
+}
+
+struct path_entry {
+	struct hashmap_entry entry;
+	char path[FLEX_ARRAY];
+};
+
+int path_entry_cmp(struct path_entry *a, struct path_entry *b, void *key)
+{
+	return strcmp(a->path, key ? key : b->path);
+}
+
+static void changed_files(struct hashmap *result, const char *index_path,
+			  const char *workdir)
+{
+	struct child_process update_index = CHILD_PROCESS_INIT;
+	struct child_process diff_files = CHILD_PROCESS_INIT;
+	struct strbuf index_env = STRBUF_INIT, buf = STRBUF_INIT;
+	const char *git_dir = absolute_path(get_git_dir()), *env[] = {
+		NULL, NULL
+	};
+	FILE *fp;
+
+	strbuf_addf(&index_env, "GIT_INDEX_FILE=%s", index_path);
+	env[0] = index_env.buf;
+
+	argv_array_pushl(&update_index.args,
+			 "--git-dir", git_dir, "--work-tree", workdir,
+			 "update-index", "--really-refresh", "-q",
+			 "--unmerged", NULL);
+	update_index.no_stdin = 1;
+	update_index.no_stdout = 1;
+	update_index.no_stderr = 1;
+	update_index.git_cmd = 1;
+	update_index.use_shell = 0;
+	update_index.clean_on_exit = 1;
+	update_index.dir = workdir;
+	update_index.env = env;
+	/* Ignore any errors of update-index */
+	run_command(&update_index);
+
+	argv_array_pushl(&diff_files.args,
+			 "--git-dir", git_dir, "--work-tree", workdir,
+			 "diff-files", "--name-only", "-z", NULL);
+	diff_files.no_stdin = 1;
+	diff_files.git_cmd = 1;
+	diff_files.use_shell = 0;
+	diff_files.clean_on_exit = 1;
+	diff_files.out = -1;
+	diff_files.dir = workdir;
+	diff_files.env = env;
+	if (start_command(&diff_files))
+		die("could not obtain raw diff");
+	fp = xfdopen(diff_files.out, "r");
+	while (!strbuf_getline_nul(&buf, fp)) {
+		struct path_entry *entry;
+		FLEX_ALLOC_STR(entry, path, buf.buf);
+		hashmap_entry_init(entry, strhash(buf.buf));
+		hashmap_add(result, entry);
+	}
+	if (finish_command(&diff_files))
+		die("diff-files did not exit properly");
+	strbuf_release(&index_env);
+	strbuf_release(&buf);
+}
+
+static NORETURN void exit_cleanup(const char *tmpdir, int exit_code)
+{
+	struct strbuf buf = STRBUF_INIT;
+	strbuf_addstr(&buf, tmpdir);
+	remove_dir_recursively(&buf, 0);
+	if (exit_code)
+		warning(_("failed: %d"), exit_code);
+	exit(exit_code);
+}
+
+static int ensure_leading_directories(char *path)
+{
+	switch (safe_create_leading_directories(path)) {
+		case SCLD_OK:
+		case SCLD_EXISTS:
+			return 0;
+		default:
+			return error(_("could not create leading directories "
+				       "of '%s'"), path);
+	}
+}
+
+static int run_dir_diff(const char *extcmd, int symlinks, const char *prefix,
+			int argc, const char **argv)
+{
+	char tmpdir[PATH_MAX];
+	struct strbuf info = STRBUF_INIT, lpath = STRBUF_INIT;
+	struct strbuf rpath = STRBUF_INIT, buf = STRBUF_INIT;
+	struct strbuf ldir = STRBUF_INIT, rdir = STRBUF_INIT;
+	struct strbuf wtdir = STRBUF_INIT;
+	size_t ldir_len, rdir_len, wtdir_len;
+	struct cache_entry *ce = xcalloc(1, sizeof(ce) + PATH_MAX + 1);
+	const char *workdir, *tmp;
+	int ret = 0, i;
+	FILE *fp;
+	struct hashmap working_tree_dups, submodules, symlinks2;
+	struct hashmap_iter iter;
+	struct pair_entry *entry;
+	enum object_type type;
+	unsigned long size;
+	struct index_state wtindex;
+	struct checkout lstate, rstate;
+	int rc, flags = RUN_GIT_CMD, err = 0;
+	struct child_process child = CHILD_PROCESS_INIT;
+	const char *helper_argv[] = { "difftool--helper", NULL, NULL, NULL };
+	struct hashmap wt_modified, tmp_modified;
+	int indices_loaded = 0;
+
+	workdir = get_git_work_tree();
+
+	/* Setup temp directories */
+	tmp = getenv("TMPDIR");
+	xsnprintf(tmpdir, sizeof(tmpdir), "%s/git-difftool.XXXXXX", tmp ? tmp : "/tmp");
+	if (!mkdtemp(tmpdir))
+		return error("could not create '%s'", tmpdir);
+	strbuf_addf(&ldir, "%s/left/", tmpdir);
+	strbuf_addf(&rdir, "%s/right/", tmpdir);
+	strbuf_addstr(&wtdir, workdir);
+	if (!wtdir.len || !is_dir_sep(wtdir.buf[wtdir.len - 1]))
+		strbuf_addch(&wtdir, '/');
+	mkdir(ldir.buf, 0700);
+	mkdir(rdir.buf, 0700);
+
+	memset(&wtindex, 0, sizeof(wtindex));
+
+	memset(&lstate, 0, sizeof(lstate));
+	lstate.base_dir = ldir.buf;
+	lstate.base_dir_len = ldir.len;
+	lstate.force = 1;
+	memset(&rstate, 0, sizeof(rstate));
+	rstate.base_dir = rdir.buf;
+	rstate.base_dir_len = rdir.len;
+	rstate.force = 1;
+
+	ldir_len = ldir.len;
+	rdir_len = rdir.len;
+	wtdir_len = wtdir.len;
+
+	hashmap_init(&working_tree_dups,
+		     (hashmap_cmp_fn)working_tree_entry_cmp, 0);
+	hashmap_init(&submodules, (hashmap_cmp_fn)pair_cmp, 0);
+	hashmap_init(&symlinks2, (hashmap_cmp_fn)pair_cmp, 0);
+
+	child.no_stdin = 1;
+	child.git_cmd = 1;
+	child.use_shell = 0;
+	child.clean_on_exit = 1;
+	child.dir = prefix;
+	child.out = -1;
+	argv_array_pushl(&child.args, "diff", "--raw", "--no-abbrev", "-z",
+			 NULL);
+	for (i = 0; i < argc; i++)
+		argv_array_push(&child.args, argv[i]);
+	if (start_command(&child))
+		die("could not obtain raw diff");
+	fp = xfdopen(child.out, "r");
+
+	/* Build index info for left and right sides of the diff */
+	i = 0;
+	while (!strbuf_getline_nul(&info, fp)) {
+		int lmode, rmode;
+		struct object_id loid, roid;
+		char status;
+		const char *src_path, *dst_path;
+		size_t src_path_len, dst_path_len;
+
+		if (starts_with(info.buf, "::"))
+			die(N_("combined diff formats('-c' and '--cc') are "
+			       "not supported in\n"
+			       "directory diff mode('-d' and '--dir-diff')."));
+
+		if (parse_index_info(info.buf, &lmode, &rmode, &loid, &roid,
+				     &status))
+			break;
+		if (strbuf_getline_nul(&lpath, fp))
+			break;
+		src_path = lpath.buf;
+		src_path_len = lpath.len;
+
+		i++;
+		if (status != 'C' && status != 'R') {
+			dst_path = src_path;
+			dst_path_len = src_path_len;
+		} else {
+			if (strbuf_getline_nul(&rpath, fp))
+				break;
+			dst_path = rpath.buf;
+			dst_path_len = rpath.len;
+		}
+
+		if (S_ISGITLINK(lmode) || S_ISGITLINK(rmode)) {
+			strbuf_reset(&buf);
+			strbuf_addf(&buf, "Subproject commit %s",
+				    oid_to_hex(&loid));
+			add_left_or_right(&submodules, src_path, buf.buf, 0);
+			strbuf_reset(&buf);
+			strbuf_addf(&buf, "Subproject commit %s",
+				    oid_to_hex(&roid));
+			if (!oidcmp(&loid, &roid))
+				strbuf_addstr(&buf, "-dirty");
+			add_left_or_right(&submodules, dst_path, buf.buf, 1);
+			continue;
+		}
+
+		if (S_ISLNK(lmode)) {
+			char *content = read_sha1_file(loid.hash, &type, &size);
+			add_left_or_right(&symlinks2, src_path, content, 0);
+			free(content);
+		}
+
+		if (S_ISLNK(rmode)) {
+			char *content = read_sha1_file(roid.hash, &type, &size);
+			add_left_or_right(&symlinks2, dst_path, content, 1);
+			free(content);
+		}
+
+		if (lmode && status != 'C') {
+			ce->ce_mode = lmode;
+			oidcpy(&ce->oid, &loid);
+			strcpy(ce->name, src_path);
+			ce->ce_namelen = src_path_len;
+			if (checkout_entry(ce, &lstate, NULL))
+				return error("could not write '%s'", src_path);
+		}
+
+		if (rmode) {
+			struct working_tree_entry *entry;
+
+			/* Avoid duplicate working_tree entries */
+			FLEX_ALLOC_STR(entry, path, dst_path);
+			hashmap_entry_init(entry, strhash(dst_path));
+			if (hashmap_get(&working_tree_dups, entry, NULL)) {
+				free(entry);
+				continue;
+			}
+			hashmap_add(&working_tree_dups, entry);
+
+			if (!use_wt_file(workdir, dst_path, &roid)) {
+				ce->ce_mode = rmode;
+				oidcpy(&ce->oid, &roid);
+				strcpy(ce->name, dst_path);
+				ce->ce_namelen = dst_path_len;
+				if (checkout_entry(ce, &rstate, NULL))
+					return error("could not write '%s'",
+						     dst_path);
+			} else if (!is_null_oid(&roid)) {
+				/*
+				 * Changes in the working tree need special
+				 * treatment since they are not part of the
+				 * index.
+				 */
+				struct cache_entry *ce2 =
+					make_cache_entry(rmode, roid.hash,
+							 dst_path, 0, 0);
+				ce_mode_from_stat(ce2, rmode);
+
+				add_index_entry(&wtindex, ce2,
+						ADD_CACHE_JUST_APPEND);
+
+				add_path(&wtdir, wtdir_len, dst_path);
+				add_path(&rdir, rdir_len, dst_path);
+				if (ensure_leading_directories(rdir.buf))
+					return error("could not create "
+						     "directory for '%s'",
+						     dst_path);
+				if (symlinks) {
+					if (symlink(wtdir.buf, rdir.buf)) {
+						ret = error_errno("could not symlink '%s' to '%s'", wtdir.buf, rdir.buf);
+						goto finish;
+					}
+				} else {
+					struct stat st;
+					if (stat(wtdir.buf, &st))
+						st.st_mode = 0644;
+					if (copy_file(rdir.buf, wtdir.buf,
+						      st.st_mode)) {
+						ret = error("could not copy '%s' to '%s'", wtdir.buf, rdir.buf);
+						goto finish;
+					}
+				}
+			}
+		}
+	}
+
+	if (finish_command(&child)) {
+		ret = error("error occurred running diff --raw");
+		goto finish;
+	}
+
+	if (!i)
+		return 0;
+
+	/*
+	 * Changes to submodules require special treatment.This loop writes a
+	 * temporary file to both the left and right directories to show the
+	 * change in the recorded SHA1 for the submodule.
+	 */
+	hashmap_iter_init(&submodules, &iter);
+	while ((entry = hashmap_iter_next(&iter))) {
+		if (*entry->left) {
+			add_path(&ldir, ldir_len, entry->path);
+			ensure_leading_directories(ldir.buf);
+			write_file(ldir.buf, "%s", entry->left);
+		}
+		if (*entry->right) {
+			add_path(&rdir, rdir_len, entry->path);
+			ensure_leading_directories(rdir.buf);
+			write_file(rdir.buf, "%s", entry->right);
+		}
+	}
+
+	/*
+	 * Symbolic links require special treatment.The standard "git diff"
+	 * shows only the link itself, not the contents of the link target.
+	 * This loop replicates that behavior.
+	 */
+	hashmap_iter_init(&symlinks2, &iter);
+	while ((entry = hashmap_iter_next(&iter))) {
+		if (*entry->left) {
+			add_path(&ldir, ldir_len, entry->path);
+			ensure_leading_directories(ldir.buf);
+			write_file(ldir.buf, "%s", entry->left);
+		}
+		if (*entry->right) {
+			add_path(&rdir, rdir_len, entry->path);
+			ensure_leading_directories(rdir.buf);
+			write_file(rdir.buf, "%s", entry->right);
+		}
+	}
+
+	strbuf_release(&buf);
+
+	strbuf_setlen(&ldir, ldir_len);
+	helper_argv[1] = ldir.buf;
+	strbuf_setlen(&rdir, rdir_len);
+	helper_argv[2] = rdir.buf;
+
+	if (extcmd) {
+		helper_argv[0] = extcmd;
+		flags = 0;
+	} else
+		setenv("GIT_DIFFTOOL_DIRDIFF", "true", 1);
+	rc = run_command_v_opt(helper_argv, flags);
+
+	/*
+	 * If the diff includes working copy files and those
+	 * files were modified during the diff, then the changes
+	 * should be copied back to the working tree.
+	 * Do not copy back files when symlinks are used and the
+	 * external tool did not replace the original link with a file.
+	 *
+	 * These hashes are loaded lazily since they aren't needed
+	 * in the common case of --symlinks and the difftool updating
+	 * files through the symlink.
+	 */
+	hashmap_init(&wt_modified, (hashmap_cmp_fn)path_entry_cmp,
+		     wtindex.cache_nr);
+	hashmap_init(&tmp_modified, (hashmap_cmp_fn)path_entry_cmp,
+		     wtindex.cache_nr);
+
+	for (i = 0; i < wtindex.cache_nr; i++) {
+		struct hashmap_entry dummy;
+		const char *name = wtindex.cache[i]->name;
+		struct stat st;
+
+		add_path(&rdir, rdir_len, name);
+		if (lstat(rdir.buf, &st))
+			continue;
+
+		if ((symlinks && S_ISLNK(st.st_mode)) || !S_ISREG(st.st_mode))
+			continue;
+
+		if (!indices_loaded) {
+			static struct lock_file lock;
+			strbuf_reset(&buf);
+			strbuf_addf(&buf, "%s/wtindex", tmpdir);
+			if (hold_lock_file_for_update(&lock, buf.buf, 0) < 0 ||
+			    write_locked_index(&wtindex, &lock, COMMIT_LOCK)) {
+				ret = error("could not write %s", buf.buf);
+				rollback_lock_file(&lock);
+				goto finish;
+			}
+			changed_files(&wt_modified, buf.buf, workdir);
+			strbuf_setlen(&rdir, rdir_len);
+			changed_files(&tmp_modified, buf.buf, rdir.buf);
+			add_path(&rdir, rdir_len, name);
+			indices_loaded = 1;
+		}
+
+		hashmap_entry_init(&dummy, strhash(name));
+		if (hashmap_get(&tmp_modified, &dummy, name)) {
+			add_path(&wtdir, wtdir_len, name);
+			if (hashmap_get(&wt_modified, &dummy, name)) {
+				warning(_("both files modified: '%s' and '%s'."),
+					wtdir.buf, rdir.buf);
+				warning(_("working tree file has been left."));
+				warning("");
+				err = 1;
+			} else if (unlink(wtdir.buf) ||
+				   copy_file(wtdir.buf, rdir.buf, st.st_mode))
+				warning_errno(_("could not copy '%s' to '%s'"),
+					      rdir.buf, wtdir.buf);
+		}
+	}
+
+	if (err) {
+		warning(_("temporary files exist in '%s'."), tmpdir);
+		warning(_("you may want to cleanup or recover these."));
+		exit(1);
+	} else
+		exit_cleanup(tmpdir, rc);
+
+finish:
+	free(ce);
+	strbuf_release(&ldir);
+	strbuf_release(&rdir);
+	strbuf_release(&wtdir);
+	strbuf_release(&buf);
+
+	return ret;
+}
+
+static int run_file_diff(int prompt, const char *prefix,
+			 int argc, const char **argv)
+{
+	struct argv_array args = ARGV_ARRAY_INIT;
+	const char *env[] = {
+		"GIT_PAGER=", "GIT_EXTERNAL_DIFF=git-difftool--helper", NULL,
+		NULL
+	};
+	int ret = 0, i;
+
+	if (prompt > 0)
+		env[2] = "GIT_DIFFTOOL_PROMPT=true";
+	else if (!prompt)
+		env[2] = "GIT_DIFFTOOL_NO_PROMPT=true";
+
+
+	argv_array_push(&args, "diff");
+	for (i = 0; i < argc; i++)
+		argv_array_push(&args, argv[i]);
+	ret = run_command_v_opt_cd_env(args.argv, RUN_GIT_CMD, prefix, env);
+	exit(ret);
+}
 
 /*
  * NEEDSWORK: this function can go once the legacy-difftool Perl script is
@@ -41,6 +640,35 @@ static int use_builtin_difftool(void) {
 
 int cmd_difftool(int argc, const char **argv, const char *prefix)
 {
+	int use_gui_tool = 0, dir_diff = 0, prompt = -1, symlinks = 0,
+	    tool_help = 0;
+	static char *difftool_cmd = NULL, *extcmd = NULL;
+	struct option builtin_difftool_options[] = {
+		OPT_BOOL('g', "gui", &use_gui_tool,
+			 N_("use `diff.guitool` instead of `diff.tool`")),
+		OPT_BOOL('d', "dir-diff", &dir_diff,
+			 N_("perform a full-directory diff")),
+		{ OPTION_SET_INT, 'y', "no-prompt", &prompt, NULL,
+			N_("do not prompt before launching a diff tool"),
+			PARSE_OPT_NOARG | PARSE_OPT_NONEG, NULL, 0},
+		{ OPTION_SET_INT, 0, "prompt", &prompt, NULL, NULL,
+			PARSE_OPT_NOARG | PARSE_OPT_NONEG | PARSE_OPT_HIDDEN,
+			NULL, 1 },
+		OPT_BOOL(0, "symlinks", &symlinks,
+			 N_("use symlinks in dir-diff mode")),
+		OPT_STRING('t', "tool", &difftool_cmd, N_("<tool>"),
+			   N_("use the specified diff tool")),
+		OPT_BOOL(0, "tool-help", &tool_help,
+			 N_("print a list of diff tools that may be used with "
+			    "`--tool`")),
+		OPT_BOOL(0, "trust-exit-code", &trust_exit_code,
+			 N_("make 'git-difftool' exit when an invoked diff "
+			    "tool returns a non - zero exit code")),
+		OPT_STRING('x', "extcmd", &extcmd, N_("<command>"),
+			   N_("specify a custom command for viewing diffs")),
+		OPT_END()
+	};
+
 	/*
 	 * NEEDSWORK: Once the builtin difftool has been tested enough
 	 * and git-legacy-difftool.perl is retired to contrib/, this preamble
@@ -58,6 +686,46 @@ int cmd_difftool(int argc, const char **argv, const char *prefix)
 	prefix = setup_git_directory();
 	trace_repo_setup(prefix);
 	setup_work_tree();
+	/* NEEDSWORK: once we no longer spawn anything, remove this */
+	setenv(GIT_DIR_ENVIRONMENT, absolute_path(get_git_dir()), 1);
+	setenv(GIT_WORK_TREE_ENVIRONMENT, absolute_path(get_git_work_tree()), 1);
+
+	git_config(difftool_config, NULL);
+	symlinks = has_symlinks;
+
+	argc = parse_options(argc, argv, prefix, builtin_difftool_options,
+			     builtin_difftool_usage, PARSE_OPT_KEEP_UNKNOWN |
+			     PARSE_OPT_KEEP_DASHDASH);
 
-	die("TODO");
+	if (tool_help)
+		return print_tool_help();
+
+	if (use_gui_tool && diff_gui_tool && *diff_gui_tool)
+		setenv("GIT_DIFF_TOOL", diff_gui_tool, 1);
+	else if (difftool_cmd) {
+		if (*difftool_cmd)
+			setenv("GIT_DIFF_TOOL", difftool_cmd, 1);
+		else
+			die(_("no <tool> given for --tool=<tool>"));
+	}
+
+	if (extcmd) {
+		if (*extcmd)
+			setenv("GIT_DIFFTOOL_EXTCMD", extcmd, 1);
+		else
+			die(_("no <cmd> given for --extcmd=<cmd>"));
+	}
+
+	setenv("GIT_DIFFTOOL_TRUST_EXIT_CODE",
+	       trust_exit_code ? "true" : "false", 1);
+
+	/*
+	 * In directory diff mode, 'git-difftool--helper' is called once
+	 * to compare the a / b directories. In file diff mode, 'git diff'
+	 * will invoke a separate instance of 'git-difftool--helper' for
+	 * each file that changed.
+	 */
+	if (dir_diff)
+		return run_dir_diff(extcmd, symlinks, prefix, argc, argv);
+	return run_file_diff(prompt, prefix, argc, argv);
 }
-- 
2.10.1.583.g721a9e0

Re: [PATCH v3 2/2] difftool: implement the functionality in the builtin

From: Jakub Narębski <hidden>
Date: 2016-11-25 21:26:05

W dniu 24.11.2016 o 21:55, Johannes Schindelin pisze:
The current version of the builtin difftool does not, however, make full
use of the internals but instead chooses to spawn a couple of Git
processes, still, to make for an easier conversion. There remains a lot
of room for improvement, left for a later date.
[...]
Sadly, the speedup is more noticable on Linux than on Windows: a quick
test shows that t7800-difftool.sh runs in (2.183s/0.052s/0.108s)
(real/user/sys) in a Linux VM, down from  (6.529s/3.112s/0.644s), while
on Windows, it is (36.064s/2.730s/7.194s), down from
(47.637s/2.407s/6.863s). The culprit is most likely the overhead
incurred from *still* having to shell out to mergetool-lib.sh and
difftool--helper.sh.
Does this mean that our shell-based testsuite is not well suited to be
benchmark suite for comparing performance on MS Windows?

Or does it mean that "builtin-difftool" spawning Git processes is the
problem?
 
quoted hunk
Signed-off-by: Johannes Schindelin <redacted>
---
 builtin/difftool.c | 670 ++++++++++++++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 669 insertions(+), 1 deletion(-)
diff --git a/builtin/difftool.c b/builtin/difftool.c
index 53870bb..3480920 100644
--- a/builtin/difftool.c
+++ b/builtin/difftool.c
@@ -11,9 +11,608 @@
  *
  * Copyright (C) 2016 Johannes Schindelin
  */
+#include "cache.h"
 #include "builtin.h"
 #include "run-command.h"
 #include "exec_cmd.h"
+#include "parse-options.h"
+#include "argv-array.h"
+#include "strbuf.h"
+#include "lockfile.h"
+#include "dir.h"
+
+static char *diff_gui_tool;
+static int trust_exit_code;
+
+static const char *const builtin_difftool_usage[] = {
+	N_("git difftool [<options>] [<commit> [<commit>]] [--] [<path>...]"),
+	NULL
+};
+
+static int difftool_config(const char *var, const char *value, void *cb)
+{
+	if (!strcmp(var, "diff.guitool")) {
Shouldn't you also read other configuration variables, like "diff.tool",
and it's mergetool fallbacks ("merge.guitool", "merge.tool")?
+		diff_gui_tool = xstrdup(value);
+		return 0;
+	}
+
+	if (!strcmp(var, "difftool.trustexitcode")) {
+		trust_exit_code = git_config_bool(var, value);
+		return 0;
+	}
Why you do not need to check "difftool.prompt"?  And "mergetool.*" fallbacks?
+
+	return git_default_config(var, value, cb);
+}
+
+static int print_tool_help(void)
+{
+	const char *argv[] = { "mergetool", "--tool-help=diff", NULL };
+	return run_command_v_opt(argv, RUN_GIT_CMD);
This looks a bit strange to me, but I guess this is to avoid recursively
invoking ourself, and { "legacy-difftool", "--tool-help", NULL }; isn't
that much better.
+}
+
+static int parse_index_info(char *p, int *mode1, int *mode2,
+			    struct object_id *oid1, struct object_id *oid2,
+			    char *status)
  There are only two hard things in Computer Science:
  cache invalidation and naming things.
    -- Phil Karlton

Why did you name function that parses "diff --raw" output (aka "diff-tree"
output) parse_index_info() instead of parse_raw_diff() or parse_diff_tree()?
I went searching for `update-index --index-info` formats...

This is not that important, because this function is file-static; though
future developers would thank you for more descriptive naming.

ADDED: Disregard this, I see this function is about index (?) part of
raw diff, that is only a part of "git diff --raw -z" output.  Though...
+{
+	if (*p != ':')
+		return error("expected ':', got '%c'", *p);
+	*mode1 = (int)strtol(p + 1, &p, 8);
+	if (*p != ' ')
+		return error("expected ' ', got '%c'", *p);
Nitpicking.

I guess because this error shouldn't really happen, and because current
implementation is transient, we don't need to worry about better error
messages (was it problem with parsing, or was it unexpected character).

For example '10064x', or '10064\n' would fail parse, but it is not
space that we were expecting...
+	*mode2 = (int)strtol(p + 1, &p, 8);
+	if (*p != ' ')
+		return error("expected ' ', got '%c'", *p);
+	if (get_oid_hex(++p, oid1))
+		return error("expected object ID, got '%s'", p + 1);
+	p += GIT_SHA1_HEXSZ;
+	if (*p != ' ')
+		return error("expected ' ', got '%c'", *p);
+	if (get_oid_hex(++p, oid2))
+		return error("expected object ID, got '%s'", p + 1);
+	p += GIT_SHA1_HEXSZ;
+	if (*p != ' ')
+		return error("expected ' ', got '%c'", *p);
+	*status = *++p;
+	if (!status || p[1])
+		return error("unexpected trailer: '%s'", p);
+	return 0;
+}
+
+/*
+ * Remove any trailing slash from $workdir
+ * before starting to avoid double slashes in symlink targets.
+ */
Err... that's not what add_path() does, in its current implementation.
It doesn't remove trailing slashes, but it checks if there is trailing
slash, and if there isn't, it adds it as separator before adding path.

Or was it original comment from the Perl implementation?  It look
like this, with '$workdir'...  If it is meant to be straight copy
of comment from legacy-difftool, a note would be nice.
+static void add_path(struct strbuf *buf, size_t base_len, const char *path)
Naming: I think strbuf_addpath() would be a better name, but I guess
it is a matter of taste.
+{
+	strbuf_setlen(buf, base_len);
+	if (buf->len && buf->buf[buf->len - 1] != '/')
+		strbuf_addch(buf, '/');
+	strbuf_addstr(buf, path);
+}
+
+/*
+ * Determine whether we can simply reuse the file in the worktree.
+ */
+static int use_wt_file(const char *workdir, const char *name,
Should it be 'name' or 'pathname'?
+		       struct object_id *oid)
+{
+	struct strbuf buf = STRBUF_INIT;
+	struct stat st;
+	int use = 0;
+
+	strbuf_addstr(&buf, workdir);
+	add_path(&buf, buf.len, name);
With proposed rename, it would IMVVVHO looks better

  +	strbuf_addstr(&buf, workdir);
  +	strbuf_addpath(&buf, buf.len, name);

But that is a matter of taste (again, the function is file-local).
+
+	if (!lstat(buf.buf, &st) && !S_ISLNK(st.st_mode)) {
+		struct object_id wt_oid;
+		int fd = open(buf.buf, O_RDONLY);
+
+		if (!index_fd(wt_oid.hash, fd, &st, OBJ_BLOB, name, 0)) {
+			if (is_null_oid(oid)) {
+				oidcpy(oid, &wt_oid);
+				use = 1;
+			} else if (!oidcmp(oid, &wt_oid))
+				use = 1;
+		}
+	}
+
+	strbuf_release(&buf);
+
+	return use;
+}
[...]
+static int ensure_leading_directories(char *path)
+{
+	switch (safe_create_leading_directories(path)) {
+		case SCLD_OK:
+		case SCLD_EXISTS:
+			return 0;
+		default:
+			return error(_("could not create leading directories "
+				       "of '%s'"), path);
+	}
+}
Nice function, I wonder if it would be useful in other places.
+
+static int run_dir_diff(const char *extcmd, int symlinks, const char *prefix,
+			int argc, const char **argv)
+{
+	char tmpdir[PATH_MAX];
+	struct strbuf info = STRBUF_INIT, lpath = STRBUF_INIT;
+	struct strbuf rpath = STRBUF_INIT, buf = STRBUF_INIT;
Nitpicking.

To be symmetric, it could be reordered like this:

  +	struct strbuf info = STRBUF_INIT, buf = STRBUF_INIT;
  +	struct strbuf lpath = STRBUF_INIT, rpath = STRBUF_INIT;

See: lpath, rpath; ldir, rdir; ldir_len, rdir_len.
+	struct strbuf ldir = STRBUF_INIT, rdir = STRBUF_INIT;
+	struct strbuf wtdir = STRBUF_INIT;
+	size_t ldir_len, rdir_len, wtdir_len;
[...]
+	argv_array_pushl(&child.args, "diff", "--raw", "--no-abbrev", "-z",
+			 NULL);
+	for (i = 0; i < argc; i++)
+		argv_array_push(&child.args, argv[i]);
+	if (start_command(&child))
+		die("could not obtain raw diff");
+	fp = xfdopen(child.out, "r");
+
+	/* Build index info for left and right sides of the diff */
+	i = 0;
+	while (!strbuf_getline_nul(&info, fp)) {
+		int lmode, rmode;
+		struct object_id loid, roid;
+		char status;
+		const char *src_path, *dst_path;
+		size_t src_path_len, dst_path_len;
+
+		if (starts_with(info.buf, "::"))
+			die(N_("combined diff formats('-c' and '--cc') are "
+			       "not supported in\n"
+			       "directory diff mode('-d' and '--dir-diff')."));
+
+		if (parse_index_info(info.buf, &lmode, &rmode, &loid, &roid,
+				     &status))
After rename it would read as:

  +		if (parse_raw_diff(info.buf, &lmode, &rmode, &loid, &roid,
  +				   &status))

Though now I see that you parse here index information of raw diff
(I think)... so disregard my musings.
+			break;
+		if (strbuf_getline_nul(&lpath, fp))
+			break;
+		src_path = lpath.buf;
+		src_path_len = lpath.len;
+
+		i++;
+		if (status != 'C' && status != 'R') {
+			dst_path = src_path;
+			dst_path_len = src_path_len;
+		} else {
+			if (strbuf_getline_nul(&rpath, fp))
+				break;
+			dst_path = rpath.buf;
+			dst_path_len = rpath.len;
+		}

[...]
+	/*
+	 * Changes to submodules require special treatment.This loop writes a
Here and in few other places you are missing space after full stop.

  +	 * Changes to submodules require special treatment. This loop writes a

+	 * temporary file to both the left and right directories to show the
+	 * change in the recorded SHA1 for the submodule.
+	 */
+	hashmap_iter_init(&submodules, &iter);
+	while ((entry = hashmap_iter_next(&iter))) {
+		if (*entry->left) {
+			add_path(&ldir, ldir_len, entry->path);
+			ensure_leading_directories(ldir.buf);
+			write_file(ldir.buf, "%s", entry->left);
+		}
+		if (*entry->right) {
+			add_path(&rdir, rdir_len, entry->path);
+			ensure_leading_directories(rdir.buf);
+			write_file(rdir.buf, "%s", entry->right);
+		}
+	}
+
+	/*
+	 * Symbolic links require special treatment.The standard "git diff"
Same here.
+	 * shows only the link itself, not the contents of the link target.
+	 * This loop replicates that behavior.
+	 */
Best,
-- 
Jakub Narębski

Re: [PATCH v3 2/2] difftool: implement the functionality in the builtin

From: Johannes Schindelin <hidden>
Date: 2016-11-27 11:10:38

Hi Jakub,

On Fri, 25 Nov 2016, Jakub Narębski wrote:
W dniu 24.11.2016 o 21:55, Johannes Schindelin pisze:
quoted
The current version of the builtin difftool does not, however, make
full use of the internals but instead chooses to spawn a couple of Git
processes, still, to make for an easier conversion. There remains a
lot of room for improvement, left for a later date.
[...]
quoted
Sadly, the speedup is more noticable on Linux than on Windows: a quick
test shows that t7800-difftool.sh runs in (2.183s/0.052s/0.108s)
(real/user/sys) in a Linux VM, down from  (6.529s/3.112s/0.644s),
while on Windows, it is (36.064s/2.730s/7.194s), down from
(47.637s/2.407s/6.863s). The culprit is most likely the overhead
incurred from *still* having to shell out to mergetool-lib.sh and
difftool--helper.sh.
Does this mean that our shell-based testsuite is not well suited to be
benchmark suite for comparing performance on MS Windows?
It is quite likely the case that shell-based testing will always be
inappropriate for performance testing. Even on Linux.
Or does it mean that "builtin-difftool" spawning Git processes is the
problem?
At the moment I would have to guess, and I'd rather not.
quoted
Signed-off-by: Johannes Schindelin <redacted>
---
 builtin/difftool.c | 670 ++++++++++++++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 669 insertions(+), 1 deletion(-)
diff --git a/builtin/difftool.c b/builtin/difftool.c
index 53870bb..3480920 100644
--- a/builtin/difftool.c
+++ b/builtin/difftool.c
@@ -11,9 +11,608 @@
  *
  * Copyright (C) 2016 Johannes Schindelin
  */
+#include "cache.h"
 #include "builtin.h"
 #include "run-command.h"
 #include "exec_cmd.h"
+#include "parse-options.h"
+#include "argv-array.h"
+#include "strbuf.h"
+#include "lockfile.h"
+#include "dir.h"
+
+static char *diff_gui_tool;
+static int trust_exit_code;
+
+static const char *const builtin_difftool_usage[] = {
+	N_("git difftool [<options>] [<commit> [<commit>]] [--] [<path>...]"),
+	NULL
+};
+
+static int difftool_config(const char *var, const char *value, void *cb)
+{
+	if (!strcmp(var, "diff.guitool")) {
Shouldn't you also read other configuration variables, like "diff.tool",
and it's mergetool fallbacks ("merge.guitool", "merge.tool")?
No, as those configuration variables are not used by the builtin difftool
directly but read by subsequently spawned commands separately. There would
be no use reading them here, for now.
quoted
+static int print_tool_help(void)
+{
+	const char *argv[] = { "mergetool", "--tool-help=diff", NULL };
+	return run_command_v_opt(argv, RUN_GIT_CMD);
This looks a bit strange to me, but I guess this is to avoid recursively
invoking ourself, and { "legacy-difftool", "--tool-help", NULL }; isn't
that much better.
This is obviously a straight translation of the Perl script (see
https://github.com/git/git/blob/v2.10.2/git-difftool.perl#L40-L46):

	sub print_tool_help
	{
		# See the comment at the bottom of file_diff() for the reason
		# behind
		# using system() followed by exit() instead of exec().
		my $rc = system(qw(git mergetool --tool-help=diff));
		exit($rc | ($rc >> 8));
	}

I read the rest of your review, but it appears that it is more about
style than about substance, while I am only willing to address the latter
issues at the moment. You see, I want to focus on getting difftool correct
first before attempting to make it pretty.

Ciao,
Dscho

Re: [PATCH v3 2/2] difftool: implement the functionality in the builtin

From: Jakub Narębski <hidden>
Date: 2016-11-27 11:21:04

Hello Johannes,

On 27 November 2016 at 12:10, Johannes Schindelin
[off-list ref] wrote:
On Fri, 25 Nov 2016, Jakub Narębski wrote:
quoted
W dniu 24.11.2016 o 21:55, Johannes Schindelin pisze:
[...]
quoted
quoted
+static int difftool_config(const char *var, const char *value, void *cb)
+{
+   if (!strcmp(var, "diff.guitool")) {
Shouldn't you also read other configuration variables, like "diff.tool",
and it's mergetool fallbacks ("merge.guitool", "merge.tool")?
No, as those configuration variables are not used by the builtin difftool
directly but read by subsequently spawned commands separately. There would
be no use reading them here, for now.
Ah, all right then.

Though NEEDSWORK comment would be nice to have here (for when
we don't spawn commands).

[...]
I read the rest of your review, but it appears that it is more about
style than about substance, while I am only willing to address the latter
issues at the moment. You see, I want to focus on getting difftool correct
first before attempting to make it pretty.

Ciao,
Dscho
Well, excet for the submodule-relates stuff, which I have skipped,
it looks good to me.
-- 
Jakub Narębski

[PATCH v4 0/4] Show Git Mailing List: a builtin difftool

From: Johannes Schindelin <hidden>
Date: 2017-01-02 16:17:02

I have been working on the builtin difftool for a while now, for two
reasons:

1. Perl is really not native on Windows. Not only is there a performance
   penalty to be paid just for running Perl scripts, we also have to deal
   with the fact that users may have different Perl installations, with
   different options, and some other Perl installation may decide to set
   PERL5LIB globally, wreaking havoc with Git for Windows' Perl (which we
   have to use because almost all other Perl distributions lack the
   Subversion bindings we need for `git svn`).

2. Perl makes for a rather large reason that Git for Windows' installer
   weighs in with >30MB. While one Perl script less does not relieve us
   of that burden, it is one step in the right direction.

Changes since v3:

- made path_entry_cmp static

- fixed a few bugs identified by Coverity

- fixed overzealous status parsing that did not expect any number after
  C or R


Johannes Schindelin (4):
  Avoid Coverity warning about unfree()d git_exec_path()
  difftool: add a skeleton for the upcoming builtin
  difftool: implement the functionality in the builtin
  t7800: run both builtin and scripted difftool, for now

 .gitignore                                    |   1 +
 Makefile                                      |   3 +-
 builtin.h                                     |   1 +
 builtin/difftool.c                            | 733 ++++++++++++++++++++++++++
 exec_cmd.c                                    |   5 +-
 git-difftool.perl => git-legacy-difftool.perl |   0
 git.c                                         |   6 +
 t/t7800-difftool.sh                           |  29 +
 8 files changed, 776 insertions(+), 2 deletions(-)
 create mode 100644 builtin/difftool.c
 rename git-difftool.perl => git-legacy-difftool.perl (100%)


base-commit: e05806da9ec4aff8adfed142ab2a2b3b02e33c8c
Published-As: https://github.com/dscho/git/releases/tag/builtin-difftool-v4
Fetch-It-Via: git fetch https://github.com/dscho/git builtin-difftool-v4

Interdiff vs v3:

 diff --git a/builtin/difftool.c b/builtin/difftool.c
 index 3480920fea..2115e548a5 100644
 --- a/builtin/difftool.c
 +++ b/builtin/difftool.c
 @@ -73,8 +73,10 @@ static int parse_index_info(char *p, int *mode1, int *mode2,
  	if (*p != ' ')
  		return error("expected ' ', got '%c'", *p);
  	*status = *++p;
 -	if (!status || p[1])
 -		return error("unexpected trailer: '%s'", p);
 +	if (!*status)
 +		return error("missing status");
 +	if (p[1] && !isdigit(p[1]))
 +		return error("unexpected trailer: '%s'", p + 1);
  	return 0;
  }
  
 @@ -107,7 +109,8 @@ static int use_wt_file(const char *workdir, const char *name,
  		struct object_id wt_oid;
  		int fd = open(buf.buf, O_RDONLY);
  
 -		if (!index_fd(wt_oid.hash, fd, &st, OBJ_BLOB, name, 0)) {
 +		if (fd >= 0 &&
 +		    !index_fd(wt_oid.hash, fd, &st, OBJ_BLOB, name, 0)) {
  			if (is_null_oid(oid)) {
  				oidcpy(oid, &wt_oid);
  				use = 1;
 @@ -162,7 +165,7 @@ static void add_left_or_right(struct hashmap *map, const char *path,
  		e->left[0] = e->right[0] = '\0';
  		hashmap_add(map, e);
  	}
 -	strcpy(is_right ? e->right : e->left, content);
 +	strlcpy(is_right ? e->right : e->left, content, PATH_MAX);
  }
  
  struct path_entry {
 @@ -170,7 +173,7 @@ struct path_entry {
  	char path[FLEX_ARRAY];
  };
  
 -int path_entry_cmp(struct path_entry *a, struct path_entry *b, void *key)
 +static int path_entry_cmp(struct path_entry *a, struct path_entry *b, void *key)
  {
  	return strcmp(a->path, key ? key : b->path);
  }
 @@ -423,17 +426,16 @@ static int run_dir_diff(const char *extcmd, int symlinks, const char *prefix,
  				struct cache_entry *ce2 =
  					make_cache_entry(rmode, roid.hash,
  							 dst_path, 0, 0);
 -				ce_mode_from_stat(ce2, rmode);
  
  				add_index_entry(&wtindex, ce2,
  						ADD_CACHE_JUST_APPEND);
  
 -				add_path(&wtdir, wtdir_len, dst_path);
  				add_path(&rdir, rdir_len, dst_path);
  				if (ensure_leading_directories(rdir.buf))
  					return error("could not create "
  						     "directory for '%s'",
  						     dst_path);
 +				add_path(&wtdir, wtdir_len, dst_path);
  				if (symlinks) {
  					if (symlink(wtdir.buf, rdir.buf)) {
  						ret = error_errno("could not symlink '%s' to '%s'", wtdir.buf, rdir.buf);
 diff --git a/exec_cmd.c b/exec_cmd.c
 index 19ac2146d0..587bd7eb48 100644
 --- a/exec_cmd.c
 +++ b/exec_cmd.c
 @@ -65,6 +65,7 @@ void git_set_argv_exec_path(const char *exec_path)
  const char *git_exec_path(void)
  {
  	const char *env;
 +	static char *system_exec_path;
  
  	if (argv_exec_path)
  		return argv_exec_path;
 @@ -74,7 +75,9 @@ const char *git_exec_path(void)
  		return env;
  	}
  
 -	return system_path(GIT_EXEC_PATH);
 +	if (!system_exec_path)
 +		system_exec_path = system_path(GIT_EXEC_PATH);
 +	return system_exec_path;
  }
  
  static void add_path(struct strbuf *out, const char *path)
 diff --git a/t/t7800-difftool.sh b/t/t7800-difftool.sh
 index e94910c563..273ab55723 100755
 --- a/t/t7800-difftool.sh
 +++ b/t/t7800-difftool.sh
 @@ -23,6 +23,20 @@ prompt_given ()
  	test "$prompt" = "Launch 'test-tool' [Y/n]? branch"
  }
  
 +for use_builtin_difftool in false true
 +do
 +
 +test_expect_success 'verify we are running the correct difftool' '
 +	if test true = '$use_builtin_difftool'
 +	then
 +		test_must_fail ok=129 git difftool -h >help &&
 +		grep "g, --gui" help
 +	else
 +		git difftool -h >help &&
 +		grep "g|--gui" help
 +	fi
 +'
 +
  # NEEDSWORK: lose all the PERL prereqs once legacy-difftool is retired.
  
  # Create a file on master and change it on branch
 @@ -606,4 +620,17 @@ test_expect_success PERL,SYMLINKS 'difftool --dir-diff symlinked directories' '
  	)
  '
  
 +test true != $use_builtin_difftool || break
 +
 +test_expect_success 'tear down for re-run' '
 +	rm -rf * .[a-z]* &&
 +	git init
 +'
 +
 +# run as builtin difftool now
 +GIT_CONFIG_PARAMETERS="'difftool.usebuiltin=true'"
 +export GIT_CONFIG_PARAMETERS
 +
 +done
 +
  test_done

-- 
2.11.0.rc3.windows.1

[PATCH v4 2/4] difftool: add a skeleton for the upcoming builtin

From: Johannes Schindelin <hidden>
Date: 2017-01-02 16:22:54

This adds a builtin difftool that still falls back to the legacy Perl
version, which has been renamed to `legacy-difftool`.

The idea is that the new, experimental, builtin difftool immediately hands
off to the legacy difftool for now, unless the config variable
difftool.useBuiltin is set to true.

This feature flag will be used in the upcoming Git for Windows v2.11.0
release, to allow early testers to opt-in to use the builtin difftool and
flesh out any bugs.

Signed-off-by: Johannes Schindelin <redacted>
---
 .gitignore                                    |  1 +
 Makefile                                      |  3 +-
 builtin.h                                     |  1 +
 builtin/difftool.c                            | 63 +++++++++++++++++++++++++++
 git-difftool.perl => git-legacy-difftool.perl |  0
 git.c                                         |  6 +++
 t/t7800-difftool.sh                           |  2 +
 7 files changed, 75 insertions(+), 1 deletion(-)
 create mode 100644 builtin/difftool.c
 rename git-difftool.perl => git-legacy-difftool.perl (100%)
diff --git a/.gitignore b/.gitignore
index 6722f78f9a..5555ae025b 100644
--- a/.gitignore
+++ b/.gitignore
@@ -76,6 +76,7 @@
 /git-init-db
 /git-interpret-trailers
 /git-instaweb
+/git-legacy-difftool
 /git-log
 /git-ls-files
 /git-ls-remote
diff --git a/Makefile b/Makefile
index d861bd9985..8cf5bef034 100644
--- a/Makefile
+++ b/Makefile
@@ -522,7 +522,7 @@ SCRIPT_LIB += git-sh-setup
 SCRIPT_LIB += git-sh-i18n
 
 SCRIPT_PERL += git-add--interactive.perl
-SCRIPT_PERL += git-difftool.perl
+SCRIPT_PERL += git-legacy-difftool.perl
 SCRIPT_PERL += git-archimport.perl
 SCRIPT_PERL += git-cvsexportcommit.perl
 SCRIPT_PERL += git-cvsimport.perl
@@ -883,6 +883,7 @@ BUILTIN_OBJS += builtin/diff-files.o
 BUILTIN_OBJS += builtin/diff-index.o
 BUILTIN_OBJS += builtin/diff-tree.o
 BUILTIN_OBJS += builtin/diff.o
+BUILTIN_OBJS += builtin/difftool.o
 BUILTIN_OBJS += builtin/fast-export.o
 BUILTIN_OBJS += builtin/fetch-pack.o
 BUILTIN_OBJS += builtin/fetch.o
diff --git a/builtin.h b/builtin.h
index b9122bc5f4..67f80519da 100644
--- a/builtin.h
+++ b/builtin.h
@@ -60,6 +60,7 @@ extern int cmd_diff_files(int argc, const char **argv, const char *prefix);
 extern int cmd_diff_index(int argc, const char **argv, const char *prefix);
 extern int cmd_diff(int argc, const char **argv, const char *prefix);
 extern int cmd_diff_tree(int argc, const char **argv, const char *prefix);
+extern int cmd_difftool(int argc, const char **argv, const char *prefix);
 extern int cmd_fast_export(int argc, const char **argv, const char *prefix);
 extern int cmd_fetch(int argc, const char **argv, const char *prefix);
 extern int cmd_fetch_pack(int argc, const char **argv, const char *prefix);
diff --git a/builtin/difftool.c b/builtin/difftool.c
new file mode 100644
index 0000000000..53870bbaf7
--- /dev/null
+++ b/builtin/difftool.c
@@ -0,0 +1,63 @@
+/*
+ * "git difftool" builtin command
+ *
+ * This is a wrapper around the GIT_EXTERNAL_DIFF-compatible
+ * git-difftool--helper script.
+ *
+ * This script exports GIT_EXTERNAL_DIFF and GIT_PAGER for use by git.
+ * The GIT_DIFF* variables are exported for use by git-difftool--helper.
+ *
+ * Any arguments that are unknown to this script are forwarded to 'git diff'.
+ *
+ * Copyright (C) 2016 Johannes Schindelin
+ */
+#include "builtin.h"
+#include "run-command.h"
+#include "exec_cmd.h"
+
+/*
+ * NEEDSWORK: this function can go once the legacy-difftool Perl script is
+ * retired.
+ *
+ * We intentionally avoid reading the config directly here, to avoid messing up
+ * the GIT_* environment variables when we need to fall back to exec()ing the
+ * Perl script.
+ */
+static int use_builtin_difftool(void) {
+	struct child_process cp = CHILD_PROCESS_INIT;
+	struct strbuf out = STRBUF_INIT;
+	int ret;
+
+	argv_array_pushl(&cp.args,
+			 "config", "--bool", "difftool.usebuiltin", NULL);
+	cp.git_cmd = 1;
+	if (capture_command(&cp, &out, 6))
+		return 0;
+	strbuf_trim(&out);
+	ret = !strcmp("true", out.buf);
+	strbuf_release(&out);
+	return ret;
+}
+
+int cmd_difftool(int argc, const char **argv, const char *prefix)
+{
+	/*
+	 * NEEDSWORK: Once the builtin difftool has been tested enough
+	 * and git-legacy-difftool.perl is retired to contrib/, this preamble
+	 * can be removed.
+	 */
+	if (!use_builtin_difftool()) {
+		const char *path = mkpath("%s/git-legacy-difftool",
+					  git_exec_path());
+
+		if (sane_execvp(path, (char **)argv) < 0)
+			die_errno("could not exec %s", path);
+
+		return 0;
+	}
+	prefix = setup_git_directory();
+	trace_repo_setup(prefix);
+	setup_work_tree();
+
+	die("TODO");
+}
diff --git a/git-difftool.perl b/git-legacy-difftool.perl
similarity index 100%
rename from git-difftool.perl
rename to git-legacy-difftool.perl
diff --git a/git.c b/git.c
index dce529fcbf..044958a780 100644
--- a/git.c
+++ b/git.c
@@ -424,6 +424,12 @@ static struct cmd_struct commands[] = {
 	{ "diff-files", cmd_diff_files, RUN_SETUP | NEED_WORK_TREE },
 	{ "diff-index", cmd_diff_index, RUN_SETUP },
 	{ "diff-tree", cmd_diff_tree, RUN_SETUP },
+	/*
+	 * NEEDSWORK: Once the redirection to git-legacy-difftool.perl in
+	 * builtin/difftool.c has been removed, this entry should be changed to
+	 * RUN_SETUP | NEED_WORK_TREE
+	 */
+	{ "difftool", cmd_difftool },
 	{ "fast-export", cmd_fast_export, RUN_SETUP },
 	{ "fetch", cmd_fetch, RUN_SETUP },
 	{ "fetch-pack", cmd_fetch_pack, RUN_SETUP },
diff --git a/t/t7800-difftool.sh b/t/t7800-difftool.sh
index 99d4123461..e94910c563 100755
--- a/t/t7800-difftool.sh
+++ b/t/t7800-difftool.sh
@@ -23,6 +23,8 @@ prompt_given ()
 	test "$prompt" = "Launch 'test-tool' [Y/n]? branch"
 }
 
+# NEEDSWORK: lose all the PERL prereqs once legacy-difftool is retired.
+
 # Create a file on master and change it on branch
 test_expect_success PERL 'setup' '
 	echo master >file &&
-- 
2.11.0.rc3.windows.1

[PATCH v4 3/4] difftool: implement the functionality in the builtin

From: Johannes Schindelin <hidden>
Date: 2017-01-02 16:22:59

This patch gives life to the skeleton added in the previous patch.

The motivation for converting the difftool is that Perl scripts are not at
all native on Windows, and that `git difftool` therefore is pretty slow on
that platform, when there is no good reason for it to be slow.

In addition, Perl does not really have access to Git's internals. That
means that any script will always have to jump through unnecessary
hoops.

The current version of the builtin difftool does not, however, make full
use of the internals but instead chooses to spawn a couple of Git
processes, still, to make for an easier conversion. There remains a lot
of room for improvement, left for a later date.

Note: to play it safe, the original difftool is still called unless the
config setting difftool.useBuiltin is set to true.

The reason: this new, experimental, builtin difftool will be shipped as
part of Git for Windows v2.11.0, to allow for easier large-scale
testing, but of course as an opt-in feature.

Sadly, the speedup is more noticable on Linux than on Windows: a quick
test shows that t7800-difftool.sh runs in (2.183s/0.052s/0.108s)
(real/user/sys) in a Linux VM, down from  (6.529s/3.112s/0.644s), while
on Windows, it is (36.064s/2.730s/7.194s), down from
(47.637s/2.407s/6.863s). The culprit is most likely the overhead
incurred from *still* having to shell out to mergetool-lib.sh and
difftool--helper.sh.

Signed-off-by: Johannes Schindelin <redacted>
---
 builtin/difftool.c | 672 ++++++++++++++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 671 insertions(+), 1 deletion(-)
diff --git a/builtin/difftool.c b/builtin/difftool.c
index 53870bbaf7..2115e548a5 100644
--- a/builtin/difftool.c
+++ b/builtin/difftool.c
@@ -11,9 +11,610 @@
  *
  * Copyright (C) 2016 Johannes Schindelin
  */
+#include "cache.h"
 #include "builtin.h"
 #include "run-command.h"
 #include "exec_cmd.h"
+#include "parse-options.h"
+#include "argv-array.h"
+#include "strbuf.h"
+#include "lockfile.h"
+#include "dir.h"
+
+static char *diff_gui_tool;
+static int trust_exit_code;
+
+static const char *const builtin_difftool_usage[] = {
+	N_("git difftool [<options>] [<commit> [<commit>]] [--] [<path>...]"),
+	NULL
+};
+
+static int difftool_config(const char *var, const char *value, void *cb)
+{
+	if (!strcmp(var, "diff.guitool")) {
+		diff_gui_tool = xstrdup(value);
+		return 0;
+	}
+
+	if (!strcmp(var, "difftool.trustexitcode")) {
+		trust_exit_code = git_config_bool(var, value);
+		return 0;
+	}
+
+	return git_default_config(var, value, cb);
+}
+
+static int print_tool_help(void)
+{
+	const char *argv[] = { "mergetool", "--tool-help=diff", NULL };
+	return run_command_v_opt(argv, RUN_GIT_CMD);
+}
+
+static int parse_index_info(char *p, int *mode1, int *mode2,
+			    struct object_id *oid1, struct object_id *oid2,
+			    char *status)
+{
+	if (*p != ':')
+		return error("expected ':', got '%c'", *p);
+	*mode1 = (int)strtol(p + 1, &p, 8);
+	if (*p != ' ')
+		return error("expected ' ', got '%c'", *p);
+	*mode2 = (int)strtol(p + 1, &p, 8);
+	if (*p != ' ')
+		return error("expected ' ', got '%c'", *p);
+	if (get_oid_hex(++p, oid1))
+		return error("expected object ID, got '%s'", p + 1);
+	p += GIT_SHA1_HEXSZ;
+	if (*p != ' ')
+		return error("expected ' ', got '%c'", *p);
+	if (get_oid_hex(++p, oid2))
+		return error("expected object ID, got '%s'", p + 1);
+	p += GIT_SHA1_HEXSZ;
+	if (*p != ' ')
+		return error("expected ' ', got '%c'", *p);
+	*status = *++p;
+	if (!*status)
+		return error("missing status");
+	if (p[1] && !isdigit(p[1]))
+		return error("unexpected trailer: '%s'", p + 1);
+	return 0;
+}
+
+/*
+ * Remove any trailing slash from $workdir
+ * before starting to avoid double slashes in symlink targets.
+ */
+static void add_path(struct strbuf *buf, size_t base_len, const char *path)
+{
+	strbuf_setlen(buf, base_len);
+	if (buf->len && buf->buf[buf->len - 1] != '/')
+		strbuf_addch(buf, '/');
+	strbuf_addstr(buf, path);
+}
+
+/*
+ * Determine whether we can simply reuse the file in the worktree.
+ */
+static int use_wt_file(const char *workdir, const char *name,
+		       struct object_id *oid)
+{
+	struct strbuf buf = STRBUF_INIT;
+	struct stat st;
+	int use = 0;
+
+	strbuf_addstr(&buf, workdir);
+	add_path(&buf, buf.len, name);
+
+	if (!lstat(buf.buf, &st) && !S_ISLNK(st.st_mode)) {
+		struct object_id wt_oid;
+		int fd = open(buf.buf, O_RDONLY);
+
+		if (fd >= 0 &&
+		    !index_fd(wt_oid.hash, fd, &st, OBJ_BLOB, name, 0)) {
+			if (is_null_oid(oid)) {
+				oidcpy(oid, &wt_oid);
+				use = 1;
+			} else if (!oidcmp(oid, &wt_oid))
+				use = 1;
+		}
+	}
+
+	strbuf_release(&buf);
+
+	return use;
+}
+
+struct working_tree_entry {
+	struct hashmap_entry entry;
+	char path[FLEX_ARRAY];
+};
+
+static int working_tree_entry_cmp(struct working_tree_entry *a,
+				  struct working_tree_entry *b, void *keydata)
+{
+	return strcmp(a->path, b->path);
+}
+
+/*
+ * The `left` and `right` entries hold paths for the symlinks hashmap,
+ * and a SHA-1 surrounded by brief text for submodules.
+ */
+struct pair_entry {
+	struct hashmap_entry entry;
+	char left[PATH_MAX], right[PATH_MAX];
+	const char path[FLEX_ARRAY];
+};
+
+static int pair_cmp(struct pair_entry *a, struct pair_entry *b, void *keydata)
+{
+	return strcmp(a->path, b->path);
+}
+
+static void add_left_or_right(struct hashmap *map, const char *path,
+			      const char *content, int is_right)
+{
+	struct pair_entry *e, *existing;
+
+	FLEX_ALLOC_STR(e, path, path);
+	hashmap_entry_init(e, strhash(path));
+	existing = hashmap_get(map, e, NULL);
+	if (existing) {
+		free(e);
+		e = existing;
+	} else {
+		e->left[0] = e->right[0] = '\0';
+		hashmap_add(map, e);
+	}
+	strlcpy(is_right ? e->right : e->left, content, PATH_MAX);
+}
+
+struct path_entry {
+	struct hashmap_entry entry;
+	char path[FLEX_ARRAY];
+};
+
+static int path_entry_cmp(struct path_entry *a, struct path_entry *b, void *key)
+{
+	return strcmp(a->path, key ? key : b->path);
+}
+
+static void changed_files(struct hashmap *result, const char *index_path,
+			  const char *workdir)
+{
+	struct child_process update_index = CHILD_PROCESS_INIT;
+	struct child_process diff_files = CHILD_PROCESS_INIT;
+	struct strbuf index_env = STRBUF_INIT, buf = STRBUF_INIT;
+	const char *git_dir = absolute_path(get_git_dir()), *env[] = {
+		NULL, NULL
+	};
+	FILE *fp;
+
+	strbuf_addf(&index_env, "GIT_INDEX_FILE=%s", index_path);
+	env[0] = index_env.buf;
+
+	argv_array_pushl(&update_index.args,
+			 "--git-dir", git_dir, "--work-tree", workdir,
+			 "update-index", "--really-refresh", "-q",
+			 "--unmerged", NULL);
+	update_index.no_stdin = 1;
+	update_index.no_stdout = 1;
+	update_index.no_stderr = 1;
+	update_index.git_cmd = 1;
+	update_index.use_shell = 0;
+	update_index.clean_on_exit = 1;
+	update_index.dir = workdir;
+	update_index.env = env;
+	/* Ignore any errors of update-index */
+	run_command(&update_index);
+
+	argv_array_pushl(&diff_files.args,
+			 "--git-dir", git_dir, "--work-tree", workdir,
+			 "diff-files", "--name-only", "-z", NULL);
+	diff_files.no_stdin = 1;
+	diff_files.git_cmd = 1;
+	diff_files.use_shell = 0;
+	diff_files.clean_on_exit = 1;
+	diff_files.out = -1;
+	diff_files.dir = workdir;
+	diff_files.env = env;
+	if (start_command(&diff_files))
+		die("could not obtain raw diff");
+	fp = xfdopen(diff_files.out, "r");
+	while (!strbuf_getline_nul(&buf, fp)) {
+		struct path_entry *entry;
+		FLEX_ALLOC_STR(entry, path, buf.buf);
+		hashmap_entry_init(entry, strhash(buf.buf));
+		hashmap_add(result, entry);
+	}
+	if (finish_command(&diff_files))
+		die("diff-files did not exit properly");
+	strbuf_release(&index_env);
+	strbuf_release(&buf);
+}
+
+static NORETURN void exit_cleanup(const char *tmpdir, int exit_code)
+{
+	struct strbuf buf = STRBUF_INIT;
+	strbuf_addstr(&buf, tmpdir);
+	remove_dir_recursively(&buf, 0);
+	if (exit_code)
+		warning(_("failed: %d"), exit_code);
+	exit(exit_code);
+}
+
+static int ensure_leading_directories(char *path)
+{
+	switch (safe_create_leading_directories(path)) {
+		case SCLD_OK:
+		case SCLD_EXISTS:
+			return 0;
+		default:
+			return error(_("could not create leading directories "
+				       "of '%s'"), path);
+	}
+}
+
+static int run_dir_diff(const char *extcmd, int symlinks, const char *prefix,
+			int argc, const char **argv)
+{
+	char tmpdir[PATH_MAX];
+	struct strbuf info = STRBUF_INIT, lpath = STRBUF_INIT;
+	struct strbuf rpath = STRBUF_INIT, buf = STRBUF_INIT;
+	struct strbuf ldir = STRBUF_INIT, rdir = STRBUF_INIT;
+	struct strbuf wtdir = STRBUF_INIT;
+	size_t ldir_len, rdir_len, wtdir_len;
+	struct cache_entry *ce = xcalloc(1, sizeof(ce) + PATH_MAX + 1);
+	const char *workdir, *tmp;
+	int ret = 0, i;
+	FILE *fp;
+	struct hashmap working_tree_dups, submodules, symlinks2;
+	struct hashmap_iter iter;
+	struct pair_entry *entry;
+	enum object_type type;
+	unsigned long size;
+	struct index_state wtindex;
+	struct checkout lstate, rstate;
+	int rc, flags = RUN_GIT_CMD, err = 0;
+	struct child_process child = CHILD_PROCESS_INIT;
+	const char *helper_argv[] = { "difftool--helper", NULL, NULL, NULL };
+	struct hashmap wt_modified, tmp_modified;
+	int indices_loaded = 0;
+
+	workdir = get_git_work_tree();
+
+	/* Setup temp directories */
+	tmp = getenv("TMPDIR");
+	xsnprintf(tmpdir, sizeof(tmpdir), "%s/git-difftool.XXXXXX", tmp ? tmp : "/tmp");
+	if (!mkdtemp(tmpdir))
+		return error("could not create '%s'", tmpdir);
+	strbuf_addf(&ldir, "%s/left/", tmpdir);
+	strbuf_addf(&rdir, "%s/right/", tmpdir);
+	strbuf_addstr(&wtdir, workdir);
+	if (!wtdir.len || !is_dir_sep(wtdir.buf[wtdir.len - 1]))
+		strbuf_addch(&wtdir, '/');
+	mkdir(ldir.buf, 0700);
+	mkdir(rdir.buf, 0700);
+
+	memset(&wtindex, 0, sizeof(wtindex));
+
+	memset(&lstate, 0, sizeof(lstate));
+	lstate.base_dir = ldir.buf;
+	lstate.base_dir_len = ldir.len;
+	lstate.force = 1;
+	memset(&rstate, 0, sizeof(rstate));
+	rstate.base_dir = rdir.buf;
+	rstate.base_dir_len = rdir.len;
+	rstate.force = 1;
+
+	ldir_len = ldir.len;
+	rdir_len = rdir.len;
+	wtdir_len = wtdir.len;
+
+	hashmap_init(&working_tree_dups,
+		     (hashmap_cmp_fn)working_tree_entry_cmp, 0);
+	hashmap_init(&submodules, (hashmap_cmp_fn)pair_cmp, 0);
+	hashmap_init(&symlinks2, (hashmap_cmp_fn)pair_cmp, 0);
+
+	child.no_stdin = 1;
+	child.git_cmd = 1;
+	child.use_shell = 0;
+	child.clean_on_exit = 1;
+	child.dir = prefix;
+	child.out = -1;
+	argv_array_pushl(&child.args, "diff", "--raw", "--no-abbrev", "-z",
+			 NULL);
+	for (i = 0; i < argc; i++)
+		argv_array_push(&child.args, argv[i]);
+	if (start_command(&child))
+		die("could not obtain raw diff");
+	fp = xfdopen(child.out, "r");
+
+	/* Build index info for left and right sides of the diff */
+	i = 0;
+	while (!strbuf_getline_nul(&info, fp)) {
+		int lmode, rmode;
+		struct object_id loid, roid;
+		char status;
+		const char *src_path, *dst_path;
+		size_t src_path_len, dst_path_len;
+
+		if (starts_with(info.buf, "::"))
+			die(N_("combined diff formats('-c' and '--cc') are "
+			       "not supported in\n"
+			       "directory diff mode('-d' and '--dir-diff')."));
+
+		if (parse_index_info(info.buf, &lmode, &rmode, &loid, &roid,
+				     &status))
+			break;
+		if (strbuf_getline_nul(&lpath, fp))
+			break;
+		src_path = lpath.buf;
+		src_path_len = lpath.len;
+
+		i++;
+		if (status != 'C' && status != 'R') {
+			dst_path = src_path;
+			dst_path_len = src_path_len;
+		} else {
+			if (strbuf_getline_nul(&rpath, fp))
+				break;
+			dst_path = rpath.buf;
+			dst_path_len = rpath.len;
+		}
+
+		if (S_ISGITLINK(lmode) || S_ISGITLINK(rmode)) {
+			strbuf_reset(&buf);
+			strbuf_addf(&buf, "Subproject commit %s",
+				    oid_to_hex(&loid));
+			add_left_or_right(&submodules, src_path, buf.buf, 0);
+			strbuf_reset(&buf);
+			strbuf_addf(&buf, "Subproject commit %s",
+				    oid_to_hex(&roid));
+			if (!oidcmp(&loid, &roid))
+				strbuf_addstr(&buf, "-dirty");
+			add_left_or_right(&submodules, dst_path, buf.buf, 1);
+			continue;
+		}
+
+		if (S_ISLNK(lmode)) {
+			char *content = read_sha1_file(loid.hash, &type, &size);
+			add_left_or_right(&symlinks2, src_path, content, 0);
+			free(content);
+		}
+
+		if (S_ISLNK(rmode)) {
+			char *content = read_sha1_file(roid.hash, &type, &size);
+			add_left_or_right(&symlinks2, dst_path, content, 1);
+			free(content);
+		}
+
+		if (lmode && status != 'C') {
+			ce->ce_mode = lmode;
+			oidcpy(&ce->oid, &loid);
+			strcpy(ce->name, src_path);
+			ce->ce_namelen = src_path_len;
+			if (checkout_entry(ce, &lstate, NULL))
+				return error("could not write '%s'", src_path);
+		}
+
+		if (rmode) {
+			struct working_tree_entry *entry;
+
+			/* Avoid duplicate working_tree entries */
+			FLEX_ALLOC_STR(entry, path, dst_path);
+			hashmap_entry_init(entry, strhash(dst_path));
+			if (hashmap_get(&working_tree_dups, entry, NULL)) {
+				free(entry);
+				continue;
+			}
+			hashmap_add(&working_tree_dups, entry);
+
+			if (!use_wt_file(workdir, dst_path, &roid)) {
+				ce->ce_mode = rmode;
+				oidcpy(&ce->oid, &roid);
+				strcpy(ce->name, dst_path);
+				ce->ce_namelen = dst_path_len;
+				if (checkout_entry(ce, &rstate, NULL))
+					return error("could not write '%s'",
+						     dst_path);
+			} else if (!is_null_oid(&roid)) {
+				/*
+				 * Changes in the working tree need special
+				 * treatment since they are not part of the
+				 * index.
+				 */
+				struct cache_entry *ce2 =
+					make_cache_entry(rmode, roid.hash,
+							 dst_path, 0, 0);
+
+				add_index_entry(&wtindex, ce2,
+						ADD_CACHE_JUST_APPEND);
+
+				add_path(&rdir, rdir_len, dst_path);
+				if (ensure_leading_directories(rdir.buf))
+					return error("could not create "
+						     "directory for '%s'",
+						     dst_path);
+				add_path(&wtdir, wtdir_len, dst_path);
+				if (symlinks) {
+					if (symlink(wtdir.buf, rdir.buf)) {
+						ret = error_errno("could not symlink '%s' to '%s'", wtdir.buf, rdir.buf);
+						goto finish;
+					}
+				} else {
+					struct stat st;
+					if (stat(wtdir.buf, &st))
+						st.st_mode = 0644;
+					if (copy_file(rdir.buf, wtdir.buf,
+						      st.st_mode)) {
+						ret = error("could not copy '%s' to '%s'", wtdir.buf, rdir.buf);
+						goto finish;
+					}
+				}
+			}
+		}
+	}
+
+	if (finish_command(&child)) {
+		ret = error("error occurred running diff --raw");
+		goto finish;
+	}
+
+	if (!i)
+		return 0;
+
+	/*
+	 * Changes to submodules require special treatment.This loop writes a
+	 * temporary file to both the left and right directories to show the
+	 * change in the recorded SHA1 for the submodule.
+	 */
+	hashmap_iter_init(&submodules, &iter);
+	while ((entry = hashmap_iter_next(&iter))) {
+		if (*entry->left) {
+			add_path(&ldir, ldir_len, entry->path);
+			ensure_leading_directories(ldir.buf);
+			write_file(ldir.buf, "%s", entry->left);
+		}
+		if (*entry->right) {
+			add_path(&rdir, rdir_len, entry->path);
+			ensure_leading_directories(rdir.buf);
+			write_file(rdir.buf, "%s", entry->right);
+		}
+	}
+
+	/*
+	 * Symbolic links require special treatment.The standard "git diff"
+	 * shows only the link itself, not the contents of the link target.
+	 * This loop replicates that behavior.
+	 */
+	hashmap_iter_init(&symlinks2, &iter);
+	while ((entry = hashmap_iter_next(&iter))) {
+		if (*entry->left) {
+			add_path(&ldir, ldir_len, entry->path);
+			ensure_leading_directories(ldir.buf);
+			write_file(ldir.buf, "%s", entry->left);
+		}
+		if (*entry->right) {
+			add_path(&rdir, rdir_len, entry->path);
+			ensure_leading_directories(rdir.buf);
+			write_file(rdir.buf, "%s", entry->right);
+		}
+	}
+
+	strbuf_release(&buf);
+
+	strbuf_setlen(&ldir, ldir_len);
+	helper_argv[1] = ldir.buf;
+	strbuf_setlen(&rdir, rdir_len);
+	helper_argv[2] = rdir.buf;
+
+	if (extcmd) {
+		helper_argv[0] = extcmd;
+		flags = 0;
+	} else
+		setenv("GIT_DIFFTOOL_DIRDIFF", "true", 1);
+	rc = run_command_v_opt(helper_argv, flags);
+
+	/*
+	 * If the diff includes working copy files and those
+	 * files were modified during the diff, then the changes
+	 * should be copied back to the working tree.
+	 * Do not copy back files when symlinks are used and the
+	 * external tool did not replace the original link with a file.
+	 *
+	 * These hashes are loaded lazily since they aren't needed
+	 * in the common case of --symlinks and the difftool updating
+	 * files through the symlink.
+	 */
+	hashmap_init(&wt_modified, (hashmap_cmp_fn)path_entry_cmp,
+		     wtindex.cache_nr);
+	hashmap_init(&tmp_modified, (hashmap_cmp_fn)path_entry_cmp,
+		     wtindex.cache_nr);
+
+	for (i = 0; i < wtindex.cache_nr; i++) {
+		struct hashmap_entry dummy;
+		const char *name = wtindex.cache[i]->name;
+		struct stat st;
+
+		add_path(&rdir, rdir_len, name);
+		if (lstat(rdir.buf, &st))
+			continue;
+
+		if ((symlinks && S_ISLNK(st.st_mode)) || !S_ISREG(st.st_mode))
+			continue;
+
+		if (!indices_loaded) {
+			static struct lock_file lock;
+			strbuf_reset(&buf);
+			strbuf_addf(&buf, "%s/wtindex", tmpdir);
+			if (hold_lock_file_for_update(&lock, buf.buf, 0) < 0 ||
+			    write_locked_index(&wtindex, &lock, COMMIT_LOCK)) {
+				ret = error("could not write %s", buf.buf);
+				rollback_lock_file(&lock);
+				goto finish;
+			}
+			changed_files(&wt_modified, buf.buf, workdir);
+			strbuf_setlen(&rdir, rdir_len);
+			changed_files(&tmp_modified, buf.buf, rdir.buf);
+			add_path(&rdir, rdir_len, name);
+			indices_loaded = 1;
+		}
+
+		hashmap_entry_init(&dummy, strhash(name));
+		if (hashmap_get(&tmp_modified, &dummy, name)) {
+			add_path(&wtdir, wtdir_len, name);
+			if (hashmap_get(&wt_modified, &dummy, name)) {
+				warning(_("both files modified: '%s' and '%s'."),
+					wtdir.buf, rdir.buf);
+				warning(_("working tree file has been left."));
+				warning("");
+				err = 1;
+			} else if (unlink(wtdir.buf) ||
+				   copy_file(wtdir.buf, rdir.buf, st.st_mode))
+				warning_errno(_("could not copy '%s' to '%s'"),
+					      rdir.buf, wtdir.buf);
+		}
+	}
+
+	if (err) {
+		warning(_("temporary files exist in '%s'."), tmpdir);
+		warning(_("you may want to cleanup or recover these."));
+		exit(1);
+	} else
+		exit_cleanup(tmpdir, rc);
+
+finish:
+	free(ce);
+	strbuf_release(&ldir);
+	strbuf_release(&rdir);
+	strbuf_release(&wtdir);
+	strbuf_release(&buf);
+
+	return ret;
+}
+
+static int run_file_diff(int prompt, const char *prefix,
+			 int argc, const char **argv)
+{
+	struct argv_array args = ARGV_ARRAY_INIT;
+	const char *env[] = {
+		"GIT_PAGER=", "GIT_EXTERNAL_DIFF=git-difftool--helper", NULL,
+		NULL
+	};
+	int ret = 0, i;
+
+	if (prompt > 0)
+		env[2] = "GIT_DIFFTOOL_PROMPT=true";
+	else if (!prompt)
+		env[2] = "GIT_DIFFTOOL_NO_PROMPT=true";
+
+
+	argv_array_push(&args, "diff");
+	for (i = 0; i < argc; i++)
+		argv_array_push(&args, argv[i]);
+	ret = run_command_v_opt_cd_env(args.argv, RUN_GIT_CMD, prefix, env);
+	exit(ret);
+}
 
 /*
  * NEEDSWORK: this function can go once the legacy-difftool Perl script is
@@ -41,6 +642,35 @@ static int use_builtin_difftool(void) {
 
 int cmd_difftool(int argc, const char **argv, const char *prefix)
 {
+	int use_gui_tool = 0, dir_diff = 0, prompt = -1, symlinks = 0,
+	    tool_help = 0;
+	static char *difftool_cmd = NULL, *extcmd = NULL;
+	struct option builtin_difftool_options[] = {
+		OPT_BOOL('g', "gui", &use_gui_tool,
+			 N_("use `diff.guitool` instead of `diff.tool`")),
+		OPT_BOOL('d', "dir-diff", &dir_diff,
+			 N_("perform a full-directory diff")),
+		{ OPTION_SET_INT, 'y', "no-prompt", &prompt, NULL,
+			N_("do not prompt before launching a diff tool"),
+			PARSE_OPT_NOARG | PARSE_OPT_NONEG, NULL, 0},
+		{ OPTION_SET_INT, 0, "prompt", &prompt, NULL, NULL,
+			PARSE_OPT_NOARG | PARSE_OPT_NONEG | PARSE_OPT_HIDDEN,
+			NULL, 1 },
+		OPT_BOOL(0, "symlinks", &symlinks,
+			 N_("use symlinks in dir-diff mode")),
+		OPT_STRING('t', "tool", &difftool_cmd, N_("<tool>"),
+			   N_("use the specified diff tool")),
+		OPT_BOOL(0, "tool-help", &tool_help,
+			 N_("print a list of diff tools that may be used with "
+			    "`--tool`")),
+		OPT_BOOL(0, "trust-exit-code", &trust_exit_code,
+			 N_("make 'git-difftool' exit when an invoked diff "
+			    "tool returns a non - zero exit code")),
+		OPT_STRING('x', "extcmd", &extcmd, N_("<command>"),
+			   N_("specify a custom command for viewing diffs")),
+		OPT_END()
+	};
+
 	/*
 	 * NEEDSWORK: Once the builtin difftool has been tested enough
 	 * and git-legacy-difftool.perl is retired to contrib/, this preamble
@@ -58,6 +688,46 @@ int cmd_difftool(int argc, const char **argv, const char *prefix)
 	prefix = setup_git_directory();
 	trace_repo_setup(prefix);
 	setup_work_tree();
+	/* NEEDSWORK: once we no longer spawn anything, remove this */
+	setenv(GIT_DIR_ENVIRONMENT, absolute_path(get_git_dir()), 1);
+	setenv(GIT_WORK_TREE_ENVIRONMENT, absolute_path(get_git_work_tree()), 1);
+
+	git_config(difftool_config, NULL);
+	symlinks = has_symlinks;
+
+	argc = parse_options(argc, argv, prefix, builtin_difftool_options,
+			     builtin_difftool_usage, PARSE_OPT_KEEP_UNKNOWN |
+			     PARSE_OPT_KEEP_DASHDASH);
 
-	die("TODO");
+	if (tool_help)
+		return print_tool_help();
+
+	if (use_gui_tool && diff_gui_tool && *diff_gui_tool)
+		setenv("GIT_DIFF_TOOL", diff_gui_tool, 1);
+	else if (difftool_cmd) {
+		if (*difftool_cmd)
+			setenv("GIT_DIFF_TOOL", difftool_cmd, 1);
+		else
+			die(_("no <tool> given for --tool=<tool>"));
+	}
+
+	if (extcmd) {
+		if (*extcmd)
+			setenv("GIT_DIFFTOOL_EXTCMD", extcmd, 1);
+		else
+			die(_("no <cmd> given for --extcmd=<cmd>"));
+	}
+
+	setenv("GIT_DIFFTOOL_TRUST_EXIT_CODE",
+	       trust_exit_code ? "true" : "false", 1);
+
+	/*
+	 * In directory diff mode, 'git-difftool--helper' is called once
+	 * to compare the a / b directories. In file diff mode, 'git diff'
+	 * will invoke a separate instance of 'git-difftool--helper' for
+	 * each file that changed.
+	 */
+	if (dir_diff)
+		return run_dir_diff(extcmd, symlinks, prefix, argc, argv);
+	return run_file_diff(prompt, prefix, argc, argv);
 }
-- 
2.11.0.rc3.windows.1

[PATCH v4 1/4] Avoid Coverity warning about unfree()d git_exec_path()

From: Johannes Schindelin <hidden>
Date: 2017-01-02 16:23:02

Technically, it is correct that git_exec_path() returns a possibly
malloc()ed string. Practically, it is *sometimes* not malloc()ed. So
let's just use a static variable to make it a singleton. That'll shut
Coverity up, hopefully.

Signed-off-by: Johannes Schindelin <redacted>
---
 exec_cmd.c | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)
diff --git a/exec_cmd.c b/exec_cmd.c
index 19ac2146d0..587bd7eb48 100644
--- a/exec_cmd.c
+++ b/exec_cmd.c
@@ -65,6 +65,7 @@ void git_set_argv_exec_path(const char *exec_path)
 const char *git_exec_path(void)
 {
 	const char *env;
+	static char *system_exec_path;
 
 	if (argv_exec_path)
 		return argv_exec_path;
@@ -74,7 +75,9 @@ const char *git_exec_path(void)
 		return env;
 	}
 
-	return system_path(GIT_EXEC_PATH);
+	if (!system_exec_path)
+		system_exec_path = system_path(GIT_EXEC_PATH);
+	return system_exec_path;
 }
 
 static void add_path(struct strbuf *out, const char *path)
-- 
2.11.0.rc3.windows.1

Re: [PATCH v4 1/4] Avoid Coverity warning about unfree()d git_exec_path()

From: Stefan Beller <hidden>
Date: 2017-01-03 20:11:30

On Mon, Jan 2, 2017 at 8:22 AM, Johannes Schindelin
[off-list ref] wrote:
Technically, it is correct that git_exec_path() returns a possibly
malloc()ed string. Practically, it is *sometimes* not malloc()ed. So
let's just use a static variable to make it a singleton. That'll shut
Coverity up, hopefully.
I picked up this patch and applied it to the coverity branch
that I maintain at github/stefanbeller/git.

I'd love to see this patch upstream as it reduces my maintenance
burden of the coverity branch by a patch.

Early on when Git was new to coverity, some arguments were made
that patches like these only clutter the main code base which is read
by a lot of people, hence we want these quirks for coverity not upstream.
And I think that still holds.

If this patch is only to appease coverity (as the commit message eludes
to) I think this may be a bad idea for upstream.  If this patch fixes an
actual problem, then the commit message needs to spell that out.

Thanks,
Stefan

Re: [PATCH v4 1/4] Avoid Coverity warning about unfree()d git_exec_path()

From: Jeff King <hidden>
Date: 2017-01-04 01:13:53

On Tue, Jan 03, 2017 at 12:11:25PM -0800, Stefan Beller wrote:
On Mon, Jan 2, 2017 at 8:22 AM, Johannes Schindelin
[off-list ref] wrote:
quoted
Technically, it is correct that git_exec_path() returns a possibly
malloc()ed string. Practically, it is *sometimes* not malloc()ed. So
let's just use a static variable to make it a singleton. That'll shut
Coverity up, hopefully.
I picked up this patch and applied it to the coverity branch
that I maintain at github/stefanbeller/git.

I'd love to see this patch upstream as it reduces my maintenance
burden of the coverity branch by a patch.
There is another lurking issue in that function, which is that the
return value of getenv() is not guaranteed to last beyond more calls to
getenv() or setenv(). It should probably xstrdup() that result, too.

At that point 2 out of 3 of the return cases would be malloc'd strings,
so we _could_ switch the third and say "caller must free the result".
But I think I prefer something like Dscho's solution (more on that
below).
Early on when Git was new to coverity, some arguments were made
that patches like these only clutter the main code base which is read
by a lot of people, hence we want these quirks for coverity not upstream.
And I think that still holds.

If this patch is only to appease coverity (as the commit message eludes
to) I think this may be a bad idea for upstream.  If this patch fixes an
actual problem, then the commit message needs to spell that out.
This is a real leak, though in all cases the program typically exits
soon afterwards. But we leak from list_commands(), for example, and it
is not immediately obvious that this is only called right before
exiting.

But I think more important is that caching the result in a static
variable communicates something (both to Coverity and to a reader of the
code). This is a value we expect to live through the life of the
program, and it is OK for it to "leak" when it goes out of scope by the
program exiting.

So even though the behavior does not really change, the annotation has
value.

-Peff

Re: [PATCH v4 1/4] Avoid Coverity warning about unfree()d git_exec_path()

From: Johannes Schindelin <hidden>
Date: 2017-01-04 09:50:08

Hi Stefan,

On Tue, 3 Jan 2017, Stefan Beller wrote:
On Mon, Jan 2, 2017 at 8:22 AM, Johannes Schindelin
[off-list ref] wrote:
quoted
Technically, it is correct that git_exec_path() returns a possibly
malloc()ed string. Practically, it is *sometimes* not malloc()ed. So
let's just use a static variable to make it a singleton. That'll shut
Coverity up, hopefully.
I picked up this patch and applied it to the coverity branch
that I maintain at github/stefanbeller/git.

I'd love to see this patch upstream as it reduces my maintenance
burden of the coverity branch by a patch.

Early on when Git was new to coverity, some arguments were made
that patches like these only clutter the main code base which is read
by a lot of people, hence we want these quirks for coverity not upstream.
And I think that still holds.

If this patch is only to appease coverity (as the commit message eludes
to) I think this may be a bad idea for upstream.  If this patch fixes an
actual problem, then the commit message needs to spell that out.
This patch was originally only to appease Coverity, but it actually *does*
plug a very real memory leak: previously, *every* call to git_exec_path()
*possibly* returned a newly-malloc()ed buffer. Now, the first call will
store that pointer in a static variable and reuse it later.

Could you maybe help me with improving the commit message?

Ciao,
Dscho

Re: [PATCH v4 1/4] Avoid Coverity warning about unfree()d git_exec_path()

From: Stefan Beller <hidden>
Date: 2017-01-04 18:16:03

On Tue, Jan 3, 2017 at 1:33 PM, Johannes Schindelin
[off-list ref] wrote:
This patch was originally only to appease Coverity, but it actually *does*
plug a very real memory leak: previously, *every* call to git_exec_path()
*possibly* returned a newly-malloc()ed buffer. Now, the first call will
store that pointer in a static variable and reuse it later.

Could you maybe help me with improving the commit message?
As someone not familiar with that area of code, this explained it
enough for me to understand, so maybe:

    exec_cmd: do not leak via git_exec_path

    Every call to git_exec_path() possibly returned a newly-malloc()ed
    buffer. Now, the first call will allocate the buffer and subsequent
    calls return a pointer to it, which then prevents leaking memory
    on each call.

The return value of a "const char *" hints to the caller, that the memory
is not owned by the caller, do we need to be explicit there (i.e. a comment
declaring the memory ownership? Probably not.)

Thanks,
Stefan

[PATCH v4 4/4] t7800: run both builtin and scripted difftool, for now

From: Johannes Schindelin <hidden>
Date: 2017-01-02 16:24:32

This is uglier than a simple

	touch "$GIT_EXEC_PATH/use-builtin-difftool"

of course. But oh well.

Signed-off-by: Johannes Schindelin <redacted>
---

	This patch implements the good ole' cross-validation technique
	(also known as "GitHub Scientist") that I already used for my
	rebase--helper work.

	I am not sure whether we want to have that patch in `master`,
	ever. But at least for the transition time, it may make sense.

 t/t7800-difftool.sh | 27 +++++++++++++++++++++++++++
 1 file changed, 27 insertions(+)
diff --git a/t/t7800-difftool.sh b/t/t7800-difftool.sh
index e94910c563..273ab55723 100755
--- a/t/t7800-difftool.sh
+++ b/t/t7800-difftool.sh
@@ -23,6 +23,20 @@ prompt_given ()
 	test "$prompt" = "Launch 'test-tool' [Y/n]? branch"
 }
 
+for use_builtin_difftool in false true
+do
+
+test_expect_success 'verify we are running the correct difftool' '
+	if test true = '$use_builtin_difftool'
+	then
+		test_must_fail ok=129 git difftool -h >help &&
+		grep "g, --gui" help
+	else
+		git difftool -h >help &&
+		grep "g|--gui" help
+	fi
+'
+
 # NEEDSWORK: lose all the PERL prereqs once legacy-difftool is retired.
 
 # Create a file on master and change it on branch
@@ -606,4 +620,17 @@ test_expect_success PERL,SYMLINKS 'difftool --dir-diff symlinked directories' '
 	)
 '
 
+test true != $use_builtin_difftool || break
+
+test_expect_success 'tear down for re-run' '
+	rm -rf * .[a-z]* &&
+	git init
+'
+
+# run as builtin difftool now
+GIT_CONFIG_PARAMETERS="'difftool.usebuiltin=true'"
+export GIT_CONFIG_PARAMETERS
+
+done
+
 test_done
-- 
2.11.0.rc3.windows.1

[PATCH v5 0/3] Turn the difftool into a builtin

From: Johannes Schindelin <hidden>
Date: 2017-01-17 16:03:37

This patch series converts the difftool from a Perl script into a
builtin, for three reasons:

1. Perl is really not native on Windows. Not only is there a performance
   penalty to be paid just for running Perl scripts, we also have to deal
   with the fact that users may have different Perl installations, with
   different options, and some other Perl installation may decide to set
   PERL5LIB globally, wreaking havoc with Git for Windows' Perl (which we
   have to use because almost all other Perl distributions lack the
   Subversion bindings we need for `git svn`).

2. As the Perl script uses Unix-y paths that are not native to Windows,
   the Perl interpreter has to go through a POSIX emulation layer (the
   MSYS2 runtime). This means that paths have to be converted from
   Unix-y paths to Windows-y paths (and vice versa) whenever crossing
   the POSIX emulation barrier, leading to quite possibly surprising path
   translation errors.

3. Perl makes for a rather large reason that Git for Windows' installer
   weighs in with >30MB. While one Perl script less does not relieve us
   of that burden, it is one step in the right direction.

Changes since v4:

- skipped the unrelated Coverity-appeasing patch.

- replaced the cross-validation with the Perl script by a patch that
  retires the Perl script instead.


Johannes Schindelin (3):
  difftool: add a skeleton for the upcoming builtin
  difftool: implement the functionality in the builtin
  Retire the scripted difftool

 Makefile                                           |   2 +-
 builtin.h                                          |   1 +
 builtin/difftool.c                                 | 692 +++++++++++++++++++++
 .../examples/git-difftool.perl                     |   0
 git.c                                              |   1 +
 t/t7800-difftool.sh                                |  92 +--
 6 files changed, 741 insertions(+), 47 deletions(-)
 create mode 100644 builtin/difftool.c
 rename git-difftool.perl => contrib/examples/git-difftool.perl (100%)


base-commit: d7dffce1cebde29a0c4b309a79e4345450bf352a
Published-As: https://github.com/dscho/git/releases/tag/builtin-difftool-v5
Fetch-It-Via: git fetch https://github.com/dscho/git builtin-difftool-v5

Interdiff vs v4:

 diff --git a/.gitignore b/.gitignore
 index 5555ae025b..6722f78f9a 100644
 --- a/.gitignore
 +++ b/.gitignore
 @@ -76,7 +76,6 @@
  /git-init-db
  /git-interpret-trailers
  /git-instaweb
 -/git-legacy-difftool
  /git-log
  /git-ls-files
  /git-ls-remote
 diff --git a/Makefile b/Makefile
 index 8cf5bef034..e9aa6ae57c 100644
 --- a/Makefile
 +++ b/Makefile
 @@ -522,7 +522,6 @@ SCRIPT_LIB += git-sh-setup
  SCRIPT_LIB += git-sh-i18n
  
  SCRIPT_PERL += git-add--interactive.perl
 -SCRIPT_PERL += git-legacy-difftool.perl
  SCRIPT_PERL += git-archimport.perl
  SCRIPT_PERL += git-cvsexportcommit.perl
  SCRIPT_PERL += git-cvsimport.perl
 diff --git a/builtin/difftool.c b/builtin/difftool.c
 index 2115e548a5..42ad9e804a 100644
 --- a/builtin/difftool.c
 +++ b/builtin/difftool.c
 @@ -616,30 +616,6 @@ static int run_file_diff(int prompt, const char *prefix,
  	exit(ret);
  }
  
 -/*
 - * NEEDSWORK: this function can go once the legacy-difftool Perl script is
 - * retired.
 - *
 - * We intentionally avoid reading the config directly here, to avoid messing up
 - * the GIT_* environment variables when we need to fall back to exec()ing the
 - * Perl script.
 - */
 -static int use_builtin_difftool(void) {
 -	struct child_process cp = CHILD_PROCESS_INIT;
 -	struct strbuf out = STRBUF_INIT;
 -	int ret;
 -
 -	argv_array_pushl(&cp.args,
 -			 "config", "--bool", "difftool.usebuiltin", NULL);
 -	cp.git_cmd = 1;
 -	if (capture_command(&cp, &out, 6))
 -		return 0;
 -	strbuf_trim(&out);
 -	ret = !strcmp("true", out.buf);
 -	strbuf_release(&out);
 -	return ret;
 -}
 -
  int cmd_difftool(int argc, const char **argv, const char *prefix)
  {
  	int use_gui_tool = 0, dir_diff = 0, prompt = -1, symlinks = 0,
 @@ -671,23 +647,6 @@ int cmd_difftool(int argc, const char **argv, const char *prefix)
  		OPT_END()
  	};
  
 -	/*
 -	 * NEEDSWORK: Once the builtin difftool has been tested enough
 -	 * and git-legacy-difftool.perl is retired to contrib/, this preamble
 -	 * can be removed.
 -	 */
 -	if (!use_builtin_difftool()) {
 -		const char *path = mkpath("%s/git-legacy-difftool",
 -					  git_exec_path());
 -
 -		if (sane_execvp(path, (char **)argv) < 0)
 -			die_errno("could not exec %s", path);
 -
 -		return 0;
 -	}
 -	prefix = setup_git_directory();
 -	trace_repo_setup(prefix);
 -	setup_work_tree();
  	/* NEEDSWORK: once we no longer spawn anything, remove this */
  	setenv(GIT_DIR_ENVIRONMENT, absolute_path(get_git_dir()), 1);
  	setenv(GIT_WORK_TREE_ENVIRONMENT, absolute_path(get_git_work_tree()), 1);
 diff --git a/git-legacy-difftool.perl b/contrib/examples/git-difftool.perl
 similarity index 100%
 rename from git-legacy-difftool.perl
 rename to contrib/examples/git-difftool.perl
 diff --git a/exec_cmd.c b/exec_cmd.c
 index 587bd7eb48..19ac2146d0 100644
 --- a/exec_cmd.c
 +++ b/exec_cmd.c
 @@ -65,7 +65,6 @@ void git_set_argv_exec_path(const char *exec_path)
  const char *git_exec_path(void)
  {
  	const char *env;
 -	static char *system_exec_path;
  
  	if (argv_exec_path)
  		return argv_exec_path;
 @@ -75,9 +74,7 @@ const char *git_exec_path(void)
  		return env;
  	}
  
 -	if (!system_exec_path)
 -		system_exec_path = system_path(GIT_EXEC_PATH);
 -	return system_exec_path;
 +	return system_path(GIT_EXEC_PATH);
  }
  
  static void add_path(struct strbuf *out, const char *path)
 diff --git a/git.c b/git.c
 index c58181e5ef..bd4d668a21 100644
 --- a/git.c
 +++ b/git.c
 @@ -424,12 +424,7 @@ static struct cmd_struct commands[] = {
  	{ "diff-files", cmd_diff_files, RUN_SETUP | NEED_WORK_TREE },
  	{ "diff-index", cmd_diff_index, RUN_SETUP },
  	{ "diff-tree", cmd_diff_tree, RUN_SETUP },
 -	/*
 -	 * NEEDSWORK: Once the redirection to git-legacy-difftool.perl in
 -	 * builtin/difftool.c has been removed, this entry should be changed to
 -	 * RUN_SETUP | NEED_WORK_TREE
 -	 */
 -	{ "difftool", cmd_difftool },
 +	{ "difftool", cmd_difftool, RUN_SETUP | NEED_WORK_TREE },
  	{ "fast-export", cmd_fast_export, RUN_SETUP },
  	{ "fetch", cmd_fetch, RUN_SETUP },
  	{ "fetch-pack", cmd_fetch_pack, RUN_SETUP },
 diff --git a/t/t7800-difftool.sh b/t/t7800-difftool.sh
 index 273ab55723..aa0ef02597 100755
 --- a/t/t7800-difftool.sh
 +++ b/t/t7800-difftool.sh
 @@ -23,24 +23,8 @@ prompt_given ()
  	test "$prompt" = "Launch 'test-tool' [Y/n]? branch"
  }
  
 -for use_builtin_difftool in false true
 -do
 -
 -test_expect_success 'verify we are running the correct difftool' '
 -	if test true = '$use_builtin_difftool'
 -	then
 -		test_must_fail ok=129 git difftool -h >help &&
 -		grep "g, --gui" help
 -	else
 -		git difftool -h >help &&
 -		grep "g|--gui" help
 -	fi
 -'
 -
 -# NEEDSWORK: lose all the PERL prereqs once legacy-difftool is retired.
 -
  # Create a file on master and change it on branch
 -test_expect_success PERL 'setup' '
 +test_expect_success 'setup' '
  	echo master >file &&
  	git add file &&
  	git commit -m "added file" &&
 @@ -52,7 +36,7 @@ test_expect_success PERL 'setup' '
  '
  
  # Configure a custom difftool.<tool>.cmd and use it
 -test_expect_success PERL 'custom commands' '
 +test_expect_success 'custom commands' '
  	difftool_test_setup &&
  	test_config difftool.test-tool.cmd "cat \"\$REMOTE\"" &&
  	echo master >expect &&
 @@ -65,21 +49,21 @@ test_expect_success PERL 'custom commands' '
  	test_cmp expect actual
  '
  
 -test_expect_success PERL 'custom tool commands override built-ins' '
 +test_expect_success 'custom tool commands override built-ins' '
  	test_config difftool.vimdiff.cmd "cat \"\$REMOTE\"" &&
  	echo master >expect &&
  	git difftool --tool vimdiff --no-prompt branch >actual &&
  	test_cmp expect actual
  '
  
 -test_expect_success PERL 'difftool ignores bad --tool values' '
 +test_expect_success 'difftool ignores bad --tool values' '
  	: >expect &&
  	test_must_fail \
  		git difftool --no-prompt --tool=bad-tool branch >actual &&
  	test_cmp expect actual
  '
  
 -test_expect_success PERL 'difftool forwards arguments to diff' '
 +test_expect_success 'difftool forwards arguments to diff' '
  	difftool_test_setup &&
  	>for-diff &&
  	git add for-diff &&
 @@ -92,40 +76,40 @@ test_expect_success PERL 'difftool forwards arguments to diff' '
  	rm for-diff
  '
  
 -test_expect_success PERL 'difftool ignores exit code' '
 +test_expect_success 'difftool ignores exit code' '
  	test_config difftool.error.cmd false &&
  	git difftool -y -t error branch
  '
  
 -test_expect_success PERL 'difftool forwards exit code with --trust-exit-code' '
 +test_expect_success 'difftool forwards exit code with --trust-exit-code' '
  	test_config difftool.error.cmd false &&
  	test_must_fail git difftool -y --trust-exit-code -t error branch
  '
  
 -test_expect_success PERL 'difftool forwards exit code with --trust-exit-code for built-ins' '
 +test_expect_success 'difftool forwards exit code with --trust-exit-code for built-ins' '
  	test_config difftool.vimdiff.path false &&
  	test_must_fail git difftool -y --trust-exit-code -t vimdiff branch
  '
  
 -test_expect_success PERL 'difftool honors difftool.trustExitCode = true' '
 +test_expect_success 'difftool honors difftool.trustExitCode = true' '
  	test_config difftool.error.cmd false &&
  	test_config difftool.trustExitCode true &&
  	test_must_fail git difftool -y -t error branch
  '
  
 -test_expect_success PERL 'difftool honors difftool.trustExitCode = false' '
 +test_expect_success 'difftool honors difftool.trustExitCode = false' '
  	test_config difftool.error.cmd false &&
  	test_config difftool.trustExitCode false &&
  	git difftool -y -t error branch
  '
  
 -test_expect_success PERL 'difftool ignores exit code with --no-trust-exit-code' '
 +test_expect_success 'difftool ignores exit code with --no-trust-exit-code' '
  	test_config difftool.error.cmd false &&
  	test_config difftool.trustExitCode true &&
  	git difftool -y --no-trust-exit-code -t error branch
  '
  
 -test_expect_success PERL 'difftool stops on error with --trust-exit-code' '
 +test_expect_success 'difftool stops on error with --trust-exit-code' '
  	test_when_finished "rm -f for-diff .git/fail-right-file" &&
  	test_when_finished "git reset -- for-diff" &&
  	write_script .git/fail-right-file <<-\EOF &&
 @@ -140,13 +124,13 @@ test_expect_success PERL 'difftool stops on error with --trust-exit-code' '
  	test_cmp expect actual
  '
  
 -test_expect_success PERL 'difftool honors exit status if command not found' '
 +test_expect_success 'difftool honors exit status if command not found' '
  	test_config difftool.nonexistent.cmd i-dont-exist &&
  	test_config difftool.trustExitCode false &&
  	test_must_fail git difftool -y -t nonexistent branch
  '
  
 -test_expect_success PERL 'difftool honors --gui' '
 +test_expect_success 'difftool honors --gui' '
  	difftool_test_setup &&
  	test_config merge.tool bogus-tool &&
  	test_config diff.tool bogus-tool &&
 @@ -157,7 +141,7 @@ test_expect_success PERL 'difftool honors --gui' '
  	test_cmp expect actual
  '
  
 -test_expect_success PERL 'difftool --gui last setting wins' '
 +test_expect_success 'difftool --gui last setting wins' '
  	difftool_test_setup &&
  	: >expect &&
  	git difftool --no-prompt --gui --no-gui >actual &&
 @@ -171,7 +155,7 @@ test_expect_success PERL 'difftool --gui last setting wins' '
  	test_cmp expect actual
  '
  
 -test_expect_success PERL 'difftool --gui works without configured diff.guitool' '
 +test_expect_success 'difftool --gui works without configured diff.guitool' '
  	difftool_test_setup &&
  	echo branch >expect &&
  	git difftool --no-prompt --gui branch >actual &&
 @@ -179,7 +163,7 @@ test_expect_success PERL 'difftool --gui works without configured diff.guitool'
  '
  
  # Specify the diff tool using $GIT_DIFF_TOOL
 -test_expect_success PERL 'GIT_DIFF_TOOL variable' '
 +test_expect_success 'GIT_DIFF_TOOL variable' '
  	difftool_test_setup &&
  	git config --unset diff.tool &&
  	echo branch >expect &&
 @@ -189,7 +173,7 @@ test_expect_success PERL 'GIT_DIFF_TOOL variable' '
  
  # Test the $GIT_*_TOOL variables and ensure
  # that $GIT_DIFF_TOOL always wins unless --tool is specified
 -test_expect_success PERL 'GIT_DIFF_TOOL overrides' '
 +test_expect_success 'GIT_DIFF_TOOL overrides' '
  	difftool_test_setup &&
  	test_config diff.tool bogus-tool &&
  	test_config merge.tool bogus-tool &&
 @@ -207,7 +191,7 @@ test_expect_success PERL 'GIT_DIFF_TOOL overrides' '
  
  # Test that we don't have to pass --no-prompt to difftool
  # when $GIT_DIFFTOOL_NO_PROMPT is true
 -test_expect_success PERL 'GIT_DIFFTOOL_NO_PROMPT variable' '
 +test_expect_success 'GIT_DIFFTOOL_NO_PROMPT variable' '
  	difftool_test_setup &&
  	echo branch >expect &&
  	GIT_DIFFTOOL_NO_PROMPT=true git difftool branch >actual &&
 @@ -216,7 +200,7 @@ test_expect_success PERL 'GIT_DIFFTOOL_NO_PROMPT variable' '
  
  # git-difftool supports the difftool.prompt variable.
  # Test that GIT_DIFFTOOL_PROMPT can override difftool.prompt = false
 -test_expect_success PERL 'GIT_DIFFTOOL_PROMPT variable' '
 +test_expect_success 'GIT_DIFFTOOL_PROMPT variable' '
  	difftool_test_setup &&
  	test_config difftool.prompt false &&
  	echo >input &&
 @@ -226,7 +210,7 @@ test_expect_success PERL 'GIT_DIFFTOOL_PROMPT variable' '
  '
  
  # Test that we don't have to pass --no-prompt when difftool.prompt is false
 -test_expect_success PERL 'difftool.prompt config variable is false' '
 +test_expect_success 'difftool.prompt config variable is false' '
  	difftool_test_setup &&
  	test_config difftool.prompt false &&
  	echo branch >expect &&
 @@ -235,7 +219,7 @@ test_expect_success PERL 'difftool.prompt config variable is false' '
  '
  
  # Test that we don't have to pass --no-prompt when mergetool.prompt is false
 -test_expect_success PERL 'difftool merge.prompt = false' '
 +test_expect_success 'difftool merge.prompt = false' '
  	difftool_test_setup &&
  	test_might_fail git config --unset difftool.prompt &&
  	test_config mergetool.prompt false &&
 @@ -245,7 +229,7 @@ test_expect_success PERL 'difftool merge.prompt = false' '
  '
  
  # Test that the -y flag can override difftool.prompt = true
 -test_expect_success PERL 'difftool.prompt can overridden with -y' '
 +test_expect_success 'difftool.prompt can overridden with -y' '
  	difftool_test_setup &&
  	test_config difftool.prompt true &&
  	echo branch >expect &&
 @@ -254,7 +238,7 @@ test_expect_success PERL 'difftool.prompt can overridden with -y' '
  '
  
  # Test that the --prompt flag can override difftool.prompt = false
 -test_expect_success PERL 'difftool.prompt can overridden with --prompt' '
 +test_expect_success 'difftool.prompt can overridden with --prompt' '
  	difftool_test_setup &&
  	test_config difftool.prompt false &&
  	echo >input &&
 @@ -264,7 +248,7 @@ test_expect_success PERL 'difftool.prompt can overridden with --prompt' '
  '
  
  # Test that the last flag passed on the command-line wins
 -test_expect_success PERL 'difftool last flag wins' '
 +test_expect_success 'difftool last flag wins' '
  	difftool_test_setup &&
  	echo branch >expect &&
  	git difftool --prompt --no-prompt branch >actual &&
 @@ -277,7 +261,7 @@ test_expect_success PERL 'difftool last flag wins' '
  
  # git-difftool falls back to git-mergetool config variables
  # so test that behavior here
 -test_expect_success PERL 'difftool + mergetool config variables' '
 +test_expect_success 'difftool + mergetool config variables' '
  	test_config merge.tool test-tool &&
  	test_config mergetool.test-tool.cmd "cat \$LOCAL" &&
  	echo branch >expect &&
 @@ -291,49 +275,49 @@ test_expect_success PERL 'difftool + mergetool config variables' '
  	test_cmp expect actual
  '
  
 -test_expect_success PERL 'difftool.<tool>.path' '
 +test_expect_success 'difftool.<tool>.path' '
  	test_config difftool.tkdiff.path echo &&
  	git difftool --tool=tkdiff --no-prompt branch >output &&
  	lines=$(grep file output | wc -l) &&
  	test "$lines" -eq 1
  '
  
 -test_expect_success PERL 'difftool --extcmd=cat' '
 +test_expect_success 'difftool --extcmd=cat' '
  	echo branch >expect &&
  	echo master >>expect &&
  	git difftool --no-prompt --extcmd=cat branch >actual &&
  	test_cmp expect actual
  '
  
 -test_expect_success PERL 'difftool --extcmd cat' '
 +test_expect_success 'difftool --extcmd cat' '
  	echo branch >expect &&
  	echo master >>expect &&
  	git difftool --no-prompt --extcmd=cat branch >actual &&
  	test_cmp expect actual
  '
  
 -test_expect_success PERL 'difftool -x cat' '
 +test_expect_success 'difftool -x cat' '
  	echo branch >expect &&
  	echo master >>expect &&
  	git difftool --no-prompt -x cat branch >actual &&
  	test_cmp expect actual
  '
  
 -test_expect_success PERL 'difftool --extcmd echo arg1' '
 +test_expect_success 'difftool --extcmd echo arg1' '
  	echo file >expect &&
  	git difftool --no-prompt \
  		--extcmd sh\ -c\ \"echo\ \$1\" branch >actual &&
  	test_cmp expect actual
  '
  
 -test_expect_success PERL 'difftool --extcmd cat arg1' '
 +test_expect_success 'difftool --extcmd cat arg1' '
  	echo master >expect &&
  	git difftool --no-prompt \
  		--extcmd sh\ -c\ \"cat\ \$1\" branch >actual &&
  	test_cmp expect actual
  '
  
 -test_expect_success PERL 'difftool --extcmd cat arg2' '
 +test_expect_success 'difftool --extcmd cat arg2' '
  	echo branch >expect &&
  	git difftool --no-prompt \
  		--extcmd sh\ -c\ \"cat\ \$2\" branch >actual &&
 @@ -341,7 +325,7 @@ test_expect_success PERL 'difftool --extcmd cat arg2' '
  '
  
  # Create a second file on master and a different version on branch
 -test_expect_success PERL 'setup with 2 files different' '
 +test_expect_success 'setup with 2 files different' '
  	echo m2 >file2 &&
  	git add file2 &&
  	git commit -m "added file2" &&
 @@ -353,7 +337,7 @@ test_expect_success PERL 'setup with 2 files different' '
  	git checkout master
  '
  
 -test_expect_success PERL 'say no to the first file' '
 +test_expect_success 'say no to the first file' '
  	(echo n && echo) >input &&
  	git difftool -x cat branch <input >output &&
  	grep m2 output &&
 @@ -362,7 +346,7 @@ test_expect_success PERL 'say no to the first file' '
  	! grep branch output
  '
  
 -test_expect_success PERL 'say no to the second file' '
 +test_expect_success 'say no to the second file' '
  	(echo && echo n) >input &&
  	git difftool -x cat branch <input >output &&
  	grep master output &&
 @@ -371,7 +355,7 @@ test_expect_success PERL 'say no to the second file' '
  	! grep br2 output
  '
  
 -test_expect_success PERL 'ending prompt input with EOF' '
 +test_expect_success 'ending prompt input with EOF' '
  	git difftool -x cat branch </dev/null >output &&
  	! grep master output &&
  	! grep branch output &&
 @@ -379,12 +363,12 @@ test_expect_success PERL 'ending prompt input with EOF' '
  	! grep br2 output
  '
  
 -test_expect_success PERL 'difftool --tool-help' '
 +test_expect_success 'difftool --tool-help' '
  	git difftool --tool-help >output &&
  	grep tool output
  '
  
 -test_expect_success PERL 'setup change in subdirectory' '
 +test_expect_success 'setup change in subdirectory' '
  	git checkout master &&
  	mkdir sub &&
  	echo master >sub/sub &&
 @@ -398,11 +382,11 @@ test_expect_success PERL 'setup change in subdirectory' '
  '
  
  run_dir_diff_test () {
 -	test_expect_success PERL "$1 --no-symlinks" "
 +	test_expect_success "$1 --no-symlinks" "
  		symlinks=--no-symlinks &&
  		$2
  	"
 -	test_expect_success PERL,SYMLINKS "$1 --symlinks" "
 +	test_expect_success SYMLINKS "$1 --symlinks" "
  		symlinks=--symlinks &&
  		$2
  	"
 @@ -524,7 +508,7 @@ do
  done >actual
  EOF
  
 -test_expect_success PERL,SYMLINKS 'difftool --dir-diff --symlink without unstaged changes' '
 +test_expect_success SYMLINKS 'difftool --dir-diff --symlink without unstaged changes' '
  	cat >expect <<-EOF &&
  	file
  	$PWD/file
 @@ -561,7 +545,7 @@ write_script modify-file <<\EOF
  echo "new content" >file
  EOF
  
 -test_expect_success PERL 'difftool --no-symlinks does not overwrite working tree file ' '
 +test_expect_success 'difftool --no-symlinks does not overwrite working tree file ' '
  	echo "orig content" >file &&
  	git difftool --dir-diff --no-symlinks --extcmd "$PWD/modify-file" branch &&
  	echo "new content" >expect &&
 @@ -574,7 +558,7 @@ echo "tmp content" >"$2/file" &&
  echo "$2" >tmpdir
  EOF
  
 -test_expect_success PERL 'difftool --no-symlinks detects conflict ' '
 +test_expect_success 'difftool --no-symlinks detects conflict ' '
  	(
  		TMPDIR=$TRASH_DIRECTORY &&
  		export TMPDIR &&
 @@ -587,7 +571,7 @@ test_expect_success PERL 'difftool --no-symlinks detects conflict ' '
  	)
  '
  
 -test_expect_success PERL 'difftool properly honors gitlink and core.worktree' '
 +test_expect_success 'difftool properly honors gitlink and core.worktree' '
  	git submodule add ./. submod/ule &&
  	test_config -C submod/ule diff.tool checktrees &&
  	test_config -C submod/ule difftool.checktrees.cmd '\''
 @@ -601,7 +585,7 @@ test_expect_success PERL 'difftool properly honors gitlink and core.worktree' '
  	)
  '
  
 -test_expect_success PERL,SYMLINKS 'difftool --dir-diff symlinked directories' '
 +test_expect_success SYMLINKS 'difftool --dir-diff symlinked directories' '
  	git init dirlinks &&
  	(
  		cd dirlinks &&
 @@ -620,17 +604,4 @@ test_expect_success PERL,SYMLINKS 'difftool --dir-diff symlinked directories' '
  	)
  '
  
 -test true != $use_builtin_difftool || break
 -
 -test_expect_success 'tear down for re-run' '
 -	rm -rf * .[a-z]* &&
 -	git init
 -'
 -
 -# run as builtin difftool now
 -GIT_CONFIG_PARAMETERS="'difftool.usebuiltin=true'"
 -export GIT_CONFIG_PARAMETERS
 -
 -done
 -
  test_done

-- 
2.11.0.windows.3

[PATCH v5 3/3] Retire the scripted difftool

From: Johannes Schindelin <hidden>
Date: 2017-01-17 16:02:56

It served its purpose, but now we have a builtin difftool. Time for the
Perl script to enjoy Florida.

Signed-off-by: Johannes Schindelin <redacted>
---
 .gitignore                                         |  1 -
 Makefile                                           |  1 -
 builtin/difftool.c                                 | 41 ----------
 .../examples/git-difftool.perl                     |  0
 git.c                                              |  7 +-
 t/t7800-difftool.sh                                | 94 +++++++++++-----------
 6 files changed, 47 insertions(+), 97 deletions(-)
 rename git-legacy-difftool.perl => contrib/examples/git-difftool.perl (100%)
diff --git a/.gitignore b/.gitignore
index 5555ae025b..6722f78f9a 100644
--- a/.gitignore
+++ b/.gitignore
@@ -76,7 +76,6 @@
 /git-init-db
 /git-interpret-trailers
 /git-instaweb
-/git-legacy-difftool
 /git-log
 /git-ls-files
 /git-ls-remote
diff --git a/Makefile b/Makefile
index 8cf5bef034..e9aa6ae57c 100644
--- a/Makefile
+++ b/Makefile
@@ -522,7 +522,6 @@ SCRIPT_LIB += git-sh-setup
 SCRIPT_LIB += git-sh-i18n
 
 SCRIPT_PERL += git-add--interactive.perl
-SCRIPT_PERL += git-legacy-difftool.perl
 SCRIPT_PERL += git-archimport.perl
 SCRIPT_PERL += git-cvsexportcommit.perl
 SCRIPT_PERL += git-cvsimport.perl
diff --git a/builtin/difftool.c b/builtin/difftool.c
index 2115e548a5..42ad9e804a 100644
--- a/builtin/difftool.c
+++ b/builtin/difftool.c
@@ -616,30 +616,6 @@ static int run_file_diff(int prompt, const char *prefix,
 	exit(ret);
 }
 
-/*
- * NEEDSWORK: this function can go once the legacy-difftool Perl script is
- * retired.
- *
- * We intentionally avoid reading the config directly here, to avoid messing up
- * the GIT_* environment variables when we need to fall back to exec()ing the
- * Perl script.
- */
-static int use_builtin_difftool(void) {
-	struct child_process cp = CHILD_PROCESS_INIT;
-	struct strbuf out = STRBUF_INIT;
-	int ret;
-
-	argv_array_pushl(&cp.args,
-			 "config", "--bool", "difftool.usebuiltin", NULL);
-	cp.git_cmd = 1;
-	if (capture_command(&cp, &out, 6))
-		return 0;
-	strbuf_trim(&out);
-	ret = !strcmp("true", out.buf);
-	strbuf_release(&out);
-	return ret;
-}
-
 int cmd_difftool(int argc, const char **argv, const char *prefix)
 {
 	int use_gui_tool = 0, dir_diff = 0, prompt = -1, symlinks = 0,
@@ -671,23 +647,6 @@ int cmd_difftool(int argc, const char **argv, const char *prefix)
 		OPT_END()
 	};
 
-	/*
-	 * NEEDSWORK: Once the builtin difftool has been tested enough
-	 * and git-legacy-difftool.perl is retired to contrib/, this preamble
-	 * can be removed.
-	 */
-	if (!use_builtin_difftool()) {
-		const char *path = mkpath("%s/git-legacy-difftool",
-					  git_exec_path());
-
-		if (sane_execvp(path, (char **)argv) < 0)
-			die_errno("could not exec %s", path);
-
-		return 0;
-	}
-	prefix = setup_git_directory();
-	trace_repo_setup(prefix);
-	setup_work_tree();
 	/* NEEDSWORK: once we no longer spawn anything, remove this */
 	setenv(GIT_DIR_ENVIRONMENT, absolute_path(get_git_dir()), 1);
 	setenv(GIT_WORK_TREE_ENVIRONMENT, absolute_path(get_git_work_tree()), 1);
diff --git a/git-legacy-difftool.perl b/contrib/examples/git-difftool.perl
similarity index 100%
rename from git-legacy-difftool.perl
rename to contrib/examples/git-difftool.perl
diff --git a/git.c b/git.c
index c58181e5ef..bd4d668a21 100644
--- a/git.c
+++ b/git.c
@@ -424,12 +424,7 @@ static struct cmd_struct commands[] = {
 	{ "diff-files", cmd_diff_files, RUN_SETUP | NEED_WORK_TREE },
 	{ "diff-index", cmd_diff_index, RUN_SETUP },
 	{ "diff-tree", cmd_diff_tree, RUN_SETUP },
-	/*
-	 * NEEDSWORK: Once the redirection to git-legacy-difftool.perl in
-	 * builtin/difftool.c has been removed, this entry should be changed to
-	 * RUN_SETUP | NEED_WORK_TREE
-	 */
-	{ "difftool", cmd_difftool },
+	{ "difftool", cmd_difftool, RUN_SETUP | NEED_WORK_TREE },
 	{ "fast-export", cmd_fast_export, RUN_SETUP },
 	{ "fetch", cmd_fetch, RUN_SETUP },
 	{ "fetch-pack", cmd_fetch_pack, RUN_SETUP },
diff --git a/t/t7800-difftool.sh b/t/t7800-difftool.sh
index e94910c563..aa0ef02597 100755
--- a/t/t7800-difftool.sh
+++ b/t/t7800-difftool.sh
@@ -23,10 +23,8 @@ prompt_given ()
 	test "$prompt" = "Launch 'test-tool' [Y/n]? branch"
 }
 
-# NEEDSWORK: lose all the PERL prereqs once legacy-difftool is retired.
-
 # Create a file on master and change it on branch
-test_expect_success PERL 'setup' '
+test_expect_success 'setup' '
 	echo master >file &&
 	git add file &&
 	git commit -m "added file" &&
@@ -38,7 +36,7 @@ test_expect_success PERL 'setup' '
 '
 
 # Configure a custom difftool.<tool>.cmd and use it
-test_expect_success PERL 'custom commands' '
+test_expect_success 'custom commands' '
 	difftool_test_setup &&
 	test_config difftool.test-tool.cmd "cat \"\$REMOTE\"" &&
 	echo master >expect &&
@@ -51,21 +49,21 @@ test_expect_success PERL 'custom commands' '
 	test_cmp expect actual
 '
 
-test_expect_success PERL 'custom tool commands override built-ins' '
+test_expect_success 'custom tool commands override built-ins' '
 	test_config difftool.vimdiff.cmd "cat \"\$REMOTE\"" &&
 	echo master >expect &&
 	git difftool --tool vimdiff --no-prompt branch >actual &&
 	test_cmp expect actual
 '
 
-test_expect_success PERL 'difftool ignores bad --tool values' '
+test_expect_success 'difftool ignores bad --tool values' '
 	: >expect &&
 	test_must_fail \
 		git difftool --no-prompt --tool=bad-tool branch >actual &&
 	test_cmp expect actual
 '
 
-test_expect_success PERL 'difftool forwards arguments to diff' '
+test_expect_success 'difftool forwards arguments to diff' '
 	difftool_test_setup &&
 	>for-diff &&
 	git add for-diff &&
@@ -78,40 +76,40 @@ test_expect_success PERL 'difftool forwards arguments to diff' '
 	rm for-diff
 '
 
-test_expect_success PERL 'difftool ignores exit code' '
+test_expect_success 'difftool ignores exit code' '
 	test_config difftool.error.cmd false &&
 	git difftool -y -t error branch
 '
 
-test_expect_success PERL 'difftool forwards exit code with --trust-exit-code' '
+test_expect_success 'difftool forwards exit code with --trust-exit-code' '
 	test_config difftool.error.cmd false &&
 	test_must_fail git difftool -y --trust-exit-code -t error branch
 '
 
-test_expect_success PERL 'difftool forwards exit code with --trust-exit-code for built-ins' '
+test_expect_success 'difftool forwards exit code with --trust-exit-code for built-ins' '
 	test_config difftool.vimdiff.path false &&
 	test_must_fail git difftool -y --trust-exit-code -t vimdiff branch
 '
 
-test_expect_success PERL 'difftool honors difftool.trustExitCode = true' '
+test_expect_success 'difftool honors difftool.trustExitCode = true' '
 	test_config difftool.error.cmd false &&
 	test_config difftool.trustExitCode true &&
 	test_must_fail git difftool -y -t error branch
 '
 
-test_expect_success PERL 'difftool honors difftool.trustExitCode = false' '
+test_expect_success 'difftool honors difftool.trustExitCode = false' '
 	test_config difftool.error.cmd false &&
 	test_config difftool.trustExitCode false &&
 	git difftool -y -t error branch
 '
 
-test_expect_success PERL 'difftool ignores exit code with --no-trust-exit-code' '
+test_expect_success 'difftool ignores exit code with --no-trust-exit-code' '
 	test_config difftool.error.cmd false &&
 	test_config difftool.trustExitCode true &&
 	git difftool -y --no-trust-exit-code -t error branch
 '
 
-test_expect_success PERL 'difftool stops on error with --trust-exit-code' '
+test_expect_success 'difftool stops on error with --trust-exit-code' '
 	test_when_finished "rm -f for-diff .git/fail-right-file" &&
 	test_when_finished "git reset -- for-diff" &&
 	write_script .git/fail-right-file <<-\EOF &&
@@ -126,13 +124,13 @@ test_expect_success PERL 'difftool stops on error with --trust-exit-code' '
 	test_cmp expect actual
 '
 
-test_expect_success PERL 'difftool honors exit status if command not found' '
+test_expect_success 'difftool honors exit status if command not found' '
 	test_config difftool.nonexistent.cmd i-dont-exist &&
 	test_config difftool.trustExitCode false &&
 	test_must_fail git difftool -y -t nonexistent branch
 '
 
-test_expect_success PERL 'difftool honors --gui' '
+test_expect_success 'difftool honors --gui' '
 	difftool_test_setup &&
 	test_config merge.tool bogus-tool &&
 	test_config diff.tool bogus-tool &&
@@ -143,7 +141,7 @@ test_expect_success PERL 'difftool honors --gui' '
 	test_cmp expect actual
 '
 
-test_expect_success PERL 'difftool --gui last setting wins' '
+test_expect_success 'difftool --gui last setting wins' '
 	difftool_test_setup &&
 	: >expect &&
 	git difftool --no-prompt --gui --no-gui >actual &&
@@ -157,7 +155,7 @@ test_expect_success PERL 'difftool --gui last setting wins' '
 	test_cmp expect actual
 '
 
-test_expect_success PERL 'difftool --gui works without configured diff.guitool' '
+test_expect_success 'difftool --gui works without configured diff.guitool' '
 	difftool_test_setup &&
 	echo branch >expect &&
 	git difftool --no-prompt --gui branch >actual &&
@@ -165,7 +163,7 @@ test_expect_success PERL 'difftool --gui works without configured diff.guitool'
 '
 
 # Specify the diff tool using $GIT_DIFF_TOOL
-test_expect_success PERL 'GIT_DIFF_TOOL variable' '
+test_expect_success 'GIT_DIFF_TOOL variable' '
 	difftool_test_setup &&
 	git config --unset diff.tool &&
 	echo branch >expect &&
@@ -175,7 +173,7 @@ test_expect_success PERL 'GIT_DIFF_TOOL variable' '
 
 # Test the $GIT_*_TOOL variables and ensure
 # that $GIT_DIFF_TOOL always wins unless --tool is specified
-test_expect_success PERL 'GIT_DIFF_TOOL overrides' '
+test_expect_success 'GIT_DIFF_TOOL overrides' '
 	difftool_test_setup &&
 	test_config diff.tool bogus-tool &&
 	test_config merge.tool bogus-tool &&
@@ -193,7 +191,7 @@ test_expect_success PERL 'GIT_DIFF_TOOL overrides' '
 
 # Test that we don't have to pass --no-prompt to difftool
 # when $GIT_DIFFTOOL_NO_PROMPT is true
-test_expect_success PERL 'GIT_DIFFTOOL_NO_PROMPT variable' '
+test_expect_success 'GIT_DIFFTOOL_NO_PROMPT variable' '
 	difftool_test_setup &&
 	echo branch >expect &&
 	GIT_DIFFTOOL_NO_PROMPT=true git difftool branch >actual &&
@@ -202,7 +200,7 @@ test_expect_success PERL 'GIT_DIFFTOOL_NO_PROMPT variable' '
 
 # git-difftool supports the difftool.prompt variable.
 # Test that GIT_DIFFTOOL_PROMPT can override difftool.prompt = false
-test_expect_success PERL 'GIT_DIFFTOOL_PROMPT variable' '
+test_expect_success 'GIT_DIFFTOOL_PROMPT variable' '
 	difftool_test_setup &&
 	test_config difftool.prompt false &&
 	echo >input &&
@@ -212,7 +210,7 @@ test_expect_success PERL 'GIT_DIFFTOOL_PROMPT variable' '
 '
 
 # Test that we don't have to pass --no-prompt when difftool.prompt is false
-test_expect_success PERL 'difftool.prompt config variable is false' '
+test_expect_success 'difftool.prompt config variable is false' '
 	difftool_test_setup &&
 	test_config difftool.prompt false &&
 	echo branch >expect &&
@@ -221,7 +219,7 @@ test_expect_success PERL 'difftool.prompt config variable is false' '
 '
 
 # Test that we don't have to pass --no-prompt when mergetool.prompt is false
-test_expect_success PERL 'difftool merge.prompt = false' '
+test_expect_success 'difftool merge.prompt = false' '
 	difftool_test_setup &&
 	test_might_fail git config --unset difftool.prompt &&
 	test_config mergetool.prompt false &&
@@ -231,7 +229,7 @@ test_expect_success PERL 'difftool merge.prompt = false' '
 '
 
 # Test that the -y flag can override difftool.prompt = true
-test_expect_success PERL 'difftool.prompt can overridden with -y' '
+test_expect_success 'difftool.prompt can overridden with -y' '
 	difftool_test_setup &&
 	test_config difftool.prompt true &&
 	echo branch >expect &&
@@ -240,7 +238,7 @@ test_expect_success PERL 'difftool.prompt can overridden with -y' '
 '
 
 # Test that the --prompt flag can override difftool.prompt = false
-test_expect_success PERL 'difftool.prompt can overridden with --prompt' '
+test_expect_success 'difftool.prompt can overridden with --prompt' '
 	difftool_test_setup &&
 	test_config difftool.prompt false &&
 	echo >input &&
@@ -250,7 +248,7 @@ test_expect_success PERL 'difftool.prompt can overridden with --prompt' '
 '
 
 # Test that the last flag passed on the command-line wins
-test_expect_success PERL 'difftool last flag wins' '
+test_expect_success 'difftool last flag wins' '
 	difftool_test_setup &&
 	echo branch >expect &&
 	git difftool --prompt --no-prompt branch >actual &&
@@ -263,7 +261,7 @@ test_expect_success PERL 'difftool last flag wins' '
 
 # git-difftool falls back to git-mergetool config variables
 # so test that behavior here
-test_expect_success PERL 'difftool + mergetool config variables' '
+test_expect_success 'difftool + mergetool config variables' '
 	test_config merge.tool test-tool &&
 	test_config mergetool.test-tool.cmd "cat \$LOCAL" &&
 	echo branch >expect &&
@@ -277,49 +275,49 @@ test_expect_success PERL 'difftool + mergetool config variables' '
 	test_cmp expect actual
 '
 
-test_expect_success PERL 'difftool.<tool>.path' '
+test_expect_success 'difftool.<tool>.path' '
 	test_config difftool.tkdiff.path echo &&
 	git difftool --tool=tkdiff --no-prompt branch >output &&
 	lines=$(grep file output | wc -l) &&
 	test "$lines" -eq 1
 '
 
-test_expect_success PERL 'difftool --extcmd=cat' '
+test_expect_success 'difftool --extcmd=cat' '
 	echo branch >expect &&
 	echo master >>expect &&
 	git difftool --no-prompt --extcmd=cat branch >actual &&
 	test_cmp expect actual
 '
 
-test_expect_success PERL 'difftool --extcmd cat' '
+test_expect_success 'difftool --extcmd cat' '
 	echo branch >expect &&
 	echo master >>expect &&
 	git difftool --no-prompt --extcmd=cat branch >actual &&
 	test_cmp expect actual
 '
 
-test_expect_success PERL 'difftool -x cat' '
+test_expect_success 'difftool -x cat' '
 	echo branch >expect &&
 	echo master >>expect &&
 	git difftool --no-prompt -x cat branch >actual &&
 	test_cmp expect actual
 '
 
-test_expect_success PERL 'difftool --extcmd echo arg1' '
+test_expect_success 'difftool --extcmd echo arg1' '
 	echo file >expect &&
 	git difftool --no-prompt \
 		--extcmd sh\ -c\ \"echo\ \$1\" branch >actual &&
 	test_cmp expect actual
 '
 
-test_expect_success PERL 'difftool --extcmd cat arg1' '
+test_expect_success 'difftool --extcmd cat arg1' '
 	echo master >expect &&
 	git difftool --no-prompt \
 		--extcmd sh\ -c\ \"cat\ \$1\" branch >actual &&
 	test_cmp expect actual
 '
 
-test_expect_success PERL 'difftool --extcmd cat arg2' '
+test_expect_success 'difftool --extcmd cat arg2' '
 	echo branch >expect &&
 	git difftool --no-prompt \
 		--extcmd sh\ -c\ \"cat\ \$2\" branch >actual &&
@@ -327,7 +325,7 @@ test_expect_success PERL 'difftool --extcmd cat arg2' '
 '
 
 # Create a second file on master and a different version on branch
-test_expect_success PERL 'setup with 2 files different' '
+test_expect_success 'setup with 2 files different' '
 	echo m2 >file2 &&
 	git add file2 &&
 	git commit -m "added file2" &&
@@ -339,7 +337,7 @@ test_expect_success PERL 'setup with 2 files different' '
 	git checkout master
 '
 
-test_expect_success PERL 'say no to the first file' '
+test_expect_success 'say no to the first file' '
 	(echo n && echo) >input &&
 	git difftool -x cat branch <input >output &&
 	grep m2 output &&
@@ -348,7 +346,7 @@ test_expect_success PERL 'say no to the first file' '
 	! grep branch output
 '
 
-test_expect_success PERL 'say no to the second file' '
+test_expect_success 'say no to the second file' '
 	(echo && echo n) >input &&
 	git difftool -x cat branch <input >output &&
 	grep master output &&
@@ -357,7 +355,7 @@ test_expect_success PERL 'say no to the second file' '
 	! grep br2 output
 '
 
-test_expect_success PERL 'ending prompt input with EOF' '
+test_expect_success 'ending prompt input with EOF' '
 	git difftool -x cat branch </dev/null >output &&
 	! grep master output &&
 	! grep branch output &&
@@ -365,12 +363,12 @@ test_expect_success PERL 'ending prompt input with EOF' '
 	! grep br2 output
 '
 
-test_expect_success PERL 'difftool --tool-help' '
+test_expect_success 'difftool --tool-help' '
 	git difftool --tool-help >output &&
 	grep tool output
 '
 
-test_expect_success PERL 'setup change in subdirectory' '
+test_expect_success 'setup change in subdirectory' '
 	git checkout master &&
 	mkdir sub &&
 	echo master >sub/sub &&
@@ -384,11 +382,11 @@ test_expect_success PERL 'setup change in subdirectory' '
 '
 
 run_dir_diff_test () {
-	test_expect_success PERL "$1 --no-symlinks" "
+	test_expect_success "$1 --no-symlinks" "
 		symlinks=--no-symlinks &&
 		$2
 	"
-	test_expect_success PERL,SYMLINKS "$1 --symlinks" "
+	test_expect_success SYMLINKS "$1 --symlinks" "
 		symlinks=--symlinks &&
 		$2
 	"
@@ -510,7 +508,7 @@ do
 done >actual
 EOF
 
-test_expect_success PERL,SYMLINKS 'difftool --dir-diff --symlink without unstaged changes' '
+test_expect_success SYMLINKS 'difftool --dir-diff --symlink without unstaged changes' '
 	cat >expect <<-EOF &&
 	file
 	$PWD/file
@@ -547,7 +545,7 @@ write_script modify-file <<\EOF
 echo "new content" >file
 EOF
 
-test_expect_success PERL 'difftool --no-symlinks does not overwrite working tree file ' '
+test_expect_success 'difftool --no-symlinks does not overwrite working tree file ' '
 	echo "orig content" >file &&
 	git difftool --dir-diff --no-symlinks --extcmd "$PWD/modify-file" branch &&
 	echo "new content" >expect &&
@@ -560,7 +558,7 @@ echo "tmp content" >"$2/file" &&
 echo "$2" >tmpdir
 EOF
 
-test_expect_success PERL 'difftool --no-symlinks detects conflict ' '
+test_expect_success 'difftool --no-symlinks detects conflict ' '
 	(
 		TMPDIR=$TRASH_DIRECTORY &&
 		export TMPDIR &&
@@ -573,7 +571,7 @@ test_expect_success PERL 'difftool --no-symlinks detects conflict ' '
 	)
 '
 
-test_expect_success PERL 'difftool properly honors gitlink and core.worktree' '
+test_expect_success 'difftool properly honors gitlink and core.worktree' '
 	git submodule add ./. submod/ule &&
 	test_config -C submod/ule diff.tool checktrees &&
 	test_config -C submod/ule difftool.checktrees.cmd '\''
@@ -587,7 +585,7 @@ test_expect_success PERL 'difftool properly honors gitlink and core.worktree' '
 	)
 '
 
-test_expect_success PERL,SYMLINKS 'difftool --dir-diff symlinked directories' '
+test_expect_success SYMLINKS 'difftool --dir-diff symlinked directories' '
 	git init dirlinks &&
 	(
 		cd dirlinks &&
-- 
2.11.0.windows.3

[PATCH v5 1/3] difftool: add a skeleton for the upcoming builtin

From: Johannes Schindelin <hidden>
Date: 2017-01-17 16:02:59

This adds a builtin difftool that still falls back to the legacy Perl
version, which has been renamed to `legacy-difftool`.

The idea is that the new, experimental, builtin difftool immediately hands
off to the legacy difftool for now, unless the config variable
difftool.useBuiltin is set to true.

This feature flag will be used in the upcoming Git for Windows v2.11.0
release, to allow early testers to opt-in to use the builtin difftool and
flesh out any bugs.

Signed-off-by: Johannes Schindelin <redacted>
---
 .gitignore                                    |  1 +
 Makefile                                      |  3 +-
 builtin.h                                     |  1 +
 builtin/difftool.c                            | 63 +++++++++++++++++++++++++++
 git-difftool.perl => git-legacy-difftool.perl |  0
 git.c                                         |  6 +++
 t/t7800-difftool.sh                           |  2 +
 7 files changed, 75 insertions(+), 1 deletion(-)
 create mode 100644 builtin/difftool.c
 rename git-difftool.perl => git-legacy-difftool.perl (100%)
diff --git a/.gitignore b/.gitignore
index 6722f78f9a..5555ae025b 100644
--- a/.gitignore
+++ b/.gitignore
@@ -76,6 +76,7 @@
 /git-init-db
 /git-interpret-trailers
 /git-instaweb
+/git-legacy-difftool
 /git-log
 /git-ls-files
 /git-ls-remote
diff --git a/Makefile b/Makefile
index d861bd9985..8cf5bef034 100644
--- a/Makefile
+++ b/Makefile
@@ -522,7 +522,7 @@ SCRIPT_LIB += git-sh-setup
 SCRIPT_LIB += git-sh-i18n
 
 SCRIPT_PERL += git-add--interactive.perl
-SCRIPT_PERL += git-difftool.perl
+SCRIPT_PERL += git-legacy-difftool.perl
 SCRIPT_PERL += git-archimport.perl
 SCRIPT_PERL += git-cvsexportcommit.perl
 SCRIPT_PERL += git-cvsimport.perl
@@ -883,6 +883,7 @@ BUILTIN_OBJS += builtin/diff-files.o
 BUILTIN_OBJS += builtin/diff-index.o
 BUILTIN_OBJS += builtin/diff-tree.o
 BUILTIN_OBJS += builtin/diff.o
+BUILTIN_OBJS += builtin/difftool.o
 BUILTIN_OBJS += builtin/fast-export.o
 BUILTIN_OBJS += builtin/fetch-pack.o
 BUILTIN_OBJS += builtin/fetch.o
diff --git a/builtin.h b/builtin.h
index b9122bc5f4..67f80519da 100644
--- a/builtin.h
+++ b/builtin.h
@@ -60,6 +60,7 @@ extern int cmd_diff_files(int argc, const char **argv, const char *prefix);
 extern int cmd_diff_index(int argc, const char **argv, const char *prefix);
 extern int cmd_diff(int argc, const char **argv, const char *prefix);
 extern int cmd_diff_tree(int argc, const char **argv, const char *prefix);
+extern int cmd_difftool(int argc, const char **argv, const char *prefix);
 extern int cmd_fast_export(int argc, const char **argv, const char *prefix);
 extern int cmd_fetch(int argc, const char **argv, const char *prefix);
 extern int cmd_fetch_pack(int argc, const char **argv, const char *prefix);
diff --git a/builtin/difftool.c b/builtin/difftool.c
new file mode 100644
index 0000000000..53870bbaf7
--- /dev/null
+++ b/builtin/difftool.c
@@ -0,0 +1,63 @@
+/*
+ * "git difftool" builtin command
+ *
+ * This is a wrapper around the GIT_EXTERNAL_DIFF-compatible
+ * git-difftool--helper script.
+ *
+ * This script exports GIT_EXTERNAL_DIFF and GIT_PAGER for use by git.
+ * The GIT_DIFF* variables are exported for use by git-difftool--helper.
+ *
+ * Any arguments that are unknown to this script are forwarded to 'git diff'.
+ *
+ * Copyright (C) 2016 Johannes Schindelin
+ */
+#include "builtin.h"
+#include "run-command.h"
+#include "exec_cmd.h"
+
+/*
+ * NEEDSWORK: this function can go once the legacy-difftool Perl script is
+ * retired.
+ *
+ * We intentionally avoid reading the config directly here, to avoid messing up
+ * the GIT_* environment variables when we need to fall back to exec()ing the
+ * Perl script.
+ */
+static int use_builtin_difftool(void) {
+	struct child_process cp = CHILD_PROCESS_INIT;
+	struct strbuf out = STRBUF_INIT;
+	int ret;
+
+	argv_array_pushl(&cp.args,
+			 "config", "--bool", "difftool.usebuiltin", NULL);
+	cp.git_cmd = 1;
+	if (capture_command(&cp, &out, 6))
+		return 0;
+	strbuf_trim(&out);
+	ret = !strcmp("true", out.buf);
+	strbuf_release(&out);
+	return ret;
+}
+
+int cmd_difftool(int argc, const char **argv, const char *prefix)
+{
+	/*
+	 * NEEDSWORK: Once the builtin difftool has been tested enough
+	 * and git-legacy-difftool.perl is retired to contrib/, this preamble
+	 * can be removed.
+	 */
+	if (!use_builtin_difftool()) {
+		const char *path = mkpath("%s/git-legacy-difftool",
+					  git_exec_path());
+
+		if (sane_execvp(path, (char **)argv) < 0)
+			die_errno("could not exec %s", path);
+
+		return 0;
+	}
+	prefix = setup_git_directory();
+	trace_repo_setup(prefix);
+	setup_work_tree();
+
+	die("TODO");
+}
diff --git a/git-difftool.perl b/git-legacy-difftool.perl
similarity index 100%
rename from git-difftool.perl
rename to git-legacy-difftool.perl
diff --git a/git.c b/git.c
index bbaa949e9c..c58181e5ef 100644
--- a/git.c
+++ b/git.c
@@ -424,6 +424,12 @@ static struct cmd_struct commands[] = {
 	{ "diff-files", cmd_diff_files, RUN_SETUP | NEED_WORK_TREE },
 	{ "diff-index", cmd_diff_index, RUN_SETUP },
 	{ "diff-tree", cmd_diff_tree, RUN_SETUP },
+	/*
+	 * NEEDSWORK: Once the redirection to git-legacy-difftool.perl in
+	 * builtin/difftool.c has been removed, this entry should be changed to
+	 * RUN_SETUP | NEED_WORK_TREE
+	 */
+	{ "difftool", cmd_difftool },
 	{ "fast-export", cmd_fast_export, RUN_SETUP },
 	{ "fetch", cmd_fetch, RUN_SETUP },
 	{ "fetch-pack", cmd_fetch_pack, RUN_SETUP },
diff --git a/t/t7800-difftool.sh b/t/t7800-difftool.sh
index 99d4123461..e94910c563 100755
--- a/t/t7800-difftool.sh
+++ b/t/t7800-difftool.sh
@@ -23,6 +23,8 @@ prompt_given ()
 	test "$prompt" = "Launch 'test-tool' [Y/n]? branch"
 }
 
+# NEEDSWORK: lose all the PERL prereqs once legacy-difftool is retired.
+
 # Create a file on master and change it on branch
 test_expect_success PERL 'setup' '
 	echo master >file &&
-- 
2.11.0.windows.3

[PATCH v5 2/3] difftool: implement the functionality in the builtin

From: Johannes Schindelin <hidden>
Date: 2017-01-17 16:03:02

This patch gives life to the skeleton added in the previous patch.

The motivation for converting the difftool is that Perl scripts are not at
all native on Windows, and that `git difftool` therefore is pretty slow on
that platform, when there is no good reason for it to be slow.

In addition, Perl does not really have access to Git's internals. That
means that any script will always have to jump through unnecessary
hoops.

The current version of the builtin difftool does not, however, make full
use of the internals but instead chooses to spawn a couple of Git
processes, still, to make for an easier conversion. There remains a lot
of room for improvement, left for a later date.

Note: to play it safe, the original difftool is still called unless the
config setting difftool.useBuiltin is set to true.

The reason: this new, experimental, builtin difftool will be shipped as
part of Git for Windows v2.11.0, to allow for easier large-scale
testing, but of course as an opt-in feature.

Sadly, the speedup is more noticable on Linux than on Windows: a quick
test shows that t7800-difftool.sh runs in (2.183s/0.052s/0.108s)
(real/user/sys) in a Linux VM, down from  (6.529s/3.112s/0.644s), while
on Windows, it is (36.064s/2.730s/7.194s), down from
(47.637s/2.407s/6.863s). The culprit is most likely the overhead
incurred from *still* having to shell out to mergetool-lib.sh and
difftool--helper.sh.

Signed-off-by: Johannes Schindelin <redacted>
---
 builtin/difftool.c | 672 ++++++++++++++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 671 insertions(+), 1 deletion(-)
diff --git a/builtin/difftool.c b/builtin/difftool.c
index 53870bbaf7..2115e548a5 100644
--- a/builtin/difftool.c
+++ b/builtin/difftool.c
@@ -11,9 +11,610 @@
  *
  * Copyright (C) 2016 Johannes Schindelin
  */
+#include "cache.h"
 #include "builtin.h"
 #include "run-command.h"
 #include "exec_cmd.h"
+#include "parse-options.h"
+#include "argv-array.h"
+#include "strbuf.h"
+#include "lockfile.h"
+#include "dir.h"
+
+static char *diff_gui_tool;
+static int trust_exit_code;
+
+static const char *const builtin_difftool_usage[] = {
+	N_("git difftool [<options>] [<commit> [<commit>]] [--] [<path>...]"),
+	NULL
+};
+
+static int difftool_config(const char *var, const char *value, void *cb)
+{
+	if (!strcmp(var, "diff.guitool")) {
+		diff_gui_tool = xstrdup(value);
+		return 0;
+	}
+
+	if (!strcmp(var, "difftool.trustexitcode")) {
+		trust_exit_code = git_config_bool(var, value);
+		return 0;
+	}
+
+	return git_default_config(var, value, cb);
+}
+
+static int print_tool_help(void)
+{
+	const char *argv[] = { "mergetool", "--tool-help=diff", NULL };
+	return run_command_v_opt(argv, RUN_GIT_CMD);
+}
+
+static int parse_index_info(char *p, int *mode1, int *mode2,
+			    struct object_id *oid1, struct object_id *oid2,
+			    char *status)
+{
+	if (*p != ':')
+		return error("expected ':', got '%c'", *p);
+	*mode1 = (int)strtol(p + 1, &p, 8);
+	if (*p != ' ')
+		return error("expected ' ', got '%c'", *p);
+	*mode2 = (int)strtol(p + 1, &p, 8);
+	if (*p != ' ')
+		return error("expected ' ', got '%c'", *p);
+	if (get_oid_hex(++p, oid1))
+		return error("expected object ID, got '%s'", p + 1);
+	p += GIT_SHA1_HEXSZ;
+	if (*p != ' ')
+		return error("expected ' ', got '%c'", *p);
+	if (get_oid_hex(++p, oid2))
+		return error("expected object ID, got '%s'", p + 1);
+	p += GIT_SHA1_HEXSZ;
+	if (*p != ' ')
+		return error("expected ' ', got '%c'", *p);
+	*status = *++p;
+	if (!*status)
+		return error("missing status");
+	if (p[1] && !isdigit(p[1]))
+		return error("unexpected trailer: '%s'", p + 1);
+	return 0;
+}
+
+/*
+ * Remove any trailing slash from $workdir
+ * before starting to avoid double slashes in symlink targets.
+ */
+static void add_path(struct strbuf *buf, size_t base_len, const char *path)
+{
+	strbuf_setlen(buf, base_len);
+	if (buf->len && buf->buf[buf->len - 1] != '/')
+		strbuf_addch(buf, '/');
+	strbuf_addstr(buf, path);
+}
+
+/*
+ * Determine whether we can simply reuse the file in the worktree.
+ */
+static int use_wt_file(const char *workdir, const char *name,
+		       struct object_id *oid)
+{
+	struct strbuf buf = STRBUF_INIT;
+	struct stat st;
+	int use = 0;
+
+	strbuf_addstr(&buf, workdir);
+	add_path(&buf, buf.len, name);
+
+	if (!lstat(buf.buf, &st) && !S_ISLNK(st.st_mode)) {
+		struct object_id wt_oid;
+		int fd = open(buf.buf, O_RDONLY);
+
+		if (fd >= 0 &&
+		    !index_fd(wt_oid.hash, fd, &st, OBJ_BLOB, name, 0)) {
+			if (is_null_oid(oid)) {
+				oidcpy(oid, &wt_oid);
+				use = 1;
+			} else if (!oidcmp(oid, &wt_oid))
+				use = 1;
+		}
+	}
+
+	strbuf_release(&buf);
+
+	return use;
+}
+
+struct working_tree_entry {
+	struct hashmap_entry entry;
+	char path[FLEX_ARRAY];
+};
+
+static int working_tree_entry_cmp(struct working_tree_entry *a,
+				  struct working_tree_entry *b, void *keydata)
+{
+	return strcmp(a->path, b->path);
+}
+
+/*
+ * The `left` and `right` entries hold paths for the symlinks hashmap,
+ * and a SHA-1 surrounded by brief text for submodules.
+ */
+struct pair_entry {
+	struct hashmap_entry entry;
+	char left[PATH_MAX], right[PATH_MAX];
+	const char path[FLEX_ARRAY];
+};
+
+static int pair_cmp(struct pair_entry *a, struct pair_entry *b, void *keydata)
+{
+	return strcmp(a->path, b->path);
+}
+
+static void add_left_or_right(struct hashmap *map, const char *path,
+			      const char *content, int is_right)
+{
+	struct pair_entry *e, *existing;
+
+	FLEX_ALLOC_STR(e, path, path);
+	hashmap_entry_init(e, strhash(path));
+	existing = hashmap_get(map, e, NULL);
+	if (existing) {
+		free(e);
+		e = existing;
+	} else {
+		e->left[0] = e->right[0] = '\0';
+		hashmap_add(map, e);
+	}
+	strlcpy(is_right ? e->right : e->left, content, PATH_MAX);
+}
+
+struct path_entry {
+	struct hashmap_entry entry;
+	char path[FLEX_ARRAY];
+};
+
+static int path_entry_cmp(struct path_entry *a, struct path_entry *b, void *key)
+{
+	return strcmp(a->path, key ? key : b->path);
+}
+
+static void changed_files(struct hashmap *result, const char *index_path,
+			  const char *workdir)
+{
+	struct child_process update_index = CHILD_PROCESS_INIT;
+	struct child_process diff_files = CHILD_PROCESS_INIT;
+	struct strbuf index_env = STRBUF_INIT, buf = STRBUF_INIT;
+	const char *git_dir = absolute_path(get_git_dir()), *env[] = {
+		NULL, NULL
+	};
+	FILE *fp;
+
+	strbuf_addf(&index_env, "GIT_INDEX_FILE=%s", index_path);
+	env[0] = index_env.buf;
+
+	argv_array_pushl(&update_index.args,
+			 "--git-dir", git_dir, "--work-tree", workdir,
+			 "update-index", "--really-refresh", "-q",
+			 "--unmerged", NULL);
+	update_index.no_stdin = 1;
+	update_index.no_stdout = 1;
+	update_index.no_stderr = 1;
+	update_index.git_cmd = 1;
+	update_index.use_shell = 0;
+	update_index.clean_on_exit = 1;
+	update_index.dir = workdir;
+	update_index.env = env;
+	/* Ignore any errors of update-index */
+	run_command(&update_index);
+
+	argv_array_pushl(&diff_files.args,
+			 "--git-dir", git_dir, "--work-tree", workdir,
+			 "diff-files", "--name-only", "-z", NULL);
+	diff_files.no_stdin = 1;
+	diff_files.git_cmd = 1;
+	diff_files.use_shell = 0;
+	diff_files.clean_on_exit = 1;
+	diff_files.out = -1;
+	diff_files.dir = workdir;
+	diff_files.env = env;
+	if (start_command(&diff_files))
+		die("could not obtain raw diff");
+	fp = xfdopen(diff_files.out, "r");
+	while (!strbuf_getline_nul(&buf, fp)) {
+		struct path_entry *entry;
+		FLEX_ALLOC_STR(entry, path, buf.buf);
+		hashmap_entry_init(entry, strhash(buf.buf));
+		hashmap_add(result, entry);
+	}
+	if (finish_command(&diff_files))
+		die("diff-files did not exit properly");
+	strbuf_release(&index_env);
+	strbuf_release(&buf);
+}
+
+static NORETURN void exit_cleanup(const char *tmpdir, int exit_code)
+{
+	struct strbuf buf = STRBUF_INIT;
+	strbuf_addstr(&buf, tmpdir);
+	remove_dir_recursively(&buf, 0);
+	if (exit_code)
+		warning(_("failed: %d"), exit_code);
+	exit(exit_code);
+}
+
+static int ensure_leading_directories(char *path)
+{
+	switch (safe_create_leading_directories(path)) {
+		case SCLD_OK:
+		case SCLD_EXISTS:
+			return 0;
+		default:
+			return error(_("could not create leading directories "
+				       "of '%s'"), path);
+	}
+}
+
+static int run_dir_diff(const char *extcmd, int symlinks, const char *prefix,
+			int argc, const char **argv)
+{
+	char tmpdir[PATH_MAX];
+	struct strbuf info = STRBUF_INIT, lpath = STRBUF_INIT;
+	struct strbuf rpath = STRBUF_INIT, buf = STRBUF_INIT;
+	struct strbuf ldir = STRBUF_INIT, rdir = STRBUF_INIT;
+	struct strbuf wtdir = STRBUF_INIT;
+	size_t ldir_len, rdir_len, wtdir_len;
+	struct cache_entry *ce = xcalloc(1, sizeof(ce) + PATH_MAX + 1);
+	const char *workdir, *tmp;
+	int ret = 0, i;
+	FILE *fp;
+	struct hashmap working_tree_dups, submodules, symlinks2;
+	struct hashmap_iter iter;
+	struct pair_entry *entry;
+	enum object_type type;
+	unsigned long size;
+	struct index_state wtindex;
+	struct checkout lstate, rstate;
+	int rc, flags = RUN_GIT_CMD, err = 0;
+	struct child_process child = CHILD_PROCESS_INIT;
+	const char *helper_argv[] = { "difftool--helper", NULL, NULL, NULL };
+	struct hashmap wt_modified, tmp_modified;
+	int indices_loaded = 0;
+
+	workdir = get_git_work_tree();
+
+	/* Setup temp directories */
+	tmp = getenv("TMPDIR");
+	xsnprintf(tmpdir, sizeof(tmpdir), "%s/git-difftool.XXXXXX", tmp ? tmp : "/tmp");
+	if (!mkdtemp(tmpdir))
+		return error("could not create '%s'", tmpdir);
+	strbuf_addf(&ldir, "%s/left/", tmpdir);
+	strbuf_addf(&rdir, "%s/right/", tmpdir);
+	strbuf_addstr(&wtdir, workdir);
+	if (!wtdir.len || !is_dir_sep(wtdir.buf[wtdir.len - 1]))
+		strbuf_addch(&wtdir, '/');
+	mkdir(ldir.buf, 0700);
+	mkdir(rdir.buf, 0700);
+
+	memset(&wtindex, 0, sizeof(wtindex));
+
+	memset(&lstate, 0, sizeof(lstate));
+	lstate.base_dir = ldir.buf;
+	lstate.base_dir_len = ldir.len;
+	lstate.force = 1;
+	memset(&rstate, 0, sizeof(rstate));
+	rstate.base_dir = rdir.buf;
+	rstate.base_dir_len = rdir.len;
+	rstate.force = 1;
+
+	ldir_len = ldir.len;
+	rdir_len = rdir.len;
+	wtdir_len = wtdir.len;
+
+	hashmap_init(&working_tree_dups,
+		     (hashmap_cmp_fn)working_tree_entry_cmp, 0);
+	hashmap_init(&submodules, (hashmap_cmp_fn)pair_cmp, 0);
+	hashmap_init(&symlinks2, (hashmap_cmp_fn)pair_cmp, 0);
+
+	child.no_stdin = 1;
+	child.git_cmd = 1;
+	child.use_shell = 0;
+	child.clean_on_exit = 1;
+	child.dir = prefix;
+	child.out = -1;
+	argv_array_pushl(&child.args, "diff", "--raw", "--no-abbrev", "-z",
+			 NULL);
+	for (i = 0; i < argc; i++)
+		argv_array_push(&child.args, argv[i]);
+	if (start_command(&child))
+		die("could not obtain raw diff");
+	fp = xfdopen(child.out, "r");
+
+	/* Build index info for left and right sides of the diff */
+	i = 0;
+	while (!strbuf_getline_nul(&info, fp)) {
+		int lmode, rmode;
+		struct object_id loid, roid;
+		char status;
+		const char *src_path, *dst_path;
+		size_t src_path_len, dst_path_len;
+
+		if (starts_with(info.buf, "::"))
+			die(N_("combined diff formats('-c' and '--cc') are "
+			       "not supported in\n"
+			       "directory diff mode('-d' and '--dir-diff')."));
+
+		if (parse_index_info(info.buf, &lmode, &rmode, &loid, &roid,
+				     &status))
+			break;
+		if (strbuf_getline_nul(&lpath, fp))
+			break;
+		src_path = lpath.buf;
+		src_path_len = lpath.len;
+
+		i++;
+		if (status != 'C' && status != 'R') {
+			dst_path = src_path;
+			dst_path_len = src_path_len;
+		} else {
+			if (strbuf_getline_nul(&rpath, fp))
+				break;
+			dst_path = rpath.buf;
+			dst_path_len = rpath.len;
+		}
+
+		if (S_ISGITLINK(lmode) || S_ISGITLINK(rmode)) {
+			strbuf_reset(&buf);
+			strbuf_addf(&buf, "Subproject commit %s",
+				    oid_to_hex(&loid));
+			add_left_or_right(&submodules, src_path, buf.buf, 0);
+			strbuf_reset(&buf);
+			strbuf_addf(&buf, "Subproject commit %s",
+				    oid_to_hex(&roid));
+			if (!oidcmp(&loid, &roid))
+				strbuf_addstr(&buf, "-dirty");
+			add_left_or_right(&submodules, dst_path, buf.buf, 1);
+			continue;
+		}
+
+		if (S_ISLNK(lmode)) {
+			char *content = read_sha1_file(loid.hash, &type, &size);
+			add_left_or_right(&symlinks2, src_path, content, 0);
+			free(content);
+		}
+
+		if (S_ISLNK(rmode)) {
+			char *content = read_sha1_file(roid.hash, &type, &size);
+			add_left_or_right(&symlinks2, dst_path, content, 1);
+			free(content);
+		}
+
+		if (lmode && status != 'C') {
+			ce->ce_mode = lmode;
+			oidcpy(&ce->oid, &loid);
+			strcpy(ce->name, src_path);
+			ce->ce_namelen = src_path_len;
+			if (checkout_entry(ce, &lstate, NULL))
+				return error("could not write '%s'", src_path);
+		}
+
+		if (rmode) {
+			struct working_tree_entry *entry;
+
+			/* Avoid duplicate working_tree entries */
+			FLEX_ALLOC_STR(entry, path, dst_path);
+			hashmap_entry_init(entry, strhash(dst_path));
+			if (hashmap_get(&working_tree_dups, entry, NULL)) {
+				free(entry);
+				continue;
+			}
+			hashmap_add(&working_tree_dups, entry);
+
+			if (!use_wt_file(workdir, dst_path, &roid)) {
+				ce->ce_mode = rmode;
+				oidcpy(&ce->oid, &roid);
+				strcpy(ce->name, dst_path);
+				ce->ce_namelen = dst_path_len;
+				if (checkout_entry(ce, &rstate, NULL))
+					return error("could not write '%s'",
+						     dst_path);
+			} else if (!is_null_oid(&roid)) {
+				/*
+				 * Changes in the working tree need special
+				 * treatment since they are not part of the
+				 * index.
+				 */
+				struct cache_entry *ce2 =
+					make_cache_entry(rmode, roid.hash,
+							 dst_path, 0, 0);
+
+				add_index_entry(&wtindex, ce2,
+						ADD_CACHE_JUST_APPEND);
+
+				add_path(&rdir, rdir_len, dst_path);
+				if (ensure_leading_directories(rdir.buf))
+					return error("could not create "
+						     "directory for '%s'",
+						     dst_path);
+				add_path(&wtdir, wtdir_len, dst_path);
+				if (symlinks) {
+					if (symlink(wtdir.buf, rdir.buf)) {
+						ret = error_errno("could not symlink '%s' to '%s'", wtdir.buf, rdir.buf);
+						goto finish;
+					}
+				} else {
+					struct stat st;
+					if (stat(wtdir.buf, &st))
+						st.st_mode = 0644;
+					if (copy_file(rdir.buf, wtdir.buf,
+						      st.st_mode)) {
+						ret = error("could not copy '%s' to '%s'", wtdir.buf, rdir.buf);
+						goto finish;
+					}
+				}
+			}
+		}
+	}
+
+	if (finish_command(&child)) {
+		ret = error("error occurred running diff --raw");
+		goto finish;
+	}
+
+	if (!i)
+		return 0;
+
+	/*
+	 * Changes to submodules require special treatment.This loop writes a
+	 * temporary file to both the left and right directories to show the
+	 * change in the recorded SHA1 for the submodule.
+	 */
+	hashmap_iter_init(&submodules, &iter);
+	while ((entry = hashmap_iter_next(&iter))) {
+		if (*entry->left) {
+			add_path(&ldir, ldir_len, entry->path);
+			ensure_leading_directories(ldir.buf);
+			write_file(ldir.buf, "%s", entry->left);
+		}
+		if (*entry->right) {
+			add_path(&rdir, rdir_len, entry->path);
+			ensure_leading_directories(rdir.buf);
+			write_file(rdir.buf, "%s", entry->right);
+		}
+	}
+
+	/*
+	 * Symbolic links require special treatment.The standard "git diff"
+	 * shows only the link itself, not the contents of the link target.
+	 * This loop replicates that behavior.
+	 */
+	hashmap_iter_init(&symlinks2, &iter);
+	while ((entry = hashmap_iter_next(&iter))) {
+		if (*entry->left) {
+			add_path(&ldir, ldir_len, entry->path);
+			ensure_leading_directories(ldir.buf);
+			write_file(ldir.buf, "%s", entry->left);
+		}
+		if (*entry->right) {
+			add_path(&rdir, rdir_len, entry->path);
+			ensure_leading_directories(rdir.buf);
+			write_file(rdir.buf, "%s", entry->right);
+		}
+	}
+
+	strbuf_release(&buf);
+
+	strbuf_setlen(&ldir, ldir_len);
+	helper_argv[1] = ldir.buf;
+	strbuf_setlen(&rdir, rdir_len);
+	helper_argv[2] = rdir.buf;
+
+	if (extcmd) {
+		helper_argv[0] = extcmd;
+		flags = 0;
+	} else
+		setenv("GIT_DIFFTOOL_DIRDIFF", "true", 1);
+	rc = run_command_v_opt(helper_argv, flags);
+
+	/*
+	 * If the diff includes working copy files and those
+	 * files were modified during the diff, then the changes
+	 * should be copied back to the working tree.
+	 * Do not copy back files when symlinks are used and the
+	 * external tool did not replace the original link with a file.
+	 *
+	 * These hashes are loaded lazily since they aren't needed
+	 * in the common case of --symlinks and the difftool updating
+	 * files through the symlink.
+	 */
+	hashmap_init(&wt_modified, (hashmap_cmp_fn)path_entry_cmp,
+		     wtindex.cache_nr);
+	hashmap_init(&tmp_modified, (hashmap_cmp_fn)path_entry_cmp,
+		     wtindex.cache_nr);
+
+	for (i = 0; i < wtindex.cache_nr; i++) {
+		struct hashmap_entry dummy;
+		const char *name = wtindex.cache[i]->name;
+		struct stat st;
+
+		add_path(&rdir, rdir_len, name);
+		if (lstat(rdir.buf, &st))
+			continue;
+
+		if ((symlinks && S_ISLNK(st.st_mode)) || !S_ISREG(st.st_mode))
+			continue;
+
+		if (!indices_loaded) {
+			static struct lock_file lock;
+			strbuf_reset(&buf);
+			strbuf_addf(&buf, "%s/wtindex", tmpdir);
+			if (hold_lock_file_for_update(&lock, buf.buf, 0) < 0 ||
+			    write_locked_index(&wtindex, &lock, COMMIT_LOCK)) {
+				ret = error("could not write %s", buf.buf);
+				rollback_lock_file(&lock);
+				goto finish;
+			}
+			changed_files(&wt_modified, buf.buf, workdir);
+			strbuf_setlen(&rdir, rdir_len);
+			changed_files(&tmp_modified, buf.buf, rdir.buf);
+			add_path(&rdir, rdir_len, name);
+			indices_loaded = 1;
+		}
+
+		hashmap_entry_init(&dummy, strhash(name));
+		if (hashmap_get(&tmp_modified, &dummy, name)) {
+			add_path(&wtdir, wtdir_len, name);
+			if (hashmap_get(&wt_modified, &dummy, name)) {
+				warning(_("both files modified: '%s' and '%s'."),
+					wtdir.buf, rdir.buf);
+				warning(_("working tree file has been left."));
+				warning("");
+				err = 1;
+			} else if (unlink(wtdir.buf) ||
+				   copy_file(wtdir.buf, rdir.buf, st.st_mode))
+				warning_errno(_("could not copy '%s' to '%s'"),
+					      rdir.buf, wtdir.buf);
+		}
+	}
+
+	if (err) {
+		warning(_("temporary files exist in '%s'."), tmpdir);
+		warning(_("you may want to cleanup or recover these."));
+		exit(1);
+	} else
+		exit_cleanup(tmpdir, rc);
+
+finish:
+	free(ce);
+	strbuf_release(&ldir);
+	strbuf_release(&rdir);
+	strbuf_release(&wtdir);
+	strbuf_release(&buf);
+
+	return ret;
+}
+
+static int run_file_diff(int prompt, const char *prefix,
+			 int argc, const char **argv)
+{
+	struct argv_array args = ARGV_ARRAY_INIT;
+	const char *env[] = {
+		"GIT_PAGER=", "GIT_EXTERNAL_DIFF=git-difftool--helper", NULL,
+		NULL
+	};
+	int ret = 0, i;
+
+	if (prompt > 0)
+		env[2] = "GIT_DIFFTOOL_PROMPT=true";
+	else if (!prompt)
+		env[2] = "GIT_DIFFTOOL_NO_PROMPT=true";
+
+
+	argv_array_push(&args, "diff");
+	for (i = 0; i < argc; i++)
+		argv_array_push(&args, argv[i]);
+	ret = run_command_v_opt_cd_env(args.argv, RUN_GIT_CMD, prefix, env);
+	exit(ret);
+}
 
 /*
  * NEEDSWORK: this function can go once the legacy-difftool Perl script is
@@ -41,6 +642,35 @@ static int use_builtin_difftool(void) {
 
 int cmd_difftool(int argc, const char **argv, const char *prefix)
 {
+	int use_gui_tool = 0, dir_diff = 0, prompt = -1, symlinks = 0,
+	    tool_help = 0;
+	static char *difftool_cmd = NULL, *extcmd = NULL;
+	struct option builtin_difftool_options[] = {
+		OPT_BOOL('g', "gui", &use_gui_tool,
+			 N_("use `diff.guitool` instead of `diff.tool`")),
+		OPT_BOOL('d', "dir-diff", &dir_diff,
+			 N_("perform a full-directory diff")),
+		{ OPTION_SET_INT, 'y', "no-prompt", &prompt, NULL,
+			N_("do not prompt before launching a diff tool"),
+			PARSE_OPT_NOARG | PARSE_OPT_NONEG, NULL, 0},
+		{ OPTION_SET_INT, 0, "prompt", &prompt, NULL, NULL,
+			PARSE_OPT_NOARG | PARSE_OPT_NONEG | PARSE_OPT_HIDDEN,
+			NULL, 1 },
+		OPT_BOOL(0, "symlinks", &symlinks,
+			 N_("use symlinks in dir-diff mode")),
+		OPT_STRING('t', "tool", &difftool_cmd, N_("<tool>"),
+			   N_("use the specified diff tool")),
+		OPT_BOOL(0, "tool-help", &tool_help,
+			 N_("print a list of diff tools that may be used with "
+			    "`--tool`")),
+		OPT_BOOL(0, "trust-exit-code", &trust_exit_code,
+			 N_("make 'git-difftool' exit when an invoked diff "
+			    "tool returns a non - zero exit code")),
+		OPT_STRING('x', "extcmd", &extcmd, N_("<command>"),
+			   N_("specify a custom command for viewing diffs")),
+		OPT_END()
+	};
+
 	/*
 	 * NEEDSWORK: Once the builtin difftool has been tested enough
 	 * and git-legacy-difftool.perl is retired to contrib/, this preamble
@@ -58,6 +688,46 @@ int cmd_difftool(int argc, const char **argv, const char *prefix)
 	prefix = setup_git_directory();
 	trace_repo_setup(prefix);
 	setup_work_tree();
+	/* NEEDSWORK: once we no longer spawn anything, remove this */
+	setenv(GIT_DIR_ENVIRONMENT, absolute_path(get_git_dir()), 1);
+	setenv(GIT_WORK_TREE_ENVIRONMENT, absolute_path(get_git_work_tree()), 1);
+
+	git_config(difftool_config, NULL);
+	symlinks = has_symlinks;
+
+	argc = parse_options(argc, argv, prefix, builtin_difftool_options,
+			     builtin_difftool_usage, PARSE_OPT_KEEP_UNKNOWN |
+			     PARSE_OPT_KEEP_DASHDASH);
 
-	die("TODO");
+	if (tool_help)
+		return print_tool_help();
+
+	if (use_gui_tool && diff_gui_tool && *diff_gui_tool)
+		setenv("GIT_DIFF_TOOL", diff_gui_tool, 1);
+	else if (difftool_cmd) {
+		if (*difftool_cmd)
+			setenv("GIT_DIFF_TOOL", difftool_cmd, 1);
+		else
+			die(_("no <tool> given for --tool=<tool>"));
+	}
+
+	if (extcmd) {
+		if (*extcmd)
+			setenv("GIT_DIFFTOOL_EXTCMD", extcmd, 1);
+		else
+			die(_("no <cmd> given for --extcmd=<cmd>"));
+	}
+
+	setenv("GIT_DIFFTOOL_TRUST_EXIT_CODE",
+	       trust_exit_code ? "true" : "false", 1);
+
+	/*
+	 * In directory diff mode, 'git-difftool--helper' is called once
+	 * to compare the a / b directories. In file diff mode, 'git diff'
+	 * will invoke a separate instance of 'git-difftool--helper' for
+	 * each file that changed.
+	 */
+	if (dir_diff)
+		return run_dir_diff(extcmd, symlinks, prefix, argc, argv);
+	return run_file_diff(prompt, prefix, argc, argv);
 }
-- 
2.11.0.windows.3

[PATCH v6 0/3] Turn the difftool into a builtin

From: Johannes Schindelin <hidden>
Date: 2017-01-19 20:32:11

This patch series converts the difftool from a Perl script into a
builtin, for three reasons:

1. Perl is really not native on Windows. Not only is there a performance
   penalty to be paid just for running Perl scripts, we also have to deal
   with the fact that users may have different Perl installations, with
   different options, and some other Perl installation may decide to set
   PERL5LIB globally, wreaking havoc with Git for Windows' Perl (which we
   have to use because almost all other Perl distributions lack the
   Subversion bindings we need for `git svn`).

2. As the Perl script uses Unix-y paths that are not native to Windows,
   the Perl interpreter has to go through a POSIX emulation layer (the
   MSYS2 runtime). This means that paths have to be converted from
   Unix-y paths to Windows-y paths (and vice versa) whenever crossing
   the POSIX emulation barrier, leading to quite possibly surprising path
   translation errors.

3. Perl makes for a rather large reason that Git for Windows' installer
   weighs in with >30MB. While one Perl script less does not relieve us
   of that burden, it is one step in the right direction.

Changes since v5:

- reworded the commit message of 2/3 to account for the change in v4
  where we no longer keep both scripted and builtin difftool working
  (with the switch difftool.useBuiltin deciding which one is used).


Johannes Schindelin (3):
  difftool: add a skeleton for the upcoming builtin
  difftool: implement the functionality in the builtin
  Retire the scripted difftool

 Makefile                                           |   2 +-
 builtin.h                                          |   1 +
 builtin/difftool.c                                 | 692 +++++++++++++++++++++
 .../examples/git-difftool.perl                     |   0
 git.c                                              |   1 +
 t/t7800-difftool.sh                                |  92 +--
 6 files changed, 741 insertions(+), 47 deletions(-)
 create mode 100644 builtin/difftool.c
 rename git-difftool.perl => contrib/examples/git-difftool.perl (100%)


base-commit: ffac48d093d4b518a0cc0e8bf1b7cb53e0c3d7a2
Published-As: https://github.com/dscho/git/releases/tag/builtin-difftool-v6
Fetch-It-Via: git fetch https://github.com/dscho/git builtin-difftool-v6

-- 
2.11.0.windows.3

[PATCH v6 1/3] difftool: add a skeleton for the upcoming builtin

From: Johannes Schindelin <hidden>
Date: 2017-01-19 20:32:14

This adds a builtin difftool that still falls back to the legacy Perl
version, which has been renamed to `legacy-difftool`.

The idea is that the new, experimental, builtin difftool immediately hands
off to the legacy difftool for now, unless the config variable
difftool.useBuiltin is set to true.

This feature flag will be used in the upcoming Git for Windows v2.11.0
release, to allow early testers to opt-in to use the builtin difftool and
flesh out any bugs.

Signed-off-by: Johannes Schindelin <redacted>
---
 .gitignore                                    |  1 +
 Makefile                                      |  3 +-
 builtin.h                                     |  1 +
 builtin/difftool.c                            | 63 +++++++++++++++++++++++++++
 git-difftool.perl => git-legacy-difftool.perl |  0
 git.c                                         |  6 +++
 t/t7800-difftool.sh                           |  2 +
 7 files changed, 75 insertions(+), 1 deletion(-)
 create mode 100644 builtin/difftool.c
 rename git-difftool.perl => git-legacy-difftool.perl (100%)
diff --git a/.gitignore b/.gitignore
index 6722f78f9a..5555ae025b 100644
--- a/.gitignore
+++ b/.gitignore
@@ -76,6 +76,7 @@
 /git-init-db
 /git-interpret-trailers
 /git-instaweb
+/git-legacy-difftool
 /git-log
 /git-ls-files
 /git-ls-remote
diff --git a/Makefile b/Makefile
index d861bd9985..8cf5bef034 100644
--- a/Makefile
+++ b/Makefile
@@ -522,7 +522,7 @@ SCRIPT_LIB += git-sh-setup
 SCRIPT_LIB += git-sh-i18n
 
 SCRIPT_PERL += git-add--interactive.perl
-SCRIPT_PERL += git-difftool.perl
+SCRIPT_PERL += git-legacy-difftool.perl
 SCRIPT_PERL += git-archimport.perl
 SCRIPT_PERL += git-cvsexportcommit.perl
 SCRIPT_PERL += git-cvsimport.perl
@@ -883,6 +883,7 @@ BUILTIN_OBJS += builtin/diff-files.o
 BUILTIN_OBJS += builtin/diff-index.o
 BUILTIN_OBJS += builtin/diff-tree.o
 BUILTIN_OBJS += builtin/diff.o
+BUILTIN_OBJS += builtin/difftool.o
 BUILTIN_OBJS += builtin/fast-export.o
 BUILTIN_OBJS += builtin/fetch-pack.o
 BUILTIN_OBJS += builtin/fetch.o
diff --git a/builtin.h b/builtin.h
index b9122bc5f4..67f80519da 100644
--- a/builtin.h
+++ b/builtin.h
@@ -60,6 +60,7 @@ extern int cmd_diff_files(int argc, const char **argv, const char *prefix);
 extern int cmd_diff_index(int argc, const char **argv, const char *prefix);
 extern int cmd_diff(int argc, const char **argv, const char *prefix);
 extern int cmd_diff_tree(int argc, const char **argv, const char *prefix);
+extern int cmd_difftool(int argc, const char **argv, const char *prefix);
 extern int cmd_fast_export(int argc, const char **argv, const char *prefix);
 extern int cmd_fetch(int argc, const char **argv, const char *prefix);
 extern int cmd_fetch_pack(int argc, const char **argv, const char *prefix);
diff --git a/builtin/difftool.c b/builtin/difftool.c
new file mode 100644
index 0000000000..53870bbaf7
--- /dev/null
+++ b/builtin/difftool.c
@@ -0,0 +1,63 @@
+/*
+ * "git difftool" builtin command
+ *
+ * This is a wrapper around the GIT_EXTERNAL_DIFF-compatible
+ * git-difftool--helper script.
+ *
+ * This script exports GIT_EXTERNAL_DIFF and GIT_PAGER for use by git.
+ * The GIT_DIFF* variables are exported for use by git-difftool--helper.
+ *
+ * Any arguments that are unknown to this script are forwarded to 'git diff'.
+ *
+ * Copyright (C) 2016 Johannes Schindelin
+ */
+#include "builtin.h"
+#include "run-command.h"
+#include "exec_cmd.h"
+
+/*
+ * NEEDSWORK: this function can go once the legacy-difftool Perl script is
+ * retired.
+ *
+ * We intentionally avoid reading the config directly here, to avoid messing up
+ * the GIT_* environment variables when we need to fall back to exec()ing the
+ * Perl script.
+ */
+static int use_builtin_difftool(void) {
+	struct child_process cp = CHILD_PROCESS_INIT;
+	struct strbuf out = STRBUF_INIT;
+	int ret;
+
+	argv_array_pushl(&cp.args,
+			 "config", "--bool", "difftool.usebuiltin", NULL);
+	cp.git_cmd = 1;
+	if (capture_command(&cp, &out, 6))
+		return 0;
+	strbuf_trim(&out);
+	ret = !strcmp("true", out.buf);
+	strbuf_release(&out);
+	return ret;
+}
+
+int cmd_difftool(int argc, const char **argv, const char *prefix)
+{
+	/*
+	 * NEEDSWORK: Once the builtin difftool has been tested enough
+	 * and git-legacy-difftool.perl is retired to contrib/, this preamble
+	 * can be removed.
+	 */
+	if (!use_builtin_difftool()) {
+		const char *path = mkpath("%s/git-legacy-difftool",
+					  git_exec_path());
+
+		if (sane_execvp(path, (char **)argv) < 0)
+			die_errno("could not exec %s", path);
+
+		return 0;
+	}
+	prefix = setup_git_directory();
+	trace_repo_setup(prefix);
+	setup_work_tree();
+
+	die("TODO");
+}
diff --git a/git-difftool.perl b/git-legacy-difftool.perl
similarity index 100%
rename from git-difftool.perl
rename to git-legacy-difftool.perl
diff --git a/git.c b/git.c
index bbaa949e9c..c58181e5ef 100644
--- a/git.c
+++ b/git.c
@@ -424,6 +424,12 @@ static struct cmd_struct commands[] = {
 	{ "diff-files", cmd_diff_files, RUN_SETUP | NEED_WORK_TREE },
 	{ "diff-index", cmd_diff_index, RUN_SETUP },
 	{ "diff-tree", cmd_diff_tree, RUN_SETUP },
+	/*
+	 * NEEDSWORK: Once the redirection to git-legacy-difftool.perl in
+	 * builtin/difftool.c has been removed, this entry should be changed to
+	 * RUN_SETUP | NEED_WORK_TREE
+	 */
+	{ "difftool", cmd_difftool },
 	{ "fast-export", cmd_fast_export, RUN_SETUP },
 	{ "fetch", cmd_fetch, RUN_SETUP },
 	{ "fetch-pack", cmd_fetch_pack, RUN_SETUP },
diff --git a/t/t7800-difftool.sh b/t/t7800-difftool.sh
index 99d4123461..e94910c563 100755
--- a/t/t7800-difftool.sh
+++ b/t/t7800-difftool.sh
@@ -23,6 +23,8 @@ prompt_given ()
 	test "$prompt" = "Launch 'test-tool' [Y/n]? branch"
 }
 
+# NEEDSWORK: lose all the PERL prereqs once legacy-difftool is retired.
+
 # Create a file on master and change it on branch
 test_expect_success PERL 'setup' '
 	echo master >file &&
-- 
2.11.0.windows.3

[PATCH v6 2/3] difftool: implement the functionality in the builtin

From: Johannes Schindelin <hidden>
Date: 2017-01-19 20:32:28

This patch gives life to the skeleton added in the previous patch.

The motivation for converting the difftool is that Perl scripts are not at
all native on Windows, and that `git difftool` therefore is pretty slow on
that platform, when there is no good reason for it to be slow.

In addition, Perl does not really have access to Git's internals. That
means that any script will always have to jump through unnecessary
hoops, and it will often need to perform unnecessary work (e.g. when
reading the entire config every time `git config` is called to query a
single config value).

The current version of the builtin difftool does not, however, make full
use of the internals but instead chooses to spawn a couple of Git
processes, still, to make for an easier conversion. There remains a lot
of room for improvement, left later.

Note: to play it safe, the original difftool is still called unless the
config setting difftool.useBuiltin is set to true.

The reason: this new, experimental, builtin difftool was shipped as part
of Git for Windows v2.11.0, to allow for easier large-scale testing, but
of course as an opt-in feature.

The speedup is actually more noticable on Linux than on Windows: a quick
test shows that t7800-difftool.sh runs in (2.183s/0.052s/0.108s)
(real/user/sys) in a Linux VM, down from  (6.529s/3.112s/0.644s), while on
Windows, it is (36.064s/2.730s/7.194s), down from (47.637s/2.407s/6.863s).
The culprit is most likely the overhead incurred from *still* having to
shell out to mergetool-lib.sh and difftool--helper.sh.

Still, it is an improvement.

Signed-off-by: Johannes Schindelin <redacted>
---
 builtin/difftool.c | 672 ++++++++++++++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 671 insertions(+), 1 deletion(-)
diff --git a/builtin/difftool.c b/builtin/difftool.c
index 53870bbaf7..2115e548a5 100644
--- a/builtin/difftool.c
+++ b/builtin/difftool.c
@@ -11,9 +11,610 @@
  *
  * Copyright (C) 2016 Johannes Schindelin
  */
+#include "cache.h"
 #include "builtin.h"
 #include "run-command.h"
 #include "exec_cmd.h"
+#include "parse-options.h"
+#include "argv-array.h"
+#include "strbuf.h"
+#include "lockfile.h"
+#include "dir.h"
+
+static char *diff_gui_tool;
+static int trust_exit_code;
+
+static const char *const builtin_difftool_usage[] = {
+	N_("git difftool [<options>] [<commit> [<commit>]] [--] [<path>...]"),
+	NULL
+};
+
+static int difftool_config(const char *var, const char *value, void *cb)
+{
+	if (!strcmp(var, "diff.guitool")) {
+		diff_gui_tool = xstrdup(value);
+		return 0;
+	}
+
+	if (!strcmp(var, "difftool.trustexitcode")) {
+		trust_exit_code = git_config_bool(var, value);
+		return 0;
+	}
+
+	return git_default_config(var, value, cb);
+}
+
+static int print_tool_help(void)
+{
+	const char *argv[] = { "mergetool", "--tool-help=diff", NULL };
+	return run_command_v_opt(argv, RUN_GIT_CMD);
+}
+
+static int parse_index_info(char *p, int *mode1, int *mode2,
+			    struct object_id *oid1, struct object_id *oid2,
+			    char *status)
+{
+	if (*p != ':')
+		return error("expected ':', got '%c'", *p);
+	*mode1 = (int)strtol(p + 1, &p, 8);
+	if (*p != ' ')
+		return error("expected ' ', got '%c'", *p);
+	*mode2 = (int)strtol(p + 1, &p, 8);
+	if (*p != ' ')
+		return error("expected ' ', got '%c'", *p);
+	if (get_oid_hex(++p, oid1))
+		return error("expected object ID, got '%s'", p + 1);
+	p += GIT_SHA1_HEXSZ;
+	if (*p != ' ')
+		return error("expected ' ', got '%c'", *p);
+	if (get_oid_hex(++p, oid2))
+		return error("expected object ID, got '%s'", p + 1);
+	p += GIT_SHA1_HEXSZ;
+	if (*p != ' ')
+		return error("expected ' ', got '%c'", *p);
+	*status = *++p;
+	if (!*status)
+		return error("missing status");
+	if (p[1] && !isdigit(p[1]))
+		return error("unexpected trailer: '%s'", p + 1);
+	return 0;
+}
+
+/*
+ * Remove any trailing slash from $workdir
+ * before starting to avoid double slashes in symlink targets.
+ */
+static void add_path(struct strbuf *buf, size_t base_len, const char *path)
+{
+	strbuf_setlen(buf, base_len);
+	if (buf->len && buf->buf[buf->len - 1] != '/')
+		strbuf_addch(buf, '/');
+	strbuf_addstr(buf, path);
+}
+
+/*
+ * Determine whether we can simply reuse the file in the worktree.
+ */
+static int use_wt_file(const char *workdir, const char *name,
+		       struct object_id *oid)
+{
+	struct strbuf buf = STRBUF_INIT;
+	struct stat st;
+	int use = 0;
+
+	strbuf_addstr(&buf, workdir);
+	add_path(&buf, buf.len, name);
+
+	if (!lstat(buf.buf, &st) && !S_ISLNK(st.st_mode)) {
+		struct object_id wt_oid;
+		int fd = open(buf.buf, O_RDONLY);
+
+		if (fd >= 0 &&
+		    !index_fd(wt_oid.hash, fd, &st, OBJ_BLOB, name, 0)) {
+			if (is_null_oid(oid)) {
+				oidcpy(oid, &wt_oid);
+				use = 1;
+			} else if (!oidcmp(oid, &wt_oid))
+				use = 1;
+		}
+	}
+
+	strbuf_release(&buf);
+
+	return use;
+}
+
+struct working_tree_entry {
+	struct hashmap_entry entry;
+	char path[FLEX_ARRAY];
+};
+
+static int working_tree_entry_cmp(struct working_tree_entry *a,
+				  struct working_tree_entry *b, void *keydata)
+{
+	return strcmp(a->path, b->path);
+}
+
+/*
+ * The `left` and `right` entries hold paths for the symlinks hashmap,
+ * and a SHA-1 surrounded by brief text for submodules.
+ */
+struct pair_entry {
+	struct hashmap_entry entry;
+	char left[PATH_MAX], right[PATH_MAX];
+	const char path[FLEX_ARRAY];
+};
+
+static int pair_cmp(struct pair_entry *a, struct pair_entry *b, void *keydata)
+{
+	return strcmp(a->path, b->path);
+}
+
+static void add_left_or_right(struct hashmap *map, const char *path,
+			      const char *content, int is_right)
+{
+	struct pair_entry *e, *existing;
+
+	FLEX_ALLOC_STR(e, path, path);
+	hashmap_entry_init(e, strhash(path));
+	existing = hashmap_get(map, e, NULL);
+	if (existing) {
+		free(e);
+		e = existing;
+	} else {
+		e->left[0] = e->right[0] = '\0';
+		hashmap_add(map, e);
+	}
+	strlcpy(is_right ? e->right : e->left, content, PATH_MAX);
+}
+
+struct path_entry {
+	struct hashmap_entry entry;
+	char path[FLEX_ARRAY];
+};
+
+static int path_entry_cmp(struct path_entry *a, struct path_entry *b, void *key)
+{
+	return strcmp(a->path, key ? key : b->path);
+}
+
+static void changed_files(struct hashmap *result, const char *index_path,
+			  const char *workdir)
+{
+	struct child_process update_index = CHILD_PROCESS_INIT;
+	struct child_process diff_files = CHILD_PROCESS_INIT;
+	struct strbuf index_env = STRBUF_INIT, buf = STRBUF_INIT;
+	const char *git_dir = absolute_path(get_git_dir()), *env[] = {
+		NULL, NULL
+	};
+	FILE *fp;
+
+	strbuf_addf(&index_env, "GIT_INDEX_FILE=%s", index_path);
+	env[0] = index_env.buf;
+
+	argv_array_pushl(&update_index.args,
+			 "--git-dir", git_dir, "--work-tree", workdir,
+			 "update-index", "--really-refresh", "-q",
+			 "--unmerged", NULL);
+	update_index.no_stdin = 1;
+	update_index.no_stdout = 1;
+	update_index.no_stderr = 1;
+	update_index.git_cmd = 1;
+	update_index.use_shell = 0;
+	update_index.clean_on_exit = 1;
+	update_index.dir = workdir;
+	update_index.env = env;
+	/* Ignore any errors of update-index */
+	run_command(&update_index);
+
+	argv_array_pushl(&diff_files.args,
+			 "--git-dir", git_dir, "--work-tree", workdir,
+			 "diff-files", "--name-only", "-z", NULL);
+	diff_files.no_stdin = 1;
+	diff_files.git_cmd = 1;
+	diff_files.use_shell = 0;
+	diff_files.clean_on_exit = 1;
+	diff_files.out = -1;
+	diff_files.dir = workdir;
+	diff_files.env = env;
+	if (start_command(&diff_files))
+		die("could not obtain raw diff");
+	fp = xfdopen(diff_files.out, "r");
+	while (!strbuf_getline_nul(&buf, fp)) {
+		struct path_entry *entry;
+		FLEX_ALLOC_STR(entry, path, buf.buf);
+		hashmap_entry_init(entry, strhash(buf.buf));
+		hashmap_add(result, entry);
+	}
+	if (finish_command(&diff_files))
+		die("diff-files did not exit properly");
+	strbuf_release(&index_env);
+	strbuf_release(&buf);
+}
+
+static NORETURN void exit_cleanup(const char *tmpdir, int exit_code)
+{
+	struct strbuf buf = STRBUF_INIT;
+	strbuf_addstr(&buf, tmpdir);
+	remove_dir_recursively(&buf, 0);
+	if (exit_code)
+		warning(_("failed: %d"), exit_code);
+	exit(exit_code);
+}
+
+static int ensure_leading_directories(char *path)
+{
+	switch (safe_create_leading_directories(path)) {
+		case SCLD_OK:
+		case SCLD_EXISTS:
+			return 0;
+		default:
+			return error(_("could not create leading directories "
+				       "of '%s'"), path);
+	}
+}
+
+static int run_dir_diff(const char *extcmd, int symlinks, const char *prefix,
+			int argc, const char **argv)
+{
+	char tmpdir[PATH_MAX];
+	struct strbuf info = STRBUF_INIT, lpath = STRBUF_INIT;
+	struct strbuf rpath = STRBUF_INIT, buf = STRBUF_INIT;
+	struct strbuf ldir = STRBUF_INIT, rdir = STRBUF_INIT;
+	struct strbuf wtdir = STRBUF_INIT;
+	size_t ldir_len, rdir_len, wtdir_len;
+	struct cache_entry *ce = xcalloc(1, sizeof(ce) + PATH_MAX + 1);
+	const char *workdir, *tmp;
+	int ret = 0, i;
+	FILE *fp;
+	struct hashmap working_tree_dups, submodules, symlinks2;
+	struct hashmap_iter iter;
+	struct pair_entry *entry;
+	enum object_type type;
+	unsigned long size;
+	struct index_state wtindex;
+	struct checkout lstate, rstate;
+	int rc, flags = RUN_GIT_CMD, err = 0;
+	struct child_process child = CHILD_PROCESS_INIT;
+	const char *helper_argv[] = { "difftool--helper", NULL, NULL, NULL };
+	struct hashmap wt_modified, tmp_modified;
+	int indices_loaded = 0;
+
+	workdir = get_git_work_tree();
+
+	/* Setup temp directories */
+	tmp = getenv("TMPDIR");
+	xsnprintf(tmpdir, sizeof(tmpdir), "%s/git-difftool.XXXXXX", tmp ? tmp : "/tmp");
+	if (!mkdtemp(tmpdir))
+		return error("could not create '%s'", tmpdir);
+	strbuf_addf(&ldir, "%s/left/", tmpdir);
+	strbuf_addf(&rdir, "%s/right/", tmpdir);
+	strbuf_addstr(&wtdir, workdir);
+	if (!wtdir.len || !is_dir_sep(wtdir.buf[wtdir.len - 1]))
+		strbuf_addch(&wtdir, '/');
+	mkdir(ldir.buf, 0700);
+	mkdir(rdir.buf, 0700);
+
+	memset(&wtindex, 0, sizeof(wtindex));
+
+	memset(&lstate, 0, sizeof(lstate));
+	lstate.base_dir = ldir.buf;
+	lstate.base_dir_len = ldir.len;
+	lstate.force = 1;
+	memset(&rstate, 0, sizeof(rstate));
+	rstate.base_dir = rdir.buf;
+	rstate.base_dir_len = rdir.len;
+	rstate.force = 1;
+
+	ldir_len = ldir.len;
+	rdir_len = rdir.len;
+	wtdir_len = wtdir.len;
+
+	hashmap_init(&working_tree_dups,
+		     (hashmap_cmp_fn)working_tree_entry_cmp, 0);
+	hashmap_init(&submodules, (hashmap_cmp_fn)pair_cmp, 0);
+	hashmap_init(&symlinks2, (hashmap_cmp_fn)pair_cmp, 0);
+
+	child.no_stdin = 1;
+	child.git_cmd = 1;
+	child.use_shell = 0;
+	child.clean_on_exit = 1;
+	child.dir = prefix;
+	child.out = -1;
+	argv_array_pushl(&child.args, "diff", "--raw", "--no-abbrev", "-z",
+			 NULL);
+	for (i = 0; i < argc; i++)
+		argv_array_push(&child.args, argv[i]);
+	if (start_command(&child))
+		die("could not obtain raw diff");
+	fp = xfdopen(child.out, "r");
+
+	/* Build index info for left and right sides of the diff */
+	i = 0;
+	while (!strbuf_getline_nul(&info, fp)) {
+		int lmode, rmode;
+		struct object_id loid, roid;
+		char status;
+		const char *src_path, *dst_path;
+		size_t src_path_len, dst_path_len;
+
+		if (starts_with(info.buf, "::"))
+			die(N_("combined diff formats('-c' and '--cc') are "
+			       "not supported in\n"
+			       "directory diff mode('-d' and '--dir-diff')."));
+
+		if (parse_index_info(info.buf, &lmode, &rmode, &loid, &roid,
+				     &status))
+			break;
+		if (strbuf_getline_nul(&lpath, fp))
+			break;
+		src_path = lpath.buf;
+		src_path_len = lpath.len;
+
+		i++;
+		if (status != 'C' && status != 'R') {
+			dst_path = src_path;
+			dst_path_len = src_path_len;
+		} else {
+			if (strbuf_getline_nul(&rpath, fp))
+				break;
+			dst_path = rpath.buf;
+			dst_path_len = rpath.len;
+		}
+
+		if (S_ISGITLINK(lmode) || S_ISGITLINK(rmode)) {
+			strbuf_reset(&buf);
+			strbuf_addf(&buf, "Subproject commit %s",
+				    oid_to_hex(&loid));
+			add_left_or_right(&submodules, src_path, buf.buf, 0);
+			strbuf_reset(&buf);
+			strbuf_addf(&buf, "Subproject commit %s",
+				    oid_to_hex(&roid));
+			if (!oidcmp(&loid, &roid))
+				strbuf_addstr(&buf, "-dirty");
+			add_left_or_right(&submodules, dst_path, buf.buf, 1);
+			continue;
+		}
+
+		if (S_ISLNK(lmode)) {
+			char *content = read_sha1_file(loid.hash, &type, &size);
+			add_left_or_right(&symlinks2, src_path, content, 0);
+			free(content);
+		}
+
+		if (S_ISLNK(rmode)) {
+			char *content = read_sha1_file(roid.hash, &type, &size);
+			add_left_or_right(&symlinks2, dst_path, content, 1);
+			free(content);
+		}
+
+		if (lmode && status != 'C') {
+			ce->ce_mode = lmode;
+			oidcpy(&ce->oid, &loid);
+			strcpy(ce->name, src_path);
+			ce->ce_namelen = src_path_len;
+			if (checkout_entry(ce, &lstate, NULL))
+				return error("could not write '%s'", src_path);
+		}
+
+		if (rmode) {
+			struct working_tree_entry *entry;
+
+			/* Avoid duplicate working_tree entries */
+			FLEX_ALLOC_STR(entry, path, dst_path);
+			hashmap_entry_init(entry, strhash(dst_path));
+			if (hashmap_get(&working_tree_dups, entry, NULL)) {
+				free(entry);
+				continue;
+			}
+			hashmap_add(&working_tree_dups, entry);
+
+			if (!use_wt_file(workdir, dst_path, &roid)) {
+				ce->ce_mode = rmode;
+				oidcpy(&ce->oid, &roid);
+				strcpy(ce->name, dst_path);
+				ce->ce_namelen = dst_path_len;
+				if (checkout_entry(ce, &rstate, NULL))
+					return error("could not write '%s'",
+						     dst_path);
+			} else if (!is_null_oid(&roid)) {
+				/*
+				 * Changes in the working tree need special
+				 * treatment since they are not part of the
+				 * index.
+				 */
+				struct cache_entry *ce2 =
+					make_cache_entry(rmode, roid.hash,
+							 dst_path, 0, 0);
+
+				add_index_entry(&wtindex, ce2,
+						ADD_CACHE_JUST_APPEND);
+
+				add_path(&rdir, rdir_len, dst_path);
+				if (ensure_leading_directories(rdir.buf))
+					return error("could not create "
+						     "directory for '%s'",
+						     dst_path);
+				add_path(&wtdir, wtdir_len, dst_path);
+				if (symlinks) {
+					if (symlink(wtdir.buf, rdir.buf)) {
+						ret = error_errno("could not symlink '%s' to '%s'", wtdir.buf, rdir.buf);
+						goto finish;
+					}
+				} else {
+					struct stat st;
+					if (stat(wtdir.buf, &st))
+						st.st_mode = 0644;
+					if (copy_file(rdir.buf, wtdir.buf,
+						      st.st_mode)) {
+						ret = error("could not copy '%s' to '%s'", wtdir.buf, rdir.buf);
+						goto finish;
+					}
+				}
+			}
+		}
+	}
+
+	if (finish_command(&child)) {
+		ret = error("error occurred running diff --raw");
+		goto finish;
+	}
+
+	if (!i)
+		return 0;
+
+	/*
+	 * Changes to submodules require special treatment.This loop writes a
+	 * temporary file to both the left and right directories to show the
+	 * change in the recorded SHA1 for the submodule.
+	 */
+	hashmap_iter_init(&submodules, &iter);
+	while ((entry = hashmap_iter_next(&iter))) {
+		if (*entry->left) {
+			add_path(&ldir, ldir_len, entry->path);
+			ensure_leading_directories(ldir.buf);
+			write_file(ldir.buf, "%s", entry->left);
+		}
+		if (*entry->right) {
+			add_path(&rdir, rdir_len, entry->path);
+			ensure_leading_directories(rdir.buf);
+			write_file(rdir.buf, "%s", entry->right);
+		}
+	}
+
+	/*
+	 * Symbolic links require special treatment.The standard "git diff"
+	 * shows only the link itself, not the contents of the link target.
+	 * This loop replicates that behavior.
+	 */
+	hashmap_iter_init(&symlinks2, &iter);
+	while ((entry = hashmap_iter_next(&iter))) {
+		if (*entry->left) {
+			add_path(&ldir, ldir_len, entry->path);
+			ensure_leading_directories(ldir.buf);
+			write_file(ldir.buf, "%s", entry->left);
+		}
+		if (*entry->right) {
+			add_path(&rdir, rdir_len, entry->path);
+			ensure_leading_directories(rdir.buf);
+			write_file(rdir.buf, "%s", entry->right);
+		}
+	}
+
+	strbuf_release(&buf);
+
+	strbuf_setlen(&ldir, ldir_len);
+	helper_argv[1] = ldir.buf;
+	strbuf_setlen(&rdir, rdir_len);
+	helper_argv[2] = rdir.buf;
+
+	if (extcmd) {
+		helper_argv[0] = extcmd;
+		flags = 0;
+	} else
+		setenv("GIT_DIFFTOOL_DIRDIFF", "true", 1);
+	rc = run_command_v_opt(helper_argv, flags);
+
+	/*
+	 * If the diff includes working copy files and those
+	 * files were modified during the diff, then the changes
+	 * should be copied back to the working tree.
+	 * Do not copy back files when symlinks are used and the
+	 * external tool did not replace the original link with a file.
+	 *
+	 * These hashes are loaded lazily since they aren't needed
+	 * in the common case of --symlinks and the difftool updating
+	 * files through the symlink.
+	 */
+	hashmap_init(&wt_modified, (hashmap_cmp_fn)path_entry_cmp,
+		     wtindex.cache_nr);
+	hashmap_init(&tmp_modified, (hashmap_cmp_fn)path_entry_cmp,
+		     wtindex.cache_nr);
+
+	for (i = 0; i < wtindex.cache_nr; i++) {
+		struct hashmap_entry dummy;
+		const char *name = wtindex.cache[i]->name;
+		struct stat st;
+
+		add_path(&rdir, rdir_len, name);
+		if (lstat(rdir.buf, &st))
+			continue;
+
+		if ((symlinks && S_ISLNK(st.st_mode)) || !S_ISREG(st.st_mode))
+			continue;
+
+		if (!indices_loaded) {
+			static struct lock_file lock;
+			strbuf_reset(&buf);
+			strbuf_addf(&buf, "%s/wtindex", tmpdir);
+			if (hold_lock_file_for_update(&lock, buf.buf, 0) < 0 ||
+			    write_locked_index(&wtindex, &lock, COMMIT_LOCK)) {
+				ret = error("could not write %s", buf.buf);
+				rollback_lock_file(&lock);
+				goto finish;
+			}
+			changed_files(&wt_modified, buf.buf, workdir);
+			strbuf_setlen(&rdir, rdir_len);
+			changed_files(&tmp_modified, buf.buf, rdir.buf);
+			add_path(&rdir, rdir_len, name);
+			indices_loaded = 1;
+		}
+
+		hashmap_entry_init(&dummy, strhash(name));
+		if (hashmap_get(&tmp_modified, &dummy, name)) {
+			add_path(&wtdir, wtdir_len, name);
+			if (hashmap_get(&wt_modified, &dummy, name)) {
+				warning(_("both files modified: '%s' and '%s'."),
+					wtdir.buf, rdir.buf);
+				warning(_("working tree file has been left."));
+				warning("");
+				err = 1;
+			} else if (unlink(wtdir.buf) ||
+				   copy_file(wtdir.buf, rdir.buf, st.st_mode))
+				warning_errno(_("could not copy '%s' to '%s'"),
+					      rdir.buf, wtdir.buf);
+		}
+	}
+
+	if (err) {
+		warning(_("temporary files exist in '%s'."), tmpdir);
+		warning(_("you may want to cleanup or recover these."));
+		exit(1);
+	} else
+		exit_cleanup(tmpdir, rc);
+
+finish:
+	free(ce);
+	strbuf_release(&ldir);
+	strbuf_release(&rdir);
+	strbuf_release(&wtdir);
+	strbuf_release(&buf);
+
+	return ret;
+}
+
+static int run_file_diff(int prompt, const char *prefix,
+			 int argc, const char **argv)
+{
+	struct argv_array args = ARGV_ARRAY_INIT;
+	const char *env[] = {
+		"GIT_PAGER=", "GIT_EXTERNAL_DIFF=git-difftool--helper", NULL,
+		NULL
+	};
+	int ret = 0, i;
+
+	if (prompt > 0)
+		env[2] = "GIT_DIFFTOOL_PROMPT=true";
+	else if (!prompt)
+		env[2] = "GIT_DIFFTOOL_NO_PROMPT=true";
+
+
+	argv_array_push(&args, "diff");
+	for (i = 0; i < argc; i++)
+		argv_array_push(&args, argv[i]);
+	ret = run_command_v_opt_cd_env(args.argv, RUN_GIT_CMD, prefix, env);
+	exit(ret);
+}
 
 /*
  * NEEDSWORK: this function can go once the legacy-difftool Perl script is
@@ -41,6 +642,35 @@ static int use_builtin_difftool(void) {
 
 int cmd_difftool(int argc, const char **argv, const char *prefix)
 {
+	int use_gui_tool = 0, dir_diff = 0, prompt = -1, symlinks = 0,
+	    tool_help = 0;
+	static char *difftool_cmd = NULL, *extcmd = NULL;
+	struct option builtin_difftool_options[] = {
+		OPT_BOOL('g', "gui", &use_gui_tool,
+			 N_("use `diff.guitool` instead of `diff.tool`")),
+		OPT_BOOL('d', "dir-diff", &dir_diff,
+			 N_("perform a full-directory diff")),
+		{ OPTION_SET_INT, 'y', "no-prompt", &prompt, NULL,
+			N_("do not prompt before launching a diff tool"),
+			PARSE_OPT_NOARG | PARSE_OPT_NONEG, NULL, 0},
+		{ OPTION_SET_INT, 0, "prompt", &prompt, NULL, NULL,
+			PARSE_OPT_NOARG | PARSE_OPT_NONEG | PARSE_OPT_HIDDEN,
+			NULL, 1 },
+		OPT_BOOL(0, "symlinks", &symlinks,
+			 N_("use symlinks in dir-diff mode")),
+		OPT_STRING('t', "tool", &difftool_cmd, N_("<tool>"),
+			   N_("use the specified diff tool")),
+		OPT_BOOL(0, "tool-help", &tool_help,
+			 N_("print a list of diff tools that may be used with "
+			    "`--tool`")),
+		OPT_BOOL(0, "trust-exit-code", &trust_exit_code,
+			 N_("make 'git-difftool' exit when an invoked diff "
+			    "tool returns a non - zero exit code")),
+		OPT_STRING('x', "extcmd", &extcmd, N_("<command>"),
+			   N_("specify a custom command for viewing diffs")),
+		OPT_END()
+	};
+
 	/*
 	 * NEEDSWORK: Once the builtin difftool has been tested enough
 	 * and git-legacy-difftool.perl is retired to contrib/, this preamble
@@ -58,6 +688,46 @@ int cmd_difftool(int argc, const char **argv, const char *prefix)
 	prefix = setup_git_directory();
 	trace_repo_setup(prefix);
 	setup_work_tree();
+	/* NEEDSWORK: once we no longer spawn anything, remove this */
+	setenv(GIT_DIR_ENVIRONMENT, absolute_path(get_git_dir()), 1);
+	setenv(GIT_WORK_TREE_ENVIRONMENT, absolute_path(get_git_work_tree()), 1);
+
+	git_config(difftool_config, NULL);
+	symlinks = has_symlinks;
+
+	argc = parse_options(argc, argv, prefix, builtin_difftool_options,
+			     builtin_difftool_usage, PARSE_OPT_KEEP_UNKNOWN |
+			     PARSE_OPT_KEEP_DASHDASH);
 
-	die("TODO");
+	if (tool_help)
+		return print_tool_help();
+
+	if (use_gui_tool && diff_gui_tool && *diff_gui_tool)
+		setenv("GIT_DIFF_TOOL", diff_gui_tool, 1);
+	else if (difftool_cmd) {
+		if (*difftool_cmd)
+			setenv("GIT_DIFF_TOOL", difftool_cmd, 1);
+		else
+			die(_("no <tool> given for --tool=<tool>"));
+	}
+
+	if (extcmd) {
+		if (*extcmd)
+			setenv("GIT_DIFFTOOL_EXTCMD", extcmd, 1);
+		else
+			die(_("no <cmd> given for --extcmd=<cmd>"));
+	}
+
+	setenv("GIT_DIFFTOOL_TRUST_EXIT_CODE",
+	       trust_exit_code ? "true" : "false", 1);
+
+	/*
+	 * In directory diff mode, 'git-difftool--helper' is called once
+	 * to compare the a / b directories. In file diff mode, 'git diff'
+	 * will invoke a separate instance of 'git-difftool--helper' for
+	 * each file that changed.
+	 */
+	if (dir_diff)
+		return run_dir_diff(extcmd, symlinks, prefix, argc, argv);
+	return run_file_diff(prompt, prefix, argc, argv);
 }
-- 
2.11.0.windows.3

[PATCH v6 3/3] Retire the scripted difftool

From: Johannes Schindelin <hidden>
Date: 2017-01-19 20:40:51

It served its purpose, but now we have a builtin difftool. Time for the
Perl script to enjoy Florida.

Signed-off-by: Johannes Schindelin <redacted>
---
 .gitignore                                         |  1 -
 Makefile                                           |  1 -
 builtin/difftool.c                                 | 41 ----------
 .../examples/git-difftool.perl                     |  0
 git.c                                              |  7 +-
 t/t7800-difftool.sh                                | 94 +++++++++++-----------
 6 files changed, 47 insertions(+), 97 deletions(-)
 rename git-legacy-difftool.perl => contrib/examples/git-difftool.perl (100%)
diff --git a/.gitignore b/.gitignore
index 5555ae025b..6722f78f9a 100644
--- a/.gitignore
+++ b/.gitignore
@@ -76,7 +76,6 @@
 /git-init-db
 /git-interpret-trailers
 /git-instaweb
-/git-legacy-difftool
 /git-log
 /git-ls-files
 /git-ls-remote
diff --git a/Makefile b/Makefile
index 8cf5bef034..e9aa6ae57c 100644
--- a/Makefile
+++ b/Makefile
@@ -522,7 +522,6 @@ SCRIPT_LIB += git-sh-setup
 SCRIPT_LIB += git-sh-i18n
 
 SCRIPT_PERL += git-add--interactive.perl
-SCRIPT_PERL += git-legacy-difftool.perl
 SCRIPT_PERL += git-archimport.perl
 SCRIPT_PERL += git-cvsexportcommit.perl
 SCRIPT_PERL += git-cvsimport.perl
diff --git a/builtin/difftool.c b/builtin/difftool.c
index 2115e548a5..42ad9e804a 100644
--- a/builtin/difftool.c
+++ b/builtin/difftool.c
@@ -616,30 +616,6 @@ static int run_file_diff(int prompt, const char *prefix,
 	exit(ret);
 }
 
-/*
- * NEEDSWORK: this function can go once the legacy-difftool Perl script is
- * retired.
- *
- * We intentionally avoid reading the config directly here, to avoid messing up
- * the GIT_* environment variables when we need to fall back to exec()ing the
- * Perl script.
- */
-static int use_builtin_difftool(void) {
-	struct child_process cp = CHILD_PROCESS_INIT;
-	struct strbuf out = STRBUF_INIT;
-	int ret;
-
-	argv_array_pushl(&cp.args,
-			 "config", "--bool", "difftool.usebuiltin", NULL);
-	cp.git_cmd = 1;
-	if (capture_command(&cp, &out, 6))
-		return 0;
-	strbuf_trim(&out);
-	ret = !strcmp("true", out.buf);
-	strbuf_release(&out);
-	return ret;
-}
-
 int cmd_difftool(int argc, const char **argv, const char *prefix)
 {
 	int use_gui_tool = 0, dir_diff = 0, prompt = -1, symlinks = 0,
@@ -671,23 +647,6 @@ int cmd_difftool(int argc, const char **argv, const char *prefix)
 		OPT_END()
 	};
 
-	/*
-	 * NEEDSWORK: Once the builtin difftool has been tested enough
-	 * and git-legacy-difftool.perl is retired to contrib/, this preamble
-	 * can be removed.
-	 */
-	if (!use_builtin_difftool()) {
-		const char *path = mkpath("%s/git-legacy-difftool",
-					  git_exec_path());
-
-		if (sane_execvp(path, (char **)argv) < 0)
-			die_errno("could not exec %s", path);
-
-		return 0;
-	}
-	prefix = setup_git_directory();
-	trace_repo_setup(prefix);
-	setup_work_tree();
 	/* NEEDSWORK: once we no longer spawn anything, remove this */
 	setenv(GIT_DIR_ENVIRONMENT, absolute_path(get_git_dir()), 1);
 	setenv(GIT_WORK_TREE_ENVIRONMENT, absolute_path(get_git_work_tree()), 1);
diff --git a/git-legacy-difftool.perl b/contrib/examples/git-difftool.perl
similarity index 100%
rename from git-legacy-difftool.perl
rename to contrib/examples/git-difftool.perl
diff --git a/git.c b/git.c
index c58181e5ef..bd4d668a21 100644
--- a/git.c
+++ b/git.c
@@ -424,12 +424,7 @@ static struct cmd_struct commands[] = {
 	{ "diff-files", cmd_diff_files, RUN_SETUP | NEED_WORK_TREE },
 	{ "diff-index", cmd_diff_index, RUN_SETUP },
 	{ "diff-tree", cmd_diff_tree, RUN_SETUP },
-	/*
-	 * NEEDSWORK: Once the redirection to git-legacy-difftool.perl in
-	 * builtin/difftool.c has been removed, this entry should be changed to
-	 * RUN_SETUP | NEED_WORK_TREE
-	 */
-	{ "difftool", cmd_difftool },
+	{ "difftool", cmd_difftool, RUN_SETUP | NEED_WORK_TREE },
 	{ "fast-export", cmd_fast_export, RUN_SETUP },
 	{ "fetch", cmd_fetch, RUN_SETUP },
 	{ "fetch-pack", cmd_fetch_pack, RUN_SETUP },
diff --git a/t/t7800-difftool.sh b/t/t7800-difftool.sh
index e94910c563..aa0ef02597 100755
--- a/t/t7800-difftool.sh
+++ b/t/t7800-difftool.sh
@@ -23,10 +23,8 @@ prompt_given ()
 	test "$prompt" = "Launch 'test-tool' [Y/n]? branch"
 }
 
-# NEEDSWORK: lose all the PERL prereqs once legacy-difftool is retired.
-
 # Create a file on master and change it on branch
-test_expect_success PERL 'setup' '
+test_expect_success 'setup' '
 	echo master >file &&
 	git add file &&
 	git commit -m "added file" &&
@@ -38,7 +36,7 @@ test_expect_success PERL 'setup' '
 '
 
 # Configure a custom difftool.<tool>.cmd and use it
-test_expect_success PERL 'custom commands' '
+test_expect_success 'custom commands' '
 	difftool_test_setup &&
 	test_config difftool.test-tool.cmd "cat \"\$REMOTE\"" &&
 	echo master >expect &&
@@ -51,21 +49,21 @@ test_expect_success PERL 'custom commands' '
 	test_cmp expect actual
 '
 
-test_expect_success PERL 'custom tool commands override built-ins' '
+test_expect_success 'custom tool commands override built-ins' '
 	test_config difftool.vimdiff.cmd "cat \"\$REMOTE\"" &&
 	echo master >expect &&
 	git difftool --tool vimdiff --no-prompt branch >actual &&
 	test_cmp expect actual
 '
 
-test_expect_success PERL 'difftool ignores bad --tool values' '
+test_expect_success 'difftool ignores bad --tool values' '
 	: >expect &&
 	test_must_fail \
 		git difftool --no-prompt --tool=bad-tool branch >actual &&
 	test_cmp expect actual
 '
 
-test_expect_success PERL 'difftool forwards arguments to diff' '
+test_expect_success 'difftool forwards arguments to diff' '
 	difftool_test_setup &&
 	>for-diff &&
 	git add for-diff &&
@@ -78,40 +76,40 @@ test_expect_success PERL 'difftool forwards arguments to diff' '
 	rm for-diff
 '
 
-test_expect_success PERL 'difftool ignores exit code' '
+test_expect_success 'difftool ignores exit code' '
 	test_config difftool.error.cmd false &&
 	git difftool -y -t error branch
 '
 
-test_expect_success PERL 'difftool forwards exit code with --trust-exit-code' '
+test_expect_success 'difftool forwards exit code with --trust-exit-code' '
 	test_config difftool.error.cmd false &&
 	test_must_fail git difftool -y --trust-exit-code -t error branch
 '
 
-test_expect_success PERL 'difftool forwards exit code with --trust-exit-code for built-ins' '
+test_expect_success 'difftool forwards exit code with --trust-exit-code for built-ins' '
 	test_config difftool.vimdiff.path false &&
 	test_must_fail git difftool -y --trust-exit-code -t vimdiff branch
 '
 
-test_expect_success PERL 'difftool honors difftool.trustExitCode = true' '
+test_expect_success 'difftool honors difftool.trustExitCode = true' '
 	test_config difftool.error.cmd false &&
 	test_config difftool.trustExitCode true &&
 	test_must_fail git difftool -y -t error branch
 '
 
-test_expect_success PERL 'difftool honors difftool.trustExitCode = false' '
+test_expect_success 'difftool honors difftool.trustExitCode = false' '
 	test_config difftool.error.cmd false &&
 	test_config difftool.trustExitCode false &&
 	git difftool -y -t error branch
 '
 
-test_expect_success PERL 'difftool ignores exit code with --no-trust-exit-code' '
+test_expect_success 'difftool ignores exit code with --no-trust-exit-code' '
 	test_config difftool.error.cmd false &&
 	test_config difftool.trustExitCode true &&
 	git difftool -y --no-trust-exit-code -t error branch
 '
 
-test_expect_success PERL 'difftool stops on error with --trust-exit-code' '
+test_expect_success 'difftool stops on error with --trust-exit-code' '
 	test_when_finished "rm -f for-diff .git/fail-right-file" &&
 	test_when_finished "git reset -- for-diff" &&
 	write_script .git/fail-right-file <<-\EOF &&
@@ -126,13 +124,13 @@ test_expect_success PERL 'difftool stops on error with --trust-exit-code' '
 	test_cmp expect actual
 '
 
-test_expect_success PERL 'difftool honors exit status if command not found' '
+test_expect_success 'difftool honors exit status if command not found' '
 	test_config difftool.nonexistent.cmd i-dont-exist &&
 	test_config difftool.trustExitCode false &&
 	test_must_fail git difftool -y -t nonexistent branch
 '
 
-test_expect_success PERL 'difftool honors --gui' '
+test_expect_success 'difftool honors --gui' '
 	difftool_test_setup &&
 	test_config merge.tool bogus-tool &&
 	test_config diff.tool bogus-tool &&
@@ -143,7 +141,7 @@ test_expect_success PERL 'difftool honors --gui' '
 	test_cmp expect actual
 '
 
-test_expect_success PERL 'difftool --gui last setting wins' '
+test_expect_success 'difftool --gui last setting wins' '
 	difftool_test_setup &&
 	: >expect &&
 	git difftool --no-prompt --gui --no-gui >actual &&
@@ -157,7 +155,7 @@ test_expect_success PERL 'difftool --gui last setting wins' '
 	test_cmp expect actual
 '
 
-test_expect_success PERL 'difftool --gui works without configured diff.guitool' '
+test_expect_success 'difftool --gui works without configured diff.guitool' '
 	difftool_test_setup &&
 	echo branch >expect &&
 	git difftool --no-prompt --gui branch >actual &&
@@ -165,7 +163,7 @@ test_expect_success PERL 'difftool --gui works without configured diff.guitool'
 '
 
 # Specify the diff tool using $GIT_DIFF_TOOL
-test_expect_success PERL 'GIT_DIFF_TOOL variable' '
+test_expect_success 'GIT_DIFF_TOOL variable' '
 	difftool_test_setup &&
 	git config --unset diff.tool &&
 	echo branch >expect &&
@@ -175,7 +173,7 @@ test_expect_success PERL 'GIT_DIFF_TOOL variable' '
 
 # Test the $GIT_*_TOOL variables and ensure
 # that $GIT_DIFF_TOOL always wins unless --tool is specified
-test_expect_success PERL 'GIT_DIFF_TOOL overrides' '
+test_expect_success 'GIT_DIFF_TOOL overrides' '
 	difftool_test_setup &&
 	test_config diff.tool bogus-tool &&
 	test_config merge.tool bogus-tool &&
@@ -193,7 +191,7 @@ test_expect_success PERL 'GIT_DIFF_TOOL overrides' '
 
 # Test that we don't have to pass --no-prompt to difftool
 # when $GIT_DIFFTOOL_NO_PROMPT is true
-test_expect_success PERL 'GIT_DIFFTOOL_NO_PROMPT variable' '
+test_expect_success 'GIT_DIFFTOOL_NO_PROMPT variable' '
 	difftool_test_setup &&
 	echo branch >expect &&
 	GIT_DIFFTOOL_NO_PROMPT=true git difftool branch >actual &&
@@ -202,7 +200,7 @@ test_expect_success PERL 'GIT_DIFFTOOL_NO_PROMPT variable' '
 
 # git-difftool supports the difftool.prompt variable.
 # Test that GIT_DIFFTOOL_PROMPT can override difftool.prompt = false
-test_expect_success PERL 'GIT_DIFFTOOL_PROMPT variable' '
+test_expect_success 'GIT_DIFFTOOL_PROMPT variable' '
 	difftool_test_setup &&
 	test_config difftool.prompt false &&
 	echo >input &&
@@ -212,7 +210,7 @@ test_expect_success PERL 'GIT_DIFFTOOL_PROMPT variable' '
 '
 
 # Test that we don't have to pass --no-prompt when difftool.prompt is false
-test_expect_success PERL 'difftool.prompt config variable is false' '
+test_expect_success 'difftool.prompt config variable is false' '
 	difftool_test_setup &&
 	test_config difftool.prompt false &&
 	echo branch >expect &&
@@ -221,7 +219,7 @@ test_expect_success PERL 'difftool.prompt config variable is false' '
 '
 
 # Test that we don't have to pass --no-prompt when mergetool.prompt is false
-test_expect_success PERL 'difftool merge.prompt = false' '
+test_expect_success 'difftool merge.prompt = false' '
 	difftool_test_setup &&
 	test_might_fail git config --unset difftool.prompt &&
 	test_config mergetool.prompt false &&
@@ -231,7 +229,7 @@ test_expect_success PERL 'difftool merge.prompt = false' '
 '
 
 # Test that the -y flag can override difftool.prompt = true
-test_expect_success PERL 'difftool.prompt can overridden with -y' '
+test_expect_success 'difftool.prompt can overridden with -y' '
 	difftool_test_setup &&
 	test_config difftool.prompt true &&
 	echo branch >expect &&
@@ -240,7 +238,7 @@ test_expect_success PERL 'difftool.prompt can overridden with -y' '
 '
 
 # Test that the --prompt flag can override difftool.prompt = false
-test_expect_success PERL 'difftool.prompt can overridden with --prompt' '
+test_expect_success 'difftool.prompt can overridden with --prompt' '
 	difftool_test_setup &&
 	test_config difftool.prompt false &&
 	echo >input &&
@@ -250,7 +248,7 @@ test_expect_success PERL 'difftool.prompt can overridden with --prompt' '
 '
 
 # Test that the last flag passed on the command-line wins
-test_expect_success PERL 'difftool last flag wins' '
+test_expect_success 'difftool last flag wins' '
 	difftool_test_setup &&
 	echo branch >expect &&
 	git difftool --prompt --no-prompt branch >actual &&
@@ -263,7 +261,7 @@ test_expect_success PERL 'difftool last flag wins' '
 
 # git-difftool falls back to git-mergetool config variables
 # so test that behavior here
-test_expect_success PERL 'difftool + mergetool config variables' '
+test_expect_success 'difftool + mergetool config variables' '
 	test_config merge.tool test-tool &&
 	test_config mergetool.test-tool.cmd "cat \$LOCAL" &&
 	echo branch >expect &&
@@ -277,49 +275,49 @@ test_expect_success PERL 'difftool + mergetool config variables' '
 	test_cmp expect actual
 '
 
-test_expect_success PERL 'difftool.<tool>.path' '
+test_expect_success 'difftool.<tool>.path' '
 	test_config difftool.tkdiff.path echo &&
 	git difftool --tool=tkdiff --no-prompt branch >output &&
 	lines=$(grep file output | wc -l) &&
 	test "$lines" -eq 1
 '
 
-test_expect_success PERL 'difftool --extcmd=cat' '
+test_expect_success 'difftool --extcmd=cat' '
 	echo branch >expect &&
 	echo master >>expect &&
 	git difftool --no-prompt --extcmd=cat branch >actual &&
 	test_cmp expect actual
 '
 
-test_expect_success PERL 'difftool --extcmd cat' '
+test_expect_success 'difftool --extcmd cat' '
 	echo branch >expect &&
 	echo master >>expect &&
 	git difftool --no-prompt --extcmd=cat branch >actual &&
 	test_cmp expect actual
 '
 
-test_expect_success PERL 'difftool -x cat' '
+test_expect_success 'difftool -x cat' '
 	echo branch >expect &&
 	echo master >>expect &&
 	git difftool --no-prompt -x cat branch >actual &&
 	test_cmp expect actual
 '
 
-test_expect_success PERL 'difftool --extcmd echo arg1' '
+test_expect_success 'difftool --extcmd echo arg1' '
 	echo file >expect &&
 	git difftool --no-prompt \
 		--extcmd sh\ -c\ \"echo\ \$1\" branch >actual &&
 	test_cmp expect actual
 '
 
-test_expect_success PERL 'difftool --extcmd cat arg1' '
+test_expect_success 'difftool --extcmd cat arg1' '
 	echo master >expect &&
 	git difftool --no-prompt \
 		--extcmd sh\ -c\ \"cat\ \$1\" branch >actual &&
 	test_cmp expect actual
 '
 
-test_expect_success PERL 'difftool --extcmd cat arg2' '
+test_expect_success 'difftool --extcmd cat arg2' '
 	echo branch >expect &&
 	git difftool --no-prompt \
 		--extcmd sh\ -c\ \"cat\ \$2\" branch >actual &&
@@ -327,7 +325,7 @@ test_expect_success PERL 'difftool --extcmd cat arg2' '
 '
 
 # Create a second file on master and a different version on branch
-test_expect_success PERL 'setup with 2 files different' '
+test_expect_success 'setup with 2 files different' '
 	echo m2 >file2 &&
 	git add file2 &&
 	git commit -m "added file2" &&
@@ -339,7 +337,7 @@ test_expect_success PERL 'setup with 2 files different' '
 	git checkout master
 '
 
-test_expect_success PERL 'say no to the first file' '
+test_expect_success 'say no to the first file' '
 	(echo n && echo) >input &&
 	git difftool -x cat branch <input >output &&
 	grep m2 output &&
@@ -348,7 +346,7 @@ test_expect_success PERL 'say no to the first file' '
 	! grep branch output
 '
 
-test_expect_success PERL 'say no to the second file' '
+test_expect_success 'say no to the second file' '
 	(echo && echo n) >input &&
 	git difftool -x cat branch <input >output &&
 	grep master output &&
@@ -357,7 +355,7 @@ test_expect_success PERL 'say no to the second file' '
 	! grep br2 output
 '
 
-test_expect_success PERL 'ending prompt input with EOF' '
+test_expect_success 'ending prompt input with EOF' '
 	git difftool -x cat branch </dev/null >output &&
 	! grep master output &&
 	! grep branch output &&
@@ -365,12 +363,12 @@ test_expect_success PERL 'ending prompt input with EOF' '
 	! grep br2 output
 '
 
-test_expect_success PERL 'difftool --tool-help' '
+test_expect_success 'difftool --tool-help' '
 	git difftool --tool-help >output &&
 	grep tool output
 '
 
-test_expect_success PERL 'setup change in subdirectory' '
+test_expect_success 'setup change in subdirectory' '
 	git checkout master &&
 	mkdir sub &&
 	echo master >sub/sub &&
@@ -384,11 +382,11 @@ test_expect_success PERL 'setup change in subdirectory' '
 '
 
 run_dir_diff_test () {
-	test_expect_success PERL "$1 --no-symlinks" "
+	test_expect_success "$1 --no-symlinks" "
 		symlinks=--no-symlinks &&
 		$2
 	"
-	test_expect_success PERL,SYMLINKS "$1 --symlinks" "
+	test_expect_success SYMLINKS "$1 --symlinks" "
 		symlinks=--symlinks &&
 		$2
 	"
@@ -510,7 +508,7 @@ do
 done >actual
 EOF
 
-test_expect_success PERL,SYMLINKS 'difftool --dir-diff --symlink without unstaged changes' '
+test_expect_success SYMLINKS 'difftool --dir-diff --symlink without unstaged changes' '
 	cat >expect <<-EOF &&
 	file
 	$PWD/file
@@ -547,7 +545,7 @@ write_script modify-file <<\EOF
 echo "new content" >file
 EOF
 
-test_expect_success PERL 'difftool --no-symlinks does not overwrite working tree file ' '
+test_expect_success 'difftool --no-symlinks does not overwrite working tree file ' '
 	echo "orig content" >file &&
 	git difftool --dir-diff --no-symlinks --extcmd "$PWD/modify-file" branch &&
 	echo "new content" >expect &&
@@ -560,7 +558,7 @@ echo "tmp content" >"$2/file" &&
 echo "$2" >tmpdir
 EOF
 
-test_expect_success PERL 'difftool --no-symlinks detects conflict ' '
+test_expect_success 'difftool --no-symlinks detects conflict ' '
 	(
 		TMPDIR=$TRASH_DIRECTORY &&
 		export TMPDIR &&
@@ -573,7 +571,7 @@ test_expect_success PERL 'difftool --no-symlinks detects conflict ' '
 	)
 '
 
-test_expect_success PERL 'difftool properly honors gitlink and core.worktree' '
+test_expect_success 'difftool properly honors gitlink and core.worktree' '
 	git submodule add ./. submod/ule &&
 	test_config -C submod/ule diff.tool checktrees &&
 	test_config -C submod/ule difftool.checktrees.cmd '\''
@@ -587,7 +585,7 @@ test_expect_success PERL 'difftool properly honors gitlink and core.worktree' '
 	)
 '
 
-test_expect_success PERL,SYMLINKS 'difftool --dir-diff symlinked directories' '
+test_expect_success SYMLINKS 'difftool --dir-diff symlinked directories' '
 	git init dirlinks &&
 	(
 		cd dirlinks &&
-- 
2.11.0.windows.3
Keyboard shortcuts
hback out one level
jnext message in thread
kprevious message in thread
ldrill in
Escclose help / fold thread tree
?toggle this help