[PATCH 0/2] Do not lock temporary files via child processes on Windows

This issue was originally reported and fixed in
https://github.com/git-for-windows/git/pull/755

The problem is that file handles to temporary files (such as
index.lock) were inherited by spawned processes. If those spawned
processes do not exit before the parent process wants to delete or
rename them, we are in big trouble.

The original use case triggering the bug is a merge driver that does
not quit, but listen to subsequent merge requests.

However, the same issue turned up in Lars Schneider's work on making
clean/smudge filters batchable (i.e. more efficient by avoiding
possibly thousands of child processes, one per file).


Ben Wijen (2):
  t6026-merge-attr: child processes must not inherit index.lock handles
  mingw: ensure temporary file handles are not inherited by child
    processes

 compat/mingw.h        |  4 ++++
 t/t6026-merge-attr.sh | 13 +++++++++++++
 tempfile.c            |  2 +-
 3 files changed, 18 insertions(+), 1 deletion(-)

Published-As: https://github.com/dscho/git/releases/tag/mingw-index-lock-v1
Fetch-It-Via: git fetch https://github.com/dscho/git mingw-index-lock-v1

-- 
2.9.2.691.g78954f3

base-commit: 07c92928f2b782330df6e78dd9d019e984d820a7

From: Ben Wijen <redacted>

On Windows, files cannot be removed unless all file handles to it have
been released. Hence it is particularly important to close handles when
spawning children (which would probably not even know that they hold on
to those handles).

The example chosen for this test is a custom merge driver that indeed
has no idea that it blocks the deletion of index.lock. The full use case
is a daemon that lives on after the merge, with subsequent invocations
handing off to the daemon, thereby avoiding hefty start-up costs. We
simulate this behavior by simply sleeping one second.

Note that the test only fails on Windows, due to the file locking issue.
Since we have no way to say "expect failure with MINGW, success
otherwise", we simply skip this test on Windows for now.

Signed-off-by: Ben Wijen <redacted>
Signed-off-by: Johannes Schindelin <redacted>
---
 t/t6026-merge-attr.sh | 13 +++++++++++++
 1 file changed, 13 insertions(+)

diff --git a/t/t6026-merge-attr.sh b/t/t6026-merge-attr.sh
index ef0cbce..3d28c78 100755
--- a/t/t6026-merge-attr.sh
+++ b/t/t6026-merge-attr.sh

@@ -181,4 +181,17 @@ test_expect_success 'up-to-date merge without common ancestor' '
 	)
 '
 
+test_expect_success !MINGW 'custom merge does not lock index' '
+	git reset --hard anchor &&
+	write_script sleep-one-second.sh <<-\EOF &&
+		sleep 1 &
+	EOF
+
+	test_write_lines >.gitattributes \
+		"* merge=ours" "text merge=sleep-one-second" &&
+	test_config merge.ours.driver true &&
+	test_config merge.sleep-one-second.driver ./sleep-one-second.sh &&
+	git merge master
+'
+
 test_done

-- 
2.9.2.691.g78954f3

From: Ben Wijen <redacted>

When the index is locked and child processes inherit the handle to
said lock and the parent process wants to remove the lock before the
child process exits, on Windows there is a problem: it won't work
because files cannot be deleted if a process holds a handle on them.
The symptom:

    Rename from 'xxx/.git/index.lock' to 'xxx/.git/index' failed.
    Should I try again? (y/n)

Spawning child processes with bInheritHandles==FALSE would not work
because no file handles would be inherited, not even the hStdXxx
handles in STARTUPINFO (stdin/stdout/stderr).

Opening every file with O_NOINHERIT does not work, either, as e.g.
git-upload-pack expects inherited file handles.

This leaves us with the only way out: creating temp files with the
O_NOINHERIT flag. This flag is Windows-specific, however. For our
purposes, it is equivalent our purposes) to O_CLOEXEC (which does not
exist on Windows), so let's just open temporary files with the
O_CLOEXEC flag and map that flag to O_NOINHERIT on Windows.

This fixes the test that we just introduced to demonstrate the problem.

Signed-off-by: Ben Wijen <redacted>
Signed-off-by: Johannes Schindelin <redacted>
---
 compat/mingw.h        | 4 ++++
 t/t6026-merge-attr.sh | 2 +-
 tempfile.c            | 2 +-
 3 files changed, 6 insertions(+), 2 deletions(-)

diff --git a/compat/mingw.h b/compat/mingw.h
index 95e128f..753e641 100644
--- a/compat/mingw.h
+++ b/compat/mingw.h

@@ -67,6 +67,10 @@ typedef int pid_t;
 #define F_SETFD 2
 #define FD_CLOEXEC 0x1
 
+#if !defined O_CLOEXEC && defined O_NOINHERIT
+#define O_CLOEXEC	O_NOINHERIT
+#endif
+
 #ifndef EAFNOSUPPORT
 #define EAFNOSUPPORT WSAEAFNOSUPPORT
 #endif

diff --git a/t/t6026-merge-attr.sh b/t/t6026-merge-attr.sh
index 3d28c78..dd8f88d 100755
--- a/t/t6026-merge-attr.sh
+++ b/t/t6026-merge-attr.sh

@@ -181,7 +181,7 @@ test_expect_success 'up-to-date merge without common ancestor' '
 	)
 '
 
-test_expect_success !MINGW 'custom merge does not lock index' '
+test_expect_success 'custom merge does not lock index' '
 	git reset --hard anchor &&
 	write_script sleep-one-second.sh <<-\EOF &&
 		sleep 1 &

diff --git a/tempfile.c b/tempfile.c
index 0af7ebf..db3981d 100644
--- a/tempfile.c
+++ b/tempfile.c

@@ -120,7 +120,7 @@ int create_tempfile(struct tempfile *tempfile, const char *path)
 	prepare_tempfile_object(tempfile);
 
 	strbuf_add_absolute_path(&tempfile->filename, path);
-	tempfile->fd = open(tempfile->filename.buf, O_RDWR | O_CREAT | O_EXCL, 0666);
+	tempfile->fd = open(tempfile->filename.buf, O_RDWR | O_CREAT | O_EXCL | O_CLOEXEC, 0666);
 	if (tempfile->fd < 0) {
 		strbuf_reset(&tempfile->filename);
 		return -1;

-- 
2.9.2.691.g78954f3

On Wed, Aug 17, 2016 at 8:41 AM, Johannes Schindelin
[off-list ref] wrote:

When the index is locked and child processes inherit the handle to
said lock and the parent process wants to remove the lock before the
child process exits, on Windows there is a problem: it won't work
because files cannot be deleted if a process holds a handle on them.
The symptom:

    Rename from 'xxx/.git/index.lock' to 'xxx/.git/index' failed.
    Should I try again? (y/n)

Spawning child processes with bInheritHandles==FALSE would not work
because no file handles would be inherited, not even the hStdXxx
handles in STARTUPINFO (stdin/stdout/stderr).

Opening every file with O_NOINHERIT does not work, either, as e.g.
git-upload-pack expects inherited file handles.

This leaves us with the only way out: creating temp files with the
O_NOINHERIT flag. This flag is Windows-specific, however. For our
purposes, it is equivalent our purposes) to O_CLOEXEC (which does not

s/our purposes)//

exist on Windows), so let's just open temporary files with the
O_CLOEXEC flag and map that flag to O_NOINHERIT on Windows.

This fixes the test that we just introduced to demonstrate the problem.

Signed-off-by: Ben Wijen <redacted>
Signed-off-by: Johannes Schindelin <redacted>

This solution works great for me. I struggled with a similar problem 
in my Git filter protocol series:
http://public-inbox.org/git/20160810130411.12419-16-larsxschneider@gmail.com/

I also tried selectively disabling inheritance for file handles but it looks like
as this is not easily possible right now: 
https://github.com/git-for-windows/git/issues/770#issuecomment-238816331

- Lars

Johannes Schindelin [off-list ref] wrote:

O_CLOEXEC only exists since Linux 2.6.23 and there are likely
still LTS (CentOS 5.x?) and non-Linux systems which do not have
it, as well as machines with could have it defined in userspace
headers but not have it in the kernel.

So I suggest something like the following: (untested)

#define GIT_O_TMP (O_RDWR | O_CREAT | O_EXCL)

#ifndef O_CLOEXEC
#  define O_CLOEXEC 0
#endif
	/* state: -1=unknown; 0=broken; 1=working */
	static int cloexec_state = O_CLOEXEC == 0 ? 0 : -1;
	static int GIT_O_ETMP = (GIT_O_TMP | O_CLOEXEC)

	int fd = open(filename, GIT_O_ETMP, 0666);

	if (fd < 0 && errno == EINVAL && cloexec_state == -1 &&
			GIT_O_ETMP != GIT_O_TMP) {
		GIT_O_ETMP = GIT_O_TMP;

		fd = open(filename, GIT_O_ETMP, 0666);
		if (fd >= 0)
			/* don't try O_CLOEXEC again */
			cloexec_state = 0;
	}

	/*
	 * This is racy in the presence of threads,
	 * but the best we can do for old *nix:
	 */
#if defined(F_GETFD) && defined(F_SETFD) && defined(FD_CLOEXEC)
	if (fd >= 0 && cloexec_state != 1) {
		int flags = fcntl(fd, F_GETFD);

		if (flags == -1)
			die_errno("F_GETFD failed");
		if (flags & O_CLOEXEC)
			cloexec_state = 1;
		else {
			flags = fcntl(fd, F_SETFD, flags | FD_CLOEXEC);
			if (flags == -1)
				die_errno("F_SETFD failed");
			cloexec_state = 0;

		}
	}
#endif
	...

This issue was originally reported and fixed in
https://github.com/git-for-windows/git/pull/755

The problem is that file handles to temporary files (such as
index.lock) were inherited by spawned processes. If those spawned
processes do not exit before the parent process wants to delete or
rename them, we are in big trouble.

The original use case triggering the bug is a merge driver that does
not quit, but listen to subsequent merge requests.

However, the same issue turned up in Lars Schneider's work on making
clean/smudge filters batchable (i.e. more efficient by avoiding
possibly thousands of child processes, one per file).

Changes since v1:

- the two commit messages have been corrected, as per Junio's and Eric's
  suggestion,
- lockfile.h and tempfile.h now sport explicit documentation that the
  current process needs to write to the files, no spawned processes.


Ben Wijen (2):
  t6026-merge-attr: child processes must not inherit index.lock handles
  mingw: ensure temporary file handles are not inherited by child
    processes

 compat/mingw.h        |  4 ++++
 lockfile.h            |  4 ++++
 t/t6026-merge-attr.sh | 13 +++++++++++++
 tempfile.c            |  2 +-
 tempfile.h            |  4 ++++
 5 files changed, 26 insertions(+), 1 deletion(-)

Published-As: https://github.com/dscho/git/releases/tag/mingw-index-lock-v2
Fetch-It-Via: git fetch https://github.com/dscho/git mingw-index-lock-v2

Interdiff vs v1:

 diff --git a/lockfile.h b/lockfile.h
 index 3d30193..d26ad27 100644
 --- a/lockfile.h
 +++ b/lockfile.h
 @@ -55,6 +55,10 @@
   *   * calling `fdopen_lock_file()` to get a `FILE` pointer for the
   *     open file and writing to the file using stdio.
   *
 + *   Note that the file descriptor returned by hold_lock_file_for_update()
 + *   is marked O_CLOEXEC, so the new contents must be written by the
 + *   current process, not a spawned one.
 + *
   * When finished writing, the caller can:
   *
   * * Close the file descriptor and rename the lockfile to its final
 diff --git a/tempfile.h b/tempfile.h
 index 4219fe4..2f0038d 100644
 --- a/tempfile.h
 +++ b/tempfile.h
 @@ -33,6 +33,10 @@
   *   * calling `fdopen_tempfile()` to get a `FILE` pointer for the
   *     open file and writing to the file using stdio.
   *
 + *   Note that the file descriptor returned by create_tempfile()
 + *   is marked O_CLOEXEC, so the new contents must be written by
 + *   the current process, not any spawned one.
 + *
   * When finished writing, the caller can:
   *
   * * Close the file descriptor and remove the temporary file by

-- 
2.9.2.691.g78954f3

base-commit: d63263a4dee8fc7da9b97bbdedf9c0d1f33024d4

Hi Eric,

On Wed, 17 Aug 2016, Eric Sunshine wrote:

On Wed, Aug 17, 2016 at 8:41 AM, Johannes Schindelin
[off-list ref] wrote:

quoted

When the index is locked and child processes inherit the handle to
said lock and the parent process wants to remove the lock before the
child process exits, on Windows there is a problem: it won't work
because files cannot be deleted if a process holds a handle on them.
The symptom:

    Rename from 'xxx/.git/index.lock' to 'xxx/.git/index' failed.
    Should I try again? (y/n)

Spawning child processes with bInheritHandles==FALSE would not work
because no file handles would be inherited, not even the hStdXxx
handles in STARTUPINFO (stdin/stdout/stderr).

Opening every file with O_NOINHERIT does not work, either, as e.g.
git-upload-pack expects inherited file handles.

This leaves us with the only way out: creating temp files with the
O_NOINHERIT flag. This flag is Windows-specific, however. For our
purposes, it is equivalent our purposes) to O_CLOEXEC (which does not

s/our purposes)//

Thanks!
Dscho

From: Ben Wijen <redacted>

When the index is locked and child processes inherit the handle to
said lock and the parent process wants to remove the lock before the
child process exits, on Windows there is a problem: it won't work
because files cannot be deleted if a process holds a handle on them.
The symptom:

    Rename from 'xxx/.git/index.lock' to 'xxx/.git/index' failed.
    Should I try again? (y/n)

Spawning child processes with bInheritHandles==FALSE would not work
because no file handles would be inherited, not even the hStdXxx
handles in STARTUPINFO (stdin/stdout/stderr).

Opening every file with O_NOINHERIT does not work, either, as e.g.
git-upload-pack expects inherited file handles.

This leaves us with the only way out: creating temp files with the
O_NOINHERIT flag. This flag is Windows-specific, however. For our
purposes, it is equivalent to O_CLOEXEC (which does not exist on
Windows), so let's just open temporary files with the O_CLOEXEC flag and
map that flag to O_NOINHERIT on Windows.

This fixes the test that we just introduced to demonstrate the problem.

Signed-off-by: Ben Wijen <redacted>
Signed-off-by: Johannes Schindelin <redacted>
---
 compat/mingw.h        | 4 ++++
 lockfile.h            | 4 ++++
 t/t6026-merge-attr.sh | 2 +-
 tempfile.c            | 2 +-
 tempfile.h            | 4 ++++
 5 files changed, 14 insertions(+), 2 deletions(-)

diff --git a/compat/mingw.h b/compat/mingw.h
index 95e128f..753e641 100644
--- a/compat/mingw.h
+++ b/compat/mingw.h

@@ -67,6 +67,10 @@ typedef int pid_t;
 #define F_SETFD 2
 #define FD_CLOEXEC 0x1
 
+#if !defined O_CLOEXEC && defined O_NOINHERIT
+#define O_CLOEXEC	O_NOINHERIT
+#endif
+
 #ifndef EAFNOSUPPORT
 #define EAFNOSUPPORT WSAEAFNOSUPPORT
 #endif

diff --git a/lockfile.h b/lockfile.h
index 3d30193..d26ad27 100644
--- a/lockfile.h
+++ b/lockfile.h

@@ -55,6 +55,10 @@
  *   * calling `fdopen_lock_file()` to get a `FILE` pointer for the
  *     open file and writing to the file using stdio.
  *
+ *   Note that the file descriptor returned by hold_lock_file_for_update()
+ *   is marked O_CLOEXEC, so the new contents must be written by the
+ *   current process, not a spawned one.
+ *
  * When finished writing, the caller can:
  *
  * * Close the file descriptor and rename the lockfile to its final

diff --git a/t/t6026-merge-attr.sh b/t/t6026-merge-attr.sh
index 3d28c78..dd8f88d 100755
--- a/t/t6026-merge-attr.sh
+++ b/t/t6026-merge-attr.sh

@@ -181,7 +181,7 @@ test_expect_success 'up-to-date merge without common ancestor' '
 	)
 '
 
-test_expect_success !MINGW 'custom merge does not lock index' '
+test_expect_success 'custom merge does not lock index' '
 	git reset --hard anchor &&
 	write_script sleep-one-second.sh <<-\EOF &&
 		sleep 1 &

diff --git a/tempfile.c b/tempfile.c
index 0af7ebf..db3981d 100644
--- a/tempfile.c
+++ b/tempfile.c

@@ -120,7 +120,7 @@ int create_tempfile(struct tempfile *tempfile, const char *path)
 	prepare_tempfile_object(tempfile);
 
 	strbuf_add_absolute_path(&tempfile->filename, path);
-	tempfile->fd = open(tempfile->filename.buf, O_RDWR | O_CREAT | O_EXCL, 0666);
+	tempfile->fd = open(tempfile->filename.buf, O_RDWR | O_CREAT | O_EXCL | O_CLOEXEC, 0666);
 	if (tempfile->fd < 0) {
 		strbuf_reset(&tempfile->filename);
 		return -1;

diff --git a/tempfile.h b/tempfile.h
index 4219fe4..2f0038d 100644
--- a/tempfile.h
+++ b/tempfile.h

@@ -33,6 +33,10 @@
  *   * calling `fdopen_tempfile()` to get a `FILE` pointer for the
  *     open file and writing to the file using stdio.
  *
+ *   Note that the file descriptor returned by create_tempfile()
+ *   is marked O_CLOEXEC, so the new contents must be written by
+ *   the current process, not any spawned one.
+ *
  * When finished writing, the caller can:
  *
  * * Close the file descriptor and remove the temporary file by

-- 
2.9.2.691.g78954f3

From: Ben Wijen <redacted>

On Windows, a file cannot be removed unless all file handles to it have
been released. Hence it is particularly important to close handles when
spawning children (which would probably not even know that they hold on
to those handles).

The example chosen for this test is a custom merge driver that indeed
has no idea that it blocks the deletion of index.lock. The full use case
is a daemon that lives on after the merge, with subsequent invocations
handing off to the daemon, thereby avoiding hefty start-up costs. We
simulate this behavior by simply sleeping one second.

Note that the test only fails on Windows, due to the file locking issue.
Since we have no way to say "expect failure with MINGW, success
otherwise", we simply skip this test on Windows for now.

Signed-off-by: Ben Wijen <redacted>
Signed-off-by: Johannes Schindelin <redacted>
---
 t/t6026-merge-attr.sh | 13 +++++++++++++
 1 file changed, 13 insertions(+)

diff --git a/t/t6026-merge-attr.sh b/t/t6026-merge-attr.sh
index ef0cbce..3d28c78 100755
--- a/t/t6026-merge-attr.sh
+++ b/t/t6026-merge-attr.sh

@@ -181,4 +181,17 @@ test_expect_success 'up-to-date merge without common ancestor' '
 	)
 '
 
+test_expect_success !MINGW 'custom merge does not lock index' '
+	git reset --hard anchor &&
+	write_script sleep-one-second.sh <<-\EOF &&
+		sleep 1 &
+	EOF
+
+	test_write_lines >.gitattributes \
+		"* merge=ours" "text merge=sleep-one-second" &&
+	test_config merge.ours.driver true &&
+	test_config merge.sleep-one-second.driver ./sleep-one-second.sh &&
+	git merge master
+'
+
 test_done

-- 
2.9.2.691.g78954f3

This issue was originally reported and fixed in
https://github.com/git-for-windows/git/pull/755

The problem is that file handles to temporary files (such as
index.lock) were inherited by spawned processes. If those spawned
processes do not exit before the parent process wants to delete or
rename them, we are in big trouble.

The original use case triggering the bug is a merge driver that does
not quit, but listen to subsequent merge requests.

However, the same issue turned up in Lars Schneider's work on making
clean/smudge filters batchable (i.e. more efficient by avoiding
possibly thousands of child processes, one per file).

Changes since v2:

- O_CLOEXEC is defined in git-compat-util.h unless already defined
- we now handle EINVAL by trying again without O_CLOEXEC


Ben Wijen (2):
  t6026-merge-attr: child processes must not inherit index.lock handles
  mingw: ensure temporary file handles are not inherited by child
    processes

 compat/mingw.h        |  4 ++++
 git-compat-util.h     |  4 ++++
 lockfile.h            |  4 ++++
 t/t6026-merge-attr.sh | 13 +++++++++++++
 tempfile.c            |  7 ++++++-
 tempfile.h            |  4 ++++
 6 files changed, 35 insertions(+), 1 deletion(-)

Published-As: https://github.com/dscho/git/releases/tag/mingw-index-lock-v3
Fetch-It-Via: git fetch https://github.com/dscho/git mingw-index-lock-v3

Interdiff vs v2:

 diff --git a/git-compat-util.h b/git-compat-util.h
 index f52e00b..db89ba7 100644
 --- a/git-compat-util.h
 +++ b/git-compat-util.h
 @@ -667,6 +667,10 @@ void *gitmemmem(const void *haystack, size_t haystacklen,
  #define getpagesize() sysconf(_SC_PAGESIZE)
  #endif
  
 +#ifndef O_CLOEXEC
 +#define O_CLOEXEC 0
 +#endif
 +
  #ifdef FREAD_READS_DIRECTORIES
  #ifdef fopen
  #undef fopen
 diff --git a/tempfile.c b/tempfile.c
 index db3981d..2990c92 100644
 --- a/tempfile.c
 +++ b/tempfile.c
 @@ -120,7 +120,12 @@ int create_tempfile(struct tempfile *tempfile, const char *path)
  	prepare_tempfile_object(tempfile);
  
  	strbuf_add_absolute_path(&tempfile->filename, path);
 -	tempfile->fd = open(tempfile->filename.buf, O_RDWR | O_CREAT | O_EXCL | O_CLOEXEC, 0666);
 +	tempfile->fd = open(tempfile->filename.buf,
 +			    O_RDWR | O_CREAT | O_EXCL | O_CLOEXEC, 0666);
 +	if (O_CLOEXEC && tempfile->fd < 0 && errno == EINVAL)
 +		/* Try again w/o O_CLOEXEC: the kernel might not support it */
 +		tempfile->fd = open(tempfile->filename.buf,
 +				    O_RDWR | O_CREAT | O_EXCL, 0666);
  	if (tempfile->fd < 0) {
  		strbuf_reset(&tempfile->filename);
  		return -1;

-- 
2.10.0.rc0.115.ged054c0

base-commit: 2632c897f74b1cc9b5533f467da459b9ec725538

From: Ben Wijen <redacted>

On Windows, a file cannot be removed unless all file handles to it have
been released. Hence it is particularly important to close handles when
spawning children (which would probably not even know that they hold on
to those handles).

The example chosen for this test is a custom merge driver that indeed
has no idea that it blocks the deletion of index.lock. The full use case
is a daemon that lives on after the merge, with subsequent invocations
handing off to the daemon, thereby avoiding hefty start-up costs. We
simulate this behavior by simply sleeping one second.

Note that the test only fails on Windows, due to the file locking issue.
Since we have no way to say "expect failure with MINGW, success
otherwise", we simply skip this test on Windows for now.

Signed-off-by: Ben Wijen <redacted>
Signed-off-by: Johannes Schindelin <redacted>
---
 t/t6026-merge-attr.sh | 13 +++++++++++++
 1 file changed, 13 insertions(+)

diff --git a/t/t6026-merge-attr.sh b/t/t6026-merge-attr.sh
index ef0cbce..3d28c78 100755
--- a/t/t6026-merge-attr.sh
+++ b/t/t6026-merge-attr.sh

@@ -181,4 +181,17 @@ test_expect_success 'up-to-date merge without common ancestor' '
 	)
 '
 
+test_expect_success !MINGW 'custom merge does not lock index' '
+	git reset --hard anchor &&
+	write_script sleep-one-second.sh <<-\EOF &&
+		sleep 1 &
+	EOF
+
+	test_write_lines >.gitattributes \
+		"* merge=ours" "text merge=sleep-one-second" &&
+	test_config merge.ours.driver true &&
+	test_config merge.sleep-one-second.driver ./sleep-one-second.sh &&
+	git merge master
+'
+
 test_done

-- 
2.10.0.rc0.115.ged054c0

From: Ben Wijen <redacted>

When the index is locked and child processes inherit the handle to
said lock and the parent process wants to remove the lock before the
child process exits, on Windows there is a problem: it won't work
because files cannot be deleted if a process holds a handle on them.
The symptom:

    Rename from 'xxx/.git/index.lock' to 'xxx/.git/index' failed.
    Should I try again? (y/n)

Spawning child processes with bInheritHandles==FALSE would not work
because no file handles would be inherited, not even the hStdXxx
handles in STARTUPINFO (stdin/stdout/stderr).

Opening every file with O_NOINHERIT does not work, either, as e.g.
git-upload-pack expects inherited file handles.

This leaves us with the only way out: creating temp files with the
O_NOINHERIT flag. This flag is Windows-specific, however. For our
purposes, it is equivalent to O_CLOEXEC (which does not exist on
Windows), so let's just open temporary files with the O_CLOEXEC flag and
map that flag to O_NOINHERIT on Windows.

As Eric Wong pointed out, we need to be careful to handle the case where
the Linux headers used to compile Git support O_CLOEXEC but the Linux
kernel used to run Git does not: it returns an EINVAL.

This fixes the test that we just introduced to demonstrate the problem.

Signed-off-by: Ben Wijen <redacted>
Signed-off-by: Johannes Schindelin <redacted>
---
 compat/mingw.h        | 4 ++++
 git-compat-util.h     | 4 ++++
 lockfile.h            | 4 ++++
 t/t6026-merge-attr.sh | 2 +-
 tempfile.c            | 7 ++++++-
 tempfile.h            | 4 ++++
 6 files changed, 23 insertions(+), 2 deletions(-)

diff --git a/compat/mingw.h b/compat/mingw.h
index 95e128f..753e641 100644
--- a/compat/mingw.h
+++ b/compat/mingw.h

@@ -67,6 +67,10 @@ typedef int pid_t;
 #define F_SETFD 2
 #define FD_CLOEXEC 0x1
 
+#if !defined O_CLOEXEC && defined O_NOINHERIT
+#define O_CLOEXEC	O_NOINHERIT
+#endif
+
 #ifndef EAFNOSUPPORT
 #define EAFNOSUPPORT WSAEAFNOSUPPORT
 #endif

diff --git a/git-compat-util.h b/git-compat-util.h
index f52e00b..db89ba7 100644
--- a/git-compat-util.h
+++ b/git-compat-util.h

@@ -667,6 +667,10 @@ void *gitmemmem(const void *haystack, size_t haystacklen,
 #define getpagesize() sysconf(_SC_PAGESIZE)
 #endif
 
+#ifndef O_CLOEXEC
+#define O_CLOEXEC 0
+#endif
+
 #ifdef FREAD_READS_DIRECTORIES
 #ifdef fopen
 #undef fopen

diff --git a/lockfile.h b/lockfile.h
index 3d30193..d26ad27 100644
--- a/lockfile.h
+++ b/lockfile.h

@@ -55,6 +55,10 @@
  *   * calling `fdopen_lock_file()` to get a `FILE` pointer for the
  *     open file and writing to the file using stdio.
  *
+ *   Note that the file descriptor returned by hold_lock_file_for_update()
+ *   is marked O_CLOEXEC, so the new contents must be written by the
+ *   current process, not a spawned one.
+ *
  * When finished writing, the caller can:
  *
  * * Close the file descriptor and rename the lockfile to its final

diff --git a/t/t6026-merge-attr.sh b/t/t6026-merge-attr.sh
index 3d28c78..dd8f88d 100755
--- a/t/t6026-merge-attr.sh
+++ b/t/t6026-merge-attr.sh

@@ -181,7 +181,7 @@ test_expect_success 'up-to-date merge without common ancestor' '
 	)
 '
 
-test_expect_success !MINGW 'custom merge does not lock index' '
+test_expect_success 'custom merge does not lock index' '
 	git reset --hard anchor &&
 	write_script sleep-one-second.sh <<-\EOF &&
 		sleep 1 &

diff --git a/tempfile.c b/tempfile.c
index 0af7ebf..2990c92 100644
--- a/tempfile.c
+++ b/tempfile.c

@@ -120,7 +120,12 @@ int create_tempfile(struct tempfile *tempfile, const char *path)
 	prepare_tempfile_object(tempfile);
 
 	strbuf_add_absolute_path(&tempfile->filename, path);
-	tempfile->fd = open(tempfile->filename.buf, O_RDWR | O_CREAT | O_EXCL, 0666);
+	tempfile->fd = open(tempfile->filename.buf,
+			    O_RDWR | O_CREAT | O_EXCL | O_CLOEXEC, 0666);
+	if (O_CLOEXEC && tempfile->fd < 0 && errno == EINVAL)
+		/* Try again w/o O_CLOEXEC: the kernel might not support it */
+		tempfile->fd = open(tempfile->filename.buf,
+				    O_RDWR | O_CREAT | O_EXCL, 0666);
 	if (tempfile->fd < 0) {
 		strbuf_reset(&tempfile->filename);
 		return -1;

diff --git a/tempfile.h b/tempfile.h
index 4219fe4..2f0038d 100644
--- a/tempfile.h
+++ b/tempfile.h

@@ -33,6 +33,10 @@
  *   * calling `fdopen_tempfile()` to get a `FILE` pointer for the
  *     open file and writing to the file using stdio.
  *
+ *   Note that the file descriptor returned by create_tempfile()
+ *   is marked O_CLOEXEC, so the new contents must be written by
+ *   the current process, not any spawned one.
+ *
  * When finished writing, the caller can:
  *
  * * Close the file descriptor and remove the temporary file by

-- 
2.10.0.rc0.115.ged054c0

Johannes Schindelin [off-list ref] wrote:

As Eric Wong pointed out, we need to be careful to handle the case where
the Linux headers used to compile Git support O_CLOEXEC but the Linux
kernel used to run Git does not: it returns an EINVAL.

Looks good to me from a Linux standpoint.  Thanks.