[Junio C Hamano] Re: Consolidate SHA1 object file close

Somehow the thread went off-list, so I'm diverting it back to the list...

On Sat, 14 Jun 2008, Junio C Hamano wrote:

Other than that I had to stop for a few seconds to think at the magic 
"20 bytes longer", they look scary-but-correct ;-).

Actually, thinking about it, the +20 was not just fairly arbitrary and 
would have been better off with a comment, it was also not a very good 
number to begin with - it just was the old tempfile pattern rounded up to 
the next "even" number.

The _correct_ number to use is +39, with a better tempfile pattern.

That needs a few comments, though. So I added those too.

This patch is totally unimportant, but it does mean that a really 
traditional filesystem can now do the final rename in-place, because the 
temporary file is not just in the same directory as the final one, but ti 
also has the same length, so old-style filesystems can literally just edit 
the name in place and mark the buffer dirty.

And more modern filesystems will generally end up doing more complex 
things regardless of what we do, but at least this won't _hurt_ either.

There is one more funny (or sad) detail to this: the old temporary 
filename was an interesting size: 14 characters. That happens to be the 
traditional path component limit on really old UNIX filesystems, so if you 
were to try to use such a filesystem, you'd always be able to create the 
temporary file, but then you'd never actually be able to do the final link 
or rename due to the final name being too long.

So take this or leave it as you want - it's really not very important, I 
just wrote this because I was thinking about what really goes on at a very 
low level when we do that final atomic rename to create the actual 
directory entry. This makes it theoretically just a tiny bit more atomic 
on old-fashioned filesystems.

		Linus

---
 sha1_file.c |   21 ++++++++++++++++++---
 1 files changed, 18 insertions(+), 3 deletions(-)

diff --git a/sha1_file.c b/sha1_file.c
index 37bcc54..f9ec069 100644
--- a/sha1_file.c
+++ b/sha1_file.c

@@ -2110,17 +2110,31 @@ static inline int directory_size(const char *filename)
  *
  * We want to avoid cross-directory filename renames, because those
  * can have problems on various filesystems (FAT, NFS, Coda).
+ *
+ * In order to make the final rename as simple as possible to do
+ * in-place for traditional filesystems, we make the temporary
+ * filename have the same size as the final one - 38 characters
+ * (19 bytes of SHA1 info - the first byte is used for the sub-
+ * directory fan-out).
+ *
+ * The temporary filename pattern is
+ *  - 8 characters of "tmp_obj_"
+ *  - 23 characters of the object filename
+ *  - 7 characters of "_XXXXXX" for mkstemp().
  */
 static int create_tmpfile(char *buffer, size_t bufsiz, const char *filename)
 {
 	int fd, dirlen = directory_size(filename);
 
-	if (dirlen + 20 > bufsiz) {
+	if (dirlen + 39 > bufsiz) {
 		errno = ENAMETOOLONG;
 		return -1;
 	}
 	memcpy(buffer, filename, dirlen);
-	strcpy(buffer + dirlen, "tmp_obj_XXXXXX");
+	memcpy(buffer + dirlen, "tmp_obj_", 8);
+	memcpy(buffer + dirlen + 8, filename + dirlen, 23);
+	memcpy(buffer + dirlen + 31, "_XXXXXX", 8);
+	
 	fd = mkstemp(buffer);
 	if (fd < 0 && dirlen) {
 		/* Make sure the directory exists */

@@ -2129,7 +2143,8 @@ static int create_tmpfile(char *buffer, size_t bufsiz, const char *filename)
 			return -1;
 
 		/* Try again */
-		strcpy(buffer + dirlen - 1, "/tmp_obj_XXXXXX");
+		buffer[dirlen-1] = '/';
+		memcpy(buffer + dirlen + 32, "XXXXXX", 6);
 		fd = mkstemp(buffer);
 	}
 	return fd;

On Sun, 15 Jun 2008, Linus Torvalds wrote:

On Sat, 14 Jun 2008, Junio C Hamano wrote:

quoted

Other than that I had to stop for a few seconds to think at the magic 
"20 bytes longer", they look scary-but-correct ;-).

BTW, isn't the same concerns valid for temporary pack file creation too?
Or maybe we just don't care in the context of packs?

Also... this change is lacking its counterpart in 
builtin-prune.c:remove_temporary_files().


Nicolas

On Sat, Jun 14, 2008 at 10:42:33PM +0000, Junio C Hamano wrote:

Somehow the thread went off-list, so I'm diverting it back to the list...

  FWIW, the person with that patch here do not have the issue with NFS
at all, whereas people with older git versions have had the issue. IOW,
I consider my issue with NFS fixed by this patch.

  Thanks a lot guys \o/

-- 
·O·  Pierre Habouzit
··O                                                madcoder@debian.org
OOO                                                http://www.madism.org