fast-import should not care about core.ignorecase

Hi,

As you now know, I'm working on a mercurial remote helper for git. As
such, it uses fast-import.

In the mercurial history of mozilla-central, there have been various
renames of files with only case changes, and it so happens that my
remote helper blows things up on case insensitive file systems. The
reason is git clone probing the file system and setting core.ignorecase
appropriately.

While it makes sense for checkouts and local commits, it doesn't make
sense to me that using git fast-import with the same import script would
have a different behavior depending on whether the file system is
case-sensitive or not.

Reduced testcase:

$ git init
$ git fast-import <<EOF
blob
mark :1
data 2
a

commit refs/FOO
committer <foo@foo> 0 +0000
data 0

M 644 :1 a

commit refs/FOO
committer <foo@foo> 0 +0000
data 0

R a A
EOF

This is what you get on a case sensitive FS:

$ git log refs/FOO -p -M
commit be1497308f30f883343eefd0da7ddf1e747133f8
Author:  <foo@foo>
Date:   Thu Jan 1 00:00:00 1970 +0000

diff --git a/a b/A
similarity index 100%
rename from a
rename to A

commit 8d37f958cfc0702c577b918c86769a902fe109f8
Author:  <foo@foo>
Date:   Thu Jan 1 00:00:00 1970 +0000

diff --git a/a b/a
new file mode 100644
index 0000000..7898192
--- /dev/null
+++ b/a

@@ -0,0 +1 @@
+a

This is what you get on a case insensitive FS:

$ git log refs/FOO -p -M
commit 208c0c4cf58cd54512301e0de33ccb8a78d6b226
Author:  <foo@foo>
Date:   Thu Jan 1 00:00:00 1970 +0000

commit 8d37f958cfc0702c577b918c86769a902fe109f8
Author:  <foo@foo>
Date:   Thu Jan 1 00:00:00 1970 +0000

diff --git a/a b/a
new file mode 100644
index 0000000..7898192
--- /dev/null
+++ b/a

@@ -0,0 +1 @@
+a

Note, this applies equally to filerename commands or filedelete +
filemodify combinations.

Mike

On Tue, Dec 09, 2014 at 09:12:11AM +0900, Mike Hommey wrote:

Hi,

As you now know, I'm working on a mercurial remote helper for git. As
such, it uses fast-import.

In the mercurial history of mozilla-central, there have been various
renames of files with only case changes, and it so happens that my
remote helper blows things up on case insensitive file systems. The
reason is git clone probing the file system and setting core.ignorecase
appropriately.

While it makes sense for checkouts and local commits, it doesn't make
sense to me that using git fast-import with the same import script would
have a different behavior depending on whether the file system is
case-sensitive or not.

Heh, I just found this thread:
http://marc.info/?t=139134708700001&r=1&w=2

It doesn't seem to have led to something actually being committed,
though.

Mike

Mike Hommey wrote on 12/8/2014 5:12 PM:

While it makes sense for checkouts and local commits, it doesn't make
sense to me that using git fast-import with the same import script would
have a different behavior depending on whether the file system is
case-sensitive or not.

I have used fast-import with Perforce inputs.  When you run a Windows Perforce server, filenames can be submitted with ANY case, but given the case insensitive nature of the file system, a synced Perforce file will end up using whatever case happens to be on the file system at that point.

That may not be clear, so here goes:

Revision 1: abc/DEF/ghi/FILE.dat

Revision 2: ABC/def/GHI/file.dat

^^ Yes, Perforce stores the filename internally in that manner and does not fold the case.

If you happen to sync Revision 2 on an empty directory tree, you'll get ABC/def/GHI/file.dat.  If you then sync Revision 1, the filename case remains ABC/def/GHI/file.dat.

Likewise, if you happen to sync Revision 1 into an empty directory tree, you'll get abc/DEF/ghi/FILE.dat.  If you then sync Revision 2, the filename case remains as abc/DEF/ghi/FILE.dat.

I was the one who originally submitted the patch for this some 4 years ago.  It was commit 50906e04e8f48215b0b09841686709b92a2ab2e4. 'git fast-import' with core.ignorecase=true will fold the case of the filename specified in Revision 2 to the case currently stored in the Git repository from Revision 1.

If it does not do this, then Git internally stores FILE.dat and file.dat, and bad things happen on case-insensitive file systems.

(Further, there are still a few paths into Git where core.ignorecase=true does not fold the case of the filename, and this can cause 'repository corruptions' on case-insensitive file systems.  One such place is in 'git update-index' directly used by 'git gui'.  I really need to get this submitted, as we've been beating on it for a long time now, but here is the partial patch for informational purposes only.

I think it has been discussed before, but maybe Git needs a core.casefold in addition to core.ignorecase.)

-Josh

---------------------------- builtin/update-index.c ----------------------------
index aaa6f78..4cfedc1 100644

@@ -99,6 +99,7 @@ static int add_one_path(const struct cache_entry *old, const char *path, int len
     memcpy(ce->name, path, len);
     ce->ce_flags = create_ce_flags(0);
     ce->ce_namelen = len;
+    fold_ce_name_case(&the_index, ce);
     fill_stat_cache_info(ce, st);
     ce->ce_mode = ce_mode_from_stat(old, st->st_mode);

@@ -234,6 +235,7 @@ static int add_cacheinfo(unsigned int mode, const unsigned char *sha1,
     memcpy(ce->name, path, len);
     ce->ce_flags = create_ce_flags(stage);
     ce->ce_namelen = len;
+    fold_ce_name_case(&the_index, ce);
     ce->ce_mode = create_ce_mode(mode);
     if (assume_unchanged)
         ce->ce_flags |= CE_VALID;

Joshua Jensen wrote:

I think it has been discussed before, but maybe Git needs a
core.casefold in addition to core.ignorecase.)

Would it work for --casefold to be a commandline flag to fast-import,
instead of a global option affecting multiple Git commands?

Curious,
Jonathan

Jonathan Nieder wrote on 12/8/2014 6:31 PM:

Joshua Jensen wrote:

quoted

I think it has been discussed before, but maybe Git needs a
core.casefold in addition to core.ignorecase.)

Would it work for --casefold to be a commandline flag to fast-import,
instead of a global option affecting multiple Git commands?

Given that core.ignorecase=true means to fold filename case in quite a number of places within Git right now, I would expect the same behavior within a repository where fast-import is being run against core.ignorecase=true.

So, I don't know what core.ignorecase should mean, but I'm pretty sure I know what core.foldcase should mean.

Would --casefold work?  Sure, but it would be a special case against the existing core.ignorecase behavior that I don't think makes much sense.

Josh