RFD: Handling case-colliding filenames on case-insensitive filesystems

Hi,

At $dayjob we recently had a problem where a developer pushed a commit 
that added new files, two of which were named "foobar.TXT" 
and "FOOBAR.txt". When this commit (or anything based on it) is checked 
out by one of our Windows developers, Git maps two files in its index 
to a single file on the filesystem, and ends up reporting a diff on one 
of those files. The diff won't go away unless one (or both) of the 
case-colliding files is removed from the repo. Obivously, the 
persisting diff prevents the developer from easily rebasing, switching 
branches, merging, bisecting and a number of other useful tasks.

The root of the problem is that the case-colliding files were added in 
the first place, and this should obviously be prevented in projects 
that aim to be compatible with case-insensitive filesystems. To that 
end, I'm currently writing an update hook which will prevent 
case-colliding files from being pushed to our central repo.

However, given that this has already happened, how can we design Git to 
handle this situation more gracefully. In other words, how can we 
better handle checking out filenames that collide on case-insensitive 
filesystems?

My first idea was to simply refuse checking out trees with 
case-colliding filenames. I.e. when core.ignoreCase is enabled, we 
check whether any of the files we're about to checkout map to the same 
filesystem representation, and if they do, we abort the checkout and 
complain loudly to the user. However, that doesn't really help the user 
at all. Failure to checkout would only make it much harder to fix the 
issue.

A colleague suggested instead that Git should notice that the collision 
will occur, and work around the failure to represent the repository 
objects in the file system with a one-to-one match. Either by checking 
out only _one_ of the colliding files, or by using a non-colliding name 
for the second file. After all, Git already has functionality for 
manipulating the file contents on checkout (CRLF conversion). Doesn't 
it make sense to add functionality for manipulating the _directory_ 
contents on checkout as well? Even if that makes sense, I'm not sure 
that implementing it will be straightforward.

Are there better suggestions on how to deal with this?


Thanks,

...Johan

-- 
Johan Herland, [off-list ref]
www.herland.net

On Wed, Feb 23, 2011 at 12:11 PM, Johan Herland [off-list ref] wrote:

A colleague suggested instead that Git should notice that the collision
will occur, and work around the failure to represent the repository
objects in the file system with a one-to-one match. Either by checking
out only _one_ of the colliding files, or by using a non-colliding name
for the second file. After all, Git already has functionality for
manipulating the file contents on checkout (CRLF conversion). Doesn't
it make sense to add functionality for manipulating the _directory_
contents on checkout as well? Even if that makes sense, I'm not sure
that implementing it will be straightforward.

Are there better suggestions on how to deal with this?

The general problem is aliasing in the working-tree, of which
case-insenitivity is the most common form, but it also happens due to
HFS's use of NFD. A search on gmane for "insensitive" or "nfd" will
return many hits.

I think the argument against remapping filenames is that it doesn't
really help the user.

Let's say (for the sake of argument) that git supported remapping
between the index and the working-tree. Further, my repo has:

$ cat Foo.c
#include "Foo.h"

$ cat foo.c
#include "foo.h"

And on a case-insensitive file-system, git has remapped foo.[ch] to
foo~2.[ch] for the purposes of avoiding collisions on checkout.

The checkout can't be compiled correctly, so what's the point of even
allowing it?

(I'm not saying this is right/wrong, just that's been one of the
arguments against remapping.)

j.

On 11-02-23 02:07 PM, Jay Soffian wrote:

On Wed, Feb 23, 2011 at 12:11 PM, Johan Herland [off-list ref] wrote:

quoted

A colleague suggested instead that Git should notice that the collision
will occur, and work around the failure to represent the repository
objects in the file system with a one-to-one match. Either by checking
out only _one_ of the colliding files, or by using a non-colliding name
for the second file. After all, Git already has functionality for
manipulating the file contents on checkout (CRLF conversion). Doesn't
it make sense to add functionality for manipulating the _directory_
contents on checkout as well? Even if that makes sense, I'm not sure
that implementing it will be straightforward.

Are there better suggestions on how to deal with this?

The general problem is aliasing in the working-tree, of which
case-insenitivity is the most common form, but it also happens due to
HFS's use of NFD. A search on gmane for "insensitive" or "nfd" will
return many hits.

I think the argument against remapping filenames is that it doesn't
really help the user.

Let's say (for the sake of argument) that git supported remapping
between the index and the working-tree. Further, my repo has:

$ cat Foo.c
#include "Foo.h"

$ cat foo.c
#include "foo.h"

And on a case-insensitive file-system, git has remapped foo.[ch] to
foo~2.[ch] for the purposes of avoiding collisions on checkout.

The checkout can't be compiled correctly, so what's the point of even
allowing it?

In our case it would be useful to still have that checkout because the people
working on the case-insensitive systems are dealing with a different part of
the tree and don't care about the part with the collision.

A build designed to exploit case-sensitivity obviously won't work on a
case-insensitive system, but there's no reason to expect a git repo to have a
single, monolithic build.  There are a couple of parts of our code tree --
parts that are out of our control -- that use case sensitive file names, but
most of it doesn't.  It would be good if git would allow people on
case-insensitive systems to work with the repository, if not the complete build.

I suggest:

1. Git should emit a warning when checking out a case-colliding file (or
directory) on a case-insensitive system.  I don't really care _what_ gets
checked out for that file -- whatever it is ain't gonna work anyway.  Let's
say it checks out the associated blob the first time it runs across
thing.foo, but then emits the warning when it tries to check out Thing.Foo.

2. Git should forbid (yes, *forbid*) a user on a case-insensitive system from
adding any change to any files stored in the repository under
case-conflicting names.  The error message should basically be "You need to
use a case-sensitive system to work on this file."

3. I'm OK with git allowing case-insensitive users to forcibly delete
case-conflicting files.  "git rm thing.foo" should, on case-insensitive
systems, fail and display all case-colliding names for
[tT][hH][iI][nN][gG].[fF][oO][oO], and tell the user to use -f if they really
want to delete *all* those files.

		M.

On Wednesday 23 February 2011, Johan Herland wrote:

Are there better suggestions on how to deal with this?

Just a small note that I forgot in the first email:

For the record, this issue has been discussed on stackoverflow <URL: 
http://stackoverflow.com/questions/2528589/git-windows-case-sensitive-file-
names-not-handled-properly > and the last comment there suggests an 
alternative way to work around the case-colliding problem without having to 
remove either file from the repo:

Use sparse-checkout to exclude the case-colliding files from the working 
tree.

Obviously, this isn't a permanent solution, but I thought I'd just throw it 
out there, as something to consider until a more permanent solution is in 
place.


...Johan

-- 
Johan Herland, [off-list ref]
www.herland.net