Re: [RFC] File system difference handling in git

"Reece Dunn" [off-list ref] writes:

  1.  File name representation

For Linux file systems ...
Therefore, you have:

   os_to_git_path( const NATIVECHAR * ospath, strbuf * gitpath );
   git_to_os_path( const char * gitpath, const NATIVECHAR * ospath, int oslen );

It is not that simple, I am afraid.  Legacy encodings can be
used in pathnames.  With bog-standard traditional UNIX pathname
semantics, all pathnames are sequences of non-NUL, non-slash
bytes, separated with slashes, so if you do not allow choices
(which is a very sensible ideal world scenario), you can declare
that the "git" encoding is UTF-8 and always check things out
as-is.

But if you want a project ("git" in your above parlance) to be
checked out in two repositories, one with legacy and the other
with UTF-8, you cannot just say os_to_git/git_to_os.  You would
need a bit more information from the repository owners what
encodings are suitable.  So your os_to_git()/git_to_os() will
not be an identity function even on Linux to support such.

I used to have a data directory on my Linux box with EUC-JP
pathname and exported as an SMB share to my wife's Windows box,
telling samba to transliterate to whatever encoding the other
end liked.  I did not want to have the pathname on the Linux end
in UTF-8 because I did not have enough energey to update my
Emacs configuration to grok Japanese in UTF-8 (even though I
finally bit the bullet and switched to UTF-8 on the Linux side
recently).

I know, this is painful.  Real life hurts.  Even on Linux,
not everybody can live in UTF-8-only world.

  2.  Case (in)sensitivity

Here, you have the following cases:
...
  3.  git says that the files are different, but the filesystem says
that the files are the same.

Allow the move, updating the git directory tree only.

Sorry, I cannot really tell what you are talking about.  You
seem to imply, with "Allow the move", that you are describing a
scenario that involves a move of one existing file to another,
but it is not clear.  E.g. did you mean, by 3, "When the user
says 'move a b', and if git says a and b are different but if
the filesystem says a and b are the same, then..."?

On 22/01/2008, Junio C Hamano [off-list ref] wrote:

"Reece Dunn" [off-list ref] writes:

quoted

  1.  File name representation

For Linux file systems ...
Therefore, you have:

   os_to_git_path( const NATIVECHAR * ospath, strbuf * gitpath );
   git_to_os_path( const char * gitpath, const NATIVECHAR * ospath, int oslen );

It is not that simple, I am afraid.  Legacy encodings can be
used in pathnames.  With bog-standard traditional UNIX pathname
semantics, all pathnames are sequences of non-NUL, non-slash
bytes, separated with slashes, so if you do not allow choices
(which is a very sensible ideal world scenario), you can declare
that the "git" encoding is UTF-8 and always check things out
as-is.

So the upshot of this is that you need to use a platform (Operating
System, filesystem, locale, etc.) that match what the git repository
was created in, otherwise there are going to be issues when
interpreting paths correctly.

The locale issue asside, can the above proposal help users working on
Mac, Linux and Windows interoperate with each other?

I understand that there is not going to be a universal magic fix; what
I'm interested in is minimising the differences between Operating
Systems. This may be a futile effort, as it is likely you will need
some knowledge of the properties of the filesystem being used (as
filesystems with different properties can be used on the same
Operating System).

quoted

  2.  Case (in)sensitivity

Here, you have the following cases:
...
  3.  git says that the files are different, but the filesystem says
that the files are the same.

Allow the move, updating the git directory tree only.

Sorry, I cannot really tell what you are talking about.  You
seem to imply, with "Allow the move", that you are describing a
scenario that involves a move of one existing file to another,
but it is not clear.  E.g. did you mean, by 3, "When the user
says 'move a b', and if git says a and b are different but if
the filesystem says a and b are the same, then..."?

This is what I am saying. For example, if you say:

   git mv myfile.H myfile.h

on a case sensitive filesystem (e.g. ext3), this will work, however on
a case insensitive filesystem (e.g. ntfs) git would complain that the
files are the same.

The workaround is to say:

   git mv myfile.H myfile.h.tmp
   git mv myfile.h.tmp myfile.h

but this is not ideal, especially if you are automating some move operations.

This also applies to the VCS importers (e.g. git-p4) that can delete a
file that is a case-only move on case insensitive filesystems.

The question then becomes what happens on Mac (with the Unicode
decomposing behaviour) if they differ in the way they are stored (e.g.
in Linus' 'ä' vs 'a¨' example)?

- Reece

On Jan 22, 2008, at 11:52 AM, Reece Dunn wrote:

quoted

quoted

  2.  Case (in)sensitivity

Here, you have the following cases:
...
  3.  git says that the files are different, but the filesystem says
that the files are the same.

Allow the move, updating the git directory tree only.

Sorry, I cannot really tell what you are talking about.  You
seem to imply, with "Allow the move", that you are describing a
scenario that involves a move of one existing file to another,
but it is not clear.  E.g. did you mean, by 3, "When the user
says 'move a b', and if git says a and b are different but if
the filesystem says a and b are the same, then..."?

This is what I am saying. For example, if you say:

   git mv myfile.H myfile.h

on a case sensitive filesystem (e.g. ext3), this will work, however on
a case insensitive filesystem (e.g. ntfs) git would complain that the
files are the same.

The workaround is to say:

   git mv myfile.H myfile.h.tmp
   git mv myfile.h.tmp myfile.h

but this is not ideal, especially if you are automating some move  
operations.

This also applies to the VCS importers (e.g. git-p4) that can delete a
file that is a case-only move on case insensitive filesystems.

The question then becomes what happens on Mac (with the Unicode
decomposing behaviour) if they differ in the way they are stored (e.g.
in Linus' 'ä' vs 'a¨' example)?

You can work around the problem as you described; but later git
will hit you again and fails unexpectedly when you try to merge
your change.

So better avoid renames that only change case until git at least
passes the two test below.

	Steffen

---- snip ---

Git behaves strangely (from a user's point of view) on filesystems
that preserve case but do not distinguish it.  The two major examples
are Windows and Mac OS X.  Simple operations such as "git mv" or "git
merge" can fail unexpectedly.

This commit adds two simple tests.  Both tests currently fail on
Windows and Mac, although they pass on Linux.

Signed-off-by: Steffen Prohaska <redacted>
---
  t/t0050-filesystems.sh |   36 ++++++++++++++++++++++++++++++++++++
  1 files changed, 36 insertions(+), 0 deletions(-)
  create mode 100755 t/t0050-filesystems.sh

diff --git a/t/t0050-filesystems.sh b/t/t0050-filesystems.sh
new file mode 100755
index 0000000..953b02b
--- /dev/null
+++ b/t/t0050-filesystems.sh

@@ -0,0 +1,36 @@
+#!/bin/sh
+
+test_description='Various filesystems issues'
+
+. ./test-lib.sh
+
+test_expect_success setup '
+
+	touch camelcase &&
+	git add camelcase &&
+	git commit -m "initial" &&
+	git tag initial &&
+	git checkout -b topic &&
+	git mv camelcase tmp &&
+	git mv tmp CamelCase &&
+	git commit -m "rename" &&
+	git checkout -f master
+
+'
+
+test_expect_success 'rename (case change)' '
+
+	git mv camelcase CamelCase &&
+	git commit -m "rename"
+
+'
+
+test_expect_success 'merge (case change)' '
+
+	git reset --hard initial &&
+	git merge topic
+
+'
+
+test_done

-- 
1.5.4.rc4

On Jan 22, 2008 10:52 AM, Reece Dunn [off-list ref] wrote:

This is what I am saying. For example, if you say:

   git mv myfile.H myfile.h

on a case sensitive filesystem (e.g. ext3), this will work, however on
a case insensitive filesystem (e.g. ntfs) git would complain that the
files are the same.

The workaround is to say:

   git mv myfile.H myfile.h.tmp
   git mv myfile.h.tmp myfile.h

but this is not ideal, especially if you are automating some move operations.

If I remember correctly, this fails when it comes to applying the
commit containing that move, at least on HFS+.  You could create 2
commits (one with the first mv, one with the second), and apply them
one at a time, but it's a pretty unpleasant workaround.