Torsten Bögershausen [off-list ref] writes:
To my undestanding we try to rename
foo/ into FOO/.
But because FOO/ already "exists" as directory,
Git tries to move foo/ into FOO/foo, which fails.
And no, the problem is probably not restricted to MacOs,
Windows and all case-insenstive file systems should show
the same, but I haven't tested yet, so it's more a suspicion.
The following diff allows to move foo/ into FOO/
If someone wants to make a patch out if, that would be good.
Is strcasecmp() sufficient for macOS whose filesystem has not just
case insensitivity but UTF-8 normalization issues?
On 2021-05-05 at 00:23:05, Junio C Hamano wrote:
Torsten Bögershausen [off-list ref] writes:
quoted
To my undestanding we try to rename
foo/ into FOO/.
But because FOO/ already "exists" as directory,
Git tries to move foo/ into FOO/foo, which fails.
And no, the problem is probably not restricted to MacOs,
Windows and all case-insenstive file systems should show
the same, but I haven't tested yet, so it's more a suspicion.
The following diff allows to move foo/ into FOO/
If someone wants to make a patch out if, that would be good.
Is strcasecmp() sufficient for macOS whose filesystem has not just
case insensitivity but UTF-8 normalization issues?
strcasecmp is locale sensitive, so I would say it is not a good choice
here. Notably, that means we have differing behavior with "i" and "I"
if the locale is Turkish versus English. Only one of those behaviors
will be a correct match for what the kernel does, since it is not locale
sensitive. Moreover, on Windows, the version of Unicode used for case
mapping is written into an NTFS file system when it's created, so a
fresh install may have different behavior from an upgrade, and
strcasecmp won't honor that.
And, as you pointed out, it doesn't handle Unicode normalization at all.
--
brian m. carlson (he/him or they/them)
Houston, Texas, US
On Wed, May 05, 2021 at 09:23:05AM +0900, Junio C Hamano wrote:
Torsten Bögershausen [off-list ref] writes:
quoted
To my undestanding we try to rename
foo/ into FOO/.
But because FOO/ already "exists" as directory,
Git tries to move foo/ into FOO/foo, which fails.
And no, the problem is probably not restricted to MacOs,
Windows and all case-insenstive file systems should show
the same, but I haven't tested yet, so it's more a suspicion.
The following diff allows to move foo/ into FOO/
If someone wants to make a patch out if, that would be good.
Is strcasecmp() sufficient for macOS whose filesystem has not just
case insensitivity but UTF-8 normalization issues?
Strictly speaking: no.
The Git code doesn't handle UTF-8 uppper/lower case at all:
git mv bar.txt BAR.TXT works because strcasecmp() is catching it.
git mv bär.txt BÄR.TXT needs the long way:
git mv bär.txt baer.txt && git mv baer.txt BÄR.TXT
We have been restricting the case-change-is-allowed to ASCII filenames
all the time.
There is no information, which code points map onto each other in Git,
since this is all file system dependent.
NTFS has one way, HFS+, APFS another, VFAT a third one, and if I expose
ext4 via SAMBA we probably have another one.
Not mentioniong that ext4 can be use case-insensitve on later Linux kernels,
which sticks to unicode.
Or Git repos running on machines using ISO-8859-1, those should be rare these
days.
That said, people are renaming files in ASCII only and are happy,
and in that sense renaming directories in ASCII can be supported
without major hassle.
And the inode approach mentioned as well:
This could go on top of strcasecmp() to cover non-ASCII filenames
or other oddities, if someone implements it.
So, I'm just a dumb Git user who doesn't even write C, so much of this
discussion is over my head, but I have a few thoughts that may be
helpful:
• The mv utility on Mac is capable of doing `mv bär.txt bÄr.txt` just
fine. Maybe `git mv` can learn something from whatever `mv` does?
• On a case-insensitive file system, `git mv somedir sOMEdir` is a
rename. But on a case-sensitive file system, it might NOT be a rename;
it might be the case that `somedir` and `sOMEdir` both exist and that
the command should put `somedir` inside `sOMEdir`. I mention this
because I can imagine some naive attempts at fixing the original bug
by doing a case-insensitive comparison of the two names ending up
breaking this behaviour on case-sensitive file systems by wrongly
treating such a command as a rename. It's probably worth having a test
that this scenario gets handled cleanly on case-sensitive file
systems? (I haven't checked whether Torsten's proposed diff falls into
this trap or not.)
• Above, Torsten mentions that there are filesystem-specific rules
about what names are equal to each other that Git can't easily handle,
because they go beyond just ASCII case changes. In that case, maybe
the right solution is to always defer the question to the filesystem
rather than Git trying to figure out the answer "in its head"?
That is: first check the inode or file ID of the src and dst passed
to `git mv`. If they are different and the second one is a folder,
move src inside the existing folder. If either they are the same or
the second one is not a folder, then do a rename.
It seems to me that this approach automatically handles stuff like
`git mv bär.txt bÄr.txt` plus any other rules about names being equal
(like two different sequences of code points that both express "à"),
all without Git ever needing to explicitly check whether two names are
case-insensitively equal. Am I missing something?
Sorry if any of the above is dumb or if I'm reiterating things others
have already said without realising it.
On Thu, May 6, 2021 at 5:34 AM Torsten Bögershausen [off-list ref] wrote:
On Wed, May 05, 2021 at 09:23:05AM +0900, Junio C Hamano wrote:
quoted
Torsten Bögershausen [off-list ref] writes:
quoted
To my undestanding we try to rename
foo/ into FOO/.
But because FOO/ already "exists" as directory,
Git tries to move foo/ into FOO/foo, which fails.
And no, the problem is probably not restricted to MacOs,
Windows and all case-insenstive file systems should show
the same, but I haven't tested yet, so it's more a suspicion.
The following diff allows to move foo/ into FOO/
If someone wants to make a patch out if, that would be good.
Is strcasecmp() sufficient for macOS whose filesystem has not just
case insensitivity but UTF-8 normalization issues?
Strictly speaking: no.
The Git code doesn't handle UTF-8 uppper/lower case at all:
git mv bar.txt BAR.TXT works because strcasecmp() is catching it.
git mv bär.txt BÄR.TXT needs the long way:
git mv bär.txt baer.txt && git mv baer.txt BÄR.TXT
We have been restricting the case-change-is-allowed to ASCII filenames
all the time.
There is no information, which code points map onto each other in Git,
since this is all file system dependent.
NTFS has one way, HFS+, APFS another, VFAT a third one, and if I expose
ext4 via SAMBA we probably have another one.
Not mentioniong that ext4 can be use case-insensitve on later Linux kernels,
which sticks to unicode.
Or Git repos running on machines using ISO-8859-1, those should be rare these
days.
That said, people are renaming files in ASCII only and are happy,
and in that sense renaming directories in ASCII can be supported
without major hassle.
And the inode approach mentioned as well:
This could go on top of strcasecmp() to cover non-ASCII filenames
or other oddities, if someone implements it.
On 06/05/21 16.12, Mark Amery wrote:
• Above, Torsten mentions that there are filesystem-specific rules
about what names are equal to each other that Git can't easily handle,
because they go beyond just ASCII case changes. In that case, maybe
the right solution is to always defer the question to the filesystem
rather than Git trying to figure out the answer "in its head"?
That is: first check the inode or file ID of the src and dst passed
to `git mv`. If they are different and the second one is a folder,
move src inside the existing folder. If either they are the same or
the second one is not a folder, then do a rename.
It seems to me that this approach automatically handles stuff like
`git mv bär.txt bÄr.txt` plus any other rules about names being equal
(like two different sequences of code points that both express "à"),
all without Git ever needing to explicitly check whether two names are
case-insensitively equal. Am I missing something?
On case when src and dst have different inodes and dst is a file, rather
than a directory, mv will overwrite dst with contents of src (on Linux).
--
An old man doll... just what I always wanted! - Clara
On Thu, May 06, 2021 at 10:12:40AM +0100, Mark Amery wrote:
So, I'm just a dumb Git user who doesn't even write C, so much of this
discussion is over my head, but I have a few thoughts that may be
helpful:
• The mv utility on Mac is capable of doing `mv bär.txt bÄr.txt` just
fine. Maybe `git mv` can learn something from whatever `mv` does?
Yes, Git can do the same.
The thing is, that Git avoids to overwrite of existing files.
Just to be nice.
In that sense, if the destination exists, Git refuses the mv,
unless you use git mv -f
Having said that,
git mv bär.txt bÄr.txt will not work (without the -f)
git mv bear.txt BEAR.txt does work,
• On a case-insensitive file system, `git mv somedir sOMEdir` is a
rename. But on a case-sensitive file system, it might NOT be a rename;
it might be the case that `somedir` and `sOMEdir` both exist and that
the command should put `somedir` inside `sOMEdir`. I mention this
because I can imagine some naive attempts at fixing the original bug
by doing a case-insensitive comparison of the two names ending up
breaking this behaviour on case-sensitive file systems by wrongly
treating such a command as a rename. It's probably worth having a test
that this scenario gets handled cleanly on case-sensitive file
systems? (I haven't checked whether Torsten's proposed diff falls into
this trap or not.)
Tests are needed - I should have started with those.
But I didn't intend to send a patch (yet), just sharing
ideas and knowledge. Which may enable someone to write a patch.
• Above, Torsten mentions that there are filesystem-specific rules
about what names are equal to each other that Git can't easily handle,
because they go beyond just ASCII case changes. In that case, maybe
the right solution is to always defer the question to the filesystem
rather than Git trying to figure out the answer "in its head"?
There are different trade-offs:
So far I am only aware of people asking for the
git mv bear.txt BEAR.txt rename.
Just because they are all SW developers ? I don't know.
And just because SW developers are developping Git,
the case-insensitive string compare is good enough,
it is working for them/us.
So things are as they are.
That is: first check the inode or file ID of the src and dst passed
to `git mv`. If they are different and the second one is a folder,
move src inside the existing folder. If either they are the same or
the second one is not a folder, then do a rename.
Yes. In short: patches are welcome.
In long: inodes don't work on Windows (without a major effort)
It seems to me that this approach automatically handles stuff like
`git mv bär.txt bÄr.txt` plus any other rules about names being equal
(like two different sequences of code points that both express "à"),
all without Git ever needing to explicitly check whether two names are
case-insensitively equal. Am I missing something?
That could be a solution. There may are situations/configurations,
where inodes don't work:
What happens if a Windows server exports a file system to MacOs ?
To Linux ?
Do we have working inodes ?
What about other networking combinations ?
Our code should handle them well as well.
Sorry if any of the above is dumb or if I'm reiterating things others
have already said without realising it.
No problem. Actually I realized that we used top-posting here,
So I remove the reset to make it more readable.
[]