Re: git-bugreport-2021-01-06-1209.txt (git can't deal with special characters)

4 messages, 2 authors, 2021-01-16 · open the first message on its own page

Re: git-bugreport-2021-01-06-1209.txt (git can't deal with special characters)

From: Daniel Troger <hidden>
Date: 2021-01-08 19:57:10

That did the trick, the test case is now reproducible here.
It seems as if there could be a patch in a couple of days, but no promises.
That's great news, thank you all very much!

I was already afraid at first it was some weird macOS filesystem bug only happening on high sierra and earlier which was never going to be fixed.

In the meantime I have managed to add "both directories" to git (with git add paulbr*). It's funny, now one directory shows up twice when viewing my repo on gitlab.

And this is the message I get when cloning:
warning: the following paths have collided (e.g. case-sensitive paths
on a case-insensitive filesystem) and only one from the same
colliding group is in the working tree:
  '[...]/paulbrunngård-springyard/main.ts'
  '[...]/paulbrunngård-springyard/main.ts'

What will happen with the two folders in the git repo after the patch?

Re: git-bugreport-2021-01-06-1209.txt (git can't deal with special characters)

From: Torsten Bögershausen <hidden>
Date: 2021-01-09 17:25:08

On Fri, Jan 08, 2021 at 08:56:08PM +0100, Daniel Troger wrote:
quoted
That did the trick, the test case is now reproducible here.
quoted
It seems as if there could be a patch in a couple of days, but no promises.
That's great news, thank you all very much!
Hej Daniel,
Not sure if I can keep my promise, please see below.
I was already afraid at first it was some weird macOS filesystem bug only happening on high sierra
and earlier which was never going to be fixed.
I haven't heard about anything special about Mac OS High Sierra.
If there are strange things, please let us know.

(And I still womder, how did you mange to create the "decomposed version of å" ?
 Mac OS itself doesn't decompose the "å", in opposite to "ä" or "ö".
 It creates a name for a file in decomposed form for "ä", "ö" and a lot of other
 characters/code points, but not for "å".
 It does, however, treat the pre- and de-composed form of "å" equivilent)
In the meantime I have managed to add "both directories" to git (with git add paulbr*).
It's funny, now one directory shows up twice when viewing my repo on gitlab.

And this is the message I get when cloning:
warning: the following paths have collided (e.g. case-sensitive paths
on a case-insensitive filesystem) and only one from the same
colliding group is in the working tree:
  '[...]/paulbrunngård-springyard/main.ts'
  '[...]/paulbrunngård-springyard/main.ts'

What will happen with the two folders in the git repo after the patch?
Now, that is another question.

First of all, I would recommend to "remove" one of the folders from Git,
because all files are tracked twice in Git, but "shadowed" by the file system.



To clean up the repo, you can do like this, explained in a dummy repo:

#One file (that is all we have) is tracked twice:
user@mac:/tmp/AAA> git ls-files
"paulbrunnga\314\212rd-springyard/horse"
"paulbrunng\303\245rd-springyard/horse"

# Remove all files from Git, but keep them on disk:
user@mac:/tmp/AAA> git rm -rf --cached .
rm 'paulbrunngård-springyard/horse'
rm 'paulbrunngård-springyard/horse'

# Re-add one version, the precomposed:
user@mac:/tmp/AAA> git add .
user@mac:/tmp/AAA> git status
On branch master
Changes to be committed:
  (use "git restore --staged <file>..." to unstage)
          deleted:    "paulbrunnga\314\212rd-springyard/horse"

user@mac:/tmp/AAA> git ls-files
"paulbrunng\303\245rd-springyard/horse"

#Not shown:
git commit -m "Remove duplicate directory"

####################

Now back to a potential patch:
That will take some time.
I digged some hours into the stuff, add lots of debug traces,
patches and stuff, and now we need a cleanup and get it into a better
shape to be acceptable.

All the best

Re: git-bugreport-2021-01-06-1209.txt (git can't deal with special characters)

From: Daniel Troger <hidden>
Date: 2021-01-13 14:58:13

Hej Torsten, 
To clean up the repo, you can do like this, explained in a dummy repo: 
your commands to remove one of the versions worked perfectly, thanks!
And I still womder, how did you mange to create the "decomposed version of å" ?
I created the directory with finder or atom I suppose, I did not find an mkdir command for that directory in my 290MB of bash history.
I digged some hours into the stuff, add lots of debug traces, patches and stuff
If I understand correctly, you already came up with a patch? That's great news! It's fine if it takes time as long as it gets fixed eventually.

Best regards

Re: git-bugreport-2021-01-06-1209.txt (git can't deal with special characters)

From: Torsten Bögershausen <hidden>
Date: 2021-01-16 17:35:06

On Wed, Jan 13, 2021 at 03:57:03PM +0100, Daniel Troger wrote:
Hej Torsten,
quoted
To clean up the repo, you can do like this, explained in a dummy repo:
your commands to remove one of the versions worked perfectly, thanks!
quoted
And I still womder, how did you mange to create the "decomposed version of å" ?
I created the directory with finder or atom I suppose, I did not find an mkdir command for that directory in my 290MB of bash history.
quoted
I digged some hours into the stuff, add lots of debug traces, patches and stuff
If I understand correctly, you already came up with a patch? That's great news! It's fine if it takes time as long as it gets fixed eventually.
That is a more tricky thing than I thought.
When Git starts, it runs getcwd() to find out where you started.
Inside that whole path there is the Git repo to be found, and the whole
path split into to parts:
The part pointing out the repo and
the part inside the repo.

This is done in setup.c, and it seems as if we need to precompose the whole
path, to work correctly.

However, the configuration saying "core.precomposeunicode" has not been read yet.

And setup.c doesn't know, where to find it, that is my understanding.

When I patch the getcwd() function to try to read the configuration,
it doesn't find it (yet).

It can find the global (or system) config.
Technically speaking, that can be used, but will probably cause other problems,
since sometimes the (repo) local configuration is different, sometimes not.

As far as I understand today, the best thing we can do is to check,
if getcwd() may need a precomposition: precompose and see if
the result is different.

Then error out, telling the user that you can not run Git inside
this directory.

I don't know, if this is worth the effort.

But: If somebody comes up with a patch, I am happy to review it
and/or get it into shape.
Keyboard shortcuts
hback out one level
jnext message in thread
kprevious message in thread
ldrill in
Escclose help / fold thread tree
?toggle this help