Re: Cloned repository has file changes -> bug?

On Tue, Feb 06 2018, Filip Jorissen jotted:

Hi all,

Thank you for your quick responses. I was able to resolve the problem based on your feedback!

Based on this experience, I would like to suggest that git is somehow able to avoid these problems by doing a case check itself rather than relying on the host OS for this?

I think it would make a lot of sense for git to ship with some standard
library of hooks for people to enable that covered common cases like
that.

But it's important to realize that:

 1) That's all this would be, this check would just be something you can
    do via a pre-receive hook now (or in this case, that big hosting
    providers like Github could offer as a pre-receive hook).

 2) Case collisions are just a subset of these issues, if you look at my
    CACBZZX4qH-w3YevoaW8RXAoFrjpfGerLWpBhetrO2tV-FGr3cQ@mail.gmail.com
    here on-list you'll see an issue relating to unicode normalizing
    that triggers a similar problem.

 3) Such hooks slow down pushes, especially on big repos, you can
    optimize things a bit (e.g. only look in the same directories), but
    pathologically you end up needing to compare the cross-product of
    changed files v.s. all existing files for each changed file.

 4) It's not something hosting providers like Github, Gitlab etc. can
    just enable, because some people want it like this. There's a lot of
    repos that have e.g. case collisions or other problems that only
    manifest on some OS-specific limitation (e.g. paths that are too
    long for VMS), and none of the users care because they don't use the
    repo on those systems.

    So it's not something that can be enabled by default, even for the
    most common case of case collisions.

Right now you're probably best off setting up a CI check on Github that
checks whether someone commits a colliding file, and poking Github about
maybe providing some opt-in feature to check for this.

I think the most performant way of checking this is:

    git --icase-pathspecs ls-files -- <changed files>

Which should never emit more lines than you give to it, you'd run that
either for the tip of the push, or if you want to be more thorough for
each pushed commit.

If someone can think of a better way to check for this I'd love to hear
about it, I'll probably add a check like this to some of our internal
repos soon. We semi-regularly have Mac users up in arms because someone
pushed a case-colliding file on Linux.

quoted

Op 28 jan. 2018, om 08:57 heeft Torsten Bögershausen [off-list ref] het volgende geschreven:

On Sat, Jan 27, 2018 at 08:59:50PM +0100, Ævar Arnfjörð Bjarmason wrote:

quoted

On Sat, Jan 27 2018, Filip Jorissen jotted:

quoted

I think our git repository is bugged. The reason why I say this is the
following. When cloning the repository, the newly cloned repository
immediately has file changes[...].

If you run this:

   git ls-files | tr '[:upper:]' '[:lower:]' | sort | uniq -D | grep '^'

You'll see that the reason is that you have files that differ only in
case.

You are using a Mac, and Macs by default think that files that are
different binary strings are the same file, since they don't consider
case to be relevant. The file FOO, foo and FoO and fOo are all the same
file as far as your Mac is concerned, but would be 4 different files on
Linux.

quoted

How can I fix the repository?

You could check it out on a OS that considers files that differ in case
to be different files, e.g. on Linux, move them around, push it, and new
clones should work on your Mac.

Alternatively I hear that you can create a loopback case-sensitive FS
image on Macs.

You can even fix the repo locally.
There are 2 files with uppercase/lowercase collisions.
I show you how to fix one off these, the other one goes similar.
After that, do a commit and a push and pull request.



Changes not staged for commit:
 (use "git add <file>..." to update what will be committed)
 (use "git checkout -- <file>..." to discard changes in working directory)

       modified:   IDEAS/Resources/ReferenceResults/Dymola/IDEAS_Fluid_HeatExchangers_GroundHeatExchangers_Borefield_Examples_MultipleBoreholesWithHeatPump.txt
       modified:   IDEAS/Resources/ReferenceResults/Dymola/IDEAS_Utilities_Psychrometrics_Functions_Examples_saturationPressure.txt

no changes added to commit (use "git add" and/or "git commit -a")
user@mac:/tmp/IDEAS> git ls-files -s | grep -i IDEAS/Resources/ReferenceResults/Dymola/IDEAS_Fluid_HeatExchangers_GroundHeatExchangers_Borefield_Examples_MultipleBoreholesWithHeatPump.txt
100644 f56cfcf14aa4b53dfc5ecfb488366f721c94c8e2 0       IDEAS/Resources/ReferenceResults/Dymola/IDEAS_Fluid_HeatExchangers_GroundHeatExchangers_Borefield_Examples_MultipleBoreholesWithHeatPump.txt
100644 e345e1372111d034b4c5a1c75eb791340b93f55e 0       IDEAS/Resources/ReferenceResults/Dymola/ideas_Fluid_HeatExchangers_GroundHeatExchangers_Borefield_Examples_MultipleBoreholesWithHeatPump.txt
user@mac:/tmp/IDEAS> git mv IDEAS/Resources/ReferenceResults/Dymola/ideas_Fluid_HeatExchangers_GroundHeatExchangers_Borefield_Examples_MultipleBoreholesWithHeatPump.txt IDEAS/Resources/ReferenceResults/Dymola/ideas_Fluid_HeatExchangers_GroundHeatExchangers_Borefield_Examples_MultipleBoreholesWithHeatPump2.txt
user@mac:/tmp/IDEAS> git checkout  IDEAS/Resources/ReferenceResults/Dymola/ideas_Fluid_HeatExchangers_GroundHeatExchangers_Borefield_Examples_MultipleBoreholesWithHeatPump2.txt
user@mac:/tmp/IDEAS> git checkout IDEAS/Resources/ReferenceResults/Dymola/IDEAS_Fluid_HeatExchangers_GroundHeatExchangers_Borefield_Examples_MultipleBoreholesWithHeatPump.txt
user@mac:/tmp/IDEAS> git status
On branch master
Your branch is up to date with 'origin/master'.

Changes to be committed:
 (use "git reset HEAD <file>..." to unstage)

       renamed:    IDEAS/Resources/ReferenceResults/Dymola/ideas_Fluid_HeatExchangers_GroundHeatExchangers_Borefield_Examples_MultipleBoreholesWithHeatPump.txt -> IDEAS/Resources/ReferenceResults/Dymola/ideas_Fluid_HeatExchangers_GroundHeatExchangers_Borefield_Examples_MultipleBoreholesWithHeatPump2.txt

Changes not staged for commit:
 (use "git add <file>..." to update what will be committed)
 (use "git checkout -- <file>..." to discard changes in working directory)

       modified:   IDEAS/Resources/ReferenceResults/Dymola/IDEAS_Utilities_Psychrometrics_Functions_Examples_saturationPressure.txt

user@mac:/tmp/IDEAS>