Hello,
Originally this bug was reported in the Git for Windows project and
contains two screenshots:
https://github.com/git-for-windows/git/issues/3321
Johannes Schindelin (dscho) is convinced that this is not a
Windows-specific issue. Following is a brief description of this bug
as I've faced it:
After running the "git submodule set-branch --branch master -- ms1"
I've noticed that the .gitmodules file is encoded with both DOS and
UNIX End-of-Line characters simultaneously: all original lines use DOS
EOL characters but the added "branch = master" line uses UNIX EOL.
Found on the following environment:
$ git --version --build-options
git version 2.32.0.windows.2
cpu: x86_64
built from commit: 3d45ac813c4adf97fe3733c1f763ab6617d5add5
sizeof-long: 4
sizeof-size_t: 8
shell-path: /bin/sh
feature: fsmonitor--daemon
$ cmd.exe /c ver
Microsoft Windows [Version 10.0.19042.1083]
$ cat /etc/install-options.txt
Editor Option: VIM
Custom Editor Path:
Default Branch Option:
Path Option: Cmd
SSH Option: OpenSSH
Tortoise Option: false
CURL Option: OpenSSL
CRLF Option: CRLFAlways
Bash Terminal Option: MinTTY
Git Pull Behavior Option: Merge
Use Credential Manager: Core
Performance Tweaks FSCache: Enabled
Enable Symlinks: Enabled
Enable Pseudo Console Support: Disabled
Enable FSMonitor: Disabled
Comments of Johannes Schindelin (dscho) in the GitHub bug report:
I don't believe that Git considers its config files free game regarding line endings.
Therefore, I think that having DOS line endings in there is already a violation of Git's
assumptions. In any case, this is not a Windows-specific issue, even if DOS line
endings are much more prevalent on Windows than on other platforms. Please take
it to the Git mailing list (send plain-text messages, HTML messages are dropped
silently).
From: brian m. carlson <hidden> Date: 2021-07-16 19:39:05
On 2021-07-15 at 21:55:07, Rostislav Krasny wrote:
Hello,
Originally this bug was reported in the Git for Windows project and
contains two screenshots:
https://github.com/git-for-windows/git/issues/3321
Johannes Schindelin (dscho) is convinced that this is not a
Windows-specific issue. Following is a brief description of this bug
as I've faced it:
After running the "git submodule set-branch --branch master -- ms1"
I've noticed that the .gitmodules file is encoded with both DOS and
UNIX End-of-Line characters simultaneously: all original lines use DOS
EOL characters but the added "branch = master" line uses UNIX EOL.
What behavior are you expecting to see here? Are you expecting Git to
write carriage returns when the file already uses carriage returns?
What about when the file already contains mixed endings?
Are you committing CRLF line endings into the repository, or are they
just created by core.autocrlf=true (or some .gitattributes setting)?
Does this cause a functional problem with Git (or maybe another tool),
or is it just aesthetically displeasing?
Comments of Johannes Schindelin (dscho) in the GitHub bug report:
quoted
I don't believe that Git considers its config files free game regarding line endings.
Therefore, I think that having DOS line endings in there is already a violation of Git's
assumptions. In any case, this is not a Windows-specific issue, even if DOS line
endings are much more prevalent on Windows than on other platforms. Please take
it to the Git mailing list (send plain-text messages, HTML messages are dropped
silently).
It appears the config parsing code handles CRLF when reading by just
turning it into a LF, so without changing the code, it wouldn't be
possible to distinguish between the different line endings. I don't
know that I agree with Dscho that CRLF line endings are right out, but I
also don't see always using LF line endings to be a problem.
Since I don't use Windows and don't see this problem, and I'm not really
partial to CRLF line endings anyway, I'm not especially interested in
making any changes here, but I'm also not opposed to seeing a patch here
if someone would like to send one to do something different.
--
brian m. carlson (he/him or they/them)
Toronto, Ontario, CA
On Fri, Jul 16, 2021 at 12:55:07AM +0300, Rostislav Krasny wrote:
Hello,
Originally this bug was reported in the Git for Windows project and
contains two screenshots:
https://github.com/git-for-windows/git/issues/3321
Johannes Schindelin (dscho) is convinced that this is not a
Windows-specific issue. Following is a brief description of this bug
as I've faced it:
After running the "git submodule set-branch --branch master -- ms1"
I've noticed that the .gitmodules file is encoded with both DOS and
UNIX End-of-Line characters simultaneously: all original lines use DOS
EOL characters but the added "branch = master" line uses UNIX EOL.
First of all: Thanks for posting this here.
Then there are some questions, at least from my side.
How did you get there ?
In which shell did you enter the command ?
Could you run
od -c .gitmodules
and post the results here ?
Or is it possible to set up a dummy repo, which does show the problem,
somewhere ?
What we appreciate is a fully reproducable receipt, so that anybody can
reproduce the problem.
I have the slight suspicion that the CR as part of CRLF had sneaked in
somewhere via the command line. But that is already a speculation.
And I don't know, if there is a problem at all, or is it just cosmetics ?
Anyway, a full set of commands would be good to have.
Found on the following environment:
$ git --version --build-options
git version 2.32.0.windows.2
cpu: x86_64
built from commit: 3d45ac813c4adf97fe3733c1f763ab6617d5add5
sizeof-long: 4
sizeof-size_t: 8
shell-path: /bin/sh
feature: fsmonitor--daemon
$ cmd.exe /c ver
Microsoft Windows [Version 10.0.19042.1083]
$ cat /etc/install-options.txt
Editor Option: VIM
Custom Editor Path:
Default Branch Option:
Path Option: Cmd
SSH Option: OpenSSH
Tortoise Option: false
CURL Option: OpenSSL
CRLF Option: CRLFAlways
Bash Terminal Option: MinTTY
Git Pull Behavior Option: Merge
Use Credential Manager: Core
Performance Tweaks FSCache: Enabled
Enable Symlinks: Enabled
Enable Pseudo Console Support: Disabled
Enable FSMonitor: Disabled
Comments of Johannes Schindelin (dscho) in the GitHub bug report:
quoted
I don't believe that Git considers its config files free game regarding line endings.
Therefore, I think that having DOS line endings in there is already a violation of Git's
assumptions. In any case, this is not a Windows-specific issue, even if DOS line
endings are much more prevalent on Windows than on other platforms. Please take
it to the Git mailing list (send plain-text messages, HTML messages are dropped
silently).
On Fri, Jul 16, 2021 at 10:39 PM brian m. carlson
[off-list ref] wrote:
On 2021-07-15 at 21:55:07, Rostislav Krasny wrote:
quoted
Hello,
Originally this bug was reported in the Git for Windows project and
contains two screenshots:
https://github.com/git-for-windows/git/issues/3321
Johannes Schindelin (dscho) is convinced that this is not a
Windows-specific issue. Following is a brief description of this bug
as I've faced it:
After running the "git submodule set-branch --branch master -- ms1"
I've noticed that the .gitmodules file is encoded with both DOS and
UNIX End-of-Line characters simultaneously: all original lines use DOS
EOL characters but the added "branch = master" line uses UNIX EOL.
What behavior are you expecting to see here? Are you expecting Git to
write carriage returns when the file already uses carriage returns?
What about when the file already contains mixed endings?
I expect it to be consistent, like the vi editor in the Git for
Windows distribution.
If the vi editor is used to create a new file or a file in the UNIX
EOL format it continues to use this format.
If the vi editor is used to edit an already existing file with the DOS
EOL format it continues to use that format.
Now if the "core.autocrlf" option is enabled (and in Git for Windows
it's enabled by default) all local copies of textual files are always
converted into the DOS format of EOL.
For example:
$ vi .gitignore
$ git add .gitignore
warning: LF will be replaced by CRLF in .gitignore.
The file will have its original line endings in your working directory
Are you committing CRLF line endings into the repository, or are they
just created by core.autocrlf=true (or some .gitattributes setting)?
In Git for Windows "core.autocrlf=true" is the default configuration.
Does this cause a functional problem with Git (or maybe another tool),
or is it just aesthetically displeasing?
I don't know, at least in the vi editor it looks broken because of all
those '^M' symbols.
I would like to take this opportunity to ask: why cloning a repository
that contains submodules sets this repository branch to its default
(master, main, whatever) but all submodules are set into a detached
HEAD mode instead of their default branches? This is actually the
reason I started to check what happens with the .gitmodules file. What
is the point in such an inconsistent behavior? There are a lot of
questions about that in Google and it seems most of the users expect
different behavior.
On Sat, Jul 17, 2021 at 8:05 PM Torsten Bögershausen [off-list ref] wrote:
On Fri, Jul 16, 2021 at 12:55:07AM +0300, Rostislav Krasny wrote:
quoted
Hello,
Originally this bug was reported in the Git for Windows project and
contains two screenshots:
https://github.com/git-for-windows/git/issues/3321
Johannes Schindelin (dscho) is convinced that this is not a
Windows-specific issue. Following is a brief description of this bug
as I've faced it:
After running the "git submodule set-branch --branch master -- ms1"
I've noticed that the .gitmodules file is encoded with both DOS and
UNIX End-of-Line characters simultaneously: all original lines use DOS
EOL characters but the added "branch = master" line uses UNIX EOL.
First of all: Thanks for posting this here.
Then there are some questions, at least from my side.
How did you get there ?
I just tried to use submodules and wanted to change the default state
of the submodules (from detached HEAD into some branch) after cloning
their parent repository together with the submodules. Take a look at
my question to Brian in this thread.
In which shell did you enter the command ?
Git Bash inside MINTTY of Git for Windows
$ bash --version
GNU bash, version 4.4.23(1)-release (x86_64-pc-msys)
Copyright (C) 2016 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software; you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Could you run
od -c .gitmodules
and post the results here ?
Sure:
$ od -c .gitmodules
0000000 [ s u b m o d u l e " m s 1 "
0000020 ] \r \n \t p a t h = m s 1 \r \n
0000040 \t u r l = . . / m s 1 \r \n \t
0000060 b r a n c h = m a s t e r \n
0000100 [ s u b m o d u l e " m s 2 "
0000120 ] \r \n \t p a t h = m s 2 \r \n
0000140 \t u r l = . . / m s 2 \r \n
0000157
Or is it possible to set up a dummy repo, which does show the problem,
somewhere ?
What we appreciate is a fully reproducable receipt, so that anybody can
reproduce the problem.
Try to do the following steps on Windows:
1. Download https://github.com/git-for-windows/git/files/6835344/git-tryouts.tar.gz
2. Extract the tarball and go into the git-tryouts/local-parent directory
3. Run the "git clone --recurse-submodules ../parent/ ." command
4. Run the "git submodule set-branch --branch master -- ms1" command
Now you can check the content of the .gitmodules file for the EOL issue.
Optional steps:
5. try to commit the new version of the .gitmodules file and push this
commit back by "git push" command
6. Delete everything in the git-tryouts/local-parent directory, for
example by the "rm -rf .git* *" command
7. Do step number 3 again
There is yet another inconsistency. Right after the commit or commit
plus push are done the .gitmodules file still has the EOL issue but
then after deleting everything and cloning the whole repository again
a different version of .gitmodules is created (because of
core.autocrlf=true). This inconsistency seems to be general and can
happen with any textual file on Windows.
I have the slight suspicion that the CR as part of CRLF had sneaked in
somewhere via the command line. But that is already a speculation.
And I don't know, if there is a problem at all, or is it just cosmetics ?
As I already answered to Brian I don't know, at least in the vi editor
it looks broken because of all
those '^M' symbols.
On Sun, Jul 18, 2021 at 01:42:26AM +0300, Rostislav Krasny wrote:
On Sat, Jul 17, 2021 at 8:05 PM Torsten B??gershausen [off-list ref] wrote:
quoted
On Fri, Jul 16, 2021 at 12:55:07AM +0300, Rostislav Krasny wrote:
quoted
Hello,
Originally this bug was reported in the Git for Windows project and
contains two screenshots:
https://github.com/git-for-windows/git/issues/3321
Johannes Schindelin (dscho) is convinced that this is not a
Windows-specific issue. Following is a brief description of this bug
as I've faced it:
After running the "git submodule set-branch --branch master -- ms1"
I've noticed that the .gitmodules file is encoded with both DOS and
UNIX End-of-Line characters simultaneously: all original lines use DOS
EOL characters but the added "branch = master" line uses UNIX EOL.
First of all: Thanks for posting this here.
Then there are some questions, at least from my side.
How did you get there ?
I just tried to use submodules and wanted to change the default state
of the submodules (from detached HEAD into some branch) after cloning
their parent repository together with the submodules. Take a look at
my question to Brian in this thread.
quoted
In which shell did you enter the command ?
Git Bash inside MINTTY of Git for Windows
$ bash --version
GNU bash, version 4.4.23(1)-release (x86_64-pc-msys)
Copyright (C) 2016 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software; you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
quoted
Could you run
od -c .gitmodules
and post the results here ?
Sure:
$ od -c .gitmodules
0000000 [ s u b m o d u l e " m s 1 "
0000020 ] \r \n \t p a t h = m s 1 \r \n
0000040 \t u r l = . . / m s 1 \r \n \t
0000060 b r a n c h = m a s t e r \n
0000100 [ s u b m o d u l e " m s 2 "
0000120 ] \r \n \t p a t h = m s 2 \r \n
0000140 \t u r l = . . / m s 2 \r \n
0000157
quoted
Or is it possible to set up a dummy repo, which does show the problem,
somewhere ?
What we appreciate is a fully reproducable receipt, so that anybody can
reproduce the problem.
Try to do the following steps on Windows:
1. Download https://github.com/git-for-windows/git/files/6835344/git-tryouts.tar.gz
2. Extract the tarball and go into the git-tryouts/local-parent directory
3. Run the "git clone --recurse-submodules ../parent/ ." command
4. Run the "git submodule set-branch --branch master -- ms1" command
Now you can check the content of the .gitmodules file for the EOL issue.
Thanks for the reproducing receipe.
I didn't manage to get there, but this was not on a Windows box.
I still suspect that this has nothing to do with core.autocrlf.
since it never produces "mixed" line endings.
(both CRLF and LF in the same file).
Is anybody else able to reproduce it ?
user@mac:/tmp/2021-07-18-git-submodules-CRLF/git-tryouts/local-parent> less .gitmodules
user@mac:/tmp/2021-07-18-git-submodules-CRLF/git-tryouts/local-parent> od -c .gitmodules
0000000 [ s u b m o d u l e " m s 1 "
0000020 ] \n \t p a t h = m s 1 \n \t u
0000040 r l = . . / m s 1 \n [ s u b
0000060 m o d u l e " m s 2 " ] \n \t p
0000100 a t h = m s 2 \n \t u r l =
0000120 . . / m s 2 \n
0000130
user@mac:/tmp/2021-07-18-git-submodules-CRLF/git-tryouts/local-parent> git config core.autocrlf true
user@mac:/tmp/2021-07-18-git-submodules-CRLF/git-tryouts/local-parent> git submodule set-branch --branch master -- ms1
user@mac:/tmp/2021-07-18-git-submodules-CRLF/git-tryouts/local-parent> od -c .gitmodules
0000000 [ s u b m o d u l e " m s 1 "
0000020 ] \n \t p a t h = m s 1 \n \t u
0000040 r l = . . / m s 1 \n \t b r a
0000060 n c h = m a s t e r \n [ s u
0000100 b m o d u l e " m s 2 " ] \n \t
0000120 p a t h = m s 2 \n \t u r l
0000140 = . . / m s 2 \n
0000151
user@mac:/tmp/2021-07-18-git-submodules-CRLF/git-tryouts/local-parent>
Optional steps:
5. try to commit the new version of the .gitmodules file and push this
commit back by "git push" command
6. Delete everything in the git-tryouts/local-parent directory, for
example by the "rm -rf .git* *" command
7. Do step number 3 again
There is yet another inconsistency. Right after the commit or commit
plus push are done the .gitmodules file still has the EOL issue but
then after deleting everything and cloning the whole repository again
a different version of .gitmodules is created (because of
core.autocrlf=true). This inconsistency seems to be general and can
happen with any textual file on Windows.
quoted
I have the slight suspicion that the CR as part of CRLF had sneaked in
somewhere via the command line. But that is already a speculation.
And I don't know, if there is a problem at all, or is it just cosmetics ?
As I already answered to Brian I don't know, at least in the vi editor
it looks broken because of all
those '^M' symbols.
On 18-Jul-2021, at 02:52, Rostislav Krasny [off-list ref] wrote:
[...]
I would like to take this opportunity to ask: why cloning a repository
that contains submodules sets this repository branch to its default
(master, main, whatever) but all submodules are set into a detached
HEAD mode instead of their default branches? This is actually the
reason I started to check what happens with the .gitmodules file. What
is the point in such an inconsistent behavior? There are a lot of
questions about that in Google and it seems most of the users expect
different behavior.
Submodules are cloned along with the parent repository only when
'submodule.recurse' is set to true, which I assume is the case for
your configuration.
A git clone that recurses submodules is equivalent to running
'git submodule update --init --recursive' right after the parent
project has been cloned.
The default mode for the update command is to checkout to the
revision that the parent project expects the submodule to be in, ie,
it checks out to the commit hash directly, and detaches the HEAD.
Why not checkout to a branch? This is because branches may change
which commits they point to.
For example, if you had a submodule 'foo'. I go into the 'foo'
folder and amend the last commit.
Now if I run 'submodule update' in the parent repository. If it had
checked out to the branch instead of the revision, I would be on a
different commit in 'foo' than the one that was registered by the
parent repo!
We want to have idempotence, ie, for a particular revision that is
registered by our parent project, we want 'update' to give the same
outcome every time, and not be dependent on whatever the state of the
branch is. This way we ensure that for a particular commit in the
parent project, the submodules will be in the same state for every
system in the world, after an 'update' is run.
I hope that makes sense.
(Anyone more knowledgeable than me in submodules, feel free to add
to what I am saying and correct me if I made an error!)
---
Atharva Raykar
ಅಥರ್ವ ರಾಯ್ಕರ್
अथर्व रायकर
From: Philippe Blain <hidden> Date: 2021-07-21 22:42:32
Hi Atharva,
Le 2021-07-21 à 03:26, Atharva Raykar a écrit :
On 18-Jul-2021, at 02:52, Rostislav Krasny [off-list ref] wrote:
quoted
[...]
I would like to take this opportunity to ask: why cloning a repository
that contains submodules sets this repository branch to its default
(master, main, whatever) but all submodules are set into a detached
HEAD mode instead of their default branches? This is actually the
reason I started to check what happens with the .gitmodules file. What
is the point in such an inconsistent behavior? There are a lot of
questions about that in Google and it seems most of the users expect
different behavior.
Submodules are cloned along with the parent repository only when
'submodule.recurse' is set to true, which I assume is the case for
your configuration.
Correction: no, 'git clone' always needs to be given the '--recurse-submodules' (or its old
alias, '--recursive') for submodules to be cloned. It does *not* honor
'submodule.recurse' (that's documented at [1]).
Should it honor it ?
Some people think yes, but others think no (mostly people that deal with repos with a huge number
of submodules which are optional, and thus would not want *all* submodules to be cloned
even if 'submodule.recurse' is set) . This has been discussed before,
and several tries were done to change that but for now the behaviour remains.
I personnally think that marking some submodules as optional should
be something that can be tracked in '.gitmodules', and a future version of Git
would clone all *non-optional* submodules if 'submodule.recurse' is set (some think
it should do so even if it's not set.) The most recent discussion about that, I think,
it at [2].
-- 8< --
I hope that makes sense.
(Anyone more knowledgeable than me in submodules, feel free to add
to what I am saying and correct me if I made an error!)
On Wed, Jul 21, 2021 at 10:26 AM Atharva Raykar [off-list ref] wrote:
The default mode for the update command is to checkout to the
revision that the parent project expects the submodule to be in, ie,
it checks out to the commit hash directly, and detaches the HEAD.
Why not checkout to a branch? This is because branches may change
which commits they point to.
For example, if you had a submodule 'foo'. I go into the 'foo'
folder and amend the last commit.
Now if I run 'submodule update' in the parent repository. If it had
checked out to the branch instead of the revision, I would be on a
different commit in 'foo' than the one that was registered by the
parent repo!
We want to have idempotence, ie, for a particular revision that is
registered by our parent project, we want 'update' to give the same
outcome every time, and not be dependent on whatever the state of the
branch is. This way we ensure that for a particular commit in the
parent project, the submodules will be in the same state for every
system in the world, after an 'update' is run.
All this is ok, but there could be other uses of sub-modules. Why not
to make this behavior configurable?
As a developer I may want to make new commits on those sub-modules and
always work with latest versions of their particular branches. In the
detached HEAD mode this is much harder to be done and also an error
prone operation. Commits that are done in the detached HEAD state may
easily be lost. Google is full of questions about how this could be
done, i.e. there is an essential demand of such a new behavior.
On Wed, Jul 21, 2021 at 10:26 AM Atharva Raykar [off-list ref] wrote:
quoted
[...]
We want to have idempotence, ie, for a particular revision that is
registered by our parent project, we want 'update' to give the same
outcome every time, and not be dependent on whatever the state of the
branch is. This way we ensure that for a particular commit in the
parent project, the submodules will be in the same state for every
system in the world, after an 'update' is run.
All this is ok, but there could be other uses of sub-modules. Why not
to make this behavior configurable?
As a developer I may want to make new commits on those sub-modules and
always work with latest versions of their particular branches. In the
detached HEAD mode this is much harder to be done and also an error
prone operation. Commits that are done in the detached HEAD state may
easily be lost. [...]
Have you tried running 'submodule update --merge' or
'submodule update --rebase'? Those modes will not detach your
submodule's HEAD.
The Pro Git book has a section that explains a workflow that sounds
similar to what you describe:
https://git-scm.com/book/en/v2/Git-Tools-Submodules
Around halfway down the page, look for the heading "Working on a
Submodule". Does that fit your use case?
[...] Google is full of questions about how this could be done, i.e. there
is an essential demand of such a new behavior.
It would be nice if you can share a specific link that captures the
demand you are talking about. When I Googled about this behaviour, I got
a range of different kinds of results, without a clearly voiced demand.
If you can show an example that best represents this demand, the Git
developers here may be able to address it better.