Difficulties of scripting git

4 messages, 2 authors, 2021-01-10 · open the first message on its own page

Difficulties of scripting git

From: Alan Mackenzie <hidden>
Date: 2021-01-09 19:21:36

Hello, git.

I want to write a bash script (more precisely, correct an existing
script) which uses git, and it is DIFFICULT!!!

I need a git pull in the script.  Fine, but how does the script know if
it's worked or not?  I couldn't find any description of a return code in
the git-pull man page (or in the git-merge man page).

The big problem is when I have modified, uncommitted files in the target
repo, and those same files are to be updated in commits coming from the
source repo.  Sadly, git is unable to merge these changes.  It just
fails, putting an error message onto stderr, but doesn't tell the
calling script in any way that I can see.

One idea would be always to call git stash before doing the pull, then
git stash pop afterwards.  Trouble is, git stash is unreliable - it
doesn't always add a new stash entry, so the stash pop at the end would
sometimes/often pop off an entry it shouldn't.  git stash doesn't have a
--force argument.  git stash doesn't set a result code, either, that I
can see.  One way around this would be to do

    $ git stash list | wc -l

both before and after the git stash and compare the answers, but really?

So, next idea, feed the output from git status --porcelain through grep
before and after the git pull, so as to find out whether there are any
modified files before the git pull (thus making a stash necessary) and
any files with conflicts after the git stash pop.  Shouldn't be too
difficult.

Except, how does one recognise a file with conflicts from this git
status output?  The man page says that

    "   For paths with merge conflicts, `X' and `Y' show the modification
    states of each side of the merge.  For paths that do not have merge
    conflicts, `X' shows the status of the index, and `Y' shows the status
    of the work tree.  For untracked paths, `XY' are `??'.  Other status
    codes can be interpreted as follows:  ...."

I've spent nearly an hour trying to make sense of this bit of man page.
How is one meant to distinguish an XY of a merge conflict from the XY of
an index/work tree entry?  I can't find that key bit of information
anywhere.

What am I missing?  Why does writing scripts using git have to be so
hard?

-- 
Alan Mackenzie (Nuremberg, Germany).

Re: Difficulties of scripting git

From: brian m. carlson <hidden>
Date: 2021-01-09 21:43:42

On 2021-01-09 at 19:14:13, Alan Mackenzie wrote:
Hello, git.

I want to write a bash script (more precisely, correct an existing
script) which uses git, and it is DIFFICULT!!!

I need a git pull in the script.  Fine, but how does the script know if
it's worked or not?  I couldn't find any description of a return code in
the git-pull man page (or in the git-merge man page).

The big problem is when I have modified, uncommitted files in the target
repo, and those same files are to be updated in commits coming from the
source repo.  Sadly, git is unable to merge these changes.  It just
fails, putting an error message onto stderr, but doesn't tell the
calling script in any way that I can see.

One idea would be always to call git stash before doing the pull, then
git stash pop afterwards.  Trouble is, git stash is unreliable - it
doesn't always add a new stash entry, so the stash pop at the end would
sometimes/often pop off an entry it shouldn't.  git stash doesn't have a
--force argument.  git stash doesn't set a result code, either, that I
can see.  One way around this would be to do

    $ git stash list | wc -l

both before and after the git stash and compare the answers, but really?
git stash doesn't consider there being nothing to stash to be an error,
so it doesn't set a nonzero error code.  Think of it as, "I want a clean
working tree," not, "I want to create a stash."  In that sense, what you
wanted is already done, so it's not an error.

You are, however, not the only person who's been unhappy with this
behavior.  A --force option may be a useful option to have if someone
wanted to create it.
So, next idea, feed the output from git status --porcelain through grep
before and after the git pull, so as to find out whether there are any
modified files before the git pull (thus making a stash necessary) and
any files with conflicts after the git stash pop.  Shouldn't be too
difficult.
Using "git status --porcelain" before and after the stash is an easy way
to find out whether there was anything to stash.  If there's a
difference, then a stash was created.
Except, how does one recognise a file with conflicts from this git
status output?  The man page says that

    "   For paths with merge conflicts, `X' and `Y' show the modification
    states of each side of the merge.  For paths that do not have merge
    conflicts, `X' shows the status of the index, and `Y' shows the status
    of the work tree.  For untracked paths, `XY' are `??'.  Other status
    codes can be interpreted as follows:  ...."

I've spent nearly an hour trying to make sense of this bit of man page.
How is one meant to distinguish an XY of a merge conflict from the XY of
an index/work tree entry?  I can't find that key bit of information
anywhere.
In the chart, the options in the first section are "normal" cases that
occur without a merge conflict.  The second section is things that
happen when there's a conflict (the "unmerged" states).

So what you're looking for to find conflicts is something maybe a little
like this:

  git status --porcelain | grep -E '^(AA|DD|[AUD]U|U[AUD])'

If you believe that you're likely to need to handle names with newlines,
then of course you'll need -z as well.

I do agree that this is a little unclear; I went ahead and simulated a
merge conflict to verify.  I'll try to send a patch making the
documentation reflect what I mentioned about the chart above.
-- 
brian m. carlson (he/him or they/them)
Houston, Texas, US

[PATCH] docs: add description of status output table

From: brian m. carlson <hidden>
Date: 2021-01-09 22:07:51

The table describing the porcelain format in git-status(1) is helpful,
but it's not completely clear what the three sections mean, even to
some contributors.  As a result, users are unable to find how to detect
common cases like merge conflicts programmatically.

Let's improve this situation by describing what each section means:
non-conflicted, conflicted, or untracked files.

Signed-off-by: brian m. carlson <redacted>
---
 Documentation/git-status.txt | 4 ++++
 1 file changed, 4 insertions(+)
diff --git a/Documentation/git-status.txt b/Documentation/git-status.txt
index 7731b45f07..63f13c201d 100644
--- a/Documentation/git-status.txt
+++ b/Documentation/git-status.txt
@@ -201,6 +201,10 @@ codes can be interpreted as follows:
 Ignored files are not listed, unless `--ignored` option is in effect,
 in which case `XY` are `!!`.
 
+In the table below, the first section indicates normal non-conflicted states for
+tracked files, the second indicates files where a merge conflict has occurred
+but not yet been resolved, and the third indicates files not tracked by Git.
+
 ....
 X          Y     Meaning
 -------------------------------------------------

[PATCH v2] docs: rephrase and clarify the git status --short format

From: brian m. carlson <hidden>
Date: 2021-01-10 19:05:40

The table describing the porcelain format in git-status(1) is helpful,
but it's not completely clear what the three sections mean, even to
some contributors.  As a result, users are unable to find how to detect
common cases like merge conflicts programmatically.

Let's improve this situation by rephrasing to be more explicit about
what each of the sections in the table means, to tell users in plain
language which cases are occurring, and to describe what "unmerged"
means.

Signed-off-by: brian m. carlson <redacted>
---
This uses text from Junio's email, so his sign-off will be required
here.  I assume that won't be a problem, but I can send a v3 if it is.

 Documentation/git-status.txt | 28 ++++++++++++++++++++--------
 1 file changed, 20 insertions(+), 8 deletions(-)
diff --git a/Documentation/git-status.txt b/Documentation/git-status.txt
index 7731b45f07..c0764e850a 100644
--- a/Documentation/git-status.txt
+++ b/Documentation/git-status.txt
@@ -184,11 +184,26 @@ characters, that field will be quoted in the manner of a C string
 literal: surrounded by ASCII double quote (34) characters, and with
 interior special characters backslash-escaped.
 
-For paths with merge conflicts, `X` and `Y` show the modification
-states of each side of the merge. For paths that do not have merge
-conflicts, `X` shows the status of the index, and `Y` shows the status
-of the work tree.  For untracked paths, `XY` are `??`.  Other status
-codes can be interpreted as follows:
+There are three different types of states that are shown using this format, and
+each one uses the `XY` syntax differently:
+
+* When a merge is occurring and the merge was successful, or outside of a merge
+	situation, `X` shows the status of the index and `Y` shows the status of the
+	working tree.
+* When a merge conflict has occurred and has not yet been resolved, `X` and `Y`
+	show the state introduced by each head of the merge, relative to the common
+	ancestor. These paths are said to be _unmerged_.
+* When a path is untracked, `X` and `Y` are always the same, since they are
+	unknown to the index. `??` is used for untracked paths. Ignored files are
+	not listed unless `--ignored` is used; if it is, ignored files are indicated
+	by `!!`.
+
+Note that the term _merge_ here also includes rebases using the default
+`--merge` strategy, cherry-picks, and anything else using the merge machinery.
+
+In the following table, these three classes are shown in separate sections, and
+these characters are used for `X` and `Y` fields for the first two sections that
+show tracked paths:
 
 * ' ' = unmodified
 * 'M' = modified
@@ -198,9 +213,6 @@ codes can be interpreted as follows:
 * 'C' = copied
 * 'U' = updated but unmerged
 
-Ignored files are not listed, unless `--ignored` option is in effect,
-in which case `XY` are `!!`.
-
 ....
 X          Y     Meaning
 -------------------------------------------------
Keyboard shortcuts
hback out one level
jnext message in thread
kprevious message in thread
ldrill in
Escclose help / fold thread tree
?toggle this help