Here's a new version of the merge normalization series that renames the
configuration variable. Since "merge.renormalize" got the best
response, I went with that.
Junio, I hope you don't mind that I squashed your patch to introduce the
config variable into the first patch. It seemed a bit out-of-place as a
separate commit.
- Eyvind
Eyvind Bernhardsen (3):
Avoid conflicts when merging branches with mixed normalization
Try normalizing files to avoid delete/modify conflicts when merging
Don't expand CRLFs when normalizing text during merge
Documentation/gitattributes.txt | 34 ++++++++++++++++++++
Documentation/merge-config.txt | 10 ++++++
builtin/merge.c | 3 ++
cache.h | 2 +
convert.c | 37 +++++++++++++++++++----
environment.c | 1 +
ll-merge.c | 15 +++++++++
merge-recursive.c | 51 +++++++++++++++++++++++++++++-
t/t6038-merge-text-auto.sh | 64 +++++++++++++++++++++++++++++++++++++++
9 files changed, 209 insertions(+), 8 deletions(-)
create mode 100755 t/t6038-merge-text-auto.sh
--
1.7.1.575.g383de
Disable CRLF expansion when convert_to_working_tree() is called from
normalize_buffer(). This improves performance when merging branches
with conflicting line endings when core.eol=crlf or core.autocrlf=true
by making the normalization act as if core.eol=lf.
Signed-off-by: Eyvind Bernhardsen <redacted>
Signed-off-by: Junio C Hamano <redacted>
---
convert.c | 27 ++++++++++++++++++++-------
1 files changed, 20 insertions(+), 7 deletions(-)
If a file is modified due to normalization on one branch, and deleted on
another, a merge of the two branches will result in a delete/modify
conflict for that file even if it is otherwise unchanged.
Try to avoid the conflict by normalizing and comparing the "base" file
and the modified file when their sha1s differ. If they compare equal,
the file is considered unmodified and is deleted.
Signed-off-by: Eyvind Bernhardsen <redacted>
Signed-off-by: Junio C Hamano <redacted>
---
merge-recursive.c | 51 ++++++++++++++++++++++++++++++++++++++++++-
t/t6038-merge-text-auto.sh | 2 +-
2 files changed, 50 insertions(+), 3 deletions(-)
@@ -1056,6 +1056,53 @@ static unsigned char *stage_sha(const unsigned char *sha, unsigned mode)return(is_null_sha1(sha)||mode==0)?NULL:(unsignedchar*)sha;}+staticintread_sha1_strbuf(constunsignedchar*sha1,structstrbuf*dst)+{+void*buf;+enumobject_typetype;+unsignedlongsize;+buf=read_sha1_file(sha1,&type,&size);+if(!buf)+returnerror("cannot read object %s",sha1_to_hex(sha1));+if(type!=OBJ_BLOB){+free(buf);+returnerror("object %s is not a blob",sha1_to_hex(sha1));+}+strbuf_attach(dst,buf,size,size+1);+return0;+}++staticintblob_unchanged(constunsignedchar*o_sha,+constunsignedchar*a_sha,+constchar*path)+{+structstrbufo=STRBUF_INIT;+structstrbufa=STRBUF_INIT;+intret=0;/* assume changed for safety */++if(sha_eq(o_sha,a_sha))+return1;+if(!merge_renormalize)+return0;++assert(o_sha&&a_sha);+if(read_sha1_strbuf(o_sha,&o)||read_sha1_strbuf(a_sha,&a))+gotoerror_return;+/*+*Note:binary|isusedsothatbothrenormalizationsare+*performed.Comparisoncanbeskippedifbothfilesare+*unchangedsincetheirsha1shavealreadybeencompared.+*/+if(renormalize_buffer(path,o.buf,o.len,&o)|+renormalize_buffer(path,a.buf,o.len,&a))+ret=(o.len==a.len&&!memcmp(o.buf,a.buf,o.len));++error_return:+strbuf_release(&o);+strbuf_release(&a);+returnret;+}+/* Per entry merge function */staticintprocess_entry(structmerge_options*o,constchar*path,structstage_data*entry)
@@ -1075,8 +1122,8 @@ static int process_entry(struct merge_options *o,if(o_sha&&(!a_sha||!b_sha)){/* Case A: Deleted in one */if((!a_sha&&!b_sha)||-(sha_eq(a_sha,o_sha)&&!b_sha)||-(!a_sha&&sha_eq(b_sha,o_sha))){+(!b_sha&&blob_unchanged(o_sha,a_sha,path))||+(!a_sha&&blob_unchanged(o_sha,b_sha,path))){/* Deleted in both or deleted in one and*unchangedintheother*/if(a_sha)
Currently, merging across changes in line ending normalization is
painful since files containing CRLF will conflict with normalized files,
even if the only difference between the two versions is the line
endings. Additionally, any "real" merge conflicts that exist are
obscured because every line in the file has a conflict.
Assume you start out with a repo that has a lot of text files with CRLF
checked in (A):
o---C
/ \
A---B---D
B: Add "* text=auto" to .gitattributes and normalize all files to
LF-only
C: Modify some of the text files
D: Try to merge C
You will get a ridiculous number of LF/CRLF conflicts when trying to
merge C into D, since the repository contents for C are "wrong" wrt the
new .gitattributes file.
Fix ll-merge so that the "base", "theirs" and "ours" stages are passed
through convert_to_worktree() and convert_to_git() before a three-way
merge. This ensures that all three stages are normalized in the same
way, removing from consideration differences that are only due to
normalization.
This feature is optional for now since it changes a low-level mechanism
and is not necessary for the majority of users. The "merge.renormalize"
config variable enables it.
Signed-off-by: Junio C Hamano <redacted>
Signed-off-by: Eyvind Bernhardsen <redacted>
---
Documentation/gitattributes.txt | 34 ++++++++++++++++++++
Documentation/merge-config.txt | 10 ++++++
builtin/merge.c | 3 ++
cache.h | 2 +
convert.c | 16 ++++++++-
environment.c | 1 +
ll-merge.c | 15 +++++++++
t/t6038-merge-text-auto.sh | 64 +++++++++++++++++++++++++++++++++++++++
8 files changed, 143 insertions(+), 2 deletions(-)
create mode 100755 t/t6038-merge-text-auto.sh
@@ -317,6 +317,17 @@ command is "cat"). smudge = cat ------------------------+For best results, `clean` should not alter its output further if it is+run twice ("clean->clean" should be equivalent to "clean"), and+multiple `smudge` commands should not alter `clean`'s output+("smudge->smudge->clean" should be equivalent to "clean"). See the+section on merging below.++The "indent" filter is well-behaved in this regard: it will not modify+input that is already correctly indented. In this case, the lack of a+smudge filter means that the clean filter _must_ accept its own output+without modifying it.+ Interaction between checkin/checkout attributes ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
@@ -331,6 +342,29 @@ In the check-out codepath, the blob content is first converted with `text`, and then `ident` and fed to `filter`.+Merging branches with differing checkin/checkout attributes+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^++If you have added attributes to a file that cause the canonical+repository format for that file to change, such as adding a+clean/smudge filter or text/eol/ident attributes, merging anything+where the attribute is not in place would normally cause merge+conflicts.++To prevent these unnecessary merge conflicts, git can be told to run a+virtual check-out and check-in of all three stages of a file when+resolving a three-way merge by setting the `merge.renormalize`+configuration variable. This prevents changes caused by check-in+conversion from causing spurious merge conflicts when a converted file+is merged with an unconverted file.++As long as a "smudge->clean" results in the same output as a "clean"+even on files that are already smudged, this strategy will+automatically resolve all filter-related conflicts. Filters that do+not act in this way may cause additional merge conflicts that must be+resolved manually.++ Generating diff text ~~~~~~~~~~~~~~~~~~~~
@@ -15,6 +15,16 @@ merge.renameLimit:: during a merge; if not specified, defaults to the value of diff.renameLimit.+merge.renormalize::+ Tell git that canonical representation of files in the+ repository has changed over time (e.g. earlier commits record+ text files with CRLF line endings, but recent ones use LF line+ endings). In such a repository, git can convert the data+ recorded in commits to a canonical form before performing a+ merge to reduce unnecessary conflicts. For more information,+ see section "Merging branches with differing checkin/checkout+ attributes" in linkgit:gitattributes[5].+ merge.stat:: Whether to print the diffstat between ORIG_HEAD and the merge result at the end of the merge. True by default.
@@ -53,6 +53,7 @@ enum object_creation_mode object_creation_mode = OBJECT_CREATION_MODE;char*notes_ref_name;intgrafts_replace_parents=1;intcore_apply_sparse_checkout;+intmerge_renormalize;/* Parallel index stat data preload? */intcore_preload_index=0;
@@ -0,0 +1,64 @@+#!/bin/sh++test_description='CRLF merge conflict across text=auto change'++../test-lib.sh++test_expect_successsetup'+gitconfigmerge.renormalizetrue&&+gitconfigcore.autocrlffalse&&+echofirstline|append_cr>file&&+echofirstline>control_file&&+echoonlyline>inert_file&&+gitaddfilecontrol_fileinert_file&&+gitcommit-m"Initial"&&+gittaginitial&&+gitbranchside&&+echo"* text=auto">.gitattributes&&+touchfile&&+gitadd.gitattributesfile&&+gitcommit-m"normalize file"&&+echosameline|append_cr>>file&&+echosameline>>control_file&&+gitaddfilecontrol_file&&+gitcommit-m"add line from a"&&+gittaga&&+gitrm.gitattributes&&+rmfile&&+gitcheckoutfile&&+gitcommit-m"remove .gitattributes"&&+gittagc&&+gitcheckoutside&&+echosameline|append_cr>>file&&+echosameline>>control_file&&+gitaddfilecontrol_file&&+gitcommit-m"add line from b"&&+gittagb&&+gitcheckoutmaster+'++test_expect_success'Check merging after setting text=auto''+gitreset--harda&&+gitmergeb&&+catfile|remove_cr>file.temp&&+test_cmpfilefile.temp+'++test_expect_success'Check merging addition of text=auto''+gitreset--hardb&&+gitmergea&&+catfile|remove_cr>file.temp&&+test_cmpfilefile.temp+'++test_expect_failure'Test delete/normalize conflict''+gitcheckoutside&&+gitreset--hardinitial&&+gitrmfile&&+gitcommit-m"remove file"&&+gitcheckoutmaster&&+gitreset--harda^&&+gitmergeside+'++test_done
From: Jonathan Nieder <hidden> Date: 2016-06-15 22:49:14
Hi Eyvind,
Eyvind Bernhardsen wrote:
Here's a new version of the merge normalization series that renames the
configuration variable. Since "merge.renormalize" got the best
response, I went with that.
I have no idea how the following branch happened in my local git tree,
but it’s here now so maybe it’s worth something. Somehow I got the
idea that the merge_renormalize variable was a nice short-term
protection but long-term scary, and so a few patches emerged to
banish it.
Please let me emphasize that in their current form I do not think
these patches should be very usable.
But for later, they tell a nice story. The idea is that the
"merge.renormalize" is not really about the behavior of the "git
merge" command but about the merge-recursive driver; and so it
should be usable everywhere else that driver is used, too.
So in an ideal world, you would be able to, e.g.:
git checkout -m -Xrenormalize otherbranch
or
git revert -Xrenormalize otherpatch
or
git pull --rebase -Xrenormalize
Of course, the plumbing is not all there yet; many commands that
use merge drivers cannot pass through arbitrary options through
yet.
Well, enough talking. Here are the patches; what do you think?
Patch 1 teaches blob_unchanged() to stop paying attention to the
merge_recursive variable. It uses a parameter instead. So from then
on, it can watch, bemused, a neutral party.
Patch 2 adds and respects a renormalize option in the merge_options
struct. It defaults to the value of the merge_recursive global to
save callers (in particular, "git merge") the pain of adjusting.
Patch 3 lets ll_merge merge callers decide whether to renormalize,
too. Most callers never renormalize, since they are not "git merge"
(and this is noted in new comments).
Patch 4 is a sort of a digression. It makes "git rerere" produce nice
help output with the "git rerere -h" option.
Patch 5 lets rerere callers decide whether to renormalize, too. Most
callers never renormalize. But looking at that patch reveals a
serious problem: there are all kinds of other merge options (e.g.,
-Xsubtree) that rerere callers are unable to control.
Following the principle that scripted users should have about as
much power as internal ones, patch 5 reluctantly exposes the
renormalize option to rerere as a new --renormalize option. It
should be -Xrenormalize instead.
Patch 6 eliminates the merge_recursive global once and for all, by
teaching merge_recursive callers to set the renormalize merge option
appropriately (and removing the backward-compatibility default).
Callers that are not "git merge" still never use renormalize.
Following the principle that scripted users should have as much
power as internal ones, it also exposes the nice interface of an
-Xrenormalize option. Some redundancy between builtin/merge and
builtin/merge-recursive is noticed but left for another topic.
Jonathan Nieder (6):
merge-trees: push choice to renormalize away from low level
merge-trees: let caller decide whether to renormalize
ll-merge: let caller decide whether to renormalize
rerere: migrate to parse-options API
rerere: let caller decide whether to renormalize
merge-recursive: add -Xrenormalize option
Documentation/merge-strategies.txt | 8 +++++
builtin/checkout.c | 13 ++++++++-
builtin/commit.c | 4 ++
builtin/merge-recursive.c | 2 +
builtin/merge.c | 24 +++++++++++----
builtin/rerere.c | 56 ++++++++++++++++++++---------------
builtin/revert.c | 18 ++++++++++-
cache.h | 1 -
environment.c | 1 -
ll-merge.c | 4 +-
ll-merge.h | 2 +-
merge-file.c | 2 +-
merge-recursive.c | 11 ++++--
merge-recursive.h | 1 +
rerere.c | 20 ++++++++----
rerere.h | 1 +
16 files changed, 117 insertions(+), 51 deletions(-)
--
1.7.2.1.544.ga752d.dirty
From: Jonathan Nieder <hidden> Date: 2016-06-15 22:49:14
The merge machinery decides whether to resmudge and clean relevant
entries based on the global merge_renormalize setting, which is set by
"git merge" based on its configuration (and left alone by other
commands).
A nicer interface would make that decision a parameter to merge_trees
so callers would pass in a choice made on a call-by-call basis.
Start by making blob_unchanged stop examining the merge_renormalize
global.
In other words, this change is a trivial no-op, but it brings us
closer to something good.
Cc: Eyvind Bernhardsen <redacted>
Signed-off-by: Jonathan Nieder <redacted>
---
merge-recursive.c | 9 +++++----
1 files changed, 5 insertions(+), 4 deletions(-)
@@ -1082,7 +1082,7 @@ static int blob_unchanged(const unsigned char *o_sha,if(sha_eq(o_sha,a_sha))return1;-if(!merge_renormalize)+if(!renormalize)return0;assert(o_sha&&a_sha);
@@ -1112,6 +1112,7 @@ static int process_entry(struct merge_options *o,print_index_entry("\tpath: ",entry);*/intclean_merge=1;+intnormalize=merge_renormalize;unsignedo_mode=entry->stages[1].mode;unsigneda_mode=entry->stages[2].mode;unsignedb_mode=entry->stages[3].mode;
@@ -1122,8 +1123,8 @@ static int process_entry(struct merge_options *o,if(o_sha&&(!a_sha||!b_sha)){/* Case A: Deleted in one */if((!a_sha&&!b_sha)||-(!b_sha&&blob_unchanged(o_sha,a_sha,path))||-(!a_sha&&blob_unchanged(o_sha,b_sha,path))){+(!b_sha&&blob_unchanged(o_sha,a_sha,normalize,path))||+(!a_sha&&blob_unchanged(o_sha,b_sha,normalize,path))){/* Deleted in both or deleted in one and*unchangedintheother*/if(a_sha)
From: Jonathan Nieder <hidden> Date: 2016-06-15 22:49:14
Add a "renormalize" option to struct merge_options so callers can
decide on a case-by-case basis whether the merge is likely to have
overlapped with a change in smudge/clean rules. The option defaults
to the global merge_renormalize setting for now.
No change in behavior intended.
Cc: Eyvind Bernhardsen <redacted>
Signed-off-by: Jonathan Nieder <redacted>
---
merge-recursive.c | 3 ++-
merge-recursive.h | 1 +
2 files changed, 3 insertions(+), 1 deletions(-)
From: Jonathan Nieder <hidden> Date: 2016-06-15 22:49:14
Add a "renormalize" parameter to ll_merge so callers can decide on a
case-by-case basis whether the merge is likely to have overlapped with
a change in smudge/clean rules. This reveals a few commands that have
not been taking that situation into account, though it does not fix
them.
No change in behavior intended.
Cc: Eyvind Bernhardsen <redacted>
Signed-off-by: Jonathan Nieder <redacted>
---
builtin/checkout.c | 6 +++++-
ll-merge.c | 4 ++--
ll-merge.h | 2 +-
merge-file.c | 2 +-
merge-recursive.c | 1 +
rerere.c | 17 +++++++++++------
6 files changed, 21 insertions(+), 11 deletions(-)
@@ -1,13 +1,16 @@#include"builtin.h"#include"cache.h"#include"dir.h"+#include"parse-options.h"#include"string-list.h"#include"rerere.h"#include"xdiff/xdiff.h"#include"xdiff-interface.h"-staticconstchargit_rerere_usage[]=-"git rerere [clear | status | diff | gc]";+staticconstchar*constrerere_usage[]={+"git rerere [clear | status | diff | gc]",+NULL,+};/* these values are days */staticintcutoff_noresolve=15;
@@ -103,25 +106,26 @@ static int diff_two(const char *file1, const char *label1,intcmd_rerere(intargc,constchar**argv,constchar*prefix){structstring_listmerge_rr={NULL,0,0,1};-inti,fd,flags=0;--if(2<argc){-if(!strcmp(argv[1],"-h"))-usage(git_rerere_usage);-if(!strcmp(argv[1],"--rerere-autoupdate"))-flags=RERERE_AUTOUPDATE;-elseif(!strcmp(argv[1],"--no-rerere-autoupdate"))-flags=RERERE_NOAUTOUPDATE;-if(flags){-argc--;-argv++;-}-}-if(argc<2)+inti,fd,autoupdate=-1,flags=0;++structoptionoptions[]={+OPT_SET_INT(0,"rerere-autoupdate",&autoupdate,+"register clean resolutions in index",1),+OPT_END(),+};++argc=parse_options(argc,argv,prefix,options,rerere_usage,0);++if(autoupdate==1)+flags=RERERE_AUTOUPDATE;+if(autoupdate==0)+flags=RERERE_NOAUTOUPDATE;++if(argc<1)returnrerere(flags);-if(!strcmp(argv[1],"forget")){-constchar**pathspec=get_pathspec(prefix,argv+2);+if(!strcmp(argv[0],"forget")){+constchar**pathspec=get_pathspec(prefix,argv+1);returnrerere_forget(pathspec);}
From: Jonathan Nieder <hidden> Date: 2016-06-15 22:49:14
Add a RERERE_RENORMALIZE flag to rerere so callers can decide
case-by-case whether the merge is likely to have overlapped with a
change in smudge/clean rules.
This is only a change in internal plumbing. Many callers do not have
a way to use that setting; this patch does not change that.
NEEDSWORK: this is a step in the wrong direction. rerere needs
an -s option to use an arbitrary merge strategy and a -X option to
pass arbitrary options to that driver. And maybe the options used
in a merge should be recorded somewhere to help rerere repeat it.
This treats the renormalize option specially anyway, to support the
existing "git merge" behavior of using -Xrenormalize in its rerere
call when configured to do so. It reluctant exposes that option in
the rerere command as --renormalize for the sake of experimentation,
too, but it does not advertise it.
Cc: Eyvind Bernhardsen <redacted>
Signed-off-by: Jonathan Nieder <redacted>
---
builtin/commit.c | 4 ++++
builtin/merge.c | 7 +++++--
builtin/rerere.c | 8 ++++++--
builtin/revert.c | 11 +++++++++--
rerere.c | 5 +++--
rerere.h | 1 +
6 files changed, 28 insertions(+), 8 deletions(-)
@@ -1377,6 +1377,10 @@ int cmd_commit(int argc, const char **argv, const char *prefix)"new_index file. Check that disk is not full or quota is\n""not exceeded, and then \"git reset HEAD\" to recover.");+/*+*NEEDSWORK:useRERERE_RENORMALIZEafteramergewith+*merge.renormalizeset+*/rerere(0);run_hook(get_index_file(),"post-commit",NULL);if(amend&&!no_post_rewrite){
@@ -822,7 +822,7 @@ static int finish_automerge(struct commit_list *common,staticintsuggest_conflicts(void){FILE*fp;-intpos;+intpos,flag;fp=fopen(git_path("MERGE_MSG"),"a");if(!fp)
@@ -841,7 +841,10 @@ static int suggest_conflicts(void)}}fclose(fp);-rerere(allow_rerere_auto);+flag=allow_rerere_auto;+if(merge_renormalize)+flag|=RERERE_RENORMALIZE;+rerere(flag);printf("Automatic merge failed; ""fix conflicts and then commit the result.\n");return1;
@@ -553,12 +553,13 @@ int setup_rerere(struct string_list *merge_rr, int flags)intrerere(intflags){structstring_listmerge_rr={NULL,0,0,1};-intfd;+intfd,renormalize;fd=setup_rerere(&merge_rr,flags);if(fd<0)return0;-returndo_plain_rerere(&merge_rr,fd,merge_renormalize);+renormalize=(flags&RERERE_RENORMALIZE)?1:0;+returndo_plain_rerere(&merge_rr,fd,renormalize);}staticintrerere_forget_one_path(constchar*path,structstring_list*rr)
From: Jonathan Nieder <hidden> Date: 2016-06-15 22:49:14
This adds a -Xrenormalize option to override the
merge.renormalize configuration. There is no way to override
it in the negative yet, but hopefully this suggests how.
The good part: this destroys the global merge_renormalize
variable.
The bad part: merge.renormalize is still not honored for
most commands. And it reveals lots of places that -X has not
been plumbed in.
Cc: Eyvind Bernhardsen <redacted>
Signed-off-by: Jonathan Nieder <redacted>
---
There you have it. As I said, this is pretty rough. Places I would
be happy to see this go:
- add tests and documentation
- -Xno-renormalize to override [merge] renormalize in the negative
- checkout -X and rerere -X
I probably will not have time to work on this topic again for a while,
so I would be very happy if someone takes ideas from it and takes it
somewhere else before I get a chance. If there are just minor
cleanups to do, I can maintain this series for a short while, though.
Thoughts?
Documentation/merge-strategies.txt | 8 ++++++++
builtin/checkout.c | 7 +++++++
builtin/merge-recursive.c | 2 ++
builtin/merge.c | 19 +++++++++++++------
builtin/revert.c | 7 +++++++
cache.h | 1 -
environment.c | 1 -
merge-recursive.c | 2 +-
8 files changed, 38 insertions(+), 9 deletions(-)
@@ -40,6 +40,14 @@ the other tree did, declaring 'our' history contains all that happened in it. theirs;; This is opposite of 'ours'.+renormalize;;+ This runs a virtual check-out and check-in of all three stages+ of a file when resolving a three-way merge. This option is+ meant to be used when merging branches with different clean+ filters or end-of-line normalization rules. See "Merging+ branches with differing checkin/checkout attributes" in+ linkgit:gitattributes[5] for details.+ subtree[=path];; This option is a more advanced form of 'subtree' strategy, where the strategy makes a guess on how two trees must be shifted to
@@ -53,7 +53,6 @@ enum object_creation_mode object_creation_mode = OBJECT_CREATION_MODE;char*notes_ref_name;intgrafts_replace_parents=1;intcore_apply_sparse_checkout;-intmerge_renormalize;/* Parallel index stat data preload? */intcore_preload_index=0;
From: Junio C Hamano <hidden> Date: 2016-06-15 22:49:14
Jonathan Nieder [off-list ref] writes:
Add a "renormalize" parameter to ll_merge so callers can decide on a
case-by-case basis whether the merge is likely to have overlapped with
a change in smudge/clean rules. This reveals a few commands that have
not been taking that situation into account, though it does not fix
them.
No change in behavior intended.
Looking nice so far; I however think a separate "renormalize" parameter
should be folded into "flag" word, perhaps with a few new macros to make
it easier to access renormalize, virtual-ancestor, and favor-side parts
it.
From: Junio C Hamano <hidden> Date: 2016-06-15 22:49:14
Jonathan Nieder [off-list ref] writes:
NEEDSWORK: this is a step in the wrong direction. rerere needs
an -s option to use an arbitrary merge strategy and a -X option to
pass arbitrary options to that driver.
If you are talking about "rerere", I strongly disagree with that.
1. "-s"
A "merge strategy" deals with the shape of the history (e.g. common
ancestor selection or synthesis for the purpose of 3-way merge) and the
shape of the trees (e.g. rename detection, subtree shifting). Starting
from two commits, it decides, based on the shape of the history, what
three trees your tree-level 3-way merge would operate on, and then
based on the shape of the trees, decides the pairing of blobs to run
the 3-way merge at the file-content level.
When "rerere" is invoked, a strategy already has dealt with all of the
above, and "rerere" only works on the (half-completed) result of that.
It is purely a three-way merge at the file-contents level and there is
no room for a "strategy" to get involved. It makes direct calls to
ll_merge() exactly for this reason.
2. "-X"
In the "merge -X<opt>" syntax, "-X" does not stand for "low level
details"; it just means "eXternal callout". It is there just to tell
the "merge" front-end "You may not understand this yourself, but the
program you call does, so just pass it along".
IOW, we want to be able to pass --<opt> through the frontend to the
backend, without having to tell all the options any possible backends
may know to the frontend. Just to make it easier to parse and tell
which ones are the front-end options and which ones are not (for both
machines and humans), we say -X<opt> to the frontend. Then "merge"
turns that -X<opt> into "--<opt>" and give it to the strategy.
A command that natively knows about an option is correct to take an
option as "--<option>", e.g. "merge--recursive --renormalize". You
trigger it by saying "merge -Xrenormalize" from the frontend.
To recap: it is absolutely the right thing to do to introduce a new
"rerere --renormalize" option, like your patch did. Doing anything else
IS a step in the wrong direction.
From: Jonathan Nieder <hidden> Date: 2016-06-15 22:49:15
Junio C Hamano wrote:
1. "-s"
[...]
When "rerere" is invoked,
[...]
It is purely a three-way merge at the file-contents level and there is
no room for a "strategy" to get involved. It makes direct calls to
ll_merge() exactly for this reason.
2. "-X"
[...]
A command that natively knows about an option is correct to take an
option as "--<option>", e.g. "merge--recursive --renormalize". You
trigger it by saying "merge -Xrenormalize" from the frontend.
To recap: it is absolutely the right thing to do to introduce a new
"rerere --renormalize" option, like your patch did. Doing anything else
IS a step in the wrong direction.
Once you say it like that, it makes sense. :)
Or rather, rerere should not take --renormalize at all. Patch 11 below
has an explanation.
Jonathan Nieder (12):
t6038 (merge.renormalize): style nitpicks
t6038 (merge.renormalize): try checkout -m and cherry-pick
t6038 (merge.renormalize): check that it can be turned off
merge-trees: push choice to renormalize away from low level
merge-trees: let caller decide whether to renormalize
Documentation/technical: document ll_merge
ll-merge: make flag easier to populate
ll-merge: let caller decide whether to renormalize
t4200 (rerere): modernize style
rerere: migrate to parse-options API
rerere: never renormalize
merge-recursive --renormalize
Documentation/merge-strategies.txt | 12 +
Documentation/technical/api-merge.txt | 73 ++++++
builtin/checkout.c | 11 +
builtin/merge-recursive.c | 4 +
builtin/merge.c | 19 ++-
builtin/rerere.c | 52 +++--
builtin/revert.c | 7 +
cache.h | 1 -
environment.c | 1 -
ll-merge.c | 11 +-
ll-merge.h | 15 ++
merge-recursive.c | 14 +-
merge-recursive.h | 1 +
rerere.c | 4 +
t/t4200-rerere.sh | 394 ++++++++++++++++++++++-----------
t/t6038-merge-text-auto.sh | 143 +++++++++++-
16 files changed, 588 insertions(+), 174 deletions(-)
create mode 100644 Documentation/technical/api-merge.txt
--
1.7.2.1.544.ga752d.dirty
From: Jonathan Nieder <hidden> Date: 2016-06-15 22:49:15
Some tweaks to simplify adding and running tests.
- Use test_tick for predictable, sort of realistic commit dates;
- Use test_cmp as "test_cmp expected actual" --- some crazy
content that was not expected should cause the test to fail;
- Remove and re-add all files at the start of each test so the
worktree is easier to think about;
- Avoid using cat where not necessary for clarity.
Signed-off-by: Jonathan Nieder <redacted>
---
t/t6038-merge-text-auto.sh | 52 +++++++++++++++++++++++++++++++++++++------
1 files changed, 44 insertions(+), 8 deletions(-)
@@ -1,58 +1,94 @@#!/bin/sh-test_description='CRLF merge conflict across text=auto change'+test_description='CRLFmergeconflictacrosstext=autochange++*[master]remove.gitattributes+![side]addlinefromb+--++[side]addlinefromb+*[master]remove.gitattributes+*[master^]addlinefroma+*[master~2]normalizefile+*+[side^]Initial+' ../test-lib.sh test_expect_successsetup'gitconfigmerge.renormalizetrue&&gitconfigcore.autocrlffalse&&+echofirstline|append_cr>file&&echofirstline>control_file&&echoonlyline>inert_file&&+gitaddfilecontrol_fileinert_file&&+test_tick&&gitcommit-m"Initial"&&gittaginitial&&gitbranchside&&+echo"* text=auto">.gitattributes&&touchfile&&gitadd.gitattributesfile&&+test_tick&&gitcommit-m"normalize file"&&+echosameline|append_cr>>file&&echosameline>>control_file&&gitaddfilecontrol_file&&+test_tick&&gitcommit-m"add line from a"&&gittaga&&+gitrm.gitattributes&&rmfile&&gitcheckoutfile&&+test_tick&&gitcommit-m"remove .gitattributes"&&gittagc&&+gitcheckoutside&&echosameline|append_cr>>file&&echosameline>>control_file&&gitaddfilecontrol_file&&+test_tick&&gitcommit-m"add line from b"&&gittagb&&+gitcheckoutmaster'-test_expect_success'Check merging after setting text=auto''+test_expect_success'Merge after setting text=auto''+cat<<-\EOF>expected&&+firstline+sameline+EOF++gitrm-fr.&&+rm-f.gitattributes&&gitreset--harda&&gitmergeb&&-catfile|remove_cr>file.temp&&-test_cmpfilefile.temp+test_cmpexpectedfile'-test_expect_success'Check merging addition of text=auto''+test_expect_success'Merge addition of text=auto''+cat<<-\EOF>expected&&+firstline+sameline+EOF++gitrm-fr.&&+rm-f.gitattributes&&gitreset--hardb&&gitmergea&&-catfile|remove_cr>file.temp&&-test_cmpfilefile.temp+test_cmpexpectedfile' test_expect_success'Test delete/normalize conflict''-gitcheckoutside&&+gitcheckout-fside&&+gitrm-fr.&&+rm-f.gitattributes&&gitreset--hardinitial&&gitrmfile&&gitcommit-m"remove file"&&
From: Jonathan Nieder <hidden> Date: 2016-06-15 22:49:15
checkout -m and cherry-pick have not been wired up to respect
merge.renormalize, but a naïve user would not know that.
Signed-off-by: Jonathan Nieder <redacted>
---
This series does not change these to expect success, but they
are part of the same thought process.
t/t6038-merge-text-auto.sh | 41 +++++++++++++++++++++++++++++++++++++++++
1 files changed, 41 insertions(+), 0 deletions(-)
@@ -85,6 +85,47 @@ test_expect_success 'Merge addition of text=auto' 'test_cmpexpectedfile'+test_expect_failure'checkout -m after setting text=auto''+cat<<-\EOF>expected&&+firstline+sameline+EOF++gitrm-fr.&&+rm-f.gitattributes&&+gitreset--hardinitial&&+gitcheckouta--.&&+gitcheckout-mb&&+test_cmpexpectedfile+'++test_expect_failure'checkout -m addition of text=auto''+cat<<-\EOF>expected&&+firstline+sameline+EOF++gitrm-fr.&&+rm-f.gitattributesfile&&+gitreset--hardinitial&&+gitcheckoutb--.&&+gitcheckout-ma&&+test_cmpexpectedfile+'++test_expect_failure'cherry-pick patch from after text=auto was added''+append_cr<<-\EOF>expected&&+firstline+sameline+EOF++gitrm-fr.&&+gitreset--hardb&&+test_must_failgitcherry-picka>err2>&1&&+grep"[Nn]othing added"err&&+test_cmpexpectedfile+'+ test_expect_success'Test delete/normalize conflict''gitcheckout-fside&&gitrm-fr.&&
From: Jonathan Nieder <hidden> Date: 2016-06-15 22:49:15
An unusual sort of person (not me) may even enjoy the conflicts
from line-ending changes. But more importantly, it is useful to
document that behavior so we can more easily notice if it changes
in an uncontrolled way while no one is watching.
Cc: Eyvind Bernhardsen <redacted>
Signed-off-by: Jonathan Nieder <redacted>
---
t/t6038-merge-text-auto.sh | 50 +++++++++++++++++++++++++++++++++++++++++++-
1 files changed, 49 insertions(+), 1 deletions(-)
@@ -119,6 +166,7 @@ test_expect_failure 'cherry-pick patch from after text=auto was added' 'samelineEOF+gitconfigmerge.renormalizetrue&&gitrm-fr.&&gitreset--hardb&&test_must_failgitcherry-picka>err2>&1&&
From: Jonathan Nieder <hidden> Date: 2016-06-15 22:49:15
The merge machinery decides whether to resmudge and clean relevant
entries based on the global merge_renormalize setting, which is set by
"git merge" based on its configuration (and left alone by other
commands).
A nicer interface would make that decision a parameter to merge_trees
so callers would pass in a choice made on a call-by-call basis.
Start by making blob_unchanged stop examining the merge_renormalize
global.
In other words, this change is a trivial no-op, but it brings us
closer to something good.
Cc: Eyvind Bernhardsen <redacted>
Signed-off-by: Jonathan Nieder <redacted>
---
Unchanged from v1.
merge-recursive.c | 9 +++++----
1 files changed, 5 insertions(+), 4 deletions(-)
@@ -1082,7 +1082,7 @@ static int blob_unchanged(const unsigned char *o_sha,if(sha_eq(o_sha,a_sha))return1;-if(!merge_renormalize)+if(!renormalize)return0;assert(o_sha&&a_sha);
@@ -1112,6 +1112,7 @@ static int process_entry(struct merge_options *o,print_index_entry("\tpath: ",entry);*/intclean_merge=1;+intnormalize=merge_renormalize;unsignedo_mode=entry->stages[1].mode;unsigneda_mode=entry->stages[2].mode;unsignedb_mode=entry->stages[3].mode;
@@ -1122,8 +1123,8 @@ static int process_entry(struct merge_options *o,if(o_sha&&(!a_sha||!b_sha)){/* Case A: Deleted in one */if((!a_sha&&!b_sha)||-(!b_sha&&blob_unchanged(o_sha,a_sha,path))||-(!a_sha&&blob_unchanged(o_sha,b_sha,path))){+(!b_sha&&blob_unchanged(o_sha,a_sha,normalize,path))||+(!a_sha&&blob_unchanged(o_sha,b_sha,normalize,path))){/* Deleted in both or deleted in one and*unchangedintheother*/if(a_sha)
From: Jonathan Nieder <hidden> Date: 2016-06-15 22:49:15
Add a "renormalize" option to struct merge_options so callers can
decide on a case-by-case basis whether the merge is likely to have
overlapped with a change in smudge/clean rules. The option defaults
to the global merge_renormalize setting for now.
No change in behavior intended.
Cc: Eyvind Bernhardsen <redacted>
Signed-off-by: Jonathan Nieder <redacted>
---
The renormalize option now is advertized to only require one bit.
Maybe some optimizer will take advantage of that.
merge-recursive.c | 3 ++-
merge-recursive.h | 1 +
2 files changed, 3 insertions(+), 1 deletions(-)
@@ -0,0 +1,70 @@+merge API+=========++The merge API helps a program to reconcile two competing sets of+improvements to some files (e.g., unregistered changes from the work+tree versus changes involved in switching to a new branch), reporting+conflicts if found. The library called through this API is+responsible for a few things.++ * determining which trees to merge (recursive ancestor consolidation);++ * lining up corresponding files in the trees to be merged (rename+ detection, subtree shifting), reporting edge cases like add/add+ and rename/rename conflicts to the user;++ * performing a three-way merge of corresponding files, taking+ path-specific merge drivers (specified in `.gitattributes`)+ into account.++Low-level (single file) merge+-----------------------------++`ll_merge`::++ Perform a three-way single-file merge in core. This is+ a thin wrapper around `xdl_merge` that takes the path and+ any merge backend specified in `.gitattributes` or+ `.git/info/attributes` into account. Returns 0 for a+ clean merge.++The caller:++1. allocates an mmbuffer_t variable for the result;+2. allocates and fills variables with the file's original content+ and two modified versions (using `read_mmfile`, for example);+3. calls ll_merge();+4. reads the output from result_buf.ptr and result_buf.size;+5. releases buffers when finished (free(ancestor.ptr); free(ours.ptr);+ free(theirs.ptr); free(result_buf.ptr);).++If the modifications do not merge cleanly, `ll_merge` will return a+nonzero value and `result_buf` will generally include a description of+the conflict bracketed by markers such as the traditional `<<<<<<<`+and `>>>>>>>`.++The `ancestor_label`, `our_label`, and `their_label` parameters are+used to label the different sides of a conflict if the merge driver+supports this.++The `flag` parameter is a bitfield:++ - The least significant bit indicates whether this is an internal+ merge to consolidate ancestors for a recursive merge.++ - The next two bits allow local conflicts to be automatically+ resolved in favor of one side or the other (as in 'git merge-file'+ `--ours`/`--theirs`/`--union` for 01, 10, and 11, respectively).++Everything else+---------------++Talk about <merge-recursive.h> and merge_file():++ - merge_trees() to merge with rename detection+ - merge_recursive() for ancestor consolidation+ - try_merge_command() for other strategies+ - conflict format+ - merge options++(Daniel, Miklos, Stephan, JC)
From: Jonathan Nieder <hidden> Date: 2016-06-15 22:49:15
ll_merge() takes its options in a flag word, which has a few
advantages:
- options flags can be cheaply passed around in registers, while
an option struct passed by pointer cannot;
- callers can easily pass 0 without trouble for no options,
while an option struct passed by value would not allow that.
The downside is that code to populate and access the flag word can be
somewhat opaque. Mitigate that with a few macros.
Cc: Avery Pennarun <redacted>
Cc: Bert Wesarg <redacted>
Signed-off-by: Jonathan Nieder <redacted>
---
Documentation/technical/api-merge.txt | 11 +++++++----
ll-merge.c | 9 +++++----
ll-merge.h | 14 ++++++++++++++
merge-recursive.c | 3 ++-
4 files changed, 28 insertions(+), 9 deletions(-)
@@ -49,12 +49,15 @@ supports this. The `flag` parameter is a bitfield:- - The least significant bit indicates whether this is an internal- merge to consolidate ancestors for a recursive merge.+ - The `LL_OPT_VIRTUAL_ANCESTOR` bit indicates whether this is an+ internal merge to consolidate ancestors for a recursive merge.- - The next two bits allow local conflicts to be automatically+ - The `LL_OPT_FAVOR_MASK` bits allow local conflicts to be automatically resolved in favor of one side or the other (as in 'git merge-file'- `--ours`/`--theirs`/`--union` for 01, 10, and 11, respectively).+ `--ours`/`--theirs`/`--union`).+ They can be populated by `create_ll_flag`, whose argument can be+ `XDL_MERGE_FAVOR_OURS`, `XDL_MERGE_FAVOR_THEIRS`, or+ `XDL_MERGE_FAVOR_UNION`. Everything else ---------------
@@ -79,7 +79,7 @@ static int ll_xdl_merge(const struct ll_merge_driver *drv_unused,memset(&xmp,0,sizeof(xmp));xmp.level=XDL_MERGE_ZEALOUS;-xmp.favor=(flag>>1)&03;+xmp.favor=ll_opt_favor(flag);if(git_xmerge_style>=0)xmp.style=git_xmerge_style;if(marker_size>0)
@@ -99,7 +99,8 @@ static int ll_union_merge(const struct ll_merge_driver *drv_unused,intflag,intmarker_size){/* Use union favor */-flag=(flag&1)|(XDL_MERGE_FAVOR_UNION<<1);+flag=(flag&LL_OPT_VIRTUAL_ANCESTOR)|+create_ll_flag(XDL_MERGE_FAVOR_UNION);returnll_xdl_merge(drv_unused,result,path_unused,orig,NULL,src1,NULL,src2,NULL,flag,marker_size);
@@ -342,7 +343,7 @@ int ll_merge(mmbuffer_t *result_buf,constchar*ll_driver_name=NULL;intmarker_size=DEFAULT_CONFLICT_MARKER_SIZE;conststructll_merge_driver*driver;-intvirtual_ancestor=flag&01;+intvirtual_ancestor=flag&LL_OPT_VIRTUAL_ANCESTOR;if(merge_renormalize){normalize_file(ancestor,path);
From: Jonathan Nieder <hidden> Date: 2016-06-15 22:49:15
Add a “renormalize” bit to the ll-merge options word so callers can
decide on a case-by-case basis whether the merge is likely to have
overlapped with a change in smudge/clean rules.
This reveals a few commands that have not been taking that situation
into account, though it does not fix them.
No functional change intended.
Cc: Eyvind Bernhardsen <redacted>
Improved-by: Junio C Hamano [off-list ref]
Signed-off-by: Jonathan Nieder <redacted>
---
Uses the flag word now.
The new option is not exposed through “git merge-file”, just like
the virtual-ancestor option isn’t, but that is only from laziness.
Exposing it would make tests this easier, too.
Some worries:
- "checkout -m" does not do convert_to_worktree() but it should;
- "rerere forget" has not been introduced to the wonderful world
of smudge filters, either.
builtin/checkout.c | 4 ++++
ll-merge.c | 6 +++---
ll-merge.h | 1 +
merge-recursive.c | 1 +
rerere.c | 15 ++++++++++-----
5 files changed, 19 insertions(+), 8 deletions(-)
@@ -99,8 +99,8 @@ static int ll_union_merge(const struct ll_merge_driver *drv_unused,intflag,intmarker_size){/* Use union favor */-flag=(flag&LL_OPT_VIRTUAL_ANCESTOR)|-create_ll_flag(XDL_MERGE_FAVOR_UNION);+flag&=~LL_OPT_FAVOR_MASK;+flag|=create_ll_flag(XDL_MERGE_FAVOR_UNION);returnll_xdl_merge(drv_unused,result,path_unused,orig,NULL,src1,NULL,src2,NULL,flag,marker_size);
@@ -345,7 +345,7 @@ int ll_merge(mmbuffer_t *result_buf,conststructll_merge_driver*driver;intvirtual_ancestor=flag&LL_OPT_VIRTUAL_ANCESTOR;-if(merge_renormalize){+if(flag&LL_OPT_RENORMALIZE){normalize_file(ancestor,path);normalize_file(ours,path);normalize_file(theirs,path);
From: Jonathan Nieder <hidden> Date: 2016-06-15 22:49:15
Guard all test code with test_expect_success to make the
script easier to follow. While at it, pick some other nits:
- use test_tick (more than we have to, to be realistic);
- 'single quotes' and \escaped HERE documents where possible
simplify review for escaping problems;
- omit whitespace after >redirection operators for
consistency with other tests;
- use "update-index --refresh" instead of testing that
"ls-files -u" output is empty, since the former produces
nicer output on failure;
- compare to expected nonempty "ls-files -u" output instead
of counting lines when it is expected to be nonempty.
Signed-off-by: Jonathan Nieder <redacted>
---
t/t4200-rerere.sh | 303 +++++++++++++++++++++++++++++++----------------------
1 files changed, 179 insertions(+), 124 deletions(-)
@@ -4,237 +4,292 @@#test_description='gitrerere++![fifth]version1+![first]first+![fourth]version1+![master]initial+![second]preferfirstoversecond+![third]version2+------++[third]version2++[fifth]version1++[fourth]version1++++[third^]third+-[second]preferfirstoversecond+++[first]first++[second^]second+++++++[master]initial' ../test-lib.sh-test_expect_success'setup'"-cat>a1<<-EOF&&+test_expect_success'setup''+cat>a1<<-\EOF&&Sometitle==========-Whether'tisnoblerinthemindtosuffer+Whether'\''tisnoblerinthemindtosufferTheslingsandarrowsofoutrageousfortune,Ortotakearmsagainstaseaoftroubles,Andbyopposingendthem?Todie:tosleep;Nomore;andbyasleeptosayweendTheheart-acheandthethousandnaturalshocks-Thatfleshisheirto,'tisaconsummation-Devoutlytobewish'd.+Thatfleshisheirto,'\''tisaconsummation+Devoutlytobewish'\''d.EOFgitadda1&&+test_tick&&gitcommit-q-a-minitial&&-gitcheckout-bfirst&&-cat>>a1<<-EOF&&+cat>>a1<<-\EOF&&Sometitle==========Todie,tosleep;-Tosleep:perchancetodream:ay,there'stherub;+Tosleep:perchancetodream:ay,there'\''stherub;ForinthatsleepofdeathwhatdreamsmaycomeWhenwehaveshuffledoffthismortalcoil,-Mustgiveuspause:there'stherespect+Mustgiveuspause:there'\''stherespectThatmakescalamityofsolonglife;EOF++gitcheckout-bfirst&&+test_tick&&gitcommit-q-a-mfirst&&gitcheckout-bsecondmaster&&gitshowfirst:a1|-sed-e's/To die, t/To die! T/'-e's/Some title/Some Title/'>a1&&-echo'* END *'>>a1&&+sed-e"s/To die, t/To die! T/"-e"s/Some title/Some Title/">a1&&+echo"* END *">>a1&&+test_tick&&gitcommit-q-a-msecond-"+' test_expect_success'nothing recorded without rerere''-(rm-rf.git/rr-cache;gitconfigrerere.enabledfalse)&&+rm-rf.git/rr-cache&&+gitconfigrerere.enabledfalse&&test_must_failgitmergefirst&&!test-d.git/rr-cache'-# activate rerere, old style-test_expect_success'conflicting merge''+test_expect_success'activate rerere, old style (conflicting merge)''gitreset--hard&&mkdir.git/rr-cache&&-gitconfig--unsetrerere.enabled&&-test_must_failgitmergefirst-'+test_might_failgitconfig--unsetrerere.enabled&&+test_must_failgitmergefirst&&-sha1=$(perl-pe's/ .*//'.git/MERGE_RR)-rr=.git/rr-cache/$sha1-test_expect_success'recorded preimage'"grep ^=======$$rr/preimage"+sha1=$(perl-pe"s/ .*//".git/MERGE_RR)&&+rr=.git/rr-cache/$sha1&&+grep"^=======\$"$rr/preimage&&+!test-f$rr/postimage&&+!test-f$rr/thisimage+' test_expect_success'rerere.enabled works, too''rm-rf.git/rr-cache&&gitconfigrerere.enabledtrue&&gitreset--hard&&test_must_failgitmergefirst&&++sha1=$(perl-pe"s/ .*//".git/MERGE_RR)&&+rr=.git/rr-cache/$sha1&&grep^=======$$rr/preimage'-test_expect_success'no postimage or thisimage yet'\-"test ! -f $rr/postimage -a ! -f $rr/thisimage"+test_expect_success'set up rr-cache''+rm-rf.git/rr-cache&&+gitconfigrerere.enabledtrue&&+gitreset--hard&&+test_must_failgitmergefirst&&+sha1=$(perl-pe"s/ .*//".git/MERGE_RR)&&+rr=.git/rr-cache/$sha1+'-test_expect_success'preimage has right number of lines''+test_expect_success'rr-cache looks sane''+# no postimage or thisimage yet+!test-f$rr/postimage&&+!test-f$rr/thisimage&&+# preimage has right number of linescnt=$(sed-ne"/^<<<<<<</,/^>>>>>>>/p"$rr/preimage|wc-l)&&+echo$cnt&&test$cnt=13-'-gitshowfirst:a1>a1--cat>expect<<EOF----a/a1-+++b/a1-@@-1,4+1,4@@--SomeTitle-+Sometitle-==========-Whether'tisnoblerinthemindtosuffer-Theslingsandarrowsofoutrageousfortune,-@@-8,21+8,11@@-Theheart-acheandthethousandnaturalshocks-Thatfleshisheirto,'tisaconsummation-Devoutlytobewish'd.--<<<<<<<--SomeTitle--==========--Todie!Tosleep;--=======-Sometitle-==========-Todie,tosleep;-->>>>>>>-Tosleep:perchancetodream:ay,there'stherub;-Forinthatsleepofdeathwhatdreamsmaycome-Whenwehaveshuffledoffthismortalcoil,-Mustgiveuspause:there'stherespect-Thatmakescalamityofsolonglife;--<<<<<<<--=======--*END*-->>>>>>>-EOF-gitrererediff>out--test_expect_success'rerere diff''test_cmp expect out'--cat>expect<<EOF-a1-EOF--gitrererestatus>out+test_expect_success'rerere diff''+gitshowfirst:a1>a1&&+cat>expect<<-\EOF&&+---a/a1++++b/a1+@@-1,4+1,4@@+-SomeTitle++Sometitle+==========+Whether'\''tisnoblerinthemindtosuffer+Theslingsandarrowsofoutrageousfortune,+@@-8,21+8,11@@+Theheart-acheandthethousandnaturalshocks+Thatfleshisheirto,'\''tisaconsummation+Devoutlytobewish'\''d.+-<<<<<<<+-SomeTitle+-==========+-Todie!Tosleep;+-=======+Sometitle+==========+Todie,tosleep;+->>>>>>>+Tosleep:perchancetodream:ay,there'\''stherub;+Forinthatsleepofdeathwhatdreamsmaycome+Whenwehaveshuffledoffthismortalcoil,+Mustgiveuspause:there'\''stherespect+Thatmakescalamityofsolonglife;+-<<<<<<<+-=======+-*END*+->>>>>>>+EOF+gitrererediff>out&&+test_cmpexpectout+'-test_expect_success'rerere status''test_cmp expect out'+test_expect_success'rerere status''+echoa1>expect&&+gitrererestatus>out&&+test_cmpexpectout+'-test_expect_success'commit succeeds'\-"git commit -q -a -m 'prefer first over second'"+test_expect_success'first postimage wins''+gitshowfirst:a1|sed"s/To die: t/To die! T/">expect&&-test_expect_success'recorded postimage'"test -f $rr/postimage"+gitcommit-q-a-m"prefer first over second"&&+test-f$rr/postimage&&-test_expect_success'another conflicting merge''gitcheckout-bthirdmaster&&-gitshowsecond^:a1|sed"s/To die: t/To die! T/">a1&&+gitshowsecond^:a1|sed"s/To die: t/To die! T/">a1&&gitcommit-q-a-mthird&&-test_must_failgitpull.first-'--gitshowfirst:a1|sed's/To die: t/To die! T/'>expect-test_expect_success'rerere kicked in'"! grep ^=======$ a1"--test_expect_success'rerere prefers first change''test_cmp a1 expect'--rm$rr/postimage-echo"$sha1 a1"|perl-pe'y/\012/\000/'>.git/MERGE_RR--test_expect_success'rerere clear''git rerere clear'--test_expect_success'clear removed the directory'"test ! -d $rr"-mkdir$rr-echoHello>$rr/preimage-echoWorld>$rr/postimage--sha2=4000000000000000000000000000000000000000-rr2=.git/rr-cache/$sha2-mkdir$rr2-echoHello>$rr2/preimage+test_must_failgitpull.first&&+# rerere kicked in+!grep"^=======\$"a1&&+test_cmpexpecta1+'-almost_15_days_ago=$((60-15*86400))-just_over_15_days_ago=$((-1-15*86400))-almost_60_days_ago=$((60-60*86400))-just_over_60_days_ago=$((-1-60*86400))+test_expect_success'rerere clear''+rm$rr/postimage&&+echo"$sha1 a1"|perl-pe"y/\012/\000/">.git/MERGE_RR&&+gitrerereclear&&+!test-d$rr+'-test-chmtime=$almost_60_days_ago$rr/preimage-test-chmtime=$almost_15_days_ago$rr2/preimage+test_expect_success'set up for garbage collection tests''+mkdir-p$rr&&+echoHello>$rr/preimage&&+echoWorld>$rr/postimage&&-test_expect_success'garbage collection (part1)''git rerere gc'+sha2=4000000000000000000000000000000000000000&&+rr2=.git/rr-cache/$sha2&&+mkdir$rr2&&+echoHello>$rr2/preimage&&-test_expect_success'young records still live'\-"test -f $rr/preimage && test -f $rr2/preimage"+almost_15_days_ago=$((60-15*86400))&&+just_over_15_days_ago=$((-1-15*86400))&&+almost_60_days_ago=$((60-60*86400))&&+just_over_60_days_ago=$((-1-60*86400))&&-test-chmtime=$just_over_60_days_ago$rr/preimage-test-chmtime=$just_over_15_days_ago$rr2/preimage+test-chmtime=$almost_60_days_ago$rr/preimage&&+test-chmtime=$almost_15_days_ago$rr2/preimage+'-test_expect_success'garbage collection (part2)''git rerere gc'+test_expect_success'garbage collection preserves young records''+gitrereregc&&+test-f$rr/preimage&&+test-f$rr2/preimage+'-test_expect_success'old records rest in peace'\-"test ! -f $rr/preimage && test ! -f $rr2/preimage"+test_expect_success'old records rest in peace''+test-chmtime=$just_over_60_days_ago$rr/preimage&&+test-chmtime=$just_over_15_days_ago$rr2/preimage&&+gitrereregc&&+!test-f$rr/preimage&&+!test-f$rr2/preimage+'-test_expect_success'file2 added differently in two branches''+test_expect_success'setup: file2 added differently in two branches''gitreset--hard&&+gitcheckout-bfourth&&-echoHallo>file2&&+echoHallo>file2&&gitaddfile2&&+test_tick&&gitcommit-mversion1&&+gitcheckoutthird&&-echoBello>file2&&+echoBello>file2&&gitaddfile2&&+test_tick&&gitcommit-mversion2&&+test_must_failgitmergefourth&&-echoCello>file2&&+echoCello>file2&&gitaddfile2&&gitcommit-mresolution' test_expect_success'resolution was recorded properly''+echoCello>expected&&+gitreset--hardHEAD~2&&gitcheckout-bfifth&&-echoHallo>file3&&++echoHallo>file3&&gitaddfile3&&+test_tick&&gitcommit-mversion1&&+gitcheckoutthird&&-echoBello>file3&&+echoBello>file3&&gitaddfile3&&+test_tick&&gitcommit-mversion2&&gittagversion2&&+test_must_failgitmergefifth&&-testCello="$(catfile3)"&&-test0!=$(gitls-files-u|wc-l)+test_cmpexpectedfile3&&+test_must_failgitupdate-index--refresh' test_expect_success'rerere.autoupdate''-gitconfigrerere.autoupdatetrue+gitconfigrerere.autoupdatetrue&&gitreset--hard&&gitcheckoutversion2&&test_must_failgitmergefifth&&-test0=$(gitls-files-u|wc-l)+gitupdate-index--refresh' test_expect_success'merge --rerere-autoupdate''-gitconfig--unsetrerere.autoupdate+test_might_failgitconfig--unsetrerere.autoupdate&&gitreset--hard&&gitcheckoutversion2&&test_must_failgitmerge--rerere-autoupdatefifth&&-test0=$(gitls-files-u|wc-l)+gitupdate-index--refresh' test_expect_success'merge --no-rerere-autoupdate''-gitconfigrerere.autoupdatetrue+headblob=$(gitrev-parseversion2:file3)&&+mergeblob=$(gitrev-parsefifth:file3)&&+cat>expected<<-EOF&&+100644$headblob2file3+100644$mergeblob3file3+EOF++gitconfigrerere.autoupdatetrue&&gitreset--hard&&gitcheckoutversion2&&test_must_failgitmerge--no-rerere-autoupdatefifth&&-test2=$(gitls-files-u|wc-l)+gitls-files-u>actual&&+test_cmpexpectedactual' test_done
From: Jonathan Nieder <hidden> Date: 2016-06-15 22:49:15
Signed-off-by: Jonathan Nieder <redacted>
---
The tests don’t pass for me with the old implementation; I think "2 < argc"
should have been "2 <= argc". Well, no use dwelling in the past.
builtin/rerere.c | 52 ++++++++++++++++--------------
t/t4200-rerere.sh | 91 +++++++++++++++++++++++++++++++++++++++++++++++++++++
2 files changed, 119 insertions(+), 24 deletions(-)
@@ -1,13 +1,16 @@#include"builtin.h"#include"cache.h"#include"dir.h"+#include"parse-options.h"#include"string-list.h"#include"rerere.h"#include"xdiff/xdiff.h"#include"xdiff-interface.h"-staticconstchargit_rerere_usage[]=-"git rerere [clear | status | diff | gc]";+staticconstchar*constrerere_usage[]={+"git rerere [clear | status | diff | gc]",+NULL,+};/* these values are days */staticintcutoff_noresolve=15;
@@ -103,25 +106,26 @@ static int diff_two(const char *file1, const char *label1,intcmd_rerere(intargc,constchar**argv,constchar*prefix){structstring_listmerge_rr={NULL,0,0,1};-inti,fd,flags=0;--if(2<argc){-if(!strcmp(argv[1],"-h"))-usage(git_rerere_usage);-if(!strcmp(argv[1],"--rerere-autoupdate"))-flags=RERERE_AUTOUPDATE;-elseif(!strcmp(argv[1],"--no-rerere-autoupdate"))-flags=RERERE_NOAUTOUPDATE;-if(flags){-argc--;-argv++;-}-}-if(argc<2)+inti,fd,autoupdate=-1,flags=0;++structoptionoptions[]={+OPT_SET_INT(0,"rerere-autoupdate",&autoupdate,+"register clean resolutions in index",1),+OPT_END(),+};++argc=parse_options(argc,argv,prefix,options,rerere_usage,0);++if(autoupdate==1)+flags=RERERE_AUTOUPDATE;+if(autoupdate==0)+flags=RERERE_NOAUTOUPDATE;++if(argc<1)returnrerere(flags);-if(!strcmp(argv[1],"forget")){-constchar**pathspec=get_pathspec(prefix,argv+2);+if(!strcmp(argv[0],"forget")){+constchar**pathspec=get_pathspec(prefix,argv+1);returnrerere_forget(pathspec);}
From: Jonathan Nieder <hidden> Date: 2016-06-15 22:49:15
plain rerere performs three tasks; let us consider how the new
merge.renormalize option should apply to each.
After an unsuccessful merge, rerere records conflict hunks from the
work tree under .git/rr-cache. If the merge was performed with
merge.renormalize enabled, both sides of the conflict hunk use the
current work tree’s end-of-line and smudge rules; there is not really
much of a choice.
After a successful manual resolution, rerere records the postimage.
Here, also, the file will be in the current work tree’s canonical
format and there is not much to do about it.
When encountering that conflict again, merge looks up the preimage
and postimage using the conflict hunk as a key and runs a three-way
merge to apply that resolution to the work tree. Since the conflict
hunk used the current work tree’s canonical format, chances are the
version in the work tree, the preimage, and the postimage will, too.
In fact using the merge.renormalize machinery is exactly the wrong
thing to do, since its result has been run through convert_to_git
and therefore is not suitable for writing to the work tree.
The only affected caller is "git merge".
NEEDSWORK: lacks test
Cc: Eyvind Bernhardsen <redacted>
Signed-off-by: Jonathan Nieder <redacted>
---
rerere.c | 11 +++++------
1 files changed, 5 insertions(+), 6 deletions(-)
@@ -365,7 +365,7 @@ static int find_conflict(struct string_list *conflict)return0;}-staticintmerge(constchar*name,intrenormalize,constchar*path)+staticintmerge(constchar*name,constchar*path){intret;mmfile_tcur={NULL,0},base={NULL,0},other={NULL,0};
@@ -380,8 +380,7 @@ static int merge(const char *name, int renormalize, const char *path)ret=1;gotoout;}-ret=ll_merge(&result,path,&base,NULL,&cur,"",&other,"",-renormalize?LL_OPT_RENORMALIZE:0);+ret=ll_merge(&result,path,&base,NULL,&cur,"",&other,"",0);if(!ret){FILE*f=fopen(path,"w");if(!f)
@@ -429,7 +428,7 @@ static int update_paths(struct string_list *update)returnstatus;}-staticintdo_plain_rerere(structstring_list*rr,intfd,intrenormalize)+staticintdo_plain_rerere(structstring_list*rr,intfd){structstring_listconflict={NULL,0,0,1};structstring_listupdate={NULL,0,0,1};
@@ -474,7 +473,7 @@ static int do_plain_rerere(struct string_list *rr, int fd, int renormalize)constchar*name=(constchar*)rr->items[i].util;if(has_rerere_resolution(name)){-if(!merge(name,renormalize,path)){+if(!merge(name,path)){if(rerere_autoupdate)string_list_insert(path,&update);fprintf(stderr,
@@ -558,7 +557,7 @@ int rerere(int flags)fd=setup_rerere(&merge_rr,flags);if(fd<0)return0;-returndo_plain_rerere(&merge_rr,fd,merge_renormalize);+returndo_plain_rerere(&merge_rr,fd);}staticintrerere_forget_one_path(constchar*path,structstring_list*rr)
From: Jonathan Nieder <hidden> Date: 2016-06-15 22:49:15
Teach "git merge-recursive" a --renormalize option to enable the
merge.renormalize configuration. The --no-renormalize option can
be used to override it in the negative.
So in the future, you might be able to, e.g.:
git checkout -m -Xrenormalize otherbranch
or
git revert -Xrenormalize otherpatch
or
git pull --rebase -Xrenormalize
The bad part: merge.renormalize is still not honored for most
commands. And it reveals lots of places that -X has not been plumbed
in (so we get "git merge -Xrenormalize" but not much else).
NEEDSWORK: tests
Cc: Eyvind Bernhardsen <redacted>
Signed-off-by: Jonathan Nieder <redacted>
---
- Commit message is rewritten
- Introduce --no-renormalize, too
- Adapt to rerere’s new lack of support for --renormalize
That’s it for this round. Thanks again.
Documentation/merge-strategies.txt | 12 ++++++++++++
builtin/checkout.c | 7 +++++++
builtin/merge-recursive.c | 4 ++++
builtin/merge.c | 19 ++++++++++++++-----
builtin/revert.c | 7 +++++++
cache.h | 1 -
environment.c | 1 -
merge-recursive.c | 2 +-
8 files changed, 45 insertions(+), 8 deletions(-)
@@ -40,6 +40,18 @@ the other tree did, declaring 'our' history contains all that happened in it. theirs;; This is opposite of 'ours'.+renormalize;;+ This runs a virtual check-out and check-in of all three stages+ of a file when resolving a three-way merge. This option is+ meant to be used when merging branches with different clean+ filters or end-of-line normalization rules. See "Merging+ branches with differing checkin/checkout attributes" in+ linkgit:gitattributes[5] for details.++no-renormalize;;+ Disables the `renormalize` option. This overrides the+ `merge.renormalize` configuration variable.+ subtree[=path];; This option is a more advanced form of 'subtree' strategy, where the strategy makes a guess on how two trees must be shifted to
@@ -53,7 +53,6 @@ enum object_creation_mode object_creation_mode = OBJECT_CREATION_MODE;char*notes_ref_name;intgrafts_replace_parents=1;intcore_apply_sparse_checkout;-intmerge_renormalize;/* Parallel index stat data preload? */intcore_preload_index=0;
On Thu, Aug 5, 2010 at 11:09, Jonathan Nieder [off-list ref] wrote:
Some tweaks to simplify adding and running tests.
- Use test_tick for predictable, sort of realistic commit dates;
git checkout side &&
echo same line | append_cr >>file &&
echo same line >>control_file &&
git add file control_file &&
+ test_tick &&
git commit -m "add line from b" &&
git tag b &&
FWIW this looks like it could use Dmitry's "test-lib.sh: introduce 4th
argument to test_commit() specifying a tag name" patch. Maybe that
goes for most of these git add/tick/commit/tag combos, i.e. they don't
really need $commit_message != $tagname.
From: Jonathan Nieder <hidden> Date: 2016-06-15 22:49:15
Ævar Arnfjörð Bjarmason wrote:
On Thu, Aug 5, 2010 at 11:09, Jonathan Nieder [off-list ref] wrote:
quoted
git checkout side &&
echo same line | append_cr >>file &&
echo same line >>control_file &&
git add file control_file &&
+ test_tick &&
git commit -m "add line from b" &&
git tag b &&
FWIW this looks like it could use Dmitry's "test-lib.sh: introduce 4th
argument to test_commit() specifying a tag name" patch.
In this example I am not confident the file has content suitable for
echo.
The discussion brings to mind something[1] I thought wise in a
different context:
“I mentioned earlier that UNIX was not especially suited
to applications involving vast quantities of data. The
reason is this: files are limited in size to 64K bytes.
The reason for this is not particularly defensible, but
it has to do with the fact that the PDP-11 word size is
16 bits.
There are a couple of ways around this problem. One of
them is simply to split one large logical file into
several smaller actual files. This approach works for a
while. The limitation here comes from the fact that
directories are searched in a linear fashion. Thus if the
are a vast number of files, it can become quite
time-consuming tosearch directories to find the files
they contain. We have not noticed this to be a problem,
so far, it is only a worry.
Another way around the small file size is to use a disk
as a special file. For various reasons, when an entire
disk drive is accessed as a special file, the size
limitation does not occur. Thus one can set up a program
which manages its own data-- in effect is its own,
special-purpose file system-- and expect reasonable results.
This again bears on the general versus special purpose
system: it probably is more efficient anyway to do your
own data management, provided the extra labor is worth
the cost.”
Of course the tradeoffs are completely different here but it is worth
bearing in mind the underlying process: sometimes a too general
facility only gets in the way unless all the facets of how it should
be used have been carefully understood (i.e., good interfaces
sometimes evolve by excluding the special cases until the missed
benefit from not including them is overwhelming).
Sorry for the ramble. Another way to say it: I am happy to see
test_commit be made more useful, but if extra-weird cases do not fit
it, please do not take that as a failing.
[1] http://cm.bell-labs.com/cm/cs/who/dmr/notes.html
On Thu, Aug 5, 2010 at 13:17, Jonathan Nieder [off-list ref] wrote:
quoted hunk
ll_merge() takes its options in a flag word, which has a few
advantages:
- options flags can be cheaply passed around in registers, while
an option struct passed by pointer cannot;
- callers can easily pass 0 without trouble for no options,
while an option struct passed by value would not allow that.
The downside is that code to populate and access the flag word can be
somewhat opaque. Mitigate that with a few macros.
Cc: Avery Pennarun <redacted>
Cc: Bert Wesarg <redacted>
Signed-off-by: Jonathan Nieder <redacted>
---
Documentation/technical/api-merge.txt | 11 +++++++----
ll-merge.c | 9 +++++----
ll-merge.h | 14 ++++++++++++++
merge-recursive.c | 3 ++-
4 files changed, 28 insertions(+), 9 deletions(-)
The `flag` parameter is a bitfield:
- - The least significant bit indicates whether this is an internal
- merge to consolidate ancestors for a recursive merge.
+ - The `LL_OPT_VIRTUAL_ANCESTOR` bit indicates whether this is an
+ internal merge to consolidate ancestors for a recursive merge.
- - The next two bits allow local conflicts to be automatically
+ - The `LL_OPT_FAVOR_MASK` bits allow local conflicts to be automatically
resolved in favor of one side or the other (as in 'git merge-file'
- `--ours`/`--theirs`/`--union` for 01, 10, and 11, respectively).
+ `--ours`/`--theirs`/`--union`).
+ They can be populated by `create_ll_flag`, whose argument can be
+ `XDL_MERGE_FAVOR_OURS`, `XDL_MERGE_FAVOR_THEIRS`, or
+ `XDL_MERGE_FAVOR_UNION`.
Everything else
---------------
These two function names do not suggests that these are symmetric. How
about get_ll_flavor() and create_ll_flavor()? Or flavor_to_ll_flag()
and ll_flag_to_flavor().
Regards,
Bert
Hi Jonathan,
Sorry I'm late to respond, I've just caught up with work after a long summer holiday. It's back to working with complicated merges across normalization boundaries now, though. This series looks like it's taking my hack in a sane direction, so for what it's worth:
Acked-by: Eyvind Bernhardsen <redacted>
Meanwhile, I've hit an annoyance: currently, files that are introduced with CRLFs on a non-normalized branch are left alone in the merged branch, causing the classical core.autocrlf problem of marking every line in such a file as changed if it is so much as touched.
I think the renormalization setting makes much more sense if the end result of the merge is normalized, so I'm working on a patch to make it do that.
- Eyvind