[PATCH] Ignore end-of-line style when computing similarity score for rename detection

Subsystems: the rest

DORMANTno replies

3 messages, 2 authors, 2016-06-15 · open the first message on its own page

[PATCH] Ignore end-of-line style when computing similarity score for rename detection

From: Steven Grimm <hidden>
Date: 2016-06-15 22:43:18

Signed-off-by: Steven Grimm <redacted>
---
A couple of source files got checked into my code base with DOS-style 
end-of-line characters. I converted them to UNIX-style (the convention 
for this project) in my branch. Then later, I renamed a couple of them.

Meanwhile, back in the original branch, someone else fixed a bug in one 
of the files and checked it in, still with DOS-style line endings.

When I merged that change into my branch, git didn't detect the rename 
because the fact that every line has a change (the end-of-line 
character) dropped the similarity score way too low.

This patch teaches git to ignore end-of-line style when looking for 
potential rename candidates. A separate question, which I expect may be 
more controversial, is what to do with conflict markers; with this 
patch, the entire file is still marked as in conflict if the end-of-line 
style changes (but it's still an improvement in that we at least detect 
the rename now.)


 diffcore-delta.c |    9 ++++++---
 1 files changed, 6 insertions(+), 3 deletions(-)
diff --git a/diffcore-delta.c b/diffcore-delta.c
index 7338a40..10bbf95 100644
--- a/diffcore-delta.c
+++ b/diffcore-delta.c
@@ -143,9 +143,12 @@ static struct spanhash_top *hash_chars(unsigned 
char *buf, unsigned int sz)
                unsigned int c = *buf++;
                unsigned int old_1 = accum1;
                sz--;
-               accum1 = (accum1 << 7) ^ (accum2 >> 25);
-               accum2 = (accum2 << 7) ^ (old_1 >> 25);
-               accum1 += c;
+               /* Ignore \r\n vs. \n when computing similarity. */
+               if (c != '\r') {
+                       accum1 = (accum1 << 7) ^ (accum2 >> 25);
+                       accum2 = (accum2 << 7) ^ (old_1 >> 25);
+                       accum1 += c;
+               }
                if (++n < 64 && c != '\n')
                        continue;
                hashval = (accum1 + accum2 * 0x61) % HASHBASE;
-- 
1.5.2.2.571.ge134

[PATCH] Ignore end-of-line style when computing similarity score for rename detection

From: Steven Grimm <hidden>
Date: 2016-06-15 22:43:18

Signed-off-by: Steven Grimm <redacted>
---
Okay, let's try this again with an MUA that won't change my tabs to
spaces -- sorry about that.

A couple of source files got checked into my code base with DOS-style
end-of-line characters. I converted them to UNIX-style (the convention
for this project) in my branch. Then later, I renamed a couple of them.

Meanwhile, back in the original branch, someone else fixed a bug in one
of the files and checked it in, still with DOS-style line endings.

When I merged that change into my branch, git didn't detect the rename
because the fact that every line has a change (the end-of-line
character) dropped the similarity score way too low.

This patch teaches git to ignore end-of-line style when looking for
potential rename candidates. A separate question, which I expect may be
more controversial, is what to do with conflict markers; with this
patch, the entire file is still marked as in conflict if the end-of-line
style changes (but it's still an improvement in that we at least detect
the rename now.)


 diffcore-delta.c |    9 ++++++---
 1 files changed, 6 insertions(+), 3 deletions(-)
diff --git a/diffcore-delta.c b/diffcore-delta.c
index 7338a40..10bbf95 100644
--- a/diffcore-delta.c
+++ b/diffcore-delta.c
@@ -143,9 +143,12 @@ static struct spanhash_top *hash_chars(unsigned char *buf, unsigned int sz)
 		unsigned int c = *buf++;
 		unsigned int old_1 = accum1;
 		sz--;
-		accum1 = (accum1 << 7) ^ (accum2 >> 25);
-		accum2 = (accum2 << 7) ^ (old_1 >> 25);
-		accum1 += c;
+		/* Ignore \r\n vs. \n when computing similarity. */
+		if (c != '\r') {
+			accum1 = (accum1 << 7) ^ (accum2 >> 25);
+			accum2 = (accum2 << 7) ^ (old_1 >> 25);
+			accum1 += c;
+		}
 		if (++n < 64 && c != '\n')
 			continue;
 		hashval = (accum1 + accum2 * 0x61) % HASHBASE;
-- 
1.5.2.2.571.ge134

Re: [PATCH] Ignore end-of-line style when computing similarity score for rename detection

From: Johannes Sixt <hidden>
Date: 2016-06-15 22:43:18

Steven Grimm wrote:
Signed-off-by: Steven Grimm <redacted>
---
Okay, let's try this again with an MUA that won't change my tabs to
spaces -- sorry about that.

A couple of source files got checked into my code base with DOS-style
end-of-line characters. I converted them to UNIX-style (the convention
for this project) in my branch. Then later, I renamed a couple of them.

Meanwhile, back in the original branch, someone else fixed a bug in one
of the files and checked it in, still with DOS-style line endings.

When I merged that change into my branch, git didn't detect the rename
because the fact that every line has a change (the end-of-line
character) dropped the similarity score way too low.

This patch teaches git to ignore end-of-line style when looking for
potential rename candidates. A separate question, which I expect may be
more controversial, is what to do with conflict markers; with this
patch, the entire file is still marked as in conflict if the end-of-line
style changes (but it's still an improvement in that we at least detect
the rename now.)
I think that nobody would object to have a use-case description like
this in the commit message...

-- Hannes
Keyboard shortcuts
hback out one level
jnext message in thread
kprevious message in thread
ldrill in
Escclose help / fold thread tree
?toggle this help