Re: mingw, windows, crlf/lf, and git

Linus Torvalds [off-list ref] writes:

Actually, my patch already had one that you didn't mention: 
 6) CR never shows up alone.

Older Macs ;-)?

So the patch I sent out basicallyhad the following rules:
 - no more than ~10% of all characters being other than regular printable 
   ASCII (where any control character except for newline/cr/tab was deemed 
   nonprintable)
 - any "lonely" CR automatically means it's binary, and I would refuse 
   to convert that to a LF (the test in the code is that CRLF count must 
   match CR count)
...
I think that to help asian languages (or strange text-files in utf8 or 
Latin1 too, for that matter: test-files with _just_ special characters), I 
should probably make the rule be that only the 0-31 range is special.

I would agree.  0-31 except HT, CR, LF and ESC would be a good
idea; that would not harm text in UTF-8, EUC based various
locales nor ISO 2022.

Patch is relative to 'pu'.
-- >8 --

diff --git a/convert.c b/convert.c
index ebcf717..b6b7c66 100644
--- a/convert.c
+++ b/convert.c

@@ -13,7 +13,7 @@ struct text_stat {
 	unsigned cr, lf, crlf;
 
 	/* These are just approximations! */
-	unsigned printable, nonprintable, nul;
+	unsigned printable, nonprintable;
 };
 
 static void gather_stats(const char *buf, unsigned long size, struct text_stat *stats)

@@ -34,13 +34,11 @@ static void gather_stats(const char *buf, unsigned long size, struct text_stat *
 			stats->lf++;
 			continue;
 		}
-		if (c == '\t' || (c >= 32 && c < 127)) {
-			stats->printable++;
+		if ((c < 32) && (c != '\t' && c != '\033')) {
+			stats->nonprintable++;
 			continue;
 		}
-		if (!c)
-			stats->nul++;
-		stats->nonprintable++;
+		stats->printable++;
 	}
 }
 

@@ -50,7 +48,7 @@ static void gather_stats(const char *buf, unsigned long size, struct text_stat *
 static int is_binary(unsigned long size, struct text_stat *stats)
 {
 
-	if (stats->nul)
+	if (stats->nonprintable)
 		return 1;
 	/*
 	 * Other heuristics? Average line length might be relevant,

On Wed, 14 Feb 2007, Junio C Hamano wrote:

Linus Torvalds [off-list ref] writes:

quoted

Actually, my patch already had one that you didn't mention: 
 6) CR never shows up alone.

Older Macs ;-)?

Yeah, I think we can ignore them..

Let's see if anybody ever complains ;)

I would agree.  0-31 except HT, CR, LF and ESC would be a good
idea; that would not harm text in UTF-8, EUC based various
locales nor ISO 2022.

You could possibly add 127 to the list too (it's ascii DEL, I don't know 
if you should ever see it in anything that has anything to do with text).

-	if (stats->nul)
+	if (stats->nonprintable)

But this is too harsh.

It's quite common to have the occasional FF character. Some things really 
do use it for page breaks. So saying that *any* nonprintable character is 
bad is not a good idea.

Same goes for BS (some programs use it to show bold and underlined text: 
man-pages, for example).

		Linus