Re: Fix UTF Encoding issue

On Mon, Dec 03, 2007 at 04:06:48AM -0800, Jakub Narebski wrote:

Ismail Dönmez [off-list ref] writes:

quoted

Monday 03 December 2007 Tarihinde 12:14:43 yazm??t?:

quoted

Benjamin Close [off-list ref] writes:

quoted

-	eval { $res = decode_utf8($str, Encode::FB_CROAK); };
-	if (defined $res) {
-		return $res;
-	} else {
-		return decode($fallback_encoding, $str, Encode::FB_DEFAULT);
-	}
+	eval { return ($res = decode_utf8($str, Encode::FB_CROAK)); };
+	return decode($fallback_encoding, $str, Encode::FB_DEFAULT);
 }

This version is broken on Debian sarge and etch. Feeding a UTF-8 and a latin1
encoding of the same character sequence yields to different results.

quoted

quoted

I thought the standard catch ... throw idiom in Perl was to do the above
like this:

	my $res;
        eval { $res = decode_utf8($str, Encode::FB_CROAK); };
        if ($@) {
        	return decode($fallback_encoding, $str, Encode::FB_DEFAULT);
	}
	return $res;

I think this is correct, but the current code in gitweb doesn't look correct 
since it checks for $res and not $@.

First version of the patch was created by Martin Koegler. I have
participated in creating the version which is now in gitweb, but I
have to say that I wrote it based on decode_utf8
documentation... which doesn't necessarily agree with facts :-(

eval { $res = decode_utf8(...); }
if ($@) 
     return decode(...);
return $res

or

eval { $res = decode_utf8(...); }
if (defined $res)
      return $res;
else
    return decode(...);

show the same (wrong) behaviour on Debian sarge. They do not always
decode non UTF-8 characters correctly, eg.
#öäü does not work
#äöüä does work

On Debian etch, both versions are working.

I'm all for the "throw idion" version. Ack.

mfg Martin Kögler