Re: Fix UTF Encoding issue

On Tue, Dec 04, 2007 at 08:16:24AM +1030, Benjamin Close wrote:

Jakub Narebski wrote:

quoted

On Mon, 3 Dec 2007, Martin Koegler wrote:

quoted

On Mon, Dec 03, 2007 at 04:06:48AM -0800, Jakub Narebski wrote:

quoted

Ismail Dönmez [off-list ref] writes:

quoted

Monday 03 December 2007 Tarihinde 12:14:43 yazm??t?:

quoted

Benjamin Close [off-list ref] writes:

quoted

-	eval { $res = decode_utf8($str, Encode::FB_CROAK); };
-	if (defined $res) {
-		return $res;
-	} else {
-		return decode($fallback_encoding, $str, 
Encode::FB_DEFAULT);
-	}
+	eval { return ($res = decode_utf8($str, Encode::FB_CROAK)); 
};
+	return decode($fallback_encoding, $str, Encode::FB_DEFAULT);
}
           

This version is broken on Debian sarge and etch. Feeding a UTF-8 and a 
latin1
encoding of the same character sequence yields to different results.

For the record, this was on a debian sid machine.

#perl --version
This is perl, v5.8.8 built for x86_64-linux-gnu-thread-multi

and the result of not using the original patch was:

<h1>Software error:</h1>
<pre>Cannot decode string with wide characters at 
/usr/lib/perl/5.8/Encode.pm line 166.
</pre>


I haven't tried the other solutions tested here.

Debian etch also has v5.8.8.

My main question is, why is the error not catched?

I'm not a perl programmer, but in your patch the first line is a
NOP. The return in eval seems to only returns from the eval block, so
any text is decoded as latin1 with the second statement.

In the original version, decode($fallback_encoding, $str,
Encode::FB_DEFAULT) can not emit an error, else it would in your
version too. 

In your version, eval is able to surpress the error of
decode_utf8($str, Encode::FB_CROAK);, but not in the original version.

Strange.

mfg Martin Kögler