Re: gitweb: charset problem

Daniel Barkalow [off-list ref] writes:

On Mon, 24 Oct 2005, Horst von Brand wrote:

quoted

I believe the Emperor Penguin decreed messages have to be
ASCII, or else UTF-8. Please don't add to the mess by using
non-portable encodings!

Should we possibly reject non-UTF-8 input to commits?

Please, don't.

IIRC, we actually define that to be UTF-8, unlike most of the
other stuff, for which we don't actually insist on a policy.

No, we do not define nor insist on a particluar policy as far as
I know.  We suggest the use of UTF-8 merely from common sense to
help interoperability, and make UTF-8 slightly easier to use
than other encodings by giving specific support for it in some
tools, namely -u flag in git-mailinfo.

It is perfectly reasonable if a company internal project that
works in Russia to standardize on KOI, or in Japan on EUC-JP.
We simply allow it without encouraging nor discouraging it.  If
gitweb can take a configuration mechanism to override the
built-in UTF-8 header, that is perfectly a valid thing to do to
help such an environment.

However, we suggest UTF-8 if the project does not have a
compelling reason to do otherwise [*1*].  If you want to be
prepared for the day your project might have wider participants
than you originally envisioned, that is the most sensible thing
to do.  This is especially true because the commit logs cannot
be re-encoded after the fact.

[Footnote]

*1* For example, I've never made GNU emacs to work well with
Japanese in UTF-8 , so if people in my company internal project
wanted to use Japanese in commit logs, I would probably
standardize on EUC-JP for such a project.  Luckily so far I have
not been forced to make that decision.

On Mon, 24 Oct 2005, Junio C Hamano wrote:

quoted

IIRC, we actually define that to be UTF-8, unlike most of the
other stuff, for which we don't actually insist on a policy.

No, we do not define nor insist on a particluar policy as far as
I know.  We suggest the use of UTF-8 merely from common sense to
help interoperability, and make UTF-8 slightly easier to use
than other encodings by giving specific support for it in some
tools, namely -u flag in git-mailinfo.

I thought we'd decided on uninterpreted byte values for blobs, filenames, 
and trees (and everything in the working tree), but using UTF-8 for tag 
and commit objects.

Consider if you started a project in EUC-JP, and then decided to switch to 
UTF-8 later (when your environment handled it cleanly, perhaps). You could 
convert all the file contents and move files to re-encoded names, but 
you'd then want to commit these changes and have the log before and after 
simultaneously intelligable.

[Footnote]

*1* For example, I've never made GNU emacs to work well with
Japanese in UTF-8 , so if people in my company internal project
wanted to use Japanese in commit logs, I would probably
standardize on EUC-JP for such a project.  Luckily so far I have
not been forced to make that decision.

It wouldn't be hard to convert at some point between the editor and the 
commit object, and you don't re-edit the commit objects like you do 
tracked files. It probably wouldn't even be hard for commit-tree to 
convert its input based on locale. (And stuff which prints commit contents 
for user consumption probably ought to re-encode it if necessary, too)

	-Daniel
*This .sig left intentionally blank*