Re: non-ascii filenames issue

On 2009-04-05 13:36 (+0400), Gregory Petrosyan wrote:

# Changes to be committed:
#   (use "git rm --cached <file>..." to unstage)
#
#  new file:   "\321\204\320\260\320\271\320\273"
#
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
                 "файл" should be here instead

It can be fixed with command:

    git config --global core.quotepath false

On Sun, Apr 05, 2009 at 12:54:28PM +0300, Teemu Likonen wrote:

On 2009-04-05 13:36 (+0400), Gregory Petrosyan wrote:

quoted

# Changes to be committed:
#   (use "git rm --cached <file>..." to unstage)
#
#  new file:   "\321\204\320\260\320\271\320\273"
#
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
                 "файл" should be here instead

It can be fixed with command:

    git config --global core.quotepath false

Thanks! That works. Does it make sence to set it to "false" by default?

	Gregory

2009/4/5 Gregory Petrosyan [off-list ref]:

On Sun, Apr 05, 2009 at 12:54:28PM +0300, Teemu Likonen wrote:

quoted

On 2009-04-05 13:36 (+0400), Gregory Petrosyan wrote:

quoted

# Changes to be committed:
#   (use "git rm --cached <file>..." to unstage)
#
#  new file:   "\321\204\320\260\320\271\320\273"
#
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
                 "файл" should be here instead

It can be fixed with command:

    git config --global core.quotepath false

Thanks! That works. Does it make sence to set it to "false" by default?

Unfortunately not, because for some absolutely crazy reason, there is
no way at all to tell what encoding the string is in.  It never
occured to anyone that it might actually be useful to be able to read
the filename in an unambiguous way.  The result is this sort of mess.
Just wait until you try to checkout that file on a new filesystem with
a different encoding.  Or try to checkout that file in Windows.  It's
like git decided to step backwards 30 years.

John

On Sun, Apr 5, 2009 at 6:51 AM, John Tapsell [off-list ref] wrote:

Unfortunately not, because for some absolutely crazy reason

Bzzt. http://article.gmane.org/gmane.comp.version-control.git/50830

And, as always, patches welcomed.

j.

John Tapsell:

Unfortunately not, because for some absolutely crazy reason, there is no 
way at all to tell what encoding the string is in.  It never occured to 
anyone that it might actually be useful to be able to read the filename in 
an unambiguous way.

It comes from the Unix tradition, unfortunately, that file names are just a 
stream of bytes, instead of a stream of characters mapped to a byte 
sequence. The "stream of bytes" think worked back when everyone used ASCII, 
but as soon as other character encodings were used (i.e back in the 1970s or 
so), that assumption broke.

The result is this sort of mess. Just wait until you try to checkout that 
file on a new filesystem with a different encoding.  Or try to checkout 
that file in Windows.  It's like git decided to step backwards 30 years.

Since most people on Linux nowadays probably are running in a UTF-8-based 
locale, I tried introducing some (very incomplete) patches for the Windows 
port to make this assumption, to allow Windows users to make use of 
non-ASCII file names (Windows uses Unicode strings for file names). Mac OS 
uses (semi-decomposed) UTF-8 strings, so it should also be able to make use 
of this.

Unfortunately, there seems to be quite some resistance towards deciding on 
a platform- and language-independent way of storing file names in Git, but 
rather just going the "Unix" way and making it someone elses problem. I find 
this a bit sad.


-- 
\\// Peter - http://www.softwolves.pp.se/

Hi,

On Mon, 6 Apr 2009, Peter Krefting wrote:

It comes from the Unix tradition, unfortunately, that file names are 
just a stream of bytes, instead of a stream of characters mapped to a 
byte sequence.

How is that different from .txt not having a defined locale?

Really, please, do not add to the non-information.

Since most people on Linux nowadays probably are running in a 
UTF-8-based locale, I tried introducing some (very incomplete) patches 
for the Windows port to make this assumption, to allow Windows users to 
make use of non-ASCII file names (Windows uses Unicode strings for file 
names). Mac OS uses (semi-decomposed) UTF-8 strings, so it should also 
be able to make use of this.

Most Russian programmers I know do not run in a UTF-8 locale.

Unfortunately, there seems to be quite some resistance towards deciding 
on a platform- and language-independent way of storing file names in 
Git, but rather just going the "Unix" way and making it someone elses 
problem. I find this a bit sad.

I find it a bit unfair that you say that, after many people participated 
in that very informative thread, and after I tried to work with you 
personally on getting the stuff into 4msysgit.git.

Actually, not just only a bit.

Ciao,
Dscho

On Mon, Apr 06, 2009 at 11:12:35AM +0200, Johannes Schindelin wrote:

Most Russian programmers I know do not run in a UTF-8 locale.

Actually, on Linux, people gradually switching to UTF-8 from koi8-r,
but on Windows MSCRT does not support UTF-8, so you do have much choice
here but to use Windows-1251. BTW, the upcoming Cygwin 1.7 is going to
have UTF-8 as the default locale. So, IMHO, UTF-8 is the only reasonable
choice for internal file name representation...

Dmitry

2009/4/6 Peter Krefting [off-list ref]:

John Tapsell:

quoted

Unfortunately not, because for some absolutely crazy reason, there is no
way at all to tell what encoding the string is in.  It never occured to
anyone that it might actually be useful to be able to read the filename in
an unambiguous way.

It comes from the Unix tradition, unfortunately, that file names are just a
stream of bytes, instead of a stream of characters mapped to a byte
sequence. The "stream of bytes" think worked back when everyone used ASCII,
but as soon as other character encodings were used (i.e back in the 1970s or
so), that assumption broke.

Those interested in this subject may find the following document on
the creation of utf8 interesting.

http://www.cl.cam.ac.uk/~mgk25/ucs/utf-8-history.txt

cheers,
Yves

-- 
perl -Mre=debug -e "/just|another|perl|hacker/"