Re: git rev-list %an, %ae, %at bug in v1.7.10.1 and beyond

On Mon, May 21, 2012 at 06:01:50PM +1000, Bryan Turner wrote:

Note that the author name, e-mail and timestamp values are all missing
(the three |'s in a row at the end).
[...]
Built from 0dbe6592ccbd1a394a69a52074e3729d546fe952, the parent of
4b340cf, and in previous versions of git (1.7.10 and earlier), I got
output like this:
[...]
Note that the author name, e-mail and timestamp are all present. The
"a" appears as ASCII here, but it's actually a UTF-16LE character (the
terminal on the Mac is being "helpful").

I'm not too surprised this is broken (in fact, I'm surprised it ever
really worked). UTF-16, especially representing pure ascii characters,
will have embedded NULs. Most of the code assumes that things like names
and emails are NUL-terminated and ascii-compatible (so ascii, or some
ascii-superset encoding like utf8, iso8859-1, etc). You can store a
commit message (and name) in utf-16 if you tell git that you are doing
so, but we should be re-encoding it before handling it.

================================ Output =====================================
aphrael:qa-resources.git bturner$ git cat-file -p
5c1ccdec5f84aa149a4978f729fdda70769f942f
tree dd173cb70baaac07bdf405f4e3db110e7fafd180
parent 02c78bc39ac6192623bf080e3e2ac892a4f5764c
author a [off-list ref] 1327876222 +1100
committer a [off-list ref] 1327876222 +1100

commit with unicode name
================================ End ========================================

There's no encoding header, so having a utf-16 character there is wrong.
How did you make such a commit in the first place, though? I believe
that git-commit treats the name as a string and would terminate on a NUL
(or am I wrong in thinking that this "a" is really U+0061, and is
actually some other unicode character that _looks_ like "a", but doesn't
contain a NUL?).

Did you create it by piping straight to git-hash-object? What does
piping the above through "xxd" or "cat -A" show?

-Peff