Re: Make "git am" properly unescape lines matching ">>*From "

On Tue, 08 Jun 2010 12:57:23 -0700, Carl Worth [off-list ref] wrote:

I'm adding support to notmuch[1] to more easily pipe a thread full of
But I noticed that "git am" wasn't removing any of these added '>'
characters, so I was getting corrupted commit messages.

I've also noticed that format-patch is generating bogus mbox files
without any escaping. (The only way it gets away with this is that
mailsplit only treats "From " lines as separators if they end with
something that looks quite a bit like the output of asctime.)

This does mean that without changing format-patch, the patched "git am"
could corrupt a commit message. This could happen if the commit message
originally contained a line matching "^From " which would previously be
passed through directly but will now be un-escaped to "From ".

This does seem less likely than a message containing a line matching
"^From " (which is the case that gets corrupted with an unpatched "git
am") so one option would be to ignore this, and apply my patch. That's
what I recommend for now.

Alternately, we could fix format-patch to add the correct, (and
reversible), escaping that is now expected by git-am.

Any attempt to add escaping to format-patch should recognize that many
users use the output of format-patch directly as content handed to their
MUA. Such users will *not* want escaping, (they are effectively treating
the format-patch output as a bare email message, not an mbox).

So if someone were to attempt this, I'd suggest first changing
format-patch to actually generate bare email messages when generating
files containing only a single message. This is instead of the invalid
mbox files it is generating now. This would be as simple as not emitting
the initial "From " line.

Then, when generating an actual mbox with multiple files, format-patch
should do the correct escaping, (which is now expected by "git am"), and
all of these cases of potential commit-message corruption should be
eliminated.

The other thing that would need to be fixed in this approach is to fix
"git send-email" to do the right thing with a bare email message. From a
quick glance at the code, it appears to be looking for an initial "From
" line, even though it doesn't appear to handle an mbox with multiple
messages. It looks for this line to distinguish an email message from
some custom "send lots of email" format. It should be simple to instead
distinguish a bare email message from the "send lots of email" format by
a first line which looks like an email header.

-Carl

-- 
carl.d.worth@intel.com

On 06/08/2010 01:47 PM, Carl Worth wrote:

On Tue, 08 Jun 2010 12:57:23 -0700, Carl Worth [off-list ref] wrote:

quoted

I'm adding support to notmuch[1] to more easily pipe a thread full of
But I noticed that "git am" wasn't removing any of these added '>'
characters, so I was getting corrupted commit messages.

I've also noticed that format-patch is generating bogus mbox files
without any escaping. (The only way it gets away with this is that
mailsplit only treats "From " lines as separators if they end with
something that looks quite a bit like the output of asctime.)

At the same time, it would be a fairly major lose to not be able to
generate individual messages easily.  I have personally considered the
fact that git format-patch produces something-vaguely-like mboxes rather
than individual plain RFC 2822 messages to be a bug; fixable by "tail"
but annoying.

	-hpa

On Tue, 08 Jun 2010 13:54:32 -0700, "H. Peter Anvin" [off-list ref] wrote:

On 06/08/2010 01:47 PM, Carl Worth wrote:

quoted

I've also noticed that format-patch is generating bogus mbox files
without any escaping. (The only way it gets away with this is that
mailsplit only treats "From " lines as separators if they end with
something that looks quite a bit like the output of asctime.)

At the same time, it would be a fairly major lose to not be able to
generate individual messages easily.  I have personally considered the
fact that git format-patch produces something-vaguely-like mboxes rather
than individual plain RFC 2822 messages to be a bug; fixable by "tail"
but annoying.

I totally agree. I said as much later on in the message. We should fix
format-patch to not emit the "From " line when generating files for
individual messages, (and we should fix send-email to accept such a bare
file).

That much is easy to agree on since it involves using mbox less, so the
whole "which mbox format to use?" question goes away, (for these uses at
least).

-Carl

-- 
carl.d.worth@intel.com