git-am doesn't strip CRLF line endings when the mbox is base64-encoded

Using git 2.24.0 from Debian testing.

It seems that git-am will strip CRLF endings from mails before applying
patches when the mail isn't encoded in any way.  It will also decode
base64-encoded mails.  But it won't strip CRLF endings from
base64-encoded mails.

Attached are two mbox files for two different recent series.
plainenc.am applies cleanly with `git am`, while base64enc.am doesn't.

Poking around the man pages, it looks like part of the issue might be
that the CRLF stripping is done in `git mailsplit`, before the base64
encoding, rather than after.

 -George

On 12/18/19 11:42 AM, George Dunlap wrote:

Using git 2.24.0 from Debian testing.

It seems that git-am will strip CRLF endings from mails before applying
patches when the mail isn't encoded in any way.  It will also decode
base64-encoded mails.  But it won't strip CRLF endings from
base64-encoded mails.

Attached are two mbox files for two different recent series.
plainenc.am applies cleanly with `git am`, while base64enc.am doesn't.

Poking around the man pages, it looks like part of the issue might be
that the CRLF stripping is done in `git mailsplit`, before the base64
encoding, rather than after.

Poking around -- it looks like the CRLF stripping would be better done
in `git mailinfo` after the decoding.

Also, this can *almost* be worked around using hooks -- there's an
`applypatch-msg` hook which can strip the CLRFs from the commit message,
but no hook (AFAICT) corresponding `applypatch-patch` which is run on
the patch itself before being applied.

 -George

George Dunlap wrote:

On 12/18/19 11:42 AM, George Dunlap wrote:

quoted

Using git 2.24.0 from Debian testing.

It seems that git-am will strip CRLF endings from mails before applying
patches when the mail isn't encoded in any way.  It will also decode
base64-encoded mails.  But it won't strip CRLF endings from
base64-encoded mails.

Attached are two mbox files for two different recent series.
plainenc.am applies cleanly with `git am`, while base64enc.am doesn't.

Poking around the man pages, it looks like part of the issue might be
that the CRLF stripping is done in `git mailsplit`, before the base64
encoding, rather than after.

Poking around -- it looks like the CRLF stripping would be better done
in `git mailinfo` after the decoding.

Also, this can *almost* be worked around using hooks -- there's an
`applypatch-msg` hook which can strip the CLRFs from the commit message,
but no hook (AFAICT) corresponding `applypatch-patch` which is run on
the patch itself before being applied.

This came up recently in [off-list ref]¹.
I don't know if any of that discussion will prove useful to
you if you want to poke at this further or not.

¹ https://lore.kernel.org/git/20191130180301.5c39d8a4@lwn.net/

-- 
Todd

On 12/18/19 12:15 PM, George Dunlap wrote:

On 12/18/19 11:42 AM, George Dunlap wrote:

quoted

Using git 2.24.0 from Debian testing.

It seems that git-am will strip CRLF endings from mails before applying
patches when the mail isn't encoded in any way.  It will also decode
base64-encoded mails.  But it won't strip CRLF endings from
base64-encoded mails.

Attached are two mbox files for two different recent series.
plainenc.am applies cleanly with `git am`, while base64enc.am doesn't.

Poking around the man pages, it looks like part of the issue might be
that the CRLF stripping is done in `git mailsplit`, before the base64
encoding, rather than after.

Poking around -- it looks like the CRLF stripping would be better done
in `git mailinfo` after the decoding.

Anyone want to take this up?  I mean, I could try to send a patch, but
since I've never looked at the git source code before, I'm sure it would
take me about 10x as much effort for me to do it as for someone already
familiar with the codebase.

(And I've already done that work for stackgit:
https://github.com/ctmarinas/stgit/pull/46)

 -George