fast-import, merges, and file changes -- lack of clarity in docs, possible minor bug, or PEBKAC?

Hi,

It appears the documentation for fast-import does not specify the
basis on which a commit command's file changes (filemodify,
filedelete, filecopy, or filerename) are relative to.  For a non-merge
commit, it's obvious, and I thought that it would be obvious for a
merge commit too -- and then found out I had assumed the wrong answer.
 I assumed the file changes would be relative to the *combined* set of
files from the parents, whereas fast-import apparently treats them as
relative to the first (or 'from') parent.  (Since the combined set of
files may have conflicts, I'd expect it to be an error if the merge
commit didn't specify an appropriate filemodify command for at least
each of the filenames with conflicts.)

In case my description above isn't clear, I've got a simple example at
the end of this email that has a file 'world' on one branch, a file
'bar' on another branch, and a merge of the two branches.  In the
merge commit, I do not need to mention the 'world' file; it will be
included automatically.  However, unless I mention the 'bar' file (by
adding a 'M 100644 :1 bar' line to the end of the input), it will not
be included in the merge commit.

Is this intentional?  I believe we could change this behavior without
breaking backward compatibility with older fast-export output streams,
since such older streams would simply be providing redundant
information.  But is there a reason for this behavior that I'm
missing?


Thanks,
Elijah


Sample input file for git fast-import (which merges two commits but
silently drops one of the files):

blob
mark :1
data 4
foo

reset refs/heads/other
commit refs/heads/other
mark :2
author Elijah Newren [off-list ref] 1234815015 -0700
committer Elijah Newren [off-list ref] 1234815015 -0700
data 29
Initial commit, other branch
M 100644 :1 bar

blob
mark :3
data 3
hi

reset refs/heads/master
commit refs/heads/master
mark :4
author Elijah Newren [off-list ref] 1234814943 -0700
committer Elijah Newren [off-list ref] 1234814943 -0700
data 30
Initial commit, master branch
M 100644 :3 world

commit refs/heads/master
mark :5
author Elijah Newren [off-list ref] 1234815029 -0700
committer Elijah Newren [off-list ref] 1234815029 -0700
data 21
Merge branch 'other'
from :4
merge :2

I think I've at least partially discovered the answer my own
question(s), so I figured I'd document them in case anyone else was
wondering...

On Mon, Feb 16, 2009 at 5:28 PM, Elijah Newren [off-list ref] wrote:

It appears the documentation for fast-import does not specify the
basis on which a commit command's file changes (filemodify,
filedelete, filecopy, or filerename) are relative to.

<snip>

Is this intentional?  I believe we could change this behavior without
breaking backward compatibility with older fast-export output streams,
since such older streams would simply be providing redundant
information.  But is there a reason for this behavior that I'm
missing?

It is documented...though implicitly in discussion of other commands.
In particular, "...a `merge` command may be used instead of `from` to
start the commit with an empty tree" and later "If the `from` command
is omitted when creating a new branch, the first `merge` commit will
be the first ancestor of the current commit, and the branch will start
out with no files."  This implies that the files of a `merge` parent
are ignored and any content must from that parent that is contained in
the commit must be explicitly spelled out to be included.

Also, as far as backward compatibility, my suggestion would not be
compatible; deletions involved in such merges would be handled by
current fast-export streams by simply omitting the file in the list of
file changes in the merge commit.  My suggestion would require
explicitly specified file deletion to ensure its removal.

So, it appears this is intentional, any behavior change such as I
suggested would require different/new syntax, and would require
support in fast-export too for my purposes.  That certainly makes my
git-fast-filter script (a script in a pipeline between fast-export and
fast-import designed to act somewhat like filter-branch but with
better performance on my large repositories) much harder.  Oh, well.


Elijah