Re: Moved files and merges

Junio C Hamano [off-list ref] writes:

"H. Peter Anvin" [off-list ref] writes:

quoted

I currently have two klibc trees,

I cloned them to take a look.  You_do_ seem to have a lot of
renames.

Well, I think I understand how your trees ancestry looks like,
but still haven't come up with a good problem definition.  I am
sorry that this message is not a solution for your problem but
would end up to be just my rambling and thinking aloud.

The ancestry looks like this:

       ----#4-#5---#7   #0: 1.0.14 released, next version is 1.0.15
      /      /              5691e96ebfccd21a1f75d3518dd55a96b311d1aa
     /---#1-#3---#6     #1: Explain why execvpe/execlpe work the way they do.
    //     /                1d774a8cbd8e8b90759491591987cb509122bd78
  #0-----#2             #2: 1.1 released, next version is 1.1.1
                            3a41b60f6730077db3f04cf2874c96a0e53da453
                        #3: Merge of #2 into #1
                            7ab38d71de2964129cf1d5bc4e071d103e807a0d
                        #4: socketcalls aren't always *.S files; they can...
                            f52be163e684fc3840e557ecf242270926136b67
                        #5: Merge of #3 into #4
                            2e2a79d62a96b6b0d4bc93697fe77cd3030cdfd9
                        #6: Warnings cleanup
                            f5260f8737517f19a03ee906cd64dfc9930221cd
                        #7: Remove obsoleted files from merge
                            59709a172ee58c9d529a8c4b6f5cf53460629cb3

and you are trying to merge #6 into #7 (or #7 into #6).  #6 does
not have usr/kinit and nfsmount at the top; #7 has nfsmount
under usr/kinit/.

The merge base of #6 and #7 is #3.  #3->#7 involves many renames
like nfsmount/sunrpc.h -> usr/kinit/nfsmount/sunrpc.h, while
#3->#6 involves no renames.  This can be seen by 

       $ git-diff-tree -M --diff-filter=R -r '#3' '#7' | wc -l

These renames are introduced by #5 merge, whose immediate
ancestors are #3 and #4.

Now, there is a question.  When you merge #6 and #7, development
between #3 and #7 have a lot of renames, while #3 and #6 do not.
What should this merge do?  Should it follow the rename, and if
so why?

Maybe I am trying to solve a wrong problem.  First I thought
there should be an automated way for merge to tell whether
renames done in one branch should be ignored, but that may be
better left under human control.  By looking at #6, #7 and #3
alone, even with looking at #5 which introduced the renames, I
do not see a good reason for the merge machinery to dictate that
these renames are to be followed or ignored when merging #6 and
#7.

So this is not what we ordinary call a 'merge', but what really
is happening is something else .  Let's say the tree you would
want to end up with is 'similar' to #6 but contains the changes
done on the branch leading to #7, except #7 has renames unwanted
for moving #6 forward.  Is that what we should be solving?

Then, I think we could do something along these lines, perhaps?

 - Ask git-diff-tree about changes betwen #3 (merge-base) and
   #7.

        git-diff-tree -M --diff-filter=R -r '#3' '#7' |
        sed -e 's/^[^	]*	//' >.rename-tmp

   This will give us list of <old-name> <new-name>, one pair on
   each line, separated with tabs.

 - check out '#7', and move files around reversing the rename we
   found in the previous step.  Make a throw-away tree out of it;
   we do not even have to have a commit.

        git checkout -b junk7 '#7'
        while read name3 name7
        do
	    test -f $name7 || { echo $name7; continue; }
	    git rename $name7 $name3
        done <.rename-tmp
	tree7_sans_rename=$(git-write-tree)
	git reset --hard junk7

 - check out '#6', and run three-way merge like this.

	git checkout -b junk6 '#6'
        git-read-tree -m -u '#3' '#6' $tree7_sans_rename

 - Notice I did not run merge-cache above.  I suspect that #6
   and #7 just have overlaps but there may be a lot of things
   missing from #7 that you would rather keep (I am not familiar
   with the relationship between these two projects, so I am
   just guessing).  First let's see what is in the tree:

    $ git-ls-files --unmerged
    100644 450e5c7f5c4f4c90df8f60fc98b96cb7db25a502 1       MCONFIG
    100644 450e5c7f5c4f4c90df8f60fc98b96cb7db25a502 2       MCONFIG
    100644 41fdd07af8c4e3eebbafcc3ced8b361882fe4c0d 1       MRULES
    100644 41fdd07af8c4e3eebbafcc3ced8b361882fe4c0d 2       MRULES
    100644 521bf99c2ab1f647156bf16a7f977e951fb9cdb0 1       ash/Makefile
    100644 521bf99c2ab1f647156bf16a7f977e951fb9cdb0 2       ash/Makefile
    100644 66ba9bb16f8289dc8c1af7d1cd49e5f563b5f624 1       gzip/MCONFIG
    100644 66ba9bb16f8289dc8c1af7d1cd49e5f563b5f624 2       gzip/MCONFIG
    100644 d5f9f94be0795accd8b02e2743ab899ac7ca661e 1       gzip/Makefile
    100644 d5f9f94be0795accd8b02e2743ab899ac7ca661e 2       gzip/Makefile
    100644 472f09205986fd9c2f9c7da6e31b45cebb9fe0af 1       ipconfig/MCONFIG
    ...

   Stage 1 is from #3, 2 is from #6, and these files does not
   exist in the faked sans-rename version of #7.  I presume that
   you would want to keep them in #6, so we keep them without
   letting git-merge-cache to remove them by:

	git-ls-files --unmerged | cut -f2 | uniq |
        xargs git-update-cache --

   This still may be slurping a lot of files that only exist in
   #7 but not in #6 in your cache (for example, I suspect you do
   not want to slurp usr/kinit/ into #6 with this 'merge').  You
   may need to remove them, using 'rm -f <file> &&
   git-update-cache --remove <file>'.

 - Then we merge-cache as usual.

	git-merge-cache -o git-merge-one-file-script -a
                      

On Sat, Sep 03, 2005 at 01:25:50AM -0700, Junio C Hamano wrote:

Junio C Hamano [off-list ref] writes:

quoted

"H. Peter Anvin" [off-list ref] writes:

quoted

I currently have two klibc trees,

I cloned them to take a look.  You_do_ seem to have a lot of
renames.

Well, I think I understand how your trees ancestry looks like,
but still haven't come up with a good problem definition.  I am
sorry that this message is not a solution for your problem but
would end up to be just my rambling and thinking aloud.

The ancestry looks like this:

       ----#4-#5---#7   #0: 1.0.14 released, next version is 1.0.15
      /      /              5691e96ebfccd21a1f75d3518dd55a96b311d1aa
     /---#1-#3---#6     #1: Explain why execvpe/execlpe work the way they do.
    //     /                1d774a8cbd8e8b90759491591987cb509122bd78
  #0-----#2             #2: 1.1 released, next version is 1.1.1
                            3a41b60f6730077db3f04cf2874c96a0e53da453
                        #3: Merge of #2 into #1
                            7ab38d71de2964129cf1d5bc4e071d103e807a0d
                        #4: socketcalls aren't always *.S files; they can...
                            f52be163e684fc3840e557ecf242270926136b67
                        #5: Merge of #3 into #4
                            2e2a79d62a96b6b0d4bc93697fe77cd3030cdfd9
                        #6: Warnings cleanup
                            f5260f8737517f19a03ee906cd64dfc9930221cd
                        #7: Remove obsoleted files from merge
                            59709a172ee58c9d529a8c4b6f5cf53460629cb3

and you are trying to merge #6 into #7 (or #7 into #6).  #6 does
not have usr/kinit and nfsmount at the top; #7 has nfsmount
under usr/kinit/.

The merge base of #6 and #7 is #3.  #3->#7 involves many renames
like nfsmount/sunrpc.h -> usr/kinit/nfsmount/sunrpc.h, while
#3->#6 involves no renames.  This can be seen by 

       $ git-diff-tree -M --diff-filter=R -r '#3' '#7' | wc -l

These renames are introduced by #5 merge, whose immediate
ancestors are #3 and #4.

Now, there is a question.  When you merge #6 and #7, development
between #3 and #7 have a lot of renames, while #3 and #6 do not.
What should this merge do?  Should it follow the rename, and if
so why?

Maybe I am missing something... but why should the merge operation
ignore renames? Is there a merge case when ignoring renames is the
Right Thing to do?

Lets say the branches A and B has the common ancestor C which contains
a file named "foo". If A has renamed "foo" to "foobar" and B has made
some content changes to "foo" shouldn't the result be a file named
"foobar" with the content changes B made to "foo" in it?

As I see it when we rename 'foo' to 'foobar' we say that "this file
should from now on be named 'foobar'". Furthermore, when we make some
content changes to some file we say "this file should have this new
content instead of its old content". In particular I do not think the
user expresses an intent to keep the name of a file just because she
makes some changes to its contents. So IMHO the merge of "this file
should from now on be named 'foobar'" and "this file should have this
new content instead of its old content" should be "this file should
from now on be named 'foobar' and have this new content".

I think it is feasible to teach the merge machinery about those rename
cases, it probably isn't trivial to implement but I certainly think it
is possible. It may be the case that the user should be warned when
such automatic merges involving renames are performed though.


- Fredrik

On Sat, Sep 03, 2005 at 01:25:50AM -0700, Junio C Hamano wrote:

Junio C Hamano [off-list ref] writes:

quoted

"H. Peter Anvin" [off-list ref] writes:

quoted

I currently have two klibc trees,

I cloned them to take a look.  You_do_ seem to have a lot of
renames.

Well, I think I understand how your trees ancestry looks like,
but still haven't come up with a good problem definition.  I am
sorry that this message is not a solution for your problem but
would end up to be just my rambling and thinking aloud.

The ancestry looks like this:

       ----#4-#5---#7   #0: 1.0.14 released, next version is 1.0.15
      /      /              5691e96ebfccd21a1f75d3518dd55a96b311d1aa
     /---#1-#3---#6     #1: Explain why execvpe/execlpe work the way they do.
    //     /                1d774a8cbd8e8b90759491591987cb509122bd78
  #0-----#2             #2: 1.1 released, next version is 1.1.1
                            3a41b60f6730077db3f04cf2874c96a0e53da453
                        #3: Merge of #2 into #1
                            7ab38d71de2964129cf1d5bc4e071d103e807a0d
                        #4: socketcalls aren't always *.S files; they can...
                            f52be163e684fc3840e557ecf242270926136b67
                        #5: Merge of #3 into #4
                            2e2a79d62a96b6b0d4bc93697fe77cd3030cdfd9
                        #6: Warnings cleanup
                            f5260f8737517f19a03ee906cd64dfc9930221cd
                        #7: Remove obsoleted files from merge
                            59709a172ee58c9d529a8c4b6f5cf53460629cb3

and you are trying to merge #6 into #7 (or #7 into #6).  #6 does
not have usr/kinit and nfsmount at the top; #7 has nfsmount
under usr/kinit/.

Hi Junio.

Ican expalin some of the background for this particular merge.
At about one month ago I cloned the current klibc.git tree and started
doing the necessary modifications needed to introduce kbuild - the
build system used in the kernel.
Futhermore we decided to move files around so they fit the directory
structure planned to be used in the kernel - when we at one point in the
future merged with mainline.
While I were modifying the build system the development continued and a
few files saw some updates in the official klibc tree.

So what we want to do in this case is:
- Merge the kbuild changes into the official tree without loosing the
  changes made to renamed files.

On purpose I did not modify any of the renamed files so the klibc-kbuild
tree contains renames only for these.

If it would be possible to merge:
libs/klibc/klibc.git and libs/klibc/sam/klibc-kbuild.git
using the above rules it would be perfect.

Then a few of the patches from libs/klibc/klibc-kbuild.git would have to
be applied again, but thats doable.

Anyway my view on it. Since Peter is the one doing the merge he may have
better ideas.

	Sam