libreoffice merge(tool?) issue #3 ...

Hi there,

        So - yet again, I'm still a completely clueless git user :-)
basically the same setup and reproduction issue as last time, still
using a stable git: version 1.7.3.4

Setup:
        git clone git://anongit.freedesktop.org/libreoffice/libs-core
        git checkout integration/dev300_m98
        git remote add stage git://anongit.freedesktop.org/libreoffice/staging/libs-core
        git fetch stage

Reproduce:
	rm -Rf *
	git reset --hard # ie. totally clean tree.
        git merge stage/dev300

	I get this output from the merge:

error: refusing to lose untracked file at 'ucb/source/ucp/ext/makefile.mk'

	Then when I run:

$ git mergetool ucb/source/ucp/ext/makefile.mk
..

Deleted merge conflict for 'ucb/source/ucp/ext/makefile.mk':
  {local}: created
  {remote}: deleted
Use (c)reated or (d)eleted file, or (a)bort?

	It seems to suggest that the file is deleted somewhere (in the branch I
am trying to merge in) it seems.

	Interestingly, though - the file is present in both
integration/dev300_m98:

	git log -n1 ucb/source/ucp/ext/makefile.mk | tee
commit 80b61d9c6762b4f195edd1246b903b11ad3f2252
Author: Thomas Arnhold [off-list ref]
Date:   Fri Jan 21 14:11:55 2011 +0100

    Remove old RCS lines.

	And also present in the branch I'm merging: stage/dev300:

commit 0c87cb97cf3790fa98bcbb0eef9d174140a4e847
Author: sb [off-list ref]
Date:   Fri Sep 10 13:10:07 2010 +0200

    sb129: #i113189# change UNO components to use passive registration


	Which makes me wonder - why the deleted / untracked file message ?
probably something obvious, but I found it rather confusing, and again
I've seen a number of examples of this.

	Thanks,

		Michael.

PS. of course, perhaps this is 'just me' - for space / time /
simplicty / certainty reasons, I do a lot of "cp -lR foo/.git baa/" to
duplicate trees - but AFAIK all git operations are atomic and use
renames rather than in-place re-writing: right ?
-- 
 michael.meeks@novell.com  <><, Pseudo Engineer, itinerant idiot

On Feb 22, 2011, at 10:34 AM, Michael Meeks wrote:

PS. of course, perhaps this is 'just me' - for space / time /
simplicty / certainty reasons, I do a lot of "cp -lR foo/.git baa/" to
duplicate trees - but AFAIK all git operations are atomic and use
renames rather than in-place re-writing: right ?

FYI: `git clone foo bar` will use hard-links to copy the object files and is both very fast and space efficient.  (See the description of `--local` in git-clone(1), which is used by default for local repositories since git 1.5.3.)  It's also guaranteed to work while the correctness of `cp -lR` depends on implementation details of git.

~~ Brian

Michael Meeks venit, vidit, dixit 22.02.2011 16:34:

Hi there,

        So - yet again, I'm still a completely clueless git user :-)
basically the same setup and reproduction issue as last time, still
using a stable git: version 1.7.3.4

<PG>
I'm sorry you're having reproduction issues. At least your git is stable.
</PG>

Setup:
        git clone git://anongit.freedesktop.org/libreoffice/libs-core

cd libs-core # I assume

        git checkout integration/dev300_m98
        git remote add stage git://anongit.freedesktop.org/libreoffice/staging/libs-core
        git fetch stage

Reproduce:
	rm -Rf *
	git reset --hard # ie. totally clean tree.

# Those two should be no-ops after following the above.

        git merge stage/dev300

Is that stage/ooo/dev300?

	I get this output from the merge:

I get thousands of conflicts. Have the branches moved since your post?
It may be better to give us sha1 or stable tags.

Are you doing any builds before merging?

PS. of course, perhaps this is 'just me' - for space / time /
simplicty / certainty reasons, I do a lot of "cp -lR foo/.git baa/" to
duplicate trees - but AFAIK all git operations are atomic and use
renames rather than in-place re-writing: right ?

Ouch. See Brian's post :)

Michael

Hi Michael,

On Tue, 2011-02-22 at 18:30 +0100, Michael J Gruber wrote:

I get thousands of conflicts. Have the branches moved since your post?
It may be better to give us sha1 or stable tags.

	Nope; there are thousands of conflicts; a subset of these are (I would
like to think ;-) erroneous; but I'm picking out individual files with
problems that I can repeat easily and that have (I hope) simple history
to try to isolate the problems for you.

Are you doing any builds before merging?

	Builds of what ? git - no; LibreOffice - sure, it builds before the
merge - and then there is a huge slew of work to do to make it build
afterwards ;-) but then that is not so suprising.

	Anyhow - thanks for looking at it; can you replicate the suprising
result in that file: ucb/source/ucp/ext/makefile.mk ? what does
'refusing to loose untracked file' mean in that context ?

	Thanks,

		Michael.

-- 
 michael.meeks@novell.com  <><, Pseudo Engineer, itinerant idiot

Michael Meeks venit, vidit, dixit 22.02.2011 19:14:

Hi Michael,

On Tue, 2011-02-22 at 18:30 +0100, Michael J Gruber wrote:

quoted

I get thousands of conflicts. Have the branches moved since your post?
It may be better to give us sha1 or stable tags.

	Nope; there are thousands of conflicts; a subset of these are (I would
like to think ;-) erroneous; but I'm picking out individual files with
problems that I can repeat easily and that have (I hope) simple history
to try to isolate the problems for you.

quoted

Are you doing any builds before merging?

	Builds of what ? git - no; LibreOffice - sure, it builds before the
merge - and then there is a huge slew of work to do to make it build
afterwards ;-) but then that is not so suprising.

Builds of LO, naturally. The point is that possibly one branch tracks
some build products that the other doesn't track - e.g., autogenerated
make or autoconf stuff etc. (This happens easily when you change your
build chain.) That would lead to a message like that:

	Anyhow - thanks for looking at it; can you replicate the suprising
result in that file: ucb/source/ucp/ext/makefile.mk ? what does
'refusing to loose untracked file' mean in that context ?

Say, you build on branch A, that generates an untracked file foo, but
branch B (which you want to merge) tracks that. Merges refuses to
overwrite foo (even though A does not have foo) so that you don't loose
the contents of the untracked file.

But I'm really wondering whether we're merging the same revs. As I
mentioned, I don't see that branch in "stage" that you're merging, only
"stage/ooo/dev300", see below. Also, I'm getting

AA ucb/source/ucp/ext/makefile.mk

which has a somewhat surprising markup (the left side introduces no
change, the right side does - should have a trivial resolution), without
building LO before.

Note that you're merging branches which are way off,

git rev-list --count --left-right
origin/integration/dev300_m98...stage/ooo/dev300
3566    3126

and that the merge base is quite old:

af61642 (#i105937# Fixed a few remaining gradient glitches, 2010-01-16)

The latter explains many of the problems (and the "surprising" above):
compared to the merge base, both branches add
ucb/source/ucp/ext/makefile.mk as a *new* file with different contents,
so that the conflict can't be resolved automatically, and that's how it
is marked up.

Is that merge really what you're after?

Michael

Branches with your recipe:

* integration/dev300_m98
  master
  remotes/origin/HEAD -> origin/master
  remotes/origin/feature/bootstrap-build
  remotes/origin/feature/currency-64bit
  remotes/origin/feature/gnumake2.1
  remotes/origin/feature/helppack
  remotes/origin/feature/layout
  remotes/origin/feature/pptx-export-ooxml11
  remotes/origin/feature/rodatastrings
  remotes/origin/feature/sqlite
  remotes/origin/feature/winshrink
  remotes/origin/integration/dev300_m98
  remotes/origin/libreoffice-3-3
  remotes/origin/libreoffice-3-3-0
  remotes/origin/libreoffice-3-3-1
  remotes/origin/master
  remotes/stage/ooo/dev300
  remotes/stage/ooo/dev300_m100
  remotes/stage/premerge/dev300_m98


Michael

Michael J Gruber venit, vidit, dixit 23.02.2011 12:18:

Note that you're merging branches which are way off,

git rev-list --count --left-right
origin/integration/dev300_m98...stage/ooo/dev300
3566    3126

and that the merge base is quite old:

af61642 (#i105937# Fixed a few remaining gradient glitches, 2010-01-16)

Following up on this:

git rev-list --count --left-right
origin/integration/dev300_m98...stage/ooo/dev300
3566    3126

git rev-list --count --left-right --no-merges
origin/integration/dev300_m98...stage/ooo/dev300
2794    2180

git rev-list --count --left-right --no-merges --cherry-pick
origin/integration/dev300_m98...stage/ooo/dev300
1136    528

That is, 2500 of these different commits are patch-equivalent
(cherry-picks). I don't think "merge" is the best tool to combine "fake"
branches like those. You may be better off with rebase... Although I'm
wondering what the branch policy was that lead to this.

Michael

Hi Brian,

	First - it seems that the issue here was entirely bogus, not least
because we had a bug with re-writing these makefiles as we checked them
in; so hopefully only 2 issues pending ;-)

	Anyhow - I tried your kind advice:

On Tue, 2011-02-22 at 10:55 -0500, Brian Gernhardt wrote:

FYI: `git clone foo bar` will use hard-links to copy the object
files and is both very fast and space efficient.  (See the
description of `--local` in git-clone(1), which is used by
default for local repositories since git 1.5.3.)  It's also
guaranteed to work while the correctness of `cp -lR` depends
on implementation details of git.

	Sounds like just what I need. Unfortunately, it didn't clone some of
the pieces I needed; eg. other configured remotes, I ended up with just
'origin' - which was unexpected (and less wonderful than cp -lR ;-).

	Is that a feature ?

	Thanks,

		Michael.

-- 
 michael.meeks@novell.com  <><, Pseudo Engineer, itinerant idiot

Michael Meeks venit, vidit, dixit 24.02.2011 17:39:

Hi Brian,

	First - it seems that the issue here was entirely bogus, not least
because we had a bug with re-writing these makefiles as we checked them
in; so hopefully only 2 issues pending ;-)

	Anyhow - I tried your kind advice:

On Tue, 2011-02-22 at 10:55 -0500, Brian Gernhardt wrote:

quoted

FYI: `git clone foo bar` will use hard-links to copy the object
files and is both very fast and space efficient.  (See the
description of `--local` in git-clone(1), which is used by
default for local repositories since git 1.5.3.)  It's also
guaranteed to work while the correctness of `cp -lR` depends
on implementation details of git.

	Sounds like just what I need. Unfortunately, it didn't clone some of
the pieces I needed; eg. other configured remotes, I ended up with just
'origin' - which was unexpected (and less wonderful than cp -lR ;-).

	Is that a feature ?

Yes, because by cloning someone else's config they could make you do
what they want (alias...).

I think in your case you can just copy over the .git/config and maybe
set up "alternates" so that you don't have to refetch the remote objects
which are not referenced by local refs. (Alternatively, clone --mirror,
then copy over config and turn into non bare.)

Maybe we do need "clone --copy" or something as a safe version of "cp -al"?

Michael

Hi,

On Thursday 24 February 2011 17:39:20 Michael Meeks wrote:

	Anyhow - I tried your kind advice:

On Tue, 2011-02-22 at 10:55 -0500, Brian Gernhardt wrote:

quoted

FYI: `git clone foo bar` will use hard-links to copy the object
files and is both very fast and space efficient.  (See the
description of `--local` in git-clone(1), which is used by
default for local repositories since git 1.5.3.)  It's also
guaranteed to work while the correctness of `cp -lR` depends
on implementation details of git.

	Sounds like just what I need. Unfortunately, it didn't clone some of
the pieces I needed; eg. other configured remotes, I ended up with just
'origin' - which was unexpected (and less wonderful than cp -lR ;-).

See the --mirror option for clone

Per default only local refs get copied over.

Andres

Hi Andres & Brian,

	Thanks for your help;

On Fri, 2011-02-25 at 21:55 +0100, Andres Freund wrote:

quoted

On Tue, 2011-02-22 at 10:55 -0500, Brian Gernhardt wrote:

quoted

FYI: `git clone foo bar` will use hard-links to copy the object
files and is both very fast and space efficient.  (See the
description of `--local` in git-clone(1), which is used by
default for local repositories since git 1.5.3.)  It's also
guaranteed to work while the correctness of `cp -lR` depends
on implementation details of git.

..

quoted

	Sounds like just what I need. Unfortunately, it didn't clone some of
the pieces I needed; eg. other configured remotes, I ended up with just
'origin' - which was unexpected (and less wonderful than cp -lR ;-).

..

See the --mirror option for clone

	I tried all of this; none of it did what I had hoped :-)

	git clone src dest

	yields a different repository, with a totally different config /
remotes setup - missing all but a new synthetic origin pointing at the
local files and which can't be git pulled from in the normal way; ie. it
behaves extremely differently to the cp -lR result.

	git clone --mirror src dest

	yields a similar problem-repository, which is for extra measure a bare
checkout, and thus also not what I need.

	I'm really looking for an equivalent of 'cp -lR foo baa' that:

	* uses hard links to save space
	* produces precisely-a-duplicate repository

	For #2 - I would use the verb 'clone' except of course the 'clone' I'm
talking about would be one that is identical, the same, with no
differences (not eg. missing a few limbs ;-) 

	Clearly cp -lR is bad & evil and all that; but it yields exactly what I
need to effectively manage my local trees, multiple checkouts, and
different builds without burning the entire disk.

	Is there a blessed 'cp -lR' wrapper for git that is functionally
identical ? [ and I'm happy of course for some slow divergence, and loss
of efficiency as I pull more changes from time to time into each tree ].

	Sorry for the noise,

		Michael.

-- 
 michael.meeks@novell.com  <><, Pseudo Engineer, itinerant idiot

Hi Michael,

On Monday, February 28, 2011 03:25:02 PM Michael Meeks wrote:

Hi Andres & Brian,
	For #2 - I would use the verb 'clone' except of course the 'clone' I'm
talking about would be one that is identical, the same, with no
differences (not eg. missing a few limbs ;-)

	Clearly cp -lR is bad & evil and all that; but it yields exactly what I
need to effectively manage my local trees, multiple checkouts, and
different builds without burning the entire disk.

	Is there a blessed 'cp -lR' wrapper for git that is functionally
identical ? [ and I'm happy of course for some slow divergence, and loss
of efficiency as I pull more changes from time to time into each tree ].

What about git clone --reference=oldrepo ssh://upstream/ ?

Andres

On Feb 28, 2011, at 9:25 AM, Michael Meeks wrote:

	I'm really looking for an equivalent of 'cp -lR foo baa' that:

	* uses hard links to save space
	* produces precisely-a-duplicate repository

I mostly handle this sort of thing by branch switching in the same checkout, which I assume you're trying not to do because of recompilation time.  Your way, eventually the repositories in question would start to diverge and you'd have to keep them in sync manually, which sounds far less than ideal.  Eventual repacks would break the hard links and you'd even lose the space savings.

What you might be interested in is the git-new-workdir script in git.git/contrib/workdir.  It uses symbolic links to create a new working directory backed by the exact same object store and config as the original.


This is not a situation git is well designed for, honestly.  The more "git-like" work flow would be to maintain a single central, probably bare, repository on your machine that pulls from everything that all your local repositories need.  Then set up the other repositories to get all the references from the central repo.  Then there's only one that needs to be updated from the remotes.  If you use the --reference option to git-clone, you're not even duplicating the object store and each clone only has the objects it needs.

~~ Brian