Hey. I'm a newbie struggling to understand git.
I'm trying to do what seems like a simple thing in darcs, monotone,
mecurial, gnu arch, etc, but seems nearly impossible in git. There's a
central repository, a long ways away on the other side of the internet.
So I want a local repository cache. I'm going to be working on a number
of different features and different machines all simultaneously so I
really don't want them all to be pulling from the central repository.
In other systems, this is a simple star network. Clone a repository,
use, push, pull, etc. But with git, I can't push unless the cache
repository is bare, but if the cache repository is bare, then a change
to the central repository will cause the two to become wedged since
neither can push or fetch the other. It seems that git is allergic to
the dual head branch solution or something, which is surprising and
disappointing.
How do other people address these situations in git?
--rich
In message [off-list ref], Rich Pixley writes:
Hey. I'm a newbie struggling to understand git.
I'm trying to do what seems like a simple thing in darcs, monotone,
mecurial, gnu arch, etc, but seems nearly impossible in git. There's a
central repository, a long ways away on the other side of the internet.
So I want a local repository cache. I'm going to be working on a number
of different features and different machines all simultaneously so I
really don't want them all to be pulling from the central repository.
Are you working with anyone else locally? If not, then what you are
probably really trying to do is save time on fetches, so that the
latest changes are more likely to be nearby than far away.
What I would do is set up a bare backup/--mirror repository of the
upstream locally and have it automatically kept up to date with cron
or something like that. Then you can have your pull URL point to this
mirror and the push URL point to the real upstream. This will work as
long as the real upstream and the local mirror are not out of date (if
they are, you will be forbidden to push without either pulling from
the real upstream or wait for the next cron fetch and pull from your
local mirror).
This works, but requires that you separate your fetch and push URLs.
Another option is to use git "alternates" to have your local
repository also look at the automatically updated repository so that
you would only fetch over-the-network-changes since the last automatic
fetch (which, unless you had the cache have an alternate for the
primary repository, would mean that the changes would be transferred
twice).
Alternates can be problematic if you start moving repositories around
or delete them or whatever since the repository with the dangling
alternate will then be bad (until the objects reappear one way or
another), so perhaps you just want a cron job to `git fetch` or `git
remote update -p` in your local repository every so often. Then you
can just `git merge` or `git rebase` to get the latest changes instead
of `git pull [--rebase]`. This is really the simplest solution. No
extra repositories, no configuration changes, just straightforward git
operations. The only trick would be race conditions between you (as a
human) reviewing the latest changes and then typing the command to
merge/rebase them into your local branch and the cron job updating the
remoteâseeing what happened afterwords would of course work. I would
probably try this first and only start using the others if this became
problematic for some reason.
None of these cases specifically handles trying to automate pushes,
mostly because it cannot always be automatically resolved (and
depending on local standards on running test suites before any change
is pushed, perhaps should not ever be automatically resolved even for
trivial conflicts) if changes appear on the real upstream between your
last pull and your next push.
Could it be done? Sure. You can push to your local upstream and then
have it push out automatically, but if there are conflicts you will
need to deal with them, and I would suggest doing so with a bare
repository, essentially by having a static preference for the
real-upstream's changes and have the cron job send mail to you telling
you to re-pull and re-push if it failed to push out due to the remote
having changed (telling you the ref/SHA1 that it failed to push).
Of course, this isn't *that* different from just sticking a `git push`
into the background which sends mail/notifies if the push failed for
some reason, and again doing so would be much easier than an
intermediary repository solution.
But with git, I can't push unless the cache repository is bare,
but if the cache repository is bare, then a change to the central
repository will cause the two to become wedged since neither can
push or fetch the other.
Not strictly speaking true. By default git will forbid pushes to
non-bare repositories (see receive.denyCurrentBranch in man
git-config) since without special automation the working directory
will get out of date. See http://bare-vs-nonbare.gitrecipes.de/ for
more information. However, I cannot think that having to perform
integration in this second repository would actually work.
It seems that git is allergic to the dual head branch solution or
something, which is surprising and disappointing.
Git tracks your version of master separately from each other remote's
master. This is exactly dual/multiple heads. What git *does* forbid
(by default) is:
1: Letting you update someone else's checked out (non-bare) repository
underneath them
2: Letting you update someone else's repository if they have more
recent changes than you do.
Both of these defaults are really good ideas, but you can disable them
if you think you know better.
-Seth Robertson
From: Jan Krüger <hidden> Date: 2016-06-15 22:53:41
Hi Rich,
On 05/01/2012 12:30 AM, Rich Pixley wrote:
I'm trying to do what seems like a simple thing in darcs, monotone,
mecurial, gnu arch, etc, but seems nearly impossible in git. There's a
central repository, a long ways away on the other side of the internet.
So I want a local repository cache. I'm going to be working on a number
of different features and different machines all simultaneously so I
really don't want them all to be pulling from the central repository.
In other systems, this is a simple star network. Clone a repository,
use, push, pull, etc. But with git, I can't push unless the cache
repository is bare, but if the cache repository is bare, then a change
to the central repository will cause the two to become wedged since
neither can push or fetch the other.
If the 'cache repository' is set up using "git clone --mirror" and you
push to the primary repository only, that makes the cache repo a
definite slave, so you can always run "git fetch" on it without any
trouble. You can even enforce this by denying all pushes to the cache
repo, thus eliminating any chance of accidental misuse.
Conveniently, git allows you to specify a different URL for fetch and
push in your local working repositories.
HTH,
Jan
Thank you for the info and the help. Just one argument...
On 4/30/12 16:31 , Seth Robertson wrote:
It seems that git is allergic to the dual head branch solution or
something, which is surprising and disappointing.
Git tracks your version of master separately from each other remote's
master. This is exactly dual/multiple heads.
No, it isn't at all.
Multiple heads are the idea that a single commit can "branch" in the
repository and that both commits can be HEADS of the same branch at once
in a single repository. This allows a potential collision to exist in
the repository and to be pushed and pulled through multiple
repositories. It also largely eliminates this entire discussion since
each of the intermediate repositories between, say, you and I can carry
the collision. Either you or I, at will, can merge these heads just
like we'd merge any other two commits, push/fetch, etc.
That would seem to be the obvious and intuitive behavior, rather than
arbitrarily preventing the transfer.
> What git *does* forbid
(by default) is:
1: Letting you update someone else's checked out (non-bare) repository
underneath them
Yeah. That "underneath them" thing is confusing. I don't see any
reason why that should necessarily be so.
Git knows what commit is checked out. That's HEAD, yes? So what's
wrong with letting it collect other commits from other repositories
while your working directory sits? You can always commit your change
right on top of what's checked out, creating a second head for that branch.
Yes, I've read that git-diff, etc, are all making assumptions that fail
in this case, but there's nothing significantly different about
collecting commits to other branches and collecting commits to the
branch you're currently checked out from. Either way, you're going to
need to merge those into your working directory before committing your
current changes will make much semantic sense. And if you don't want to
do that, you can always commit them directly onto HEAD, and thereby
create a new branch, at least temporarily. That's one of the huge
advantages of the daggy architecture.
2: Letting you update someone else's repository if they have more
recent changes than you do.
Again, if they have more recent changes, then my line of changes should
create a fresh HEAD on that branch. Then the repositories hold all of
our changes to be merged at our leisure.
From a UI perspective, that request has a valid, and relatively obvious
semantic. That git simply refuses to do anything except produce a
cryptic error message seems... well, sad.
Both of these defaults are really good ideas, but you can disable them
if you think you know better.
I know better for source code control systems that support the multiple
HEAD concept. I don't know better for git. So far, it looks to me as
though git is just plain failing here.
I thank you for your suggestions. It'll take me a few readings before I
follow them all. Regardless of how I think git _should_ behave, I'll
still need to figure something, so thank you.
--rich
I've been reading the thread with interest.
People who know far more than I do about git, its innards, and its
design have been responding in this thread so consider this a git
*user*'s point of view:
On Tue, May 1, 2012 at 6:45 AM, Rich Pixley [off-list ref] wrote:
Multiple heads are the idea that a single commit can "branch" in the
repository and that both commits can be HEADS of the same branch at once in
a single repository. This allows a potential collision to exist in the
repository and to be pushed and pulled through multiple repositories. It
That is bizarre; I have no other word for it.
I teach git (occasionally), and if this feature existed I would
totally ignore it in my teaching material because I wouldn't know how
to defend or explain the need for "hydra branches".
It's like having two people with the same first name *and* last name
(a situation that is not impossible in real life, but is rare and
almost always requires special handling).
Does Hg do this? That would explain why my (admittedly half-hearted)
attempts to learn it have failed -- whatever tutorial I used must have
been written with the idea that hydra branches are intuitive and
logical and sane, but did not express the concept as clearly and
succinctly as you did.
Thanks for this insight; my next attempt to understand Hg, should I
ever be forced into it, might actually succeed!
On Tue, May 01, 2012 at 09:14:24AM +0530, Sitaram Chamarty wrote:
quoted
Multiple heads are the idea that a single commit can "branch" in the
repository and that both commits can be HEADS of the same branch at once in
a single repository. This allows a potential collision to exist in the
repository and to be pushed and pulled through multiple repositories. It
That is bizarre; I have no other word for it.
I teach git (occasionally), and if this feature existed I would
totally ignore it in my teaching material because I wouldn't know how
to defend or explain the need for "hydra branches".
I wouldn't use the verb branch (and certainly not "hydra branch"),
because it's confusing to someone who thinks this has something to do
with noun "branch". But that's a confusion because of the english, or
rather the terminology that was used.
I would put it this way. Every non-merge commit has a parent (we'll
ignore merge commits for now). When you look at that commit via "git
show <commit-id>", what you see is the diff between its parent and the
state of the source tree as described by that commit-id. If you put
it this way, it becomes obvious that a particular parent commit can
have multiple child commits. (This seems to be what you are calling
"hydra branches".)
A branch is a pointer to a commit. When you add a commit to a branch,
you are adding a new commit whose parent is pointing to the current
branch head, and afterwards, the branch head pointer is changed to
point at the new commit.
Does Hg do this? That would explain why my (admittedly half-hearted)
attempts to learn it have failed -- whatever tutorial I used must have
been written with the idea that hydra branches are intuitive and
logical and sane, but did not express the concept as clearly and
succinctly as you did.
What Hg does is it requires that all terminal commits (commits that do
not have children) must be named by a branch pointer. So when you
pull in some changes from Hg, there may be a non-terminal commit, but
before the hg pull finishes, it will create a merge commit which
merges the current branch pointer and the newly pulled in commits, so
that when you are done, the branch pointer points at the new merge
commit, and the requirement that there be no non-named terminal
commits is maintained.
Git differs in that you can have a child commit which is not pointed
to by a branch pointer, and which is referred to only by commit-id.
These child commits can disappear on you, when you do a garbage
collection; but it allows you to have multiple child commits hanging
off of a single parent commit, and you can do diffs, cherry picks,
etc. But they *do* have a unique name --- the commit id, which is a
SHA1 hash of the contents of the diff.
Does this help?
- Ted
On Tue, May 1, 2012 at 4:44 PM, Ted Ts'o [off-list ref] wrote:
On Tue, May 01, 2012 at 09:14:24AM +0530, Sitaram Chamarty wrote:
quoted
quoted
Multiple heads are the idea that a single commit can "branch" in the
repository and that both commits can be HEADS of the same branch at once in
a single repository. This allows a potential collision to exist in the
repository and to be pushed and pulled through multiple repositories. It
That is bizarre; I have no other word for it.
I teach git (occasionally), and if this feature existed I would
totally ignore it in my teaching material because I wouldn't know how
to defend or explain the need for "hydra branches".
I wouldn't use the verb branch (and certainly not "hydra branch"),
I coined that phrase for what was described as "[multiple] HEADS of
the same branch at once in a single repository".
it this way, it becomes obvious that a particular parent commit can
have multiple child commits. (This seems to be what you are calling
"hydra branches".)
No. In git, the multiple child commits you mention are all either
different branches, tags, or they are detached/subject to GC. At no
time do you actually have the situation he was talking about: a single
branch representing more than one leaf (no children) commit *at the
same time*.
Git differs in that you can have a child commit which is not pointed
to by a branch pointer, and which is referred to only by commit-id.
These child commits can disappear on you, when you do a garbage
collection; but it allows you to have multiple child commits hanging
off of a single parent commit, and you can do diffs, cherry picks,
etc. But they *do* have a unique name --- the commit id, which is a
SHA1 hash of the contents of the diff.
Sure.
What the original poster wants is that all these unnamed commits be
magically associated with the branch they were born in, and be
propagated via pushes and pulls.
As I understand it, he would like a one -> many relationship between
branch name and SHA.
On Tue, May 01, 2012 at 09:14:24AM +0530, Sitaram Chamarty wrote:
quoted
Does Hg do this? That would explain why my (admittedly half-hearted)
attempts to learn it have failed -- whatever tutorial I used must have
been written with the idea that hydra branches are intuitive and
logical and sane, but did not express the concept as clearly and
succinctly as you did.
What Hg does is it requires that all terminal commits (commits that do
not have children) must be named by a branch pointer.
No more so than git does. It's entirely possible to have commits that
have no branch pointer pointing to them.
So when you
pull in some changes from Hg, there may be a non-terminal commit, but
before the hg pull finishes, it will create a merge commit which
merges the current branch pointer and the newly pulled in commits, so
that when you are done, the branch pointer points at the new merge
commit, and the requirement that there be no non-named terminal
commits is maintained.
Not so. What happens is that any commit to a non-terminal commit simply
succeeds and creates an additional childless commit. If the new commit
had a branch pointer, then it continues to have that branch pointer,
even if another commit already has that branch pointer. There are just
multiple childless commits with that branch pointer.
Any merges are initiated manually. But merging any other childless
commits is the default for "hg merge". (And merge commits have two
parents).
The only merges that are done automatically are the same ones that git
does on a pull. These are sort of degenerate merges in the sense that
they exist entirely in the source code repository graph, there are no
lexical or file content collisions.
In hg, I can have revision 1 checked out, you can push, (or I can pull),
revisions 2, 3, and 4 into my repository, and my next update will merge
2, 3, and 4 into my current working directory, much like with
subversion. In git, your push is refused and I can only fetch if I'm
also willing to merge at that very moment.
Git differs in that you can have a child commit which is not pointed
to by a branch pointer, and which is referred to only by commit-id.
Hg can do this too.
These child commits can disappear on you, when you do a garbage
collection; but it allows you to have multiple child commits hanging
off of a single parent commit, and you can do diffs, cherry picks,
etc. But they *do* have a unique name --- the commit id, which is a
SHA1 hash of the contents of the diff.
Same with hg, except that they are persistent and don't disappear on
garbage collection.
--rich
On Tue, May 1, 2012 at 4:44 PM, Ted Ts'o[off-list ref] wrote:
quoted
I wouldn't use the verb branch (and certainly not "hydra branch"),
I coined that phrase for what was described as "[multiple] HEADS of
the same branch at once in a single repository".
...keeping in mind here that HEAD is a misnomer as HEAD points to the
currently checked out commit, regardless of where that commit might live
in the commit graph. It might be childless, but it might have children.
What I'm talking about is the situation where a branch can have
multiple, childless commits. I've switched to calling these "tips" for
this discussion.
Sure. What the original poster wants is that all these unnamed commits
be magically associated with the branch they were born in, and be
propagated via pushes and pulls. As I understand it, he would like a
one -> many relationship between branch name and SHA.
Really, what I want is for the push semantic to have a meaning. I want
push to work. I want pull to work even without merging. I want to be
able to share a branch between different repositories and different
users while the source code control system tracks this for me. And I
want to create arbitrary network graphs of repositories who all share
code via push/pull without manual intervention.
These are facilities that we've had in other source code control systems
for at least a decade.
It seems as though git is tracking all of the info that is needed -
excepting multiple tips. The fact that there are converters, including
dynamic, real time converters, between git repositories and the hg user
interface suggest that there is, indeed, a near one-to-one mapping
between mercurial and git. The only thing that's missing in git is the
user interface to provide this semantic. Instead, git simply doesn't do
what it is asked to do, which seems like a silly user interface choice.
--rich
From: Michael Witten <hidden> Date: 2016-06-15 22:53:42
On Tue, May 1, 2012 at 18:15, Rich Pixley [off-list ref] wrote:
I want pull to work even without merging. I want to be able to share a
branch between different repositories and different users while the source
code control system tracks this for me
I believe you are missing the point that a `pull' in git is a `fetch'
followed by a `merge'. You should read about the `fetch' command by
reading (`git help fetch'), and make sure you understand how to use
refspecs; you will probably find it very instructive to play around
by specifying explicit refspecs to `git fetch' rather than relying
on the implicit rules (which can be somewhat confusing).
On Tue, May 1, 2012 at 18:15, Rich Pixley[off-list ref] wrote:
quoted
I want pull to work even without merging. I want to be able to share a
branch between different repositories and different users while the source
code control system tracks this for me
I believe you are missing the point that a `pull' in git is a `fetch'
followed by a `merge'. You should read about the `fetch' command by
reading (`git help fetch'), and make sure you understand how to use
refspecs; you will probably find it very instructive to play around
by specifying explicit refspecs to `git fetch' rather than relying
on the implicit rules (which can be somewhat confusing).
Yes, I'm aware of the distinction within git. Confusing is an
understatement. It seems that in most cases git has no defaults nor
implicit rules and when it does, they are frequently surprising or
unfathomable. I suppose it's nice that they can be set explicitly, but
sad that they pretty much must be.
That git uses the word "pull" to mean something different than previous
source code control systems only adds to the confusion. I was using
"pull" in the more general sense of pushing and pulling data, not in the
very narrow meaning of "git fetch + git merge".
I'm still pretty much lost on refspecs and refs. The terms are
apparently not used in the manuals I've been reading and they don't seem
to be used consistently even within git error messages.
Is "refspec" the git word for the branch pointer that points to the
childless commit that defines a branch?
--rich
Hi Rich,
On 05/01/2012 12:30 AM, Rich Pixley wrote:
quoted
I'm trying to do what seems like a simple thing in darcs, monotone,
mecurial, gnu arch, etc, but seems nearly impossible in git. There's a
central repository, a long ways away on the other side of the internet.
So I want a local repository cache. I'm going to be working on a number
of different features and different machines all simultaneously so I
really don't want them all to be pulling from the central repository.
In other systems, this is a simple star network. Clone a repository,
use, push, pull, etc. But with git, I can't push unless the cache
repository is bare, but if the cache repository is bare, then a change
to the central repository will cause the two to become wedged since
neither can push or fetch the other.
If the 'cache repository' is set up using "git clone --mirror" and you
push to the primary repository only, that makes the cache repo a
definite slave, so you can always run "git fetch" on it without any
trouble. You can even enforce this by denying all pushes to the cache
repo, thus eliminating any chance of accidental misuse.
Conveniently, git allows you to specify a different URL for fetch and
push in your local working repositories.
Thank you.
For completeness, Michael Witten posted details for a comparable
architecture where the data flow is in the other direction. Leaf
repositories push/pull to/from the cache, pull from the central
repository in order to merge changes, then push to the cache to share
locally. Eventually, some leaf repository will push to the central.
Michael's approach has the advantage that the cache repository can be
unattended and that my changes can be circulated locally before becoming
visible to the wider, central repository audience.
Both approaches cleverly avoid potential collisions in the cache
repository by working around them. And, of course, some combinations of
the two will work too.
--rich
From: Philippe Vaucher <hidden> Date: 2016-06-15 22:53:42
In other systems, this is a simple star network. Clone a repository, use, push, pull, etc. But with git, I can't push unless the cache repository is bare, but if the cache repository is bare, then a change to the central repository will cause the two to become wedged since neither can push or fetch the other. It seems that git is allergic to the dual head branch solution or something, which is surprising and disappointing.
If I understand correctly, you're looking for a system where you have
a lot of systems that needs to share the modifications on their repo
*without* having a central repo. Basically you want a lot of non-bare
repositories pushing/pulling from each others.
It sounds to me that your problem could easily be solved if your
restrict yourself to "pulling" only. Whenever you want the latest
change from repo on machine A, just pull from machine A and benefit
from the automagic merging etc.
Tell me if there's something I'm missing, or maybe describe a simple
scenario we can relate with in terms of ideal commands to type and
things happening.
Thanks,
Philippe
From: Jakub Narebski <hidden> Date: 2016-06-15 22:53:43
Rich Pixley [off-list ref] writes:
On 5/1/12 11:20 , Michael Witten wrote:
quoted
On Tue, May 1, 2012 at 18:15, Rich Pixley[off-list ref] wrote:
quoted
I want pull to work even without merging. I want to be able to share a
branch between different repositories and different users while the source
code control system tracks this for me
I believe you are missing the point that a `pull' in git is a `fetch'
followed by a `merge'. You should read about the `fetch' command by
reading (`git help fetch'), and make sure you understand how to use
refspecs; you will probably find it very instructive to play around
by specifying explicit refspecs to `git fetch' rather than relying
on the implicit rules (which can be somewhat confusing).
[...]
That git uses the word "pull" to mean something different than
previous source code control systems only adds to the confusion. I
was using "pull" in the more general sense of pushing and pulling
data, not in the very narrow meaning of "git fetch + git merge".
In using "pull" vs "fetch" Git follows the convention of BitKeeper
(proprietary distributed version control system which was used for
Linux kernel development 'till "BitKeeper fiasco", and which Git
replaced).
I'm still pretty much lost on refspecs and refs. The terms are
apparently not used in the manuals I've been reading and they don't
seem to be used consistently even within git error messages.
Is "refspec" the git word for the branch pointer that points to the
childless commit that defines a branch?
"Ref" in Git is a named reference (pointer) to a commit in DAG of
revisions, i.e. either [local] branch, tag, remote-tracking branch,
etc.
"Refspec" is a specification of mapping between ref name in remote
repository and "tracking" ref in local repository, e.g.
refs/heads/*:refs/remotes/origin/*
refs/tags/*:refs/tags/*
See any of git-pull(1), git-fetch(1) and git-push(1) manpages.
--
Jakub Narebski