Re: erratic behavior commit --allow-empty

10 messages, 4 authors, 2016-06-15 · open the first message on its own page

Re: erratic behavior commit --allow-empty

From: Matthieu Moy <hidden>
Date: 2016-06-15 22:54:55

Angelo Borsotti [off-list ref] writes:
quoted
You still didn't tell us where the problem was.
I described it few mails above. I wanted to create an orphan branch.
And you did. The branch happens to point to the same commit as another
existing commit, but this is a very common situation. Try this:

# do arbitrary hacking and commit on branch master
git checkout -b new-branch
gitk

You will see branches "master" and "new-branch" pointing to the same
commit (but you HEAD points to new-branch, as "git branch" will tell
you).

You still did not describe a _problem_. Up to now, the only "problem" I
see is that you have twice the same sha1 showing up, but you did not
describe somethine concrete that you wanted to do and did not work.
However, the branch is not actually created until a commit is done on
it.
Right, but the definition of "done" in your sentence includes "reusing
an object in the object database".

I just tried this:

rm -fr test
git init test
cd test
date > foo.txt
git add .
git commit --allow-empty -m foo
git checkout --orphan new-branch
git commit --allow-empty -m foo

I ended up with a branch "master" and a branch "new-branch", both
pointing to the same commit. The new branch _is_ created.

(BTW, --allow-empty is useless here as you have no parent)

-- 
Matthieu Moy
http://www-verimag.imag.fr/~moy/

Re: erratic behavior commit --allow-empty

From: Angelo Borsotti <hidden>
Date: 2016-06-15 22:54:55

Hi
quoted
quoted
You still didn't tell us where the problem was.
I thought I did, but here it is: I have private and a public
repositories. In the private ones the developers keep both the sources
and the binaries. In the public ones they keep only the sources. They
do not want the binaries there because binaries are very large and
requite much time to be pushed. Besides that, they are not even needed
because they must be rebuilt anyway.
To push the sources only they keep in the private repositories an
orphan branch in which commits are done taking the relevant commits in
the (say) master branch and removing the binaries from the index.
Pushing directly the master branch would push also the binaries even
if they were removed from its index (the  history gets pushed): thence
the need for an orphan branch. Scripts have been provided to do this
easily and safely. Now, it could happen that a developer does not have
(yet) binaries, but want to push all the same. The script has to take
care for this special case, in which no binaries are removed, but a
commit on the orphan branch is done all the same. And here is the
problem since git commit does not produce a brand new, different &
unique commit all the times, making then the orphan branch point to
the master one, i.e. becoming a non-orphan one.
I ended up with a branch "master" and a branch "new-branch", both
pointing to the same commit. The new branch _is_ created.
Exactly, it is created, but it is not an orphan ... or more precisely,
it is sometimes, depending on how fast you are to enter the second
commit command. This time-dependent behaviour is what I am talking
about.

-Angelo

Re: erratic behavior commit --allow-empty

From: Philip Oakley <hidden>
Date: 2016-06-15 22:54:56

From: "Angelo Borsotti" <redacted>
Sent: Wednesday, October 03, 2012 12:52 PM
Hi
quoted
quoted
quoted
You still didn't tell us where the problem was.
I've split up the explanation of your problem you have seen, to see if I 
can understand where the 'missing' aspect is within the extended 
dicussions.
I thought I did, but here it is:
I have private and a public
repositories. In the private ones the developers keep both the sources
and the binaries. In the public ones they keep only the sources. They
do not want the binaries there because binaries are very large and
requite much time to be pushed. Besides that, they are not even needed
because they must be rebuilt anyway.
To push the sources only, they keep in the private repositories an
orphan branch in which commits are done taking the relevant commits in
the (say) master branch and removing the binaries from the index.
Pushing directly the master branch would push also the binaries even
if they were removed from its index (the  history gets pushed): thence
the need for an orphan branch.
Scripts have been provided to do this
easily and safely. Now, it could happen that a developer does not have
(yet) binaries, but want to push all the same.
The script has to take
care for this special case, in which no binaries are removed, but a
commit on the orphan branch is done all the same.
And here is the
problem since git commit does not produce a brand new, different &
unique commit all the times, making then the orphan branch point to
the master one, i.e. becoming a non-orphan one.
What isn't clear is how the master branch is created and maintained at 
this point.

Does the script create it afresh each time, so that it is also, 
implicitly, an --orphan branch?
quoted
I ended up with a branch "master" and a branch "new-branch", both
pointing to the same commit. The new branch _is_ created.
In such a case (a new master being created every time the script runs), 
then you can suffer the situation you describe where you have a common 
sentinel commit being used for both branches, even though you thought 
they were orphaned from each other. - a very special case.

However one has to ask how the rest of the script would work in such 
situations with such a truncated master branch.

If the master branch has a true history, then you would get different 
commits being created on the two branches because the parents would be 
different.

Or finally, you have a truly special test (initialisation) case when you 
are starting master (which will later grow) and comparing it to the very 
first test case of the --orphan branch and in that special case you 
could get a common commit. But that is a one off special case, and would 
not recur in practice.

Can you say more about the script?
Exactly, it is created, but it is not an orphan ... or more precisely,
it is sometimes, depending on how fast you are to enter the second
commit command. This time-dependent behaviour is what I am talking
about.

-Angelo
--

Re: erratic behavior commit --allow-empty

From: Phil Hord <hidden>
Date: 2016-06-15 22:54:56

On Thu, Oct 4, 2012 at 3:07 AM, Angelo Borsotti
[off-list ref] wrote:
...
The operation that caused problems was nr. 4. In all the cases
enlisted above, a git commit creates a brand new and unique commit
because either it has a parent that is different from that of any
other commit, or because its tree is different. All, except case nr 3
when there are no binaries:

     source branch         A'
                                   :
     topic branch        A

In this case the parent is the same as that of A, i.e. none, and also
the tree is the same.
And why is this a problem?

Is there a process or person watching the server for a new commit?

Is it not enough to notice that the pushed-to branch has a new HEAD?

Phil

Re: erratic behavior commit --allow-empty

From: Angelo Borsotti <hidden>
Date: 2016-06-15 22:54:56

Hi Philip and all,

let me explain in full what is the problem that I tried to solve, and
how along the way I stumbled in something that seems to me a git bug
(at least a documentation one).

There is an R&D team developing software using a workflow that is
similar to the integerator-manager one (the one described by Scott
Chacon in chapter 5 of ProGit).
Developers implement features using a local repository hosted on their
workstations, and when finished push on a server; integrators pull
from it and put all the contributions together.
Since integrators rebuild always the software after merging all
contribution, there is no need for the developers to push the
binaries. Not pushing them speeds up uploading.
In order to make life simpler and safer, scripts are provided to
perform the pushing, pulling, etc. operations. So, most of the git
commands shown below are actually run from within scripts.
The development of each feature is done in a dedicated topic branch,
and the commits done in it contain both the sources and the binaries
(to allow to recover fully a previous snapshot when a later change
broke a previous one). When pushing, there are these needs:

      1. push the sources only
      2. push only the last commit of the topic branch (not the whole history)

A note on point 2: the integrators are not interested in seeing all
the commits that developers did while implementing their features.
Having all the history makes their repositories cluttered.

In order to avoid pushing all the history, orphan branches are used to
parallel the topic ones.
When pushing, first a commit is done on the topic branch, and then a
snapshot is created in the parallel branch with the same files,
binaries removed. The general case is:

     source branch                              D'
                                                        :
     topic branch        A----B----C---D

In the picture, the developer made 4 commits, and pushed the sources
of the last one, D.
A D' is created on the source branch (the relationship with D is
indicated with a dotted line).
The push script must cope with all the cases that may occur:

     1.  the general one (the one in the previous figure)
     2.  none of the commits in the topic branch with binaries (i.e. D
and D' with the same tree)
     3.  push done immediately after the first commit (A)
     4.  a push done after another

The script:

     1.  creates the source branch if it does not exist yet (git
checkout --orphan),
          otherwise makes HEAD point to it
     2.  sets a .git/info/exclude file that excludes the binaries
     3.  removes the binaries from the index (git rm)
     4.  creates a commit on the source branch
     5.  pushes it
     6.  restores the HEAD and index as they were before

The operation that caused problems was nr. 4. In all the cases
enlisted above, a git commit creates a brand new and unique commit
because either it has a parent that is different from that of any
other commit, or because its tree is different. All, except case nr 3
when there are no binaries:

     source branch         A'
                                   :
     topic branch        A

In this case the parent is the same as that of A, i.e. none, and also
the tree is the same. In order to try to force the creation of a brand
new and unique commit even when the trees are the same --allow-empty
has been used, but this did not avail because git commit creates a
brand new one only when the seconds of the system clock have ticked
before it.

Some of you have suggested to create an A' that is not orphan in such
a case, which is a workaround, and some others to change the message
in it, and this is another. I choose the latter because it allows to
keep the source branch orphan in all cases. So, there are workarounds,
and the script has eventually been implemented and tested, but the
unexpected, time-dependent behavior of git commit is there and someone
could stumble on it sooner or later.

-Angelo

Re: erratic behavior commit --allow-empty

From: Angelo Borsotti <hidden>
Date: 2016-06-15 22:54:56

Hi Phil,

\>
And why is this a problem?

Is there a process or person watching the server for a new commit?

Is it not enough to notice that the pushed-to branch has a new HEAD?
Yes, the developers use the git gui to see the graph of branches and commits.
The simpler and uniform it is, the better.

-Angelo

Re: erratic behavior commit --allow-empty

From: Philip Oakley <hidden>
Date: 2016-06-15 22:54:56

From: "Angelo Borsotti" <redacted>
Sent: Thursday, October 04, 2012 8:07 AM
Hi Philip and all,

let me explain in full what is the problem that I tried to solve, and
how along the way I stumbled in something that seems to me a git bug
(at least a documentation one).

There is an R&D team developing software using a workflow that is
similar to the integerator-manager one (the one described by Scott
Chacon in chapter 5 of ProGit).
This has the developers having a full copy/history of the integrators 
relevant branches, so that when the pull of the developers branch occurs 
there is a proper link to the integrators history.
Developers implement features using a local repository hosted on their
workstations, and when finished push on a server; integrators pull
from it and put all the contributions together.
Since integrators rebuild always the software after merging all
contribution, there is no need for the developers to push the
binaries. Not pushing them speeds up uploading.
In order to make life simpler and safer, scripts are provided to
perform the pushing, pulling, etc. operations. So, most of the git
commands shown below are actually run from within scripts.
The development of each feature is done in a dedicated topic branch,
and the commits done in it contain both the sources and the binaries
(to allow to recover fully a previous snapshot when a later change
broke a previous one). When pushing, there are these needs:

     1. push the sources only
     2. push only the last commit of the topic branch (not the whole 
history)

A note on point 2: the integrators are not interested in seeing all
the commits that developers did while implementing their features.
Having all the history makes their repositories cluttered.

In order to avoid pushing all the history, orphan branches are used to
parallel the topic ones.
There are other ways to create a branch which has all the developers 
feature history removed, rather tha using an --orphan, which removes the 
integrators history as well.
When pushing, first a commit is done on the topic branch, and then a
snapshot is created in the parallel branch with the same files,
binaries removed. The general case is:

    source branch                              D'
                                                       :
    topic branch        A----B----C---D

In the picture, the developer made 4 commits, and pushed the sources
of the last one, D.
A D' is created on the source branch (the relationship with D is
indicated with a dotted line).
The disconnection of the D' source branch makes it sound like you have a 
second SCM system that you have to put stuff into, which is independent 
of the development teams git repos. I have this [hassle] at my 
$dayjob -one almost has to hide git from the powers-that-be.
The push script must cope with all the cases that may occur:

    1.  the general one (the one in the previous figure)
    2.  none of the commits in the topic branch with binaries (i.e. D
and D' with the same tree)
    3.  push done immediately after the first commit (A)
    4.  a push done after another

The script:

    1.  creates the source branch if it does not exist yet (git
checkout --orphan),
         otherwise makes HEAD point to it
    2.  sets a .git/info/exclude file that excludes the binaries
    3.  removes the binaries from the index (git rm)
    4.  creates a commit on the source branch
    5.  pushes it
    6.  restores the HEAD and index as they were before

The operation that caused problems was nr. 4. In all the cases
enlisted above, a git commit creates a brand new and unique commit
because either it has a parent that is different from that of any
other commit, or because its tree is different. All, except case nr 3
when there are no binaries:

    source branch         A'
                                  :
    topic branch        A

In this case the parent is the same as that of A, i.e. none, and also
the tree is the same.
True.
In order to try to force the creation of a brand
new and unique commit even when the trees are the same --allow-empty
has been used, but this did not avail because
It was --orphan,  --allow-empty (a common tree), the --root commit, and 
scripted with both branches using the same clock tick...
git commit creates a
brand new one only when the seconds of the system clock have ticked
before it.

Some of you have suggested to create an A' that is not orphan in such
a case, which is a workaround, and some others to change the message
in it, and this is another. I choose the latter
A reasonable solution. You can also create a sentinel (--root) commit 
for any time that you need to create the source branch, just so it (the 
real source code commit) has a different parent when on source branch to 
that on the binaries branch.

However, personally, I'd have wanted the source branch to show real 
history and actually match with the integrators repo history, but no 
doubt local conditions & politics have their influence.
because it allows to
keep the source branch orphan in all cases. So, there are workarounds,
and the script has eventually been implemented and tested, but the
unexpected, time-dependent behavior of git commit is there and someone
could stumble on it sooner or later.

-Angelo

Re: erratic behavior commit --allow-empty

From: Angelo Borsotti <hidden>
Date: 2016-06-15 22:54:56

Hi Philip,
This has the developers having a full copy/history of the integrators
relevant branches, so that when the pull of the developers branch occurs
there is a proper link to the integrators history.
True.
There are other ways to create a branch which has all the developers feature
history removed, rather tha using an --orphan, which removes the integrators
history as well.
The topic branches are populated only by the developers. The integrators merge
all the topic branches into branches dedicated to the integration. In
case of need,
the developers can pull these (with all the integrators' history).
The disconnection of the D' source branch makes it sound like you have a
second SCM system that you have to put stuff into, which is independent of
the development teams git repos. I have this [hassle] at my $dayjob -one
almost has to hide git from the powers-that-be.
Well, there is another way to see this: think to a distributed SCM in
which there are some parts of the contents that are shared and some
that are not.
The technique to use disconnected branches is only a way of implementing this.
If, say, git push had an option to filter out the binaries there would
be no need for disconnected branches.
A reasonable solution. You can also create a sentinel (--root) commit for
any time that you need to create the source branch, just so it (the real
source code commit) has a different parent when on source branch to that on
the binaries branch.
Do you mean I could create an empty root commit to be used as parent for the
real source commit? Or that there is some --root option to be used?

-Angelo

Re: erratic behavior commit --allow-empty

From: Philip Oakley <hidden>
Date: 2016-06-15 22:54:56

From: "Angelo Borsotti" <redacted>
Sent: Thursday, October 04, 2012 11:09 PM
quoted
A reasonable solution. You can also create a sentinel (--root) commit
for
any time that you need to create the source branch, just so it (the
real
source code commit) has a different parent when on source branch to
that on
the binaries branch.
Do you mean I could create an empty root commit to be used as parent
for the
real source commit? Or that there is some --root option to be used?
I was using "--root" in a colloquial way. It is used in some other
commands when the very first commit is to be included in its operation.

At the point where you do the 'git checkout --orphan  <new_branch>
<start_point>' you could have separate start points ready for the source
branch and the binaries branch, and immediately do a 'git commit' to
create the unique sentinel commit before you re-checkout the developers
latest and greatest (with --force), and then do your commits on the 
source branch as before.

Another technique could be to simply switch to the sources branch, and 
then use a 'git clean -x' with an updated .gitignore ('reset' the file 
from the source branch)[or use the exclude file] to remove those now 
ignored binaries, before doing the commit.

Philip

Re: erratic behavior commit --allow-empty

From: Angelo Borsotti <hidden>
Date: 2016-06-15 22:54:56

Hi Phil,
Another technique could be to simply switch to the sources branch, and then
use a 'git clean -x' with an updated .gitignore ('reset' the file from the
source branch)[or use the exclude file] to remove those now ignored
binaries, before doing the commit.
Actually, the first time I make a git checkout --orphan to create the
branch, and the following times a git symbolic-ref HEAD to switch to
it. Then I set a proper exclude file and do a list=`git ls-files -c -i
--exclude-standard` to get the paths of the files to remove from the
index. Then I remove them with git rm --cached. Then all is ready to
make a git commit. At this point I restore the HEAD and the index as
they were before.
This allows me to keep the work tree pristine, no files removed or
loaded in it from the repo,
which makes the script quite fast.

-Angelo
Keyboard shortcuts
hback out one level
jnext message in thread
kprevious message in thread
ldrill in
Escclose help / fold thread tree
?toggle this help