Hi all:
I'm writing a daily build script for all the OpenRISC components, so every day I need to check out several git repositories with the source code of many tools that depend on each other.
The hole process takes hours. In order to minimize the risk of repository skew, I thought I could just take the current date and time, clone all repositories to my local PC and check them all out at that particular timestamp.
I figured out that something like git "checkout HEAD@{2011-08-21 10:00:00}" does not really cut it. I'm getting this warning:
warning: Log for 'HEAD' only goes back to Sun, 21 Aug 2011 10:00:02 +0200.
The reason is, the timestamp was taken at 10:00, and the repository was cloned 2 seconds later, which means that 10:00 is earlier than the repository. That was totally unexpected, but then I found this in the documentation for "gitrevisions":
Note that this looks up the state of your local ref at
a given time; e.g., what was in your local master branch last week.
If you want to look at commits made during certain times,
see --since and --until.
I guess that means the HEAD@{date} syntax does not do what I expected. But hey, it's not the first time I find the git docs hard to follow... }8-)
By the way, it would be nice if the gitrevisions documentation could be improved, as I still don't understand what that really means. Say, for example, an hour ago I had temporarily checked out last year's versions, but half an hour ago I went back to this year's versions. If I check out at HEAD@{1 hour ago}, will I get then last year's version, or this year's?
Anyway, my real problem is with the mentioned --until option. "git checkout" does not understand that option, so I guess I need to feed the date to some other git command in order to get the commit ID for "git checkout", right? Can someone help me here?
Or even better, can someone add this kind of explanation to the "git checkout" documentation? If you are used to other version control systems, and wish to checkout the versions at a particular date, that's the documentation page you first look at.
For extra karma points, git checkout could understand the --until option itself.
Many thanks in advance,
R. Diez
From: Thomas Rast <hidden> Date: 2016-06-15 22:51:51
I'll basically reply from bottom up so you can see the motivation and
then my suggestions for the solution.
R. Diez wrote:
Note that this looks up the state of your local ref at
a given time; e.g., what was in your local master branch last week.
If you want to look at commits made during certain times,
see --since and --until.
[...]
Say, for example, an hour ago I had temporarily checked out last
year's versions, but half an hour ago I went back to this year's
versions. If I check out at HEAD@{1 hour ago}, will I get then last
year's version, or this year's?
The @{date} and @{n} syntax refers to the reflog, which as the name
tries to imply, is a log of where *your local ref* was at that
time/step. Since the HEAD ref is by definition what you have checked
out at the moment, HEAD@{1 hour ago} indeed refers to last year's
version.
Anyway, my real problem is with the mentioned --until option. "git
checkout" does not understand that option, so I guess I need to feed
the date to some other git command in order to get the commit ID for
"git checkout", right? Can someone help me here?
It is a git-log option (or more precisely, revision walker option).
The main problem is that your request is not very well-defined: in
nonlinear history there will in general be more than one commit at the
time requested.
---a----b----c----M---- (M is a merge)
\ /
d-----e----f
^---- April 1st
Suppose you ask git for "the newest commit as of April 1st" in this
history. Is it supposed to give you b or d? [If you think nonlinear
history is easy, try to figure out a good rule in the presence of time
skew, where misconfigured clocks/timezones resulted in parents being
younger than children.]
Hence:
For extra karma points, git checkout could understand the --until option itself.
It probably never will, because that is an ill-defined request.
You can indeed say
git log -1 --until="april 1"
to get *one* commit that happened before April 1st, but which one is
up to the order internally used by git. You can also say
git log -1 --first-parent --until="april 1"
to get the first such commit along the first-parent ancestry, which
might suit the ticket.
But there is a more fundamental issue. Let me explain.
I'm writing a daily build script for all the OpenRISC components, so
every day I need to check out several git repositories with the
source code of many tools that depend on each other.
The hole process takes hours. In order to minimize the risk of
repository skew
Step back and consider the real problem here. In the simplest case
you are getting two components A and B which depend on each other,
e.g., A depends on B. But there is a race condition in the case where
a user updates an API between them in a backward-incompatible way: she
has to update both A and B, and an unfortunate coworker/buildbot may
pull old-A and new-B (or vice versa) and get a broken build.
[Incidentally this seems to be a frequent problem with SVN externals.]
You might say: if only we had a way to record the fact that the
"blessed" version of B to go with old-A is old-B, and for new-A it's
new-B.
And indeed we do. Submodules were invented to allow B to be "linked"
into A's repository, such that a checkout of any commit of A "knows"
the correct corresponding version of B. A user who updates the API
can record the update to B inside the API-changing commit in A.
So while you can kludge your way around the problem with clever use of
'git log --until', submodules would be the "correct" solution.
--
Thomas Rast
trast@{inf,student}.ethz.ch
Hallo Thomas Rast:
Thanks for your quick answer. Please see mine below.
The @{date} and @{n} syntax refers to the reflog, which as
the name
tries to imply, is a log of where *your local ref* was at
that
time/step. Since the HEAD ref is by definition what
you have checked
out at the moment, HEAD@{1 hour ago} indeed refers to last
year's version.
OK, thanks. That kind of example would be nice to have in the "git checkout" documentation page. In the meantime, I've seen on the Internet that other people also got caught by this... let's say... 'unintuitive' behaviour or documentation. 8-)
It is a git-log option (or more precisely, revision walker
option).
In the meantime, I've seen this done with "git rev-list" instead, like this:
git rev-list -n 1 --before="2010-11-01 11:45:16 +0000" master
Is that the same as with git-log ?
The main problem is that your request is not very
well-defined: in
nonlinear history there will in general be more than one
commit at the
time requested.
---a----b----c----M---- (M is a merge)
\ /
d-----e----f
^----
April 1st
Suppose you ask git for "the newest commit as of April 1st"
in this history. Is it supposed to give you b or d?
I still don't quite understand how git works, but let me risk a naive statement here. If "a-b-c-M" were 'master', and "d-e-f" were 'new-feature', then on April 1st the current version on 'master' is 'b', because I merged the 'new-feature' branch at a later point in time. Does that make sense?
Step back and consider the real problem here. In the
You're right in saying there is a race condition here between developers, and the right solution would be of course to tag which versions work well with each other.
But that problem the daily build is trying to solve is precisely that it's too hard to keep track of all component versions in all repositories. Things just move too fast, and as far as I understand it, git submodules require manual intervention. If I ever tag anything manually, it must have already passed the daily build!
The development model looks like this: the latest HEAD versions of all components should always work well with each other. If something breaks, the daily build will let the developer know by the next day. If two developers make incompatible changes, they'll speak to each other and commit their changes within a few hours. During that time, they will be trouble, but that's quite alright (at least for the moment).
I just didn't want the daily build to add to the uncertainty of what went wrong by introducing a few hours' worth of random time skew to the mix.
The daily build server needs to check out from git the head status at say 02:00 am on all repositories, as if the server had so many CPUs that it had ran a "git pull" for all of them simultaneously. That's close enough for my purposes. Like stated above, if someone merges some old branch at 02:01 am, a user that did a "git pull" on master at 02:00 am would not have seen that merge, that's the effect I would like to achieve.
In the future there will probably be a stable HEAD branch and a development branch with the same name across all git repositories, and the daily build can do both every day. Or maybe the stable versions will not come from git any more, but from .tar.gz files. Another solution to automate releases without so much human intervention would be as follows: if the automated build and automated testing succeeded 3 hours ago, then that timestamp can be entered in the database of "pretty sure it works" versions. The timestamp becomes effectively the version number. Manually coordinating all participants is hard if you don't have so many human resources.
Thanks again,
R. Diez
But that problem the daily build is trying to solve is precisely that it's
too hard to keep track of all component versions in all repositories. Things
just move too fast, and as far as I understand it, git submodules require
manual intervention. If I ever tag anything manually, it must have already
passed the daily build!
Submodules can easily be scripted too. Why don't you let your buildsystem
automatically create a commit with the current HEADs in the superproject
when building and testing all repositories was successful? Then each
developer can use one of those superproject commits as starting point and
happily hack away on a repository. And as a bonus he can see in the
superproject how many changes he did from the nightly build he started
with. And if he doesn't care, he can forget about the superproject until
he needs to sync again.
The development model looks like this: the latest HEAD versions of all
components should always work well with each other. If something breaks,
the daily build will let the developer know by the next day. If two
developers make incompatible changes, they'll speak to each other and
commit their changes within a few hours. During that time, they will be
trouble, but that's quite alright (at least for the moment).
You can decide later if you want to use the superproject to coordinate
such possibly conflicting changes, but that would mean your developers
would have to commit their changes in the superproject too.
From: Michael Witten <hidden> Date: 2016-06-15 22:51:52
On Mon, Aug 22, 2011 at 15:18, R. Diez [off-list ref] wrote:
quoted
The main problem is that your request is not very
well-defined: in
nonlinear history there will in general be more than one
commit at the
time requested.
---a----b----c----M---- (M is a merge)
\ /
d-----e----f
^----April 1st
Suppose you ask git for "the newest commit as of April 1st"
in this history. Is it supposed to give you b or d?
I still don't quite understand how git works, but let me
risk a naive statement here. If "a-b-c-M" were 'master',
and "d-e-f" were 'new-feature', then on April 1st the
current version on 'master' is 'b', because I merged the
'new-feature' branch at a later point in time. Does that
make sense?
O! for the love all that is Holy!
You see, guys? The term `branch' was a TERRIBLE choice.
What git calls `branch master' in your example is just a pointer to
the commit object `M'; it has nothing to do with particular lineages
like `a-b-c-M'.
Please see my discussion with Hilco, starting here:
http://marc.info/?l=git&m=131364675708355&w=2
Message-ID: CAMOZ1BsZvXsnnWAPXR7UGKdqOMwuGB-ffaAPk55U_1dcjZUcDw@mail.gmail.com
and this email in particular:
http://marc.info/?l=git&m=131396006222173&w=2
Message-ID: CAMOZ1BvpnP_729YOHrrPW3B8wa5c4cLyD_qAQ5rTuy0JqNiiXg@mail.gmail.com
which also includes the following very germane link:
http://slashdot.org/comments.pl?sid=2350536&cid=36903136
From: Jonathan Nieder <hidden> Date: 2016-06-15 22:51:52
Michael Witten wrote:
On Mon, Aug 22, 2011 at 15:18, R. Diez [off-list ref] wrote:
quoted
I still don't quite understand how git works, but let me
risk a naive statement here. If "a-b-c-M" were 'master',
and "d-e-f" were 'new-feature', then on April 1st the
current version on 'master' is 'b', because I merged the
'new-feature' branch at a later point in time. Does that
make sense?
O! for the love all that is Holy!
Wait, what's wrong with what R. Diez said? It's exactly what
--first-parent gives you.
From: Andreas Ericsson <hidden> Date: 2016-06-15 22:51:54
On 08/23/2011 05:54 PM, Michael Witten wrote:
On Mon, Aug 22, 2011 at 15:18, R. Diez[off-list ref] wrote:
quoted
quoted
The main problem is that your request is not very
well-defined: in
nonlinear history there will in general be more than one
commit at the
time requested.
---a----b----c----M---- (M is a merge)
\ /
d-----e----f
^----April 1st
Suppose you ask git for "the newest commit as of April 1st"
in this history. Is it supposed to give you b or d?
I still don't quite understand how git works, but let me
risk a naive statement here. If "a-b-c-M" were 'master',
and "d-e-f" were 'new-feature', then on April 1st the
current version on 'master' is 'b', because I merged the
'new-feature' branch at a later point in time. Does that
make sense?
O! for the love all that is Holy!
You see, guys? The term `branch' was a TERRIBLE choice.
What git calls `branch master' in your example is just a pointer to
the commit object `M'; it has nothing to do with particular lineages
like `a-b-c-M'.
Back in 2005 when git was young and fresh, there was a discussion about
what to call things. If memory serves (which it might not), I think
the consensus was that "branch" works just fine, and when someone who
doesn't like it comes along we can just tell them that it's short for
"tip-of-branch pointer", which is far more accurate. A "ref" is always
local though, which is why the reflog (which is used for such date
resolving problems) is never even considered to work on remote refs.
They *can* work on remotes' refs though, which is a slightly different
thing.
Whatever, really. The fact that pretty much everyone seems to know
what a branch is and how it works in git after a (very) brief intro
to it means it's either right on target or that people are so used to
the fact that branch means something different in every scm that they
don't even bother loading the word with some preconceived notion that
used to be right in cvs.
--
Andreas Ericsson andreas.ericsson@op5.se
OP5 AB www.op5.se
Tel: +46 8-230225 Fax: +46 8-230231
Considering the successes of the wars on alcohol, poverty, drugs and
terror, I think we should give some serious thought to declaring war
on peace.