Re: git and time

Matthew L Foster [off-list ref] writes:

quoted

PS Nit: Git doesn't work with changesets, it works with snapshots,
building a directed graph of snapshots. Maybe that is the source of your
confusion

It's true I don't know much about git, what is the difference
between a changeset and a snapshot?  Are you saying timestamps
should be tracked separately or tracked by an scm system built
on top of git? Does/should git care about the when of a
snapshot?

I do not know what Jeff meant by snapshot vs changeset, so I
would not comment on this part.

Perhaps my question is directed more toward gitweb.cgi, it
seems to me the timestamp of when a snapshot was merged into
this repository should somehow be tracked and that is what
gitweb.cgi should default to display. For example, if someone
wants to know if security bugfix X was merged into linus'
kernel tree they also want to know when that happened, don't
they?

Each commit object in git records two timestamps.  When the
author made that change, and when the change was made into a
commit object in _some_ repository.  I _think_ gitweb shows the
latter, but I haven't checked, so Jakub is CC'ed.

What you want to know, when a particular change has become part
of the history of one branch in one repository, is not something
a git commit object records.  Enough people wanted to know that
information, so ref-log was introduced.  When it is enabled on a
branch, ref-log records when the tip of the branch changed from
what commit to what other commit.  But it primarily is meant to
answer this question: "what commit was at the tip of this branch
at time T?"

So if Linus had enabled ref-log in his public repository, and if
gitweb knew to look at ref-log, then gitweb _could_ iterate over
ref-log records for the "master" branch of Linus's repository,
find the earliest one that makes the tip of the "master" branch
a descendant of that security fix X, and report the time of that
change.  It could certainly do that.

I have to warn you that this is fairly expensive, and also I
happen to know that Linus does not have ref-log enabled on his
public repository.

Having said that, I doubt the question "when did Linus's tree
saw this security fix X commit?" has much practical value.  For
one thing, the time he merges the side branch that has the
security fix and the time he pushes out the resulting mess^Wtree
out to the public repository is different.  After pushed out to
the public repository, it takes time for that to mirror out for
public view.  From the consumer's point of view, the time it
finishes mirroring out _is_ the only timestamp that matters, but
that mirroing happens outside of git so even if ref-log showed
timestamp from the repository Linus pushes to, it would not
reflect the time the general public first saw "Linus's official
version that contained that fix".

What people would often want to know is "Does v2.6.18-rc5
include that fix?" which is a similar but different question.
This is something gitweb _could_ answer without using ref-log,
and gitk already knows how to answer it.

I somehow thought that it was possible to get "the latest tag
that precedes this commit" (aka "git describe") for each commit
by visiting its commitdiff_plain page, but I do not see it now.
Can somebody tell me if I am hallucinating?

Junio C Hamano wrote:

Matthew L Foster [off-list ref] writes:

quoted

quoted

PS Nit: Git doesn't work with changesets, it works with snapshots,
building a directed graph of snapshots. Maybe that is the source of your
confusion

It's true I don't know much about git, what is the difference
between a changeset and a snapshot?  Are you saying timestamps
should be tracked separately or tracked by an scm system built
on top of git? Does/should git care about the when of a
snapshot?

I do not know what Jeff meant by snapshot vs changeset, so I
would not comment on this part.

Me neither, but I've seen this distinction before on the mailing-list.

To my mind, a changeset is the patch that brings some form of data from 
one state (snapshot) to another. In this respect, git is certainly both 
snapshot- and changeset-based.

-- 
Andreas Ericsson                   andreas.ericsson@op5.se
OP5 AB                             www.op5.se
Tel: +46 8-230225                  Fax: +46 8-230231

Each commit object in git records two timestamps.  When the
author made that change, and when the change was made into a
commit object in _some_ repository. 

Perhaps git should record three(+) timestamps, adding when the change was committed into this
repository? Last weekend there was a committ in Linus' kernel tree with a timestamp ~2 days into
the future, which could be a problem in a scenario involving when an important bug fix was
merged/published in Linus' kernel tree, situations like that should be impossible. How can git be
said to keep an accurate record of history if time is uncertain?

-Matt

__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around 
http://mail.yahoo.com 

Matthew L Foster wrote:

How can git be
said to keep an accurate record of history if time is uncertain?

Because git doesn't care about timestamps. It stores them as comments 
(albeit auto-formatted comments) and relies on the dependency chain to 
provide history.

In the same way that contributors are expected to write clear and 
concise commit-messages, they are also expected to keep their system 
clocks somewhat in sync. Sometimes one or the other fails, and this is 
as inevitable as it can be annoying (although commit-messages along the 
line of "fixed some bugs causing some random crashes" for a commit that 
touches 2384 lines are indefinitely worse than a bad timestamp).

What's beautiful about git is that it's designed to present a correct 
history even if random-contributor-X's system clock is out of sync with 
the rest of the world, as it inevitably will be at one point or another. 
It handles content, and the order in which each piece of content was 
added/removed/mutilated/transformed into something else, and it does a 
good job at that.

All that aside though, would you rather have that fix pronto with a bad 
timestamp, or three days later when the contributor had time to set up 
ntp properly?

-- 
Andreas Ericsson                   andreas.ericsson@op5.se
OP5 AB                             www.op5.se
Tel: +46 8-230225                  Fax: +46 8-230231

On Wed, Sep 27, 2006 at 10:42:52AM +0200, Andreas Ericsson wrote:

quoted

quoted

It's true I don't know much about git, what is the difference
between a changeset and a snapshot?  Are you saying timestamps

I do not know what Jeff meant by snapshot vs changeset, so I
would not comment on this part.

Me neither, but I've seen this distinction before on the mailing-list.

To my mind, a changeset is the patch that brings some form of data from 
one state (snapshot) to another. In this respect, git is certainly both 
snapshot- and changeset-based.

I was talking specifically about the core data structure of git. The
commit object doesn't say "I'm based on commit X, and the deltas are Y."
It says "Here are the complete contents of the tree at this point, and
the previous complete contents were X."

Of course, git often shows changesets (patches, git-whatchanged, etc)
because that's what's useful to users. But the context of the discussion
was fetching commits to a repository. In that case it's important to
note that you're just grabbing the new state (albeit optimizing the
process by skipping things you have) and not "re-committing" changesets
(which is what the OP seemed to think was happening).

-Peff

On Wed, 27 Sep 2006, Matthew L Foster wrote:

Perhaps git should record three(+) timestamps, adding when the change was committed into this
repository?

If you ask for logging, that's exactly what you get.

Well, almost. 

You cannot connect the time to a _commit_, since a commit (very much by 
design) is totally immutable. Once you have created any git object, it's 
done. It can never be changed ever again, and this is not just a small 
detail, it's what the whole system builds up on, and it's where the 
security and trustworthiness fundamentally comes from (it's also where the 
naming scheme comes from - things are named by their contents, so if you 
were to ever change them again, they'd have to be renamed - they 
effectively _become_ something different).

Also, you fundamentally couldn't do it anyway, since the third time isn't 
actually even well-defined in a distributed manner. It only makes sense 
very much in a local way, and as such can never be part of the distributed 
data - and trying to add something like that to the commit would be 
fundamentally incorrect.

BUT.

What you CAN do is to connect (in any particular private repository) a 
_branch_update_ with the time it was done. That is Shawn Pierces "reflog" 
work - you can track a particular branch _locally_. It's purely local to 
that _one_ repository, though. It by definition makes no sense anywhere 
else, and it's not tracking commits, it's literally tracking how branches 
changed in a local copy.

To enable it, just add a

	[core]
		logAllRefUpdates=true

thing to your .git/config file (or, if you want to do it for _all_ the 
projects you track, you can just do it in your ~/.gitconfig file, and it 
should be the default for everything you do).

(Althernatively, you can choose to log just a _single_ branch by just 
creating the ".git/logs/refs/heads/<branchname>" file - git should start 
logging that branch automatically)

This is a reasonably new feature, so old git versions need not apply.

			Linus

Linus Torvalds wrote:

On Wed, 27 Sep 2006, Matthew L Foster wrote:

quoted

Perhaps git should record three(+) timestamps, adding when the change was committed into this
repository?

What you CAN do is to connect (in any particular private repository) a 
_branch_update_ with the time it was done. That is Shawn Pierces "reflog" 
work - you can track a particular branch _locally_. It's purely local to 
that _one_ repository, though. It by definition makes no sense anywhere 
else, and it's not tracking commits, it's literally tracking how branches 
changed in a local copy.

Well, I would simply look at the filesystem's mtime of the commit object
resp. the pack containing the commit.  IMHO good enough most of the time.

Ciao, ET.

Because git doesn't care about timestamps. It stores them as comments 
(albeit auto-formatted comments) and relies on the dependency chain to 
provide history.

Ok, the word "history" in the context of git primarily means the order of changes not the when?
Would it be a conceptual or technical issue for git to directly track the local time of
merges/changesets?

-Matt

__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around 
http://mail.yahoo.com 

On Wed, 27 Sep 2006, Edgar Toernig wrote:

Well, I would simply look at the filesystem's mtime of the commit object
resp. the pack containing the commit.  IMHO good enough most of the time.

Nope. The moment you repack, you're toast. And if you don't repack, please 
don't use large repositories.

Here's a real-world example: in the week since 2.6.18 was released, the 
kernel has gotten over twenty _thousand_ new objects. If you don't repack, 
you'll have lost about 40MB of diskspace. It adds up.

So yes, mtime works for a bit. And then it stops working ;)

		Linus

On Wed, 27 Sep 2006, Matthew L Foster wrote:

Ok, the word "history" in the context of git primarily means the order of changes not the when?
Would it be a conceptual or technical issue for git to directly track the local time of
merges/changesets?

True merges _get_ tracked - they are commits too (they just have multiple 
parents).

But it's only the time the merge was done that gets tracked, not the time 
the merge was then pushed out to somebody else.

		Linus

Matthew L Foster wrote:

quoted

Because git doesn't care about timestamps. It stores them as comments 
(albeit auto-formatted comments) and relies on the dependency chain to 
provide history.

Ok, the word "history" in the context of git primarily means the order of changes not the when?
Would it be a conceptual or technical issue for git to directly track the local time of
merges/changesets?

It is tracking the local times of each change as it is added to the
dependancy chain.  This chain then moves about between repositories
carrying its stamp with it.  When we merge a set of changes into a trunk
such as Linus does that merge will be stamped by him saying when he
merged it.  So there is plenty of time stuff in there.

Of course none of it tells you when the kernel you are running has it
in.  The only way to know that is to know when the thing was released,
under what version#, and what version you are running.

Now when we make a signed tag, doen't that make a new object too and I
assume that has a tagged date in it.  That time might really actually
mean something and a fix's relation ship to those tags might also mean
something.

-apw

--- Linus Torvalds <torvalds@osdl.org> wrote:

quoted

Ok, the word "history" in the context of git primarily means the order of changes not the

when?

quoted

Would it be a conceptual or technical issue for git to directly track the local time of
merges/changesets?

True merges _get_ tracked - they are commits too (they just have multiple 
parents).

But it's only the time the merge was done that gets tracked, not the time 
the merge was then pushed out to somebody else.

What is the difference between a merge and a "merge then pushed out"? There are at least some
situations where a repo would prefer to know its local time of a merge or pulled in merge and
anyway a local repo probably should not in any way be dependent on nor _trust_ all remote repos
timestamps...?

-Matt


__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around 
http://mail.yahoo.com 

On Wed, 27 Sep 2006, Matthew L Foster wrote:

What is the difference between a merge and a "merge then pushed out"? There are at least some
situations where a repo would prefer to know its local time of a merge or pulled in merge and
anyway a local repo probably should not in any way be dependent on nor _trust_ all remote repos
timestamps...?

Look into the ref-logging. It's exactly what you ask for.

The fact is, in a distributed system, you can _never_ make sense of 
"time". Just live with it. That's basic "distributed programming 101", and 
it's the one thing every such course should start with on the very first 
day.

So in short, you _cannot_ depend on time in a distributed environment. 
Really. Stop even asking. Please.

You can ask when some local reference was changed, and we support that 
already, and I pointed you to how to enable it in a repository you care 
about. But it's _always_ going to be just about your local repository, the 
whole question doesn't make sense any other way.

And no, it's _never_ going to tag individual merges or commits, since the 
same merge or commit can show up at DIFFERENT times in different branches, 
even within the same local repository.

So as long as you continue to ask for "commit times", you cannot get what 
you ask for. The _only_ commit time that makes sense is the time ON THE 
MACHINE that the commit was made. That's the time that git already saves 
in the commit itself. And if you don't trust that timeframe, then tough 
luck.

Git itself doesn't trust it, because git knows better. But it's there.

		Linus

On Wed, 27 Sep 2006, Matthew L Foster wrote:

What is the difference between a merge and a "merge then pushed out"?

To answer just this technical question, which came up today:

 - I might do a merge at 8:30 in the morning, but since I do it on my 
   machine, and nobody else actually saw my merge.

 - at 9:30, somebody sends me a patch that I already have (through the 
   merge), and I reply saying "I got this already", but I realize I 
   haven't pushed it out.

 - so I replicate my home machine tree to the one on master.kernel.org, 
   and now others can see it.

When did the merge happen? It happened at 8:30 on my machine, and that's 
what is recorded. End of story. No ifs, buts, maybes about it. That's the 
only time you can _ever_ see for that merge.

When did everybody else see it? It only became _visible_ in the git 
archive on kernel.org at 9:30, when I pushed out _all_ the merges that I 
had done (some of them at 8:30, others at 8:00, yet others perhaps at some 
other time).

So all those different merges, which were done at different times, only 
became visible in another git repository all at the same time, at 9:30.

There's _zero_ commits or merges that themselves happened at 9:30. There's 
no development at all that happened then. The only actual thing that 
changed at 9:30 was that the _reference_ that made all those old changes 
visible finally made it to the repository at kernel.org.

This is why you can use the ref-logging code to say "Ok, how did my 
repository look at 9:25 vs 9:35" and you could see the difference. BUT NO 
ACTUAL COMMIT OR MERGE WILL EVER HAVE THAT TIMESTAMP. That's purely a 
"when did it show up" question, and the whole question only makes sense 
within a particular repository (ie the answers are different: on _my_ 
machine, a merge showed up on 8:30 when it was made. On other machines, 
the time will always be different).

		Linus

--- Linus Torvalds <torvalds@osdl.org> wrote:

The fact is, in a distributed system, you can _never_ make sense of 
"time". Just live with it. That's basic "distributed programming 101", and 
it's the one thing every such course should start with on the very first 
day.

I agree which is exactly why I think git should conceptually prefer a repo's _local_ time. 

Commit/merge times could be specific to each repo and not generally distributed?  
 
-Matt

__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around 
http://mail.yahoo.com 

--- Linus Torvalds <torvalds@osdl.org> wrote:

 - so I replicate my home machine tree to the one on master.kernel.org, 
   and now others can see it.

When did the merge happen? It happened at 8:30 on my machine, and that's 
what is recorded. End of story. No ifs, buts, maybes about it. That's the 
only time you can _ever_ see for that merge.

Ok, so it's more complex because of the workflow issue of delayed/pseudo mirroring/replication
between private and public repos? This cloning/replication is not done through git? Are you saying
it's impossible for master.kernel.org's git to track the local time of each
commit/merge/replication? Perhaps replication time is precisely what should/could be tracked
(locally)? 

From an integrity or at least gitweb.cgi's viewpoint it seems very important to me that commit

order also be per repo consistent with time order.

-Matt

__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around 
http://mail.yahoo.com 

On Wed, 27 Sep 2006, Matthew L Foster wrote:

quoted

When did the merge happen? It happened at 8:30 on my machine, and that's 
what is recorded. End of story. No ifs, buts, maybes about it. That's the 
only time you can _ever_ see for that merge.

Ok, so it's more complex because of the workflow issue of delayed/pseudo mirroring/replication
between private and public repos? This cloning/replication is not done through git?

No, it very much happened with git.

But git will _refuse_ to rewrite history. That means that if a commit says 
it happened at 8:30AM on machine X, git will _not_ rewrite history to say 
that it happened at 9:30AM on machine Y just because that's when it made 
it to that machine.

Are you saying it's impossible for master.kernel.org's git to track the 
local time of each commit/merge/replication?

No, I'm saying that that would be _lying_.

The actual action happened at 8:30. And git tracks only truth. It doesn't 
rewrite the truth afterwards.

		Linus

Matthew L Foster [off-list ref] wrote:

No.  Its no different than anything else.  Linus' personal repository
is just as visible to you as say Andrew Morton's personal repository
(read: neither one is visible to you).  Therefore the date/time
that a given commit hits either one of those repositories is only
of interest to the people with direct access to it.  Which is Linus
and Andrew respectively.  And probably nobody else.

When Linus pushes all of those merges out to kernel.org those
commits are arriving at "his" kernel.org repository at a date/time
that is absolutely after when they first arrived in Linus' personal
repository.  Note that the kernel.org repository is a completely
different repository from Linus' personal work repository!

But due to clock skew, time zone differences, etc. between systems
those commits may actually appear to arrive at kernel.org before,
at the same time as, or after they arrived on Linus' personal system.
:-)

This cloning/replication is not done through git?

I don't know.  I'd wager Linus is probably using Git to push changes
to some repository on a master system behind the kernel.org domain
name, and that master then gets replicated out to mirror systems
through some form of replication.

Are you saying it's impossible for master.kernel.org's git to track the local time of each
commit/merge/replication? Perhaps replication time is precisely what should/could be tracked
(locally)? 

I don't think the replication time is really important here.  If it
was then Git should be used for the replication of Git repositories
being mirrored on kernel.org.  I doubt you'd get every mirror
operator to do that however.

From an integrity or at least gitweb.cgi's viewpoint it seems very important to me that commit
order also be per repo consistent with time order.

I'm not sure I follow you.  Time order has nothing to do with
anything.  That's largely been the entire point of this thread.

Because of the potentical for clock skew even on a single system
you can't take much stock in a timestamp.  But with Git you can at
least completely trust the commit graph, provided that you trust
those who made commits before your own commit.  Of course this
trust is only possible because the commit graph cannot be altered
once a node has been added into it.

As such the commit graph is consistent between repositories (assuming
they have the same head commits), but the timestamps of the reflogs
within each will widely differ.  They could widely differ even on
the same system due to ntpd updating the clock at the exact wrong
moment for example.  :)

-- 
Shawn.

--- Linus Torvalds <torvalds@osdl.org> wrote:

The actual action happened at 8:30. And git tracks only truth. It doesn't 
rewrite the truth afterward.

So the separate action of replication is not tracked? Replication/sub merges are denied the
possibility of "truth"?

To be clear I think there are actually two separate though semi-related issues:

- 1. release/replication time of a mirrored private --> public repo

- 2. A repo's commit order being inconsistent with local time order

My questions are primarily focused on #2. Last weekend even your private repo's commit order was
out of sync with your local time order because a remote git server's time was grossly
misconfigured, right? Integrity wise that shouldn't happen. I am merely asking if tracking
commits/merges using local repo time could solve both issues? If the local merge time information
is already available in the ref-log then gitweb.cgi might only need to be made aware of it.

-Matt


__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around 
http://mail.yahoo.com 

--- Shawn Pearce <spearce@spearce.org> wrote:

Because of the potentical for clock skew even on a single system
you can't take much stock in a timestamp.  But with Git you can at
least completely trust the commit graph, provided that you trust
those who made commits before your own commit.  Of course this
trust is only possible because the commit graph cannot be altered
once a node has been added into it.

As such the commit graph is consistent between repositories (assuming
they have the same head commits), but the timestamps of the reflogs
within each will widely differ.  They could widely differ even on
the same system due to ntpd updating the clock at the exact wrong
moment for example.  :)

I am not arguing for git to try to achieve "exact" time just merely locally time consistent commit
order. This might all just be a gitweb.cgi time display issue, it should be more impossible for a
commit to appear as being made 2 days in the future and impossible for local time order to be out
of sync with commit order. If each git repo used local time to track commits/merges you wouldn't
have to worry if any remote git server's time was grossly misconfigured. Time doesn't need to be
exact, all I am saying is each git repo should trust/prefer its local time rather than a remote
git server's timestamp.

-Matt


__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around 
http://mail.yahoo.com 

On Wed, 27 Sep 2006, Matthew L Foster wrote:

So the separate action of replication is not tracked?

Correct. Replication without changes is a no-op.

Replication/sub merges are denied the possibility of "truth"?

No, it's actually much deeper than that.

To git, pure replication simply isn't an action at all, so trying to track 
it would be like trying to track all the voices in my head - something 
that doesn't exist. It wouldn't be "truth", it would be insanity.

And the thing is, _not_ tracking it is really fundamental. If you actually 
track the issue of copying a git repository, you'd end up in a technically 
untenable and insane situation. You could never "merge" two git trees ever 
again without going into an infinite bouncing back-and-forth of "A merged 
the changes from B" and "B merged the fact that A merged the changes from 
B" and "A merged the fact that B merged the fact that A merged the changes 
from B" and so on ad infinitum.

There's another reason too, namely that if you track where things came 
from and when, suddenly it matters whether you cloned from the _original_ 
repository or from somewhere else. And that's also fundamnetally wrong, 
since I don't actually want to give _anybody_ access to the actual 
original repository on my machine, so everything always has to go through 
an intermediate repository. If we tracked that, we'd just confuse 
everything, and it wouldn't be seamless any more.

There's one final reason, namely that I wanted to design git to just track 
_contents_. So the design philosophy is very much against tracking exactly 
which repository something has been in, since that has nothing to do with 
the deeper issue of what you are actually tracking.

		Linus

Matthew L Foster [off-list ref] wrote:

Git does has local time order, so long as the user doesn't screw
around with their clock.

Each time a commit gets made (or a merge gets performed) Git takes
the local system clock and dumps into the new commit as part of
the "committer" comment.  But as has been stated many times before
in this thread, this is no more trustworthy than the username and
email address also appearing in that "committer" line and its not
relied upon by Git.  Some interfaces may try to sort based on this
timestamp but only to help it break ties.  The dependency graph
always wins as that's always correct.

I have a Git repository from which I'm tracking an SVN repository.
Recently that SVN repository gave me "(no date)" as a datestamp.
Git converted that to Jan 1, 1970.  That commit has a parent and
has a child, both of which have sane timestamps.  But viewing this
in gitk or git log its obvious that the Jan 1, 1970 commit is in
the right position within the graph, it just has a timestamp that
I can't trust as its 36 years in the past.

There's nothing I can do about that timestamp.  It came from another
system that I have no control over.  But Git accurately recorded
what that other system provided it.  The "truth" here is that other
system provided a bogus timestamp.

-- 
Shawn.

Linus Torvalds [off-list ref] wrote:

On Wed, 27 Sep 2006, Matthew L Foster wrote:

quoted

So the separate action of replication is not tracked?

Correct. Replication without changes is a no-op.

I think it could be reasonably argued that knowing when the head(s)
of your public repository on a mirror of kernel.org changed to a
given value is useful.  Especially when there is a mirroring lag.

-- 
Shawn.

On Wed, 27 Sep 2006, Linus Torvalds wrote:

No, it's actually much deeper than that.

To git, pure replication simply isn't an action at all, so trying to track 
it would be like trying to track all the voices in my head - something 
that doesn't exist. It wouldn't be "truth", it would be insanity.

Another reason it's not an action at all: git in many ways does not 
actually care at all about the difference of a "local branch" and a 
"remote branch on another host".

Of course there _is_ a difference, in that the remote branch has to be 
fetched from that other repository, but it's possible (and some of the 
original design came from this) to share the repository data between 
multiple separate repositories. They can even be on different machines, if 
there is a networked filesystem in between (and, unlike most systems, the 
git database format should even be happy about _disconnected_ networked 
filesystems).

So git from the ground up is designed so that there is no real difference 
between "remote branch" and "local branch", other than simply physically 
where the data might be.

By that token, "cloning" a repository is pretty much by definition a 
no-op as far as the repository contents is concerned. In fact, if you use 
"git clone -l -s", all the cost is just checking out the new copy (so if 
you add "-n" to avoid checking out the new state, you basically have a 
zero-cost clone).

	[torvalds@g5 ~]$ time git clone -n -l -s v2.6/linux empty-clone

	real    0m0.129s
	user    0m0.084s
	sys     0m0.048s

That's it. I created a "clone" of the whole kernel repo in 0.129 seconds.

Exactly because cloning doesn't actually _do_ something (of course, 0.129 
seconds in git speak is pretty slow, so I suspect we are doing something 
stupid here with shell-script).

		Linus

Ignoring the separate issue of replication for a momment, can someone respond to my time integrity
question about whether a future version of git could trust/prefer its local time rather than a
remote/sub/parent (non replicated) git server's timestamp? How do we fix gitweb.cgi, ref-log? How
useful is gitweb.cgi if timestamps are all over the place? It does not make sense that commit
order is currently out of sync with time order in the main linux kernel tree git repo on
kernel.org. Why must each and every repo be dependent on time being set properly on all other git
servers? How useful is change history or commit order without some concept of (local) time order?

-Matt 


__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around 
http://mail.yahoo.com 

On Wed, Sep 27, 2006 at 05:12:41PM -0700, Matthew L Foster wrote:

Ignoring the separate issue of replication for a momment, can someone
respond to my time integrity question about whether a future version
of git could trust/prefer its local time rather than a
remote/sub/parent (non replicated) git server's timestamp?

Yes, it could. But it would not involve rewriting the commit object or
adding a new commit object (for reasons I hope have been adequately
explained by Linus). Instead, an external mapping would be made between
a commit SHA1 and a timestamp (or a (branch,sha1) pair and a timestamp).

In fact, this is how reflogs work (but they make a map only when the
head changes, not marking each commit that enters the repo).

How do we fix gitweb.cgi, ref-log?

To "fix" gitweb, keep a database in each local repository as described
above (either based on reflog, or one that is more comprehensive). Have
gitweb report that date rather than the timestamp contained in the
commit.

Nobody has created such a patch; you can always try and see what the
response is.

How useful is gitweb.cgi if timestamps are all over the place? It does

Quite useful, according to many git users.

-Peff

Matthew L Foster [off-list ref] wrote:

Ignoring the separate issue of replication for a momment, can someone respond to my time integrity
question about whether a future version of git could trust/prefer its local time rather than a
remote/sub/parent (non replicated) git server's timestamp? How do we fix gitweb.cgi, ref-log? How
useful is gitweb.cgi if timestamps are all over the place? It does not make sense that commit
order is currently out of sync with time order in the main linux kernel tree git repo on
kernel.org. Why must each and every repo be dependent on time being set properly on all other git
servers? How useful is change history or commit order without some concept of (local) time order?

Dependency order is all that matters.

It doesn't matter if K. Hacker makes a bug fix at 8 am his local
time or 3 days ago.  All that matters is that K. Hacker made it by
changing version A to version B.  Therefore commit B (containing
the bug fix) depends on commit A and only commit A (which may in
turn depend on commit A^, etc.).

That dependency in turn implies that you can't have bug fix B without
whatever feature/bug fix was A, and what that dependend on, etc.
Thus you know you have some particular chain of events as a result
of having B.  That's all that's interesting.


It _may_ matter to me that I received commit B (and maybe commit A)
at 3 pm my local time.  It may not.

In general I don't care too much about when a commit comes to me and
when it doesn't or when it was written, though I do look at

	git log next@{yesterday}..next

to see what Junio has pushed out recently.  Since I tend to
fetch only once per day (and usually around the same time of day)
this works reasonably well.

-- 
Shawn.

On Wed, Sep 27, 2006 at 05:12:41PM -0700, Matthew L Foster wrote:

Ignoring the separate issue of replication for a momment, can
someone respond to my time integrity question about whether a future
version of git could trust/prefer its local time rather than a
remote/sub/parent (non replicated) git server's timestamp? 

No, it can't.  In order to do that it would have to change the commit,
and that would be rewriting history.

Consider a more complicated case of replication, where git is used
exclusively, so that a patch gets commited by developer A, pushed to
device driver maintainer B, which then gets pushed to subsystem
maintainer C, which then pushed to Linus's private tree, and then
Linus pushes it to the publically visible repository.  Now assume for
the sake of argument that developers A, B, and C all keep publically
visible repositories, all of which are accessible via gitweb.cgi, and
assume for the sake of argument that the patch is pushed exclusively
via git.  

In that case, the time that the changeset was made is the time that
developer A created the commit.  That is, of course, the time on his
local machine, which may or may not be accurate.  And of course, the
e-mail address which he gives may or may not be accurate as well.
There can be no integrity guarantees because we are not using
cryptography.  There are no time notaries, no public key signatures.
Both the time and the author name/e-mail are merely metadata which is
attached to the patch, and the time, author name/e-mail, and patch are
hashed together to form a SHA checksum of the commit.  So while we
don't have any kind of assurance about the time or author e-mail, what
we *do* have is an assurance that a commit's name --- which is the SHA
checksum --- refers to a specific patch, and with a specific claimed
timestamp and claimed e-mail name.  So if we refer to a particular
commit by that SHA hash value, we do know globally that we are always
referring to the same commit.  Changing the time would change the SHA
hash, which would make it impossible for two machines to know whether
or not a commit is the same or not.

More to the point, that particular commit will travel from developer
A, to B, to C, to Linus, to public repository on master.kernel.org.
During that whole time, it is invariant.  It cannot change, including
the commit timestamp.  Now, at each stage, as the patch gets pushed
along, all of the information ---- the patch itself, the claimed time,
and the claimed author date --- are used to evalute the patch and
decide whether or not the patch should be pushed higher up the trust
hierarchy.  In Linus, the most important thing which is used is an
evaluation of the patch itself.  If the patch is good, the fact that
the commit time might be six months in the past isn't necessarily a
problem.  

And of course, the commit time might be *right*.  It could be that
developer A did do the work six months ago, but merely sat on it for a
while before submitting it to maintainer B, and maybe maintainer B sat
on it because rc1 had passed and he was being a good doobie and not
forwarding patches on so that tree could stablize, and by the time rc7
was released, he had gotten tired of waiting, so pushed it on to
lieutenant C, and it finally got pushed to Linus after 2.6.18 was
released, many months later.  

How do we fix gitweb.cgi, ref-log? 
How useful is gitweb.cgi if timestamps are
all over the place? It does not make sense that commit order is
currently out of sync with time order in the main linux kernel tree
git repo on kernel.org. 

Your problem is that you are putting too much faith in the time stamp.
It's there as a help, but it is no more trustworthy than the author
name or the patch description --- which is to say, if people in the
git hierarchy are happy with the commit, it will get pushed, and if
they aren't it won't.  And most people don't care about the time; they
care about whether the patch is correct.  Now, the time is mostly
correct, so if you look at the patch dependencies you can usually
determine when a patch was likely committed to the first local
repoistory to within a few hours in all likelihood.

Now, I could imagine a system where every changeset also had metadata
associated with it that involved a receipt from a time notary, and a
digital signature so that each patch could be cryptographically shown
to have come form some individual (or at least someone who posses that
particular individual's key).  This would add a lot more overhead to
the SCM, but at least it's theoretically possible.  The question is it
worth it?

At least within the Linux commmunity, the most important thing is
whether or not the patch is good, not whether it is cryptographically
signed by some particular developer.  And times are considered even
less important --- and given that a time notary is complex, and
requires out of band facilities such as publishing lists of checksums
and checksums of checksums in publically norepudiable places such as
classifie dadds in the New York Times, LA Times, and Washington Post,
it won't be something which is zero cost, either.  And given that no
one cares enough about times to spend that kind of resources, I very
much doubt git will ever support time notary support.

Why must each and every repo be dependent on
time being set properly on all other git servers? 

They don't.

How useful is change history or commit order without some concept of
(local) time order?

Very useful.  :-)

							- Ted

--- Shawn Pearce <spearce@spearce.org> wrote:

Dependency order is all that matters.

It doesn't matter if K. Hacker makes a bug fix at 8 am his local
time or 3 days ago.  All that matters is that K. Hacker made it by
changing version A to version B.  Therefore commit B (containing
the bug fix) depends on commit A and only commit A (which may in
turn depend on commit A^, etc.).

From a web display/generic notion of integrity perspective time order matters to me but it looks

like I am the only one. Keeping track of _local_ commit time would not add any dependencies. 

-Matt


__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around 
http://mail.yahoo.com 

On Wed, 27 Sep 2006, Matthew L Foster wrote:

From a web display/generic notion of integrity perspective time order 
matters to me but it looks like I am the only one. Keeping track of 
_local_ commit time would not add any dependencies.

Actually, I think one problem here is that anybody why looks at just the 
gitweb interface may not realize how git works.

If you use gitk as your primary way of learning about a git problem, the 
whole time issue just goes away, because gitk shows the _real_ 
relationships so well.

I used gitk in all my initial explanations of git, because it turned a 
fairly abstract "here, let me explain how it works" into a "See? Look at 
this" kind of situation.

I think gitweb is great (in a way I have _never_ felt about any of the CVS 
web interfaces I have ever seen), but gitweb doesn't really explain how 
things work as well as gitk does.

		Linus

--- Theodore Tso <tytso@mit.edu> wrote:

On Wed, Sep 27, 2006 at 05:12:41PM -0700, Matthew L Foster wrote:

quoted

Ignoring the separate issue of replication for a momment, can
someone respond to my time integrity question about whether a future
version of git could trust/prefer its local time rather than a
remote/sub/parent (non replicated) git server's timestamp? 

No, it can't.  In order to do that it would have to change the commit,
and that would be rewriting history.

Perhaps the actual change itself should not contain a "commit time", only "local commit time"
should matter or be tracked locally (if time is tracked/matters any). To repeat from a previous
mail, I am not saying timestamps (local or other) should be tracked in a git distributed way,
quite the opposite, local commit time should be tracked locally. 

Replication is a separate issue if I understand git any. Please correct me if I misunderstand: the
kernel.org gitweb.cgi linux repo time inconsistencies happened when Linus pulled into his
_private_ repo from a remote git server with misconfigured time, _not_ when he later replicated
those errant timestamps from his private repo to the public kernel.org one. I don't care nearly as
much about replication not being time aware, I care about a _merge_ from a remote misconfigured
git server making timestamps inconsistent with commit order. Using/prefering local time could
solve this but perhaps I am the only one that thinks local time is most important.

-Matt


__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around 
http://mail.yahoo.com 

On Wed, 27 Sep 2006, Matthew L Foster wrote:

What I think you want and what you should talk about is that you're 
interested into the "local appearance time" for a given commit and not 
"local commit time".  Using that terminology is probably much less 
confusing in the GIT world.

To do so you'll need a GIT command that doesn'T exist yet.  Let's call 
it git-local-arrival.  It could be defined as follows:

SYNOPSIS

	git-local-arrival <committish>

DESCRIPTION

	The command displays the time when given commit appeared in the 
	local repository.

Is that what you want?  That's certainly something _I_ would be 
interested in.  But such a command would have to do some commit graph 
walking, based on the recorded reflog data, (there is not much 
documentation about reflog unfortunately) to find out exactly when given 
commit actually was fetched into the local repository.  While that would 
be perfectly acceptable to use on your own machine, I don't think it 
would be a good idea to let gitweb use it due to the computing cost 
required.

But again that's something possible but for which there is currently no 
code.

[ thinking out loud: maybe git-rev-list could provide that local 
  appearance time quite easily though... ]


Nicolas

Linus Torvalds wrote:

On Wed, 27 Sep 2006, Matthew L Foster wrote:

quoted

From a web display/generic notion of integrity perspective time order 
matters to me but it looks like I am the only one. Keeping track of 
_local_ commit time would not add any dependencies.

Actually, I think one problem here is that anybody why looks at just the 
gitweb interface may not realize how git works.

If you use gitk as your primary way of learning about a git problem, the 
whole time issue just goes away, because gitk shows the _real_ 
relationships so well.

I used gitk in all my initial explanations of git, because it turned a 
fairly abstract "here, let me explain how it works" into a "See? Look at 
this" kind of situation.

True that. I would have had a hard time introducing git as The SCM in 
the company if it hadn't been for gitk and qgit. They both let you just 
skip over 90% of that initial steep part of the learning curve and jump 
straight to work.

I think gitweb is great (in a way I have _never_ felt about any of the CVS 
web interfaces I have ever seen), but gitweb doesn't really explain how 
things work as well as gitk does.

Someone started hacking on a web-thingie to show the graph. Whatever 
happened to that? If it's no longer alive, perhaps we could add some 
qgit/gitk screenshots to the git wiki/docs so the people who spend most 
of their lives in browsers can get some visual aid in understanding the 
way git works.

-- 
Andreas Ericsson                   andreas.ericsson@op5.se
OP5 AB                             www.op5.se
Tel: +46 8-230225                  Fax: +46 8-230231

Hi,

On Fri, 29 Sep 2006, Andreas Ericsson wrote:

Someone started hacking on a web-thingie to show the graph. Whatever 
happened to that?

It is called git-browser, and was done by Artem Khodush. See 
http://straytree.com/.

I asked Artem what the plans are, since some features are not yet 
implemented, but he said that the thing is too slow, and he'll probably 
not continue to work on it.

Ciao,
Dscho

Johannes Schindelin wrote:

Hi,

On Fri, 29 Sep 2006, Andreas Ericsson wrote:

quoted

Someone started hacking on a web-thingie to show the graph. Whatever 
happened to that?

It is called git-browser, and was done by Artem Khodush. See 
http://straytree.com/.

I asked Artem what the plans are, since some features are not yet 
implemented, but he said that the thing is too slow, and he'll probably 
not continue to work on it.

Ah well. I hope he keeps that page running though, and I hope the 
"time-is-importan" people find it.

For reference, it gives a crude (and indeed slow) picture of what gitk 
and qgit does.

-- 
Andreas Ericsson                   andreas.ericsson@op5.se
OP5 AB                             www.op5.se
Tel: +46 8-230225                  Fax: +46 8-230231

Matthew L. Foster wrote:

If the local merge time information is already available in the
ref-log then gitweb.cgi might only need to be made aware of it. 

It is planned to add reflog support (view) to gitweb.

But of course the repository that is under gitweb has to have reflog 
_enabled_ to be able to view it.
-- 
Jakub Narebski
Poland

Junio C. Hamano wrote:

I somehow thought that it was possible to get "the latest tag
that precedes this commit" (aka "git describe") for each commit
by visiting its commitdiff_plain page, but I do not see it now.
Can somebody tell me if I am hallucinating?

No, as of now "commitdiff_plain" or "commit_plain" view shows either 
git-name-rev information, or just tag if the tag points exactly at 
given [child] commit, not git-describe information. Although it would 
be fairly easy to add this information, though...
-- 
Jakub Narebski
Poland