unable to pull from remote if commit date is in the future

Hi,

I'm following an upstream repo on github.  Today morning I saw a new
commit there, but a `git pull` in my clone did not fetch it and
instead said "Already up-to-date.".  On closer inspection, github
reports commit time as 2152-06-19. The same project has some other
commits with commit time in the future that were fetched.  My guess is
that happened when those commits got a child with commit date in the
past.

Is there any way to force git pulling that request?  (Perhaps I should
try to tell git that it's really 2152?)

For the record: the faulty commit is
https://github.com/seandepagnier/weather_routing_pi/commit/23c07cc5d2be7ce68349f4b3719b6fa6fe90e0bf

Hmm.  On closer inspection that commit 23c07cc that github shows with
date 2152-06-19 is already in my local branch.  I got confused because
locally it is shown with a different date: `git log -1 --format='%ci'
23c07cc` shows "2106-02-07 06:28:56 -40643156" which is invalid.

My system is running Debian unstable 64bit.  Is git using the time
rendering methods from the C library (glibc 2.22-12)?

On Mon, Jun 20, 2016 at 11:46 AM, Stefan Beller [off-list ref] wrote:

On Mon, Jun 20, 2016 at 11:41 AM, Norbert Kiesel [off-list ref] wrote:

quoted

Hi,

I'm following an upstream repo on github.  Today morning I saw a new
commit there, but a `git pull` in my clone did not fetch it and
instead said "Already up-to-date.".  On closer inspection, github
reports commit time as 2152-06-19. The same project has some other
commits with commit time in the future that were fetched.  My guess is
that happened when those commits got a child with commit date in the
past.

git-pull doesn't care about the commit/author date/time at all.

All it takes into consideration
is the graph structure of the commits on the local and the remote branch,
i.e. Are there any commits on the remote branch that are not part of the local
branch history?

quoted

Is there any way to force git pulling that request?  (Perhaps I should
try to tell git that it's really 2152?)

You need to see if that commit is part of the history of the
remote branch you pulled.

quoted

For the record: the faulty commit is
https://github.com/seandepagnier/weather_routing_pi/commit/23c07cc5d2be7ce68349f4b3719b6fa6fe90e0bf
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

On Mon, Jun 20, 2016 at 11:41 AM, Norbert Kiesel [off-list ref] wrote:

Hi,

I'm following an upstream repo on github.  Today morning I saw a new
commit there, but a `git pull` in my clone did not fetch it and
instead said "Already up-to-date.".  On closer inspection, github
reports commit time as 2152-06-19. The same project has some other
commits with commit time in the future that were fetched.  My guess is
that happened when those commits got a child with commit date in the
past.

git-pull doesn't care about the commit/author date/time at all.

All it takes into consideration
is the graph structure of the commits on the local and the remote branch,
i.e. Are there any commits on the remote branch that are not part of the local
branch history?

Is there any way to force git pulling that request?  (Perhaps I should
try to tell git that it's really 2152?)

You need to see if that commit is part of the history of the
remote branch you pulled.

For the record: the faulty commit is
https://github.com/seandepagnier/weather_routing_pi/commit/23c07cc5d2be7ce68349f4b3719b6fa6fe90e0bf
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

On Mon, Jun 20, 2016 at 12:05:07PM -0700, Norbert Kiesel wrote:

Hmm.  On closer inspection that commit 23c07cc that github shows with
date 2152-06-19 is already in my local branch.  I got confused because
locally it is shown with a different date: `git log -1 --format='%ci'
23c07cc` shows "2106-02-07 06:28:56 -40643156" which is invalid.

My system is running Debian unstable 64bit.  Is git using the time
rendering methods from the C library (glibc 2.22-12)?

No, git's time code is (mostly) internal routines. Can you show us the
output of:

    git cat-file commit 23c07cc | egrep '^author|committer'

Note also that some interfaces (like "git log", and GitHub) will show
the author date by default, which might be different than the committer
date. The "-40643156" timezone definitely looks suspicious, though. I'm
curious if it is bad handling in the time code, or if the commit has
corrupt ident lines.

-Peff

On Mon, Jun 20, 2016 at 12:39 PM, Jeff King [off-list ref] wrote:

git cat-file commit 23c07cc | egrep '^author|committer'

author Sean D'Epagnier [off-list ref] 5758122296 -40643156
committer Sean D'Epagnier [off-list ref] 5758122296 -40643156

date --date='@5758122296' returns "Mon Jun 19 15:24:56 PDT 2152" which
is what is shown by github.

On Mon, Jun 20, 2016 at 03:39:28PM -0400, Jeff King wrote:

On Mon, Jun 20, 2016 at 12:05:07PM -0700, Norbert Kiesel wrote:

quoted

Hmm.  On closer inspection that commit 23c07cc that github shows with
date 2152-06-19 is already in my local branch.  I got confused because
locally it is shown with a different date: `git log -1 --format='%ci'
23c07cc` shows "2106-02-07 06:28:56 -40643156" which is invalid.

My system is running Debian unstable 64bit.  Is git using the time
rendering methods from the C library (glibc 2.22-12)?

No, git's time code is (mostly) internal routines. Can you show us the
output of:

    git cat-file commit 23c07cc | egrep '^author|committer'

Note also that some interfaces (like "git log", and GitHub) will show
the author date by default, which might be different than the committer
date. The "-40643156" timezone definitely looks suspicious, though. I'm
curious if it is bad handling in the time code, or if the commit has
corrupt ident lines.

Actually, I just noticed in your earlier message a link to the public
GitHub repository.

So to answer my own questions:

  $ git cat-file commit 23c07cc | egrep '^author|committer'
  author Sean D'Epagnier [off-list ref] 5758122296 -40643156
  committer Sean D'Epagnier [off-list ref] 5758122296 -40643156

Yes, the timezone really is that ridiculous value. No, the author and
committer aren't different. According to GNU date, the correct timestamp
is actually in 2152. Offhand, I'd guess that the timestamp exceeding
2^32 is getting converted somewhere inside git to a bogus value, and
that's how we end up with 2106.

-Peff

On Mon, 20 Jun 2016 15:39:28 -0400, Jeff King wrote:

On Mon, Jun 20, 2016 at 12:05:07PM -0700, Norbert Kiesel wrote:

quoted

Hmm.  On closer inspection that commit 23c07cc that github shows with
date 2152-06-19 is already in my local branch.  I got confused because
locally it is shown with a different date: `git log -1 --format='%ci'
23c07cc` shows "2106-02-07 06:28:56 -40643156" which is invalid.

My system is running Debian unstable 64bit.  Is git using the time
rendering methods from the C library (glibc 2.22-12)?

No, git's time code is (mostly) internal routines. Can you show us the
output of:

    git cat-file commit 23c07cc | egrep '^author|committer'

Note also that some interfaces (like "git log", and GitHub) will show
the author date by default, which might be different than the committer
date. The "-40643156" timezone definitely looks suspicious, though. I'm
curious if it is bad handling in the time code, or if the commit has
corrupt ident lines.

-Peff

  2106 is the year of unsigned 32-bit unix time bug, would there be any relation ?

-- 
  Eric Deplagne

On Mon, Jun 20, 2016 at 03:46:49PM -0400, Jeff King wrote:

So to answer my own questions:

  $ git cat-file commit 23c07cc | egrep '^author|committer'
  author Sean D'Epagnier [off-list ref] 5758122296 -40643156
  committer Sean D'Epagnier [off-list ref] 5758122296 -40643156

Yes, the timezone really is that ridiculous value. No, the author and
committer aren't different. According to GNU date, the correct timestamp
is actually in 2152. Offhand, I'd guess that the timestamp exceeding
2^32 is getting converted somewhere inside git to a bogus value, and
that's how we end up with 2106.

Ah, nope. Everything is working as designed.

5758122296 _is_ in 2152, but that's before we apply the author's
timezone offset. :)

Timezones are supposed to be [+-]HHMM. So the -40643156 timezone is
parsed as -406431 hours, 56 minutes. Which is about 46 years. Hence git
printing 2106.

You _should_ be able to get the right answer by asking git for
--date=local, but it doesn't seem to work. Looks like it is because our
tm_to_time_t hits this code:

  if (year < 0 || year > 129) /* algo only works for 1970-2099 */
	return -1;

and the caller does not actually check the error. The resulting timezone
is the screwed-up -40643156, which is perhaps how it got into the commit
in the first place.

-Peff

On Mon, Jun 20, 2016 at 09:49:47PM +0200, Eric Deplagne wrote:

On Mon, 20 Jun 2016 15:39:28 -0400, Jeff King wrote:

quoted

On Mon, Jun 20, 2016 at 12:05:07PM -0700, Norbert Kiesel wrote:

quoted

Hmm.  On closer inspection that commit 23c07cc that github shows with
date 2152-06-19 is already in my local branch.  I got confused because
locally it is shown with a different date: `git log -1 --format='%ci'
23c07cc` shows "2106-02-07 06:28:56 -40643156" which is invalid.

My system is running Debian unstable 64bit.  Is git using the time
rendering methods from the C library (glibc 2.22-12)?

No, git's time code is (mostly) internal routines. Can you show us the
output of:

    git cat-file commit 23c07cc | egrep '^author|committer'

Note also that some interfaces (like "git log", and GitHub) will show
the author date by default, which might be different than the committer
date. The "-40643156" timezone definitely looks suspicious, though. I'm
curious if it is bad handling in the time code, or if the commit has
corrupt ident lines.

-Peff

  2106 is the year of unsigned 32-bit unix time bug, would there be any relation ?

In an extremely roundabout way, yes. That -40643156 time zone really is
"minus 46 years", but it was generated by _different_ code trying to
compute the author timezone on the fly and using a stray "-1". So I
suspect that no matter what time you ask for in the year 2152 (or
later), the same process would end up with the 2106 time, as the
timezone is custom-computed to end up back at the same error point.

-Peff

We ended up testing some of these date formats throughout
the rest of the suite (e.g., via for-each-ref's
"$(authordate:...)" format), but we never did so
systematically. t0006 is the right place for unit-testing of
our date-handling code.

Signed-off-by: Jeff King <redacted>
---
 t/helper/test-date.c | 26 ++++++++++++++++++++++++++
 t/t0006-date.sh      | 21 +++++++++++++++++++++
 2 files changed, 47 insertions(+)

diff --git a/t/helper/test-date.c b/t/helper/test-date.c
index 8ebcded..d9ab360 100644
--- a/t/helper/test-date.c
+++ b/t/helper/test-date.c

@@ -2,6 +2,7 @@
 
 static const char *usage_msg = "\n"
 "  test-date relative [time_t]...\n"
+"  test-date show:<format> [time_t]...\n"
 "  test-date parse [date]...\n"
 "  test-date approxidate [date]...\n";
 

@@ -17,6 +18,29 @@ static void show_relative_dates(char **argv, struct timeval *now)
 	strbuf_release(&buf);
 }
 
+static void show_dates(char **argv, const char *format)
+{
+	struct date_mode mode;
+
+	parse_date_format(format, &mode);
+	for (; *argv; argv++) {
+		char *arg = *argv;
+		time_t t;
+		int tz;
+
+		/*
+		 * Do not use our normal timestamp parsing here, as the point
+		 * is to test the formatting code in isolation.
+		 */
+		t = strtol(arg, &arg, 10);
+		while (*arg == ' ')
+			arg++;
+		tz = atoi(arg);
+
+		printf("%s -> %s\n", *argv, show_date(t, tz, &mode));
+	}
+}
+
 static void parse_dates(char **argv, struct timeval *now)
 {
 	struct strbuf result = STRBUF_INIT;

@@ -63,6 +87,8 @@ int main(int argc, char **argv)
 		usage(usage_msg);
 	if (!strcmp(*argv, "relative"))
 		show_relative_dates(argv+1, &now);
+	else if (skip_prefix(*argv, "show:", &x))
+		show_dates(argv+1, x);
 	else if (!strcmp(*argv, "parse"))
 		parse_dates(argv+1, &now);
 	else if (!strcmp(*argv, "approxidate"))

diff --git a/t/t0006-date.sh b/t/t0006-date.sh
index fa05269..57033dd 100755
--- a/t/t0006-date.sh
+++ b/t/t0006-date.sh

@@ -27,6 +27,27 @@ check_relative 630000000 '20 years ago'
 check_relative 31449600 '12 months ago'
 check_relative 62985600 '2 years ago'
 
+check_show () {
+	format=$1
+	time=$2
+	expect=$3
+	test_expect_${4:-success} "show date ($format:$time)" '
+		echo "$time -> $expect" >expect &&
+		test-date show:$format "$time" >actual &&
+		test_cmp expect actual
+	'
+}
+
+# arbitrary but sensible time for examples
+TIME='1466000000 +0200'
+check_show iso8601 "$TIME" '2016-06-15 16:13:20 +0200'
+check_show iso8601-strict "$TIME" '2016-06-15T16:13:20+02:00'
+check_show rfc2822 "$TIME" 'Wed, 15 Jun 2016 16:13:20 +0200'
+check_show short "$TIME" '2016-06-15'
+check_show default "$TIME" 'Wed Jun 15 16:13:20 2016 +0200'
+check_show raw "$TIME" '1466000000 +0200'
+check_show iso-local "$TIME" '2016-06-15 14:13:20 +0000'
+
 check_parse() {
 	echo "$1 -> $2" >expect
 	test_expect_${4:-success} "parse date ($1${3:+ TZ=$3})" "

-- 
2.9.0.167.g9e4667c

When we want to know the local timezone offset at a given
timestamp, we compute it by asking for localtime() at the
given time, and comparing the offset to GMT at that time.
However, there's some juggling between time_t and "struct
tm" which happens, which involves calling our own
tm_to_time_t().

If that function returns an error (e.g., because it only
handles dates up to the year 2099), it returns "-1", which
we treat as a time_t, and is clearly bogus, leading to
bizarre timestamps (that seem to always adjust the time back
to (time_t)(uint32_t)-1, in the year 2106).

It's not a good idea for local_tzoffset() to simply die
here; it would make it hard to run "git log" on a repository
with funny timestamps. Instead, let's just treat such cases
as "zero offset".

Reported-by: Norbert Kiesel <redacted>
Signed-off-by: Jeff King <redacted>
---
 date.c          | 2 ++
 t/t0006-date.sh | 5 +++++
 2 files changed, 7 insertions(+)

diff --git a/date.c b/date.c
index 7c9f769..4c7aa9b 100644
--- a/date.c
+++ b/date.c

@@ -74,6 +74,8 @@ static int local_tzoffset(unsigned long time)
 	localtime_r(&t, &tm);
 	t_local = tm_to_time_t(&tm);
 
+	if (t_local == -1)
+		return 0; /* error; just use +0000 */
 	if (t_local < t) {
 		eastwest = -1;
 		offset = t - t_local;

diff --git a/t/t0006-date.sh b/t/t0006-date.sh
index 57033dd..04ce535 100755
--- a/t/t0006-date.sh
+++ b/t/t0006-date.sh

@@ -48,6 +48,11 @@ check_show default "$TIME" 'Wed Jun 15 16:13:20 2016 +0200'
 check_show raw "$TIME" '1466000000 +0200'
 check_show iso-local "$TIME" '2016-06-15 14:13:20 +0000'
 
+# arbitrary time absurdly far in the future
+FUTURE="5758122296 -0400"
+check_show iso       "$FUTURE" "2152-06-19 18:24:56 -0400"
+check_show iso-local "$FUTURE" "2152-06-19 22:24:56 +0000"
+
 check_parse() {
 	echo "$1 -> $2" >expect
 	test_expect_${4:-success} "parse date ($1${3:+ TZ=$3})" "

-- 
2.9.0.167.g9e4667c

On Mon, Jun 20, 2016 at 04:00:12PM -0400, Jeff King wrote:

You _should_ be able to get the right answer by asking git for
--date=local, but it doesn't seem to work. Looks like it is because our
tm_to_time_t hits this code:

  if (year < 0 || year > 129) /* algo only works for 1970-2099 */
	return -1;

and the caller does not actually check the error. The resulting timezone
is the screwed-up -40643156, which is perhaps how it got into the commit
in the first place.

So here's a patch to fix that (along with some test infrastructure to
support it). I still don't know how that screwed-up timestamp got _into_
a commit, so perhaps there is another bug lurking.  I couldn't convince
git to parse anything beyond 2100, and committing with
GIT_AUTHOR_DATE='@5758122296 +0000' works just fine.

  [1/3]: t0006: rename test-date's "show" to "relative"
  [2/3]: t0006: test various date formats
  [3/3]: local_tzoffset: detect errors from tm_to_time_t

-Peff

The "show" tests are really only checking relative formats;
we should make that more clear.

This also frees up the "show" name to later check other
formats. We could later fold "relative" into a more generic
"show" command, but it's not worth it.  Relative times are a
special case already because we have to munge the concept of
"now" in our tests.

Signed-off-by: Jeff King <redacted>
---
 t/helper/test-date.c |  8 ++++----
 t/t0006-date.sh      | 26 +++++++++++++-------------
 2 files changed, 17 insertions(+), 17 deletions(-)

diff --git a/t/helper/test-date.c b/t/helper/test-date.c
index 63f3735..8ebcded 100644
--- a/t/helper/test-date.c
+++ b/t/helper/test-date.c

@@ -1,11 +1,11 @@
 #include "cache.h"
 
 static const char *usage_msg = "\n"
-"  test-date show [time_t]...\n"
+"  test-date relative [time_t]...\n"
 "  test-date parse [date]...\n"
 "  test-date approxidate [date]...\n";
 
-static void show_dates(char **argv, struct timeval *now)
+static void show_relative_dates(char **argv, struct timeval *now)
 {
 	struct strbuf buf = STRBUF_INIT;
 

@@ -61,8 +61,8 @@ int main(int argc, char **argv)
 	argv++;
 	if (!*argv)
 		usage(usage_msg);
-	if (!strcmp(*argv, "show"))
-		show_dates(argv+1, &now);
+	if (!strcmp(*argv, "relative"))
+		show_relative_dates(argv+1, &now);
 	else if (!strcmp(*argv, "parse"))
 		parse_dates(argv+1, &now);
 	else if (!strcmp(*argv, "approxidate"))

diff --git a/t/t0006-date.sh b/t/t0006-date.sh
index fac0986..fa05269 100755
--- a/t/t0006-date.sh
+++ b/t/t0006-date.sh

@@ -6,26 +6,26 @@ test_description='test date parsing and printing'
 # arbitrary reference time: 2009-08-30 19:20:00
 TEST_DATE_NOW=1251660000; export TEST_DATE_NOW
 
-check_show() {
+check_relative() {
 	t=$(($TEST_DATE_NOW - $1))
 	echo "$t -> $2" >expect
 	test_expect_${3:-success} "relative date ($2)" "
-	test-date show $t >actual &&
+	test-date relative $t >actual &&
 	test_i18ncmp expect actual
 	"
 }
 
-check_show 5 '5 seconds ago'
-check_show 300 '5 minutes ago'
-check_show 18000 '5 hours ago'
-check_show 432000 '5 days ago'
-check_show 1728000 '3 weeks ago'
-check_show 13000000 '5 months ago'
-check_show 37500000 '1 year, 2 months ago'
-check_show 55188000 '1 year, 9 months ago'
-check_show 630000000 '20 years ago'
-check_show 31449600 '12 months ago'
-check_show 62985600 '2 years ago'
+check_relative 5 '5 seconds ago'
+check_relative 300 '5 minutes ago'
+check_relative 18000 '5 hours ago'
+check_relative 432000 '5 days ago'
+check_relative 1728000 '3 weeks ago'
+check_relative 13000000 '5 months ago'
+check_relative 37500000 '1 year, 2 months ago'
+check_relative 55188000 '1 year, 9 months ago'
+check_relative 630000000 '20 years ago'
+check_relative 31449600 '12 months ago'
+check_relative 62985600 '2 years ago'
 
 check_parse() {
 	echo "$1 -> $2" >expect

-- 
2.9.0.167.g9e4667c