git-2.13.0: log --date=format:%z not working

The following commands work as expected (using commit b06d364310
in the git://git.kernel.org/pub/scm/git/git.git repo as test case):

$ export TZ=Europe/Berlin
$ git --no-pager log -1 --pretty="%ad" --date=iso b06d364310
2017-05-09 23:26:02 +0900
$ git --no-pager log -1 --pretty="%ad" --date=iso-local b06d364310
2017-05-09 16:26:02 +0200

However, if I use explicit format: then the output of the time zone is
wrong:

$ git --no-pager log -1 --pretty="%ad" --date="format:%F %T %z" b06d364310
2017-05-09 23:26:02 +0000
$ git --no-pager log -1 --pretty="%ad" --date="format-local:%F %T %z" b06d364310
2017-05-09 16:26:02 +0000

I would expect the output to be the same as in the first two examples.

On Fri, May 26, 2017 at 8:33 PM, Ulrich Mueller [off-list ref] wrote:

The following commands work as expected (using commit b06d364310
in the git://git.kernel.org/pub/scm/git/git.git repo as test case):

$ export TZ=Europe/Berlin
$ git --no-pager log -1 --pretty="%ad" --date=iso b06d364310
2017-05-09 23:26:02 +0900
$ git --no-pager log -1 --pretty="%ad" --date=iso-local b06d364310
2017-05-09 16:26:02 +0200

However, if I use explicit format: then the output of the time zone is
wrong:

$ git --no-pager log -1 --pretty="%ad" --date="format:%F %T %z" b06d364310
2017-05-09 23:26:02 +0000
$ git --no-pager log -1 --pretty="%ad" --date="format-local:%F %T %z" b06d364310
2017-05-09 16:26:02 +0000

I would expect the output to be the same as in the first two examples.

This patch solves half of your problem, i.e. makes the latter
format-local case work:

diff --git a/date.c b/date.c
index 63fa99685e..469306ebf5 100644
--- a/date.c
+++ b/date.c

@@ -70,6 +70,12 @@ static struct tm *time_to_tm(timestamp_t time, int tz)
        return gmtime(&t);
 }

+static struct tm *time_to_local_tm(timestamp_t time, int tz)
+{
+       time_t t = gm_time_t(time, tz);
+       return localtime(&t);
+}
+
 /*
  * What value of "tz" was in effect back then at "time" in the
  * local timezone?

@@ -214,10 +220,14 @@ const char *show_date(timestamp_t time, int tz,

const struct date_mode *mode)
                return timebuf.buf;
        }

-       tm = time_to_tm(time, tz);
-       if (!tm) {
-               tm = time_to_tm(0, 0);
-               tz = 0;
+       if (mode->type == DATE_STRFTIME) {
+               tm = time_to_local_tm(time, tz);
+       } else {
+               tm = time_to_tm(time, tz);
+               if (!tm) {
+                       tm = time_to_tm(0, 0);
+                       tz = 0;
+               }
         }

        strbuf_reset(&timebuf);

There's another test which breaks if we just s/gmtime/localtime/g. As
far as I can tell to make the non-local case work we'd need to do a
whole dance where we set the TZ variable to e.g. UTC$offset, then call
strftime(), then call it again. Maybe there's some way to just specify
the tz offset, but I didn't find any in a quick skimming of time.h.

On Sat, May 27, 2017 at 06:57:08PM +0200, Ævar Arnfjörð Bjarmason wrote:

There's another test which breaks if we just s/gmtime/localtime/g. As
far as I can tell to make the non-local case work we'd need to do a
whole dance where we set the TZ variable to e.g. UTC$offset, then call
strftime(), then call it again. Maybe there's some way to just specify
the tz offset, but I didn't find any in a quick skimming of time.h.

There isn't. At least on _some_ platforms, the zone information is
embedded in "struct tm" and stored by gmtime() and localtime(), but the
fields aren't publicly accessible. Which is why your patch worked for
format-local (it swaps out gmtime() for localtime() which sets those
fields behind the scenes). But:

  - I'm not sure that's guaranteed by the standard; strftime() might get
    its zone information elsewhere (if it needs to reliably distinguish
    between gmtime() and localtime() results it has to at least set a
    bit in the "struct tm", but that bit may not be the full zone info).

  - Even if it does work, you're stuck with only the local timezone. In
    theory you could temporarily tweak the process's timezone, call
    localtime(), and then tweak it back. I was never able to get that to
    work (links below).

On glibc, at least, you can access the zone fields in "struct tm" by
compiling with _DEFAULT_SOURCE.

So I think the best we could do is probably to have a feature macro like
TM_HAS_GMTOFF, and set tm->tm_gmtoff and tm->tm_zone on platforms that
support it. I'm not sure what we'd put in the latter, though; we don't
actually have the timezone name at all (we just have "+0200" or whatever
we parsed from the git object, but that would be better than nothing).

That leaves other platforms still broken, but like I said, I don't think
there's a portable solution.

Here are some links to past explorations:

  http://public-inbox.org/git/20160208152858.GA17226@sigill.intra.peff.net/

  http://public-inbox.org/git/87vb2d37ea.fsf@web.de/

-Peff

On Sat, May 27, 2017 at 11:46 PM, Jeff King [off-list ref] wrote:

On Sat, May 27, 2017 at 06:57:08PM +0200, Ævar Arnfjörð Bjarmason wrote:

quoted

There's another test which breaks if we just s/gmtime/localtime/g. As
far as I can tell to make the non-local case work we'd need to do a
whole dance where we set the TZ variable to e.g. UTC$offset, then call
strftime(), then call it again. Maybe there's some way to just specify
the tz offset, but I didn't find any in a quick skimming of time.h.

There isn't. At least on _some_ platforms, the zone information is
embedded in "struct tm" and stored by gmtime() and localtime(), but the
fields aren't publicly accessible. Which is why your patch worked for
format-local (it swaps out gmtime() for localtime() which sets those
fields behind the scenes). But:

  - I'm not sure that's guaranteed by the standard; strftime() might get
    its zone information elsewhere (if it needs to reliably distinguish
    between gmtime() and localtime() results it has to at least set a
    bit in the "struct tm", but that bit may not be the full zone info).

  - Even if it does work, you're stuck with only the local timezone. In
    theory you could temporarily tweak the process's timezone, call
    localtime(), and then tweak it back. I was never able to get that to
    work (links below).

On glibc, at least, you can access the zone fields in "struct tm" by
compiling with _DEFAULT_SOURCE.

So I think the best we could do is probably to have a feature macro like
TM_HAS_GMTOFF, and set tm->tm_gmtoff and tm->tm_zone on platforms that
support it. I'm not sure what we'd put in the latter, though; we don't
actually have the timezone name at all (we just have "+0200" or whatever
we parsed from the git object, but that would be better than nothing).

That leaves other platforms still broken, but like I said, I don't think
there's a portable solution.

Here are some links to past explorations:

  http://public-inbox.org/git/20160208152858.GA17226@sigill.intra.peff.net/

  http://public-inbox.org/git/87vb2d37ea.fsf@web.de/

There's a third and possibly least shitty option that isn't covered in
those threads; We could just make a pass over the strftime format
ourselves and replace %z and %Z with the timezone (as done for
DATE_ISO8601_STRICT in date.c), then hand the rest off to strftime().

I'm not going to pursue it though, Ulrich, are you maybe interested in
hacking on that?

Am 27.05.2017 um 23:46 schrieb Jeff King:

On Sat, May 27, 2017 at 06:57:08PM +0200, Ævar Arnfjörð Bjarmason wrote:

quoted

There's another test which breaks if we just s/gmtime/localtime/g. As
far as I can tell to make the non-local case work we'd need to do a
whole dance where we set the TZ variable to e.g. UTC$offset, then call
strftime(), then call it again. Maybe there's some way to just specify
the tz offset, but I didn't find any in a quick skimming of time.h.

There isn't.

Right.  We could handle %z internally, though.  %Z would be harder (left
as an exercise for readers..).

First we'd have to undo 0a0416a3 (strbuf_expand: convert "%%" to "%"),
though, in order to give full control back to strbuf_expand callbacks.

2-pack patch:

--- >8 ---
 builtin/cat-file.c         |  5 +++++
 convert.c                  |  1 +
 daemon.c                   |  3 +++
 date.c                     |  2 +-
 ll-merge.c                 |  5 +++--
 pretty.c                   |  3 +++
 strbuf.c                   | 39 ++++++++++++++++++++++++++++++---------
 strbuf.h                   | 11 +++++------
 t/t6006-rev-list-format.sh | 12 ++++++++++++
 9 files changed, 63 insertions(+), 18 deletions(-)

diff --git a/builtin/cat-file.c b/builtin/cat-file.c
index 1890d7a639..9cf2e4291d 100644
--- a/builtin/cat-file.c
+++ b/builtin/cat-file.c

@@ -252,6 +252,11 @@ static size_t expand_format(struct strbuf *sb, const char *start, void *data)
 {
 	const char *end;
 
+	if (*start == '%') {
+		strbuf_addch(sb, '%');
+		return 1;
+	}
+
 	if (*start != '(')
 		return 0;
 	end = strchr(start + 1, ')');

diff --git a/convert.c b/convert.c
index 8d652bf27c..8336fff694 100644
--- a/convert.c
+++ b/convert.c

@@ -399,6 +399,7 @@ static int filter_buffer_or_fd(int in, int out, void *data)
 	struct strbuf path = STRBUF_INIT;
 	struct strbuf_expand_dict_entry dict[] = {
 		{ "f", NULL, },
+		{ "%", "%" },
 		{ NULL, NULL, },
 	};
 

diff --git a/daemon.c b/daemon.c
index ac7181a483..191fac2716 100644
--- a/daemon.c
+++ b/daemon.c

@@ -148,6 +148,9 @@ static size_t expand_path(struct strbuf *sb, const char *placeholder, void *ctx)
 	case 'D':
 		strbuf_addstr(sb, context->directory);
 		return 1;
+	case '%':
+		strbuf_addch(sb, '%');
+		return 1;
 	}
 	return 0;
 }

diff --git a/date.c b/date.c
index 63fa99685e..d16a88eea5 100644
--- a/date.c
+++ b/date.c

@@ -246,7 +246,7 @@ const char *show_date(timestamp_t time, int tz, const struct date_mode *mode)
 			month_names[tm->tm_mon], tm->tm_year + 1900,
 			tm->tm_hour, tm->tm_min, tm->tm_sec, tz);
 	else if (mode->type == DATE_STRFTIME)
-		strbuf_addftime(&timebuf, mode->strftime_fmt, tm);
+		strbuf_addftime(&timebuf, mode->strftime_fmt, tm, tz);
 	else
 		strbuf_addf(&timebuf, "%.3s %.3s %d %02d:%02d:%02d %d%c%+05d",
 				weekday_names[tm->tm_wday],

diff --git a/ll-merge.c b/ll-merge.c
index ac0d4a5d78..e6202c7397 100644
--- a/ll-merge.c
+++ b/ll-merge.c

@@ -172,7 +172,7 @@ static int ll_ext_merge(const struct ll_merge_driver *fn,
 {
 	char temp[4][50];
 	struct strbuf cmd = STRBUF_INIT;
-	struct strbuf_expand_dict_entry dict[6];
+	struct strbuf_expand_dict_entry dict[7];
 	struct strbuf path_sq = STRBUF_INIT;
 	const char *args[] = { NULL, NULL };
 	int status, fd, i;

@@ -185,7 +185,8 @@ static int ll_ext_merge(const struct ll_merge_driver *fn,
 	dict[2].placeholder = "B"; dict[2].value = temp[2];
 	dict[3].placeholder = "L"; dict[3].value = temp[3];
 	dict[4].placeholder = "P"; dict[4].value = path_sq.buf;
-	dict[5].placeholder = NULL; dict[5].value = NULL;
+	dict[5].placeholder = "%"; dict[5].value = "%";
+	dict[6].placeholder = NULL; dict[6].value = NULL;
 
 	if (fn->cmdline == NULL)
 		die("custom merge driver %s lacks command line.", fn->name);

diff --git a/pretty.c b/pretty.c
index 587d48371b..56872bfa25 100644
--- a/pretty.c
+++ b/pretty.c

@@ -1428,6 +1428,9 @@ static size_t format_commit_item(struct strbuf *sb, /* in UTF-8 */
 	} magic = NO_MAGIC;
 
 	switch (placeholder[0]) {
+	case '%':
+		strbuf_addch(sb, '%');
+		return 1;
 	case '-':
 		magic = DEL_LF_BEFORE_EMPTY;
 		break;

diff --git a/strbuf.c b/strbuf.c
index 00457940cf..206dff6037 100644
--- a/strbuf.c
+++ b/strbuf.c

@@ -309,12 +309,6 @@ void strbuf_expand(struct strbuf *sb, const char *format, expand_fn_t fn,
 			break;
 		format = percent + 1;
 
-		if (*format == '%') {
-			strbuf_addch(sb, '%');
-			format++;
-			continue;
-		}
-
 		consumed = fn(sb, format, context);
 		if (consumed)
 			format += consumed;

@@ -785,7 +779,28 @@ char *xstrfmt(const char *fmt, ...)
 	return ret;
 }
 
-void strbuf_addftime(struct strbuf *sb, const char *fmt, const struct tm *tm)
+struct date_format_context {
+	int tz;
+};
+
+static size_t expand_date_format(struct strbuf *sb, const char *placeholder,
+				 void *ctx)
+{
+	struct date_format_context *c = ctx;
+
+	switch (placeholder[0]) {
+	case '%':
+		strbuf_addstr(sb, "%%");
+		return 1;
+	case 'z':
+		strbuf_addf(sb, "%+05d", c->tz);
+		return 1;
+	}
+	return 0;
+}
+
+void strbuf_addftime(struct strbuf *sb, const char *fmt, const struct tm *tm,
+		     int tz)
 {
 	size_t hint = 128;
 	size_t len;

@@ -794,7 +809,10 @@ void strbuf_addftime(struct strbuf *sb, const char *fmt, const struct tm *tm)
 		return;
 
 	strbuf_grow(sb, hint);
-	len = strftime(sb->buf + sb->len, sb->alloc - sb->len, fmt, tm);
+	if (strstr(fmt, "%z"))
+		len = 0;
+	else
+		len = strftime(sb->buf + sb->len, sb->alloc - sb->len, fmt, tm);
 
 	if (!len) {
 		/*

@@ -805,7 +823,10 @@ void strbuf_addftime(struct strbuf *sb, const char *fmt, const struct tm *tm)
 		 * character before returning.
 		 */
 		struct strbuf munged_fmt = STRBUF_INIT;
-		strbuf_addf(&munged_fmt, "%s ", fmt);
+		struct date_format_context ctx;
+		ctx.tz = tz;
+		strbuf_expand(&munged_fmt, fmt, expand_date_format, &ctx);
+		strbuf_addch(&munged_fmt, ' ');
 		while (!len) {
 			hint *= 2;
 			strbuf_grow(sb, hint);

diff --git a/strbuf.h b/strbuf.h
index 80047b1bb7..fc4c796a2b 100644
--- a/strbuf.h
+++ b/strbuf.h

@@ -281,10 +281,6 @@ extern void strbuf_adddup(struct strbuf *sb, size_t pos, size_t len);
  * the length of the placeholder recognized and `strbuf_expand()` skips
  * over it.
  *
- * The format `%%` is automatically expanded to a single `%` as a quoting
- * mechanism; callers do not need to handle the `%` placeholder themselves,
- * and the callback function will not be invoked for this placeholder.
- *
  * All other characters (non-percent and not skipped ones) are copied
  * verbatim to the strbuf.  If the callback returned zero, meaning that the
  * placeholder is unknown, then the percent sign is copied, too.

@@ -339,9 +335,12 @@ __attribute__((format (printf,2,0)))
 extern void strbuf_vaddf(struct strbuf *sb, const char *fmt, va_list ap);
 
 /**
- * Add the time specified by `tm`, as formatted by `strftime`.
+ * Add the time specified by `tm` and `tz`, as formatted by `strftime`.
+ * The time zone offset is specified like hhmm, so e.g. -600 means six
+ * hours west of Greenwich.
  */
-extern void strbuf_addftime(struct strbuf *sb, const char *fmt, const struct tm *tm);
+extern void strbuf_addftime(struct strbuf *sb, const char *fmt,
+			    const struct tm *tm, int tz);
 
 /**
  * Read a given size of data from a FILE* pointer to the buffer.

diff --git a/t/t6006-rev-list-format.sh b/t/t6006-rev-list-format.sh
index a1dcdb81d7..dc0bed8d90 100755
--- a/t/t6006-rev-list-format.sh
+++ b/t/t6006-rev-list-format.sh

@@ -483,4 +483,16 @@ test_expect_success 'unused %G placeholders are passed through' '
 	test_cmp expect actual
 '
 
+test_expect_success 'date format "%F %T %z" is the same as iso' '
+	git log -1 --format="%ad" --date=iso >expect &&
+	git log -1 --format="%ad" --date="format:%F %T %z" >actual &&
+	test_cmp expect actual
+'
+
+test_expect_success 'date format "%%z" expands to percent zed' '
+	echo "%z" >expect &&
+	git log -1 --format="%ad" --date="format:%%z" >actual &&
+	test_cmp expect actual
+'
+
 test_done