From: Junio C Hamano <hidden> Date: 2016-06-15 23:07:48
Jeff King [off-list ref] writes:
So it's not wrong, but it's perhaps more complicated than it needs to
be. We could scrap this patch in favor of just:
if (!skip_prefix(author, "Author: ", &v) &&
!skip_prefix(author, "author ", &v))
continue;
That is technically more strict (it does not take "author: ", which is
accepted by the current code), but matches "git log" and "git log --raw"
output, and misses nothing that git has ever generated. And it extends
naturally to:
if (!skip_prefix(author, "Commit: ", &v) &&
!skip_prefix(author, "committer ", &v))
continue;
Yeah, I agree that the above long-hand would be more readable.
From: Jeff King <hidden> Date: 2016-06-15 23:07:48
On Mon, Jan 18, 2016 at 11:55:18AM -0800, Junio C Hamano wrote:
Jeff King [off-list ref] writes:
quoted
So it's not wrong, but it's perhaps more complicated than it needs to
be. We could scrap this patch in favor of just:
if (!skip_prefix(author, "Author: ", &v) &&
!skip_prefix(author, "author ", &v))
continue;
That is technically more strict (it does not take "author: ", which is
accepted by the current code), but matches "git log" and "git log --raw"
output, and misses nothing that git has ever generated. And it extends
naturally to:
if (!skip_prefix(author, "Commit: ", &v) &&
!skip_prefix(author, "committer ", &v))
continue;
Yeah, I agree that the above long-hand would be more readable.
OK. Here it is again (the whole series, since the change creates minor
conflicts some of the later patches).
The interdiff is:
From: Jeff King <hidden> Date: 2016-06-15 23:07:48
The original git-shortlog could read both the normal "git
log" output as well as "git log --format=raw". However, when
it was converted to C by b8ec592 (Build in shortlog,
2006-10-22), the trailing colon became mandatory, and we no
longer matched the raw output.
Given the amount of intervening time without any bug
reports, it's probable that nobody cares. But it's
relatively easy to fix, and the end result is hopefully more
readable than the original.
Note that this no longer matches "author: ", which we did
before, but that has never been a format generated by git.
Signed-off-by: Jeff King <redacted>
---
builtin/shortlog.c | 7 ++++---
t/t4201-shortlog.sh | 6 ++++++
2 files changed, 10 insertions(+), 3 deletions(-)
@@ -120,6 +120,12 @@ test_expect_success !MINGW 'shortlog from non-git directory' 'test_cmpexpectout'+test_expect_success!MINGW'shortlog can read --format=raw output''+gitlog--format=rawHEAD>log&&+GIT_DIR=non-existinggitshortlog-w<log>out&&+test_cmpexpectout+'+ test_expect_success'shortlog should add newline when input line matches wraplen''cat>expect<<\EOF&& AUThor(2):
From: Jeff King <hidden> Date: 2016-06-15 23:07:48
When gathering the author and oneline subject for each
commit, we hand-parse the commit headers to find the
"author" line, and then continue past to the blank line at
the end of the header.
We can replace this tricky hand-parsing by simply asking the
pretty-printer for the relevant items. This also decouples
the author and oneline parsing, opening up some new
optimizations in further commits.
One reason to avoid the pretty-printer is that it might be
less efficient than hand-parsing. However, I measured no
slowdown at all running "git shortlog -ns HEAD" on
linux.git.
As a bonus, we also fix a memory leak in the (uncommon) case
that the author field is blank.
Signed-off-by: Jeff King <redacted>
---
builtin/shortlog.c | 62 +++++++++++++++++++++++-------------------------------
1 file changed, 26 insertions(+), 36 deletions(-)
@@ -113,45 +113,35 @@ static void read_from_stdin(struct shortlog *log)voidshortlog_add_commit(structshortlog*log,structcommit*commit){-constchar*author=NULL,*buffer;-structstrbufbuf=STRBUF_INIT;-structstrbufufbuf=STRBUF_INIT;--pp_commit_easy(CMIT_FMT_RAW,commit,&buf);-buffer=buf.buf;-while(*buffer&&*buffer!='\n'){-constchar*eol=strchr(buffer,'\n');--if(eol==NULL)-eol=buffer+strlen(buffer);-else-eol++;--if(starts_with(buffer,"author "))-author=buffer+7;-buffer=eol;-}-if(!author){+structstrbufauthor=STRBUF_INIT;+structstrbufoneline=STRBUF_INIT;+structpretty_print_contextctx={0};++ctx.fmt=CMIT_FMT_USERFORMAT;+ctx.abbrev=log->abbrev;+ctx.subject="";+ctx.after_subject="";+ctx.date_mode.type=DATE_NORMAL;+ctx.output_encoding=get_log_output_encoding();++format_commit_message(commit,"%an <%ae>",&author,&ctx);+/* we can detect a total failure only by seeing " <>" in the output */+if(author.len<=3){warning(_("Missing author: %s"),oid_to_hex(&commit->object.oid));-return;-}-if(log->user_format){-structpretty_print_contextctx={0};-ctx.fmt=CMIT_FMT_USERFORMAT;-ctx.abbrev=log->abbrev;-ctx.subject="";-ctx.after_subject="";-ctx.date_mode.type=DATE_NORMAL;-ctx.output_encoding=get_log_output_encoding();-pretty_print_commit(&ctx,commit,&ufbuf);-buffer=ufbuf.buf;-}elseif(*buffer){-buffer++;+gotoout;}-insert_one_record(log,author,!*buffer?"<none>":buffer);-strbuf_release(&ufbuf);-strbuf_release(&buf);++if(log->user_format)+pretty_print_commit(&ctx,commit,&oneline);+else+format_commit_message(commit,"%s",&oneline,&ctx);++insert_one_record(log,author.buf,oneline.len?oneline.buf:"<none>");++out:+strbuf_release(&author);+strbuf_release(&oneline);}staticvoidget_from_rev(structrev_info*rev,structshortlog*log)
From: Jeff King <hidden> Date: 2016-06-15 23:07:48
We currently use fixed-size buffers with fgets(), which
could lead to incorrect results in the unlikely event that a
line had something like "Author:" at exactly its 1024th
character.
But it's easy to convert this to a strbuf, and because we
can reuse the same buffer through the loop, we don't even
pay the extra allocation cost.
Signed-off-by: Jeff King <redacted>
---
builtin/shortlog.c | 21 ++++++++++++---------
1 file changed, 12 insertions(+), 9 deletions(-)
From: Jeff King <hidden> Date: 2016-06-15 23:07:48
If we are in --summary mode, we will always pass <none> to
insert_one_record, which will then do some normalization
(e.g., cutting out "[PATCH]"). There's no point in doing so
if we aren't going to use the result anyway.
This drops my best-of-five for "git shortlog -ns HEAD" on
linux.git from:
real 0m5.257s
user 0m5.104s
sys 0m0.156s
to:
real 0m5.194s
user 0m5.028s
sys 0m0.168s
That's only 1%, but arguably the result is clearer to read,
as we're able to group our variable declarations inside the
conditional block. It also opens up further optimization
possibilities for future patches.
Signed-off-by: Jeff King <redacted>
---
builtin/shortlog.c | 63 +++++++++++++++++++++++++++++-------------------------
1 file changed, 34 insertions(+), 29 deletions(-)
@@ -59,34 +55,43 @@ static void insert_one_record(struct shortlog *log,if(item->util==NULL)item->util=xcalloc(1,sizeof(structstring_list));-/* Skip any leading whitespace, including any blank lines. */-while(*oneline&&isspace(*oneline))-oneline++;-eol=strchr(oneline,'\n');-if(!eol)-eol=oneline+strlen(oneline);-if(starts_with(oneline,"[PATCH")){-char*eob=strchr(oneline,']');-if(eob&&(!eol||eob<eol))-oneline=eob+1;-}-while(*oneline&&isspace(*oneline)&&*oneline!='\n')-oneline++;-format_subject(&subject,oneline," ");-buffer=strbuf_detach(&subject,NULL);--if(dot3){-intdot3len=strlen(dot3);-if(dot3len>5){-while((p=strstr(buffer,dot3))!=NULL){-inttaillen=strlen(p)-dot3len;-memcpy(p,"/.../",5);-memmove(p+5,p+dot3len,taillen+1);+if(log->summary)+string_list_append(item->util,xstrdup(""));+else{+constchar*dot3=log->common_repo_prefix;+char*buffer,*p;+structstrbufsubject=STRBUF_INIT;+constchar*eol;++/* Skip any leading whitespace, including any blank lines. */+while(*oneline&&isspace(*oneline))+oneline++;+eol=strchr(oneline,'\n');+if(!eol)+eol=oneline+strlen(oneline);+if(starts_with(oneline,"[PATCH")){+char*eob=strchr(oneline,']');+if(eob&&(!eol||eob<eol))+oneline=eob+1;+}+while(*oneline&&isspace(*oneline)&&*oneline!='\n')+oneline++;+format_subject(&subject,oneline," ");+buffer=strbuf_detach(&subject,NULL);++if(dot3){+intdot3len=strlen(dot3);+if(dot3len>5){+while((p=strstr(buffer,dot3))!=NULL){+inttaillen=strlen(p)-dot3len;+memcpy(p,"/.../",5);+memmove(p+5,p+dot3len,taillen+1);+}}}-}-string_list_append(item->util,buffer);+string_list_append(item->util,buffer);+}}staticvoidread_from_stdin(structshortlog*log)
From: Jeff King <hidden> Date: 2016-06-15 23:07:48
If the user asked us only to show counts for each author,
rather than the individual summary lines, then there is no
point in us generating the summaries only to throw them
away. With this patch, I measured the following speedup for
"git shortlog -ns HEAD" on linux.git (best-of-five):
[before]
real 0m5.644s
user 0m5.472s
sys 0m0.176s
[after]
real 0m5.257s
user 0m5.104s
sys 0m0.156s
That's only ~7%, but it's so easy to do, there's no good
reason not to. We don't have to touch any downstream code,
since we already fill in the magic string "<none>" to handle
commits without a message.
Signed-off-by: Jeff King <redacted>
---
builtin/shortlog.c | 10 ++++++----
1 file changed, 6 insertions(+), 4 deletions(-)
From: Jeff King <hidden> Date: 2016-06-15 23:07:48
If we are in "--summary" mode, then we do not care about the
actual list of subject onelines associated with each author.
We care only about the number. So rather than store a
string-list for each author full of "<none>", let's just
keep a count.
This drops my best-of-five for "git shortlog -ns HEAD" on
linux.git from:
real 0m5.194s
user 0m5.028s
sys 0m0.168s
to:
real 0m5.057s
user 0m4.916s
sys 0m0.144s
That's about 2.5%.
Signed-off-by: Jeff King <redacted>
---
builtin/shortlog.c | 43 +++++++++++++++++++++++++++++++------------
1 file changed, 31 insertions(+), 12 deletions(-)
From: Jeff King <hidden> Date: 2016-06-15 23:07:48
On Mon, Jan 18, 2016 at 03:02:48PM -0500, Jeff King wrote:
+ format_commit_message(commit, "%an <%ae>", &author, &ctx);
+ /* we can detect a total failure only by seeing " <>" in the output */
+ if (author.len <= 3) {
warning(_("Missing author: %s"),
oid_to_hex(&commit->object.oid));
[...]
+ goto out;
}
One note on this. In the linux.git tree, this warning actually triggers,
because there is a commit with a bogus empty author:
$ git log -1 --format=raw af25e94d4dc | grep ^author
author <> 1120285620 -0700
Whereas in the original code, you really do get a line with a blank
name.
I think what the new code does is quite reasonable, but I'm not sure if:
1. People really want a syntactically valid empty name to be
represented.
and
2. Regardless of the output, if kernel folks will be annoyed by the
warning whenever they run a full-repo shortlog.
-Peff
From: Jeff King <hidden> Date: 2016-06-15 23:07:48
On Mon, Jan 18, 2016 at 03:13:37PM -0500, Jeff King wrote:
On Mon, Jan 18, 2016 at 03:02:48PM -0500, Jeff King wrote:
quoted
+ format_commit_message(commit, "%an <%ae>", &author, &ctx);
+ /* we can detect a total failure only by seeing " <>" in the output */
+ if (author.len <= 3) {
warning(_("Missing author: %s"),
oid_to_hex(&commit->object.oid));
[...]
+ goto out;
}
One note on this. In the linux.git tree, this warning actually triggers,
because there is a commit with a bogus empty author:
$ git log -1 --format=raw af25e94d4dc | grep ^author
author <> 1120285620 -0700
Whereas in the original code, you really do get a line with a blank
name.
I think what the new code does is quite reasonable, but I'm not sure if:
1. People really want a syntactically valid empty name to be
represented.
and
2. Regardless of the output, if kernel folks will be annoyed by the
warning whenever they run a full-repo shortlog.
After thinking on this, I'm in favor of removing this warning entirely.
My reasons are given in the commit message below, which can apply on top
of the series. It could also be squashed in to 2/6, but given that it
is removing the test added by cd4f09e (shortlog: ignore commits with
missing authors, 2013-09-18), I think it's worth recording as a separate
commit.
-- >8 --
Subject: [PATCH] shortlog: don't warn on empty author
Git tries to avoid creating a commit with an empty author
name or email. However, commits created by older, less
strict versions of git may still be in the history. There's
not much point in issuing a warning to stderr for an empty
author. The user can't do anything about it now, and we are
better off to simply include it in the shortlog output as an
empty name/email, and let the caller process it however they
see fit.
Older versions of shortlog differentiated between "author
header not present" (which complained) and "author
name/email are blank" (which included the empty ident in the
output). But since switching to format_commit_message, we
complain to stderr about either case (linux.git has a blank
author deep in its history which triggers this).
We could try to restore the older behavior (complaining only
about the missing header), but in retrospect, there's not
much point in differentiating these cases. A missing
author header is bogus, but as for the "blank" case, the
only useful behavior is to add it to the "empty name"
collection.
Signed-off-by: Jeff King <redacted>
---
builtin/shortlog.c | 8 --------
t/t4201-shortlog.sh | 16 ----------------
2 files changed, 24 deletions(-)
@@ -149,13 +149,6 @@ void shortlog_add_commit(struct shortlog *log, struct commit *commit)ctx.output_encoding=get_log_output_encoding();format_commit_message(commit,"%an <%ae>",&author,&ctx);-/* we can detect a total failure only by seeing " <>" in the output */-if(author.len<=3){-warning(_("Missing author: %s"),-oid_to_hex(&commit->object.oid));-gotoout;-}-if(!log->summary){if(log->user_format)pretty_print_commit(&ctx,commit,&oneline);