Re: [PATCH 4/7] commit-graph: fix bogus counter in "Scanning merged commits" progress line

On Sat, Jun 26 2021, René Scharfe wrote:

Am 21.06.21 um 22:08 schrieb Ævar Arnfjörð Bjarmason:

quoted

On Mon, Jun 21 2021, René Scharfe wrote:

quoted

Am 21.06.21 um 00:13 schrieb Ævar Arnfjörð Bjarmason:

quoted

On Sun, Jun 20 2021, SZEDER Gábor wrote:

quoted

The final value of the counter of the "Scanning merged commits"
progress line is always one less than its expected total, e.g.:

  Scanning merged commits:  83% (5/6), done.

This happens because while iterating over an array the loop variable
is passed to display_progress() as-is, but while C arrays (and thus
the loop variable) start at 0 and end at N-1, the progress counter
must end at N.  This causes the failures of the tests
'fetch.writeCommitGraph' and 'fetch.writeCommitGraph with submodules'
in 't5510-fetch.sh' when run with GIT_TEST_CHECK_PROGRESS=1.

Fix this by passing 'i + 1' to display_progress(), like most other
callsites do.

Signed-off-by: SZEDER Gábor <redacted>
---
 commit-graph.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/commit-graph.c b/commit-graph.c
index 2bcb4e0f89..3181906368 100644
--- a/commit-graph.c
+++ b/commit-graph.c

@@ -2096,7 +2096,7 @@ static void sort_and_scan_merged_commits(struct write_commit_graph_context *ctx)

 	ctx->num_extra_edges = 0;
 	for (i = 0; i < ctx->commits.nr; i++) {
-		display_progress(ctx->progress, i);
+		display_progress(ctx->progress, i + 1);

 		if (i && oideq(&ctx->commits.list[i - 1]->object.oid,
 			  &ctx->commits.list[i]->object.oid)) {

I think this fix makes sense, but FWIW there's a large thread starting
at [1] where René disagrees with me, and thinks the fix for this sort of
thing would be to display_progress(..., i + 1) at the end of that
for-loop, or just before the stop_progress().

I don't agree, but just noting the disagreement, and that if that
argument wins then a patch like this would involve changing the other
20-some calls to display_progress() in commit-graph.c to work
differently (and to be more complex, we'd need to deal with loop
break/continue etc.).

1. https://lore.kernel.org/git/patch-2.2-042f598826-20210607T144206Z-avarab@gmail.com/ (local)

*sigh*  (And sorry, Ævar.)

Before an item is done, it should be reported as not done.  After an
item is done, it should be reported as done.  One loop iteration
finishes one item.  Thus the number of items to report at the bottom of
the loop is one higher than at the top.  i is the correct number to
report at the top of a zero-based loop, i+1 at the bottom.

quoted

Anyone with more time than sense can go and read over our linked back &
forth thread where we're disagreeing on that point :). I think the pattern
in commit-graph.c makes sense, you don't.

Thanks for this comment, I think I got it now: Work doesn't count in the
commit-graph.c model of measuring progress, literally.  I.e. progress is
the same before and after one item of work.

The progress isn't the same, we update the count. Or do you mean in the
time it takes us to go from the end of the for-loop & jump to the start
of it and update the count?

Instead it counts the number of loop iterations.  The model I describe
above counts finished work items instead.  The results of the two
models differ by at most one despite their inverted axiom regarding
the value of work.

Phew, that took me a while.

For what it's worth I had some extensive examples in our initial
thread[1][2] (search for "apple" and "throughput", respectively), that
you cut out when replying to the relevant E-Mails. I'd think we could
probably have gotten here earlier :)

I'm a bit confused about this "value of work" comment.

If you pick up a copy of say a video game like Mario Kart you'll find
that for a 3-lap race you start at 1/3, and still have an entire lap to
go when the count is at 3/3.

So it's just a question of whether you report progress on item N or work
finished on item N, not whether laps in a race have more or less
value.

To reference my earlier E-Mail[1] are you eating the first apple or the
zeroeth apple? I don't think one is more or less right in the
mathematical sense, I just think for UX aimed at people counting "laps"
makes more sense than counting completed items.

quoted

Anyway, aside from that. I think, and I really would be advocating this
too, even if our respective positions were reversed, that *in this case*
it makes sense to just take something like SZEDER's patch here
as-is. Because in that file there's some dozen occurrences of that exact
pattern.

The code without the patch either forgets to report the last work item
in the count-work-items model or is one short in the count-iterations
model, so a fix is needed either way.

It won't be one short, for a loop of 2 items we'll go from:

     0/2
     1/2
     1/2, done

To:

     1/2
     2/2
     2/2, done

Just like the rest of the uses of the progress API in that file.

Which is one of the two reasons I prefer this pattern, i.e. this is less
verbose:

    start_progress()
    for i in (0..X-1):
        display_progress(i+1)
        work()
    stop_progress()

Than one of these, which AFAICT would be your recommendation:

    # Simplest, but stalls on work()
    start_progress()
    for i in (0..X-1):
        work()
        display_progress(i+1)
    stop_progress()

    # More verbose, but doesn't:
    start_progress()
    for i in (0..X-1):
        display_progress(i)
        work()
        display_progress(i+1)
    stop_progress()

    # Ditto:
    start_progress()
    display_progress(0)
    for i in (0..X-1):
        work()
        display_progress(i+1)
    stop_progress()

And of course if your loop continues or whatever you'll need a last
"display_progress(X)" before the "stop_progress()".

The other is that if you count laps you can have your progress bar
optionally show progress on that item. E.g. we could if we stall show
seconds spend that we're hung on that item, or '3/3 ETA 40s". I have a
patch[3] that takes an initial step towards that, with some more queued
locally.

The number of the other occurrences wouldn't matter if they were
buggy, but in this case they indicate that Stolee consistently used
the count-iterations model.  Thus using it in the patch as well makes
sense.

quoted

Let's just bring this one case in line with the rest, if we then want to
argue that one or the other use of the progress.c API is wrong as a
general thing, I think it makes more sense to discuss that as some
follow-up series that changes these various API uses en-masse than
holding back isolated fixes that leave the state of the progress bar it
!= 100%.

Agreed.

Sorry to go on about this again :)

1. https://lore.kernel.org/git/87lf7k2bem.fsf@evledraar.gmail.com/ (local)
2. https://lore.kernel.org/git/87o8c8z105.fsf@evledraar.gmail.com/ (local)
3. https://lore.kernel.org/git/patch-18.25-e21fc66623f-20210623T155626Z-avarab@gmail.com/ (local)