Re: An idea for "git bisect" and a GSoC enquiry

Jacopo Notarstefano [off-list ref] writes:

On Thu, Feb 27, 2014 at 12:18 PM, Michael Haggerty [off-list ref] wrote:

quoted

I don't understand the benefit of adding a new command "mark" rather
than continuing to use "good", "bad", plus new commands "unfixed" and
"fixed".  Does this solve any problems?

As Matthieu Moy remarked in a previous email, the main reason is
extensibility: I prefer having a single command to assign new
descriptive labels instead of having to patch git-bisect.sh to create
new labels like fixed, unfixed, fast, slow...

quoted

What happens if the user mixes, say, "good" and "fixed" in a single
bisect session?

I don't think that's an issue. If the user uses the label "fixed"
instead of "bad" she will have a hard time remembering to use it every
time she needs it,...

I am not sure I understand what you are trying to say.  Are you
saying that we should stick to good/bad and allow the users use
nothing else, because allowing "fixed" will be confusing?

and maybe the output of "git bisect" will look very
confusing, but what can git do? This is a semantic user input error,
not a syntax one.

For a young tool or a feature, catering to perfect human perfectly
is a good first goal---if it does not work well even for error-free
human input, it would be worthless.  However, its second goal after
achieving that first goal ought to be to help imperfect humans.

I can very well imagine somebody start hunting for an earlier bugfix
(perhaps trying to find it to backport to an older maintenance
track), start saying "fixed", "broken", "broken", ..., continue
after leaving for lunch for a while, and then try to mark the next
version he tests as "bad" because it has a bug.

It technically may be an user error, in the sense that in such a
"where is the fix?" session, you want to mark a "still-has-bug" one
as "broken" and mark a "no-longer-has-bug" one as "fixed" (just like
"still-broken" as "bad" and "no-longer-broken" as "good" in regular
bisection).  But at that point, the tool *knows* that the user
earlier used "fixed" (or "broken") to mark some commits *already*.

Why do you think there is nothing it can do to help the user?  Upon
seeing "bad", the tool should at least be able to say "Excuse me,
but you earlier said 'fixed' for one of the commits, so your
vocabulary now is limited to 'fixed' and 'broken'".  I think it also
should be able to add "Did you mean to say 'broken'?", or even "I'd
assume that you meant 'broken' and will continue."

I have always wondered if we can introduce a value neutral synonyms
to good and bad.  For a bisect session, we care only about two
states: "still-X" and "no-longer-X" where X may be 'working' for the
normal bug-hunting bisection and 'broken' for the fix-hunting one.

	$ git bisect still-working v1.6.0
        $ git bisect no-longer-working v1.8.0

would be a way to find a bug that was introduced during v1.6.0..v1.8.0,
while

        $ git bisect still-broken v1.6.0
        $ git bisect no-longer-broken v1.8.0

would be a way to find a fix in the same range.  The lowest-level
bisection machinery could just _ignore_ anything after still/no-longer
and do its thing, while the end-user facing layer could enforce,
once one commit is marked as still-X (or no-longer-X), that nothing
other than the same X is used, and issue an error message, perhaps
like this:

	$ git bisect still-broken v1.6.0
        $ git bisect still-working v1.8.0
        error: You earlier marked v1.6.0 as "still-broken" and
        error: are hunting for the first commit that can be marked
        error: as "no-longer-broken".  Say either "still-broken" or
        error: "no-longer-broken", not "still-working".

and that can be done without having to understand that "broken" is
the opposite of "working" (of course if we understood that, we could
even offer to guess that the user meant "no-longer-broken" by
"still-working", but we do not want to go there).

I am not sure I understand what you are trying to say.  Are you
saying that we should stick to good/bad and allow the users use
nothing else, because allowing "fixed" will be confusing?

No! Pretty much the opposite of that. My idea (the "mark" subcommand)
is to let people define their own pairs of labels to represent two
opposite states of a commit. My point was that, if people choose pairs
of words that are not opposites (such as "good" and "fixed") then it's
their error, not something that git should attempt to fix or detect.

For a young tool or a feature, catering to perfect human perfectly
is a good first goal---if it does not work well even for error-free
human input, it would be worthless.  However, its second goal after
achieving that first goal ought to be to help imperfect humans.

Loved this.

Why do you think there is nothing it can do to help the user?  Upon
seeing "bad", the tool should at least be able to say "Excuse me,
but you earlier said 'fixed' for one of the commits, so your
vocabulary now is limited to 'fixed' and 'broken'".  I think it also
should be able to add "Did you mean to say 'broken'?", or even "I'd
assume that you meant 'broken' and will continue."

I haven't said this, but this is pretty much what I had in mind.
Suppose a user wants to find a bugfix between HEAD and HEAD~10, this
is what she would do:

$ git bisect start
$ git bisect mark working HEAD
$ git bisect mark broken HEAD~10

[git will now start bisecting as usual. Suppose that she is now at HEAD~5]

$ git bisect mark bad
-> Error: unrecognized label 'bad'. You previously used 'working' and
'fixed' to describe commits in this git bisect session. Please mark
commits with one of these labels.

I suppose that we could cater a little better to imperfect humans if
we had two predefined parallel list of antonyms in which to search for
given labels and infer whether they are positive or negative labels,
but this is beyond the scope of my proposal.

I have always wondered if we can introduce a value neutral synonyms
to good and bad.  For a bisect session, we care only about two
states: "still-X" and "no-longer-X" where X may be 'working' for the
normal bug-hunting bisection and 'broken' for the fix-hunting one.

        $ git bisect still-working v1.6.0
        $ git bisect no-longer-working v1.8.0

would be a way to find a bug that was introduced during v1.6.0..v1.8.0,
while

        $ git bisect still-broken v1.6.0
        $ git bisect no-longer-broken v1.8.0

would be a way to find a fix in the same range.  The lowest-level
bisection machinery could just _ignore_ anything after still/no-longer
and do its thing, [...]

This is remarkably similar to my proposal. Using "mark", these would be:

$ git bisect mark working v1.6.0
$ git bisect mark not-working v1.8.0

and

$ git bisect mark broken v1.6.0
$ git bisect mark not-broken v1.8.0

while the end-user facing layer could enforce,
once one commit is marked as still-X (or no-longer-X), that nothing
other than the same X is used, and issue an error message, perhaps
like this:

        $ git bisect still-broken v1.6.0
        $ git bisect still-working v1.8.0
        error: You earlier marked v1.6.0 as "still-broken" and
        error: are hunting for the first commit that can be marked
        error: as "no-longer-broken".  Say either "still-broken" or
        error: "no-longer-broken", not "still-working".

and that can be done without having to understand that "broken" is
the opposite of "working" (of course if we understood that, we could
even offer to guess that the user meant "no-longer-broken" by
"still-working", but we do not want to go there).

Here my proposal differs in that I have no way of knowing which label
is good and which label is bad: I blindly accept two distinct labels
and bisect with those. I gave an example of this behaviour above.