Re: [RFC] git integrated bugtracking

Pierre Habouzit writes:

  Hi, I'm currently trying to think about a bug tracking system that
would be tightly integrated with git, meaning that it would have to be
somehow decentralized.

  Though there is a few design issues I have, that block me from doing
first decisions about how to implement some kind of Proof of Concept. My
main problem is: should I put this bug tracking system in the repository
it tracks bugs for, or not.

  I mean, the immediate idea is to have some .bugs/ directories (or
alike). This has many good properties, e.g. for projects like the linux
kernel with its many subsystems or driver, it would make sense to have
per driver/subsystems/... bug packs, and move bugs from one pack to
another would be the way of assigning bugs to different modules.

  Also, a good thing is that when you "report" a bug, it gets commited
in the repository, and taints all the commit chilren, until you commit
the closing patch. This allow a release manager to rapidly _see_ if his
stable branch has this or this bug.

  OTOH it comes with many problems. First, and most obvious IMHO, it's
that it'll mean bugs will have to be pulled into the mainlines. Let's
take example with the linux repository, I'm not sure Linus would be
really keen on doing rounds of bugs pulls, not to mention it'll bloat
the repository somehow.

  The other problem I see is that at the time a bug gets reported, the
user knows it's found at a commit say 'X'. But it could in fact have
been generated at a commit Y, with this pattern:

  --o---o---Y---o---o---o---o---X---o---o--> master
                     \
                      o---o---o---o---o---o--> branch B

Mainly for that reason, I would suggest having it outside the code
base's namespace: probably a different root in the same $GIT_DIR, but
I can see people wanting to have a separate $GIT_DIR.  If the database
tracks bugs by what commit(s) introduce or expose the bug -- at least
once that is known -- then you get nearly free tracking of which
branches have the bug without having to check out largely redundant
trees.

Michael Poole

On Sun, Jun 03, 2007 at 08:59:18AM -0400, Michael Poole wrote:

Pierre Habouzit writes:

quoted

  The other problem I see is that at the time a bug gets reported, the
user knows it's found at a commit say 'X'. But it could in fact have
been generated at a commit Y, with this pattern:

  --o---o---Y---o---o---o---o---X---o---o--> master
                     \
                      o---o---o---o---o---o--> branch B

Mainly for that reason, I would suggest having it outside the code
base's namespace: probably a different root in the same $GIT_DIR, but
I can see people wanting to have a separate $GIT_DIR.  If the database
tracks bugs by what commit(s) introduce or expose the bug -- at least
once that is known -- then you get nearly free tracking of which
branches have the bug without having to check out largely redundant
trees.

  Sure, but if it's completely out-of-tree, then cloning a repository
don't allow you to get the bug databases with it for free. I mean it'd
be great to have it somehow linked to the repository, but also I agree
that not everybody wants to clone the whole bugs databases. So maybe it
should just be in another shadow branch that annotates the devel ones.
Hmmm I definitely need to read the git-note thread...

-- 
·O·  Pierre Habouzit
··O                                                madcoder@debian.org
OOO                                                http://www.madism.org

On Sunday 03 June 2007, Pierre Habouzit wrote:

On Sun, Jun 03, 2007 at 08:59:18AM -0400, Michael Poole wrote:

quoted

Pierre Habouzit writes:

quoted

  The other problem I see is that at the time a bug gets reported, the
user knows it's found at a commit say 'X'. But it could in fact have
been generated at a commit Y, with this pattern:

  --o---o---Y---o---o---o---o---X---o---o--> master
                     \
                      o---o---o---o---o---o--> branch B

Mainly for that reason, I would suggest having it outside the code
base's namespace: probably a different root in the same $GIT_DIR, but
I can see people wanting to have a separate $GIT_DIR.  If the database
tracks bugs by what commit(s) introduce or expose the bug -- at least
once that is known -- then you get nearly free tracking of which
branches have the bug without having to check out largely redundant
trees.

  Sure, but if it's completely out-of-tree, then cloning a repository
don't allow you to get the bug databases with it for free. I mean it'd
be great to have it somehow linked to the repository, but also I agree
that not everybody wants to clone the whole bugs databases. So maybe it
should just be in another shadow branch that annotates the devel ones.
Hmmm I definitely need to read the git-note thread...

I guess I'm the one responsible for starting that git-note thread...

For the moment, I'm busy implementing some concepts that came out of that 
discussion (refactoring tag objects and building some infrastructure needed 
to support notes without the drawbacks present in my first version).

Hopefully I'll have a proof-of-concept ready before too long. In the 
meantime I'll be happy to answer questions you might have.

Regarding the notes themselves, I thought about possibly using them as a 
link between the repo and the bug tracker, with some glue code in between 
for making the connections. I haven't thought about integrating them more 
deeply into a bug tracker, but it might be worth thinking along those 
lines, especially for the kind of system you're proposing.

Now, back to hacking...


Have fun! :)

...Johan


-- 
Johan Herland, [off-list ref]
www.herland.net

On Sun, Jun 03, 2007 at 03:48:29PM +0200, Johan Herland wrote:

On Sunday 03 June 2007, Pierre Habouzit wrote:

quoted

On Sun, Jun 03, 2007 at 08:59:18AM -0400, Michael Poole wrote:

quoted

Pierre Habouzit writes:

quoted

  The other problem I see is that at the time a bug gets reported, the
user knows it's found at a commit say 'X'. But it could in fact have
been generated at a commit Y, with this pattern:

  --o---o---Y---o---o---o---o---X---o---o--> master
                     \
                      o---o---o---o---o---o--> branch B

Mainly for that reason, I would suggest having it outside the code
base's namespace: probably a different root in the same $GIT_DIR, but
I can see people wanting to have a separate $GIT_DIR.  If the database
tracks bugs by what commit(s) introduce or expose the bug -- at least
once that is known -- then you get nearly free tracking of which
branches have the bug without having to check out largely redundant
trees.

  Sure, but if it's completely out-of-tree, then cloning a repository
don't allow you to get the bug databases with it for free. I mean it'd
be great to have it somehow linked to the repository, but also I agree
that not everybody wants to clone the whole bugs databases. So maybe it
should just be in another shadow branch that annotates the devel ones.
Hmmm I definitely need to read the git-note thread...

I guess I'm the one responsible for starting that git-note thread...

For the moment, I'm busy implementing some concepts that came out of that 
discussion (refactoring tag objects and building some infrastructure needed 
to support notes without the drawbacks present in my first version).

Hopefully I'll have a proof-of-concept ready before too long. In the 
meantime I'll be happy to answer questions you might have.

Regarding the notes themselves, I thought about possibly using them as a 
link between the repo and the bug tracker, with some glue code in between 
for making the connections. I haven't thought about integrating them more 
deeply into a bug tracker, but it might be worth thinking along those 
lines, especially for the kind of system you're proposing.

  Yeah, now that I read that thread, well yeah, I think notes are a hell
of a good concept for my ideas. I mean, a bug report would be basically
a collection of notes:
  * the bug has been found at this commit ;
  * the bug has been not-found at this commit ;
  * this commit is a fix for that bug ;
  * this commit enhance features for that wish and so on.
Some other bits are more followup comments and are disconnected to
commits, but could be attached to the "bug object" whatever would be
used for bugs, the whole concept is blurry anyways, we're just
discussing ideas :)

  Though, for a good bug tracking system, you need to be able to answer
some kind of questions fast enough:
  * list bugs that affect a given stage of the repository
    (tag/branch/commit/...) ;
  * be able to trace history of a given bug (yes, it's fairly obvious,
    but unlike many other bug tracking systems, for us it would not be
    contiguous, so it's not necessarily a O(1) operation) ;

  Other things nice to have is textual search, git-grep can help of
course, but that's an operation you do often with a bug tracking system,
and you expect it to be faster than git-grep is (though maybe an
optionnal index can be built around the notes for that purpose and not
be versionned ?).

  Another think that notes do not address are another operation we
usually do on bugs: merge (or duplicates). There is a think I hate in
bugzilla and love in debbugs, it's that duplicate bugs are closed in the
former, and merged in the latter. When two bugs are the same, their
history are often *both* valuable, and you really don't want to lose one
half, you want to merge them. And you also want the option to "unmerge"
them, but for that the better option is to have the ability to duplicate
a bug (aka debbugs cloning).

  Anyways it's just gossip, but maybe someone will have a brilliant
ideas, so I'm just throwing my thoughts into this mail :)

-- 
·O·  Pierre Habouzit
··O                                                madcoder@debian.org
OOO                                                http://www.madism.org

On Sun, Jun 03, 2007 at 05:19:21PM +0200, Pierre Habouzit wrote:

  Another think that notes do not address are another operation we
usually do on bugs: merge (or duplicates). There is a think I hate in
bugzilla and love in debbugs, it's that duplicate bugs are closed in the
former, and merged in the latter. When two bugs are the same, their
history are often *both* valuable, and you really don't want to lose one
half, you want to merge them. And you also want the option to "unmerge"
them, but for that the better option is to have the ability to duplicate
a bug (aka debbugs cloning).

I can't wait for visualising the history of bugreports in gitk, along
with duplicates/forks and merges :)

  Anyways it's just gossip, but maybe someone will have a brilliant
ideas, so I'm just throwing my thoughts into this mail :)

/me wishes all gossip would be as productive as this one :)

Best regards,
-- 
Yann

One thing that I just thought about is how such a system would play
with eg. StGIT.

Precisely, if I stick a BTS note (found/not-found probably makes no
sense here, so that'd mostly be fixes/enhances notes) to a given
version of an StGIT patch, I probably want the BTS to be able to tell
that the next version of the patch "probably has" the same note - if a
given patch fixes a given bug, I infer the patch is primarily written
as a fix for that bug, so later revisions of the patch (eg. cleanups)
would also fix it.  But maybe we'd want the patch author to
explicitely make such an assertion, in case a cleanup would have
broken the fix - both options could be useful in different cases.

Currently, old versions of an StGIT patch are only available through
StGIT patchlogs, which are not ancestors of later versions of the same
patch.  That is, even the "subsequent versions inherit notes" policy
would need extra StGIT-aware stuff to work.

This may be something to keep in mind here - but that issue could also
be seen as belonging to the more general (and with ongoing work)
aspect of git/StGIT interactions.  Even better if the 2 ongoing
reflexions cross-fertilisate :)

Best regards,
-- 
Yann.

On Sun, Jun 03, 2007 at 10:04:31PM +0200, Yann Dirson wrote:

One thing that I just thought about is how such a system would play
with eg. StGIT.

  I read the rest of the mail, but I don't know stgit at all, so I'm not
really able to tell how well it would integrate, especially since well,
it's still early.

  But for sure, I'm launching that brainstorming thread because I want
as much input as possible to know what are people needs, and add the one
that make sense to the todo list, the one that would be good to the
wishlist. FWIW playing well with usual git tools (like stgit, guilt, and
whatever other evolved "thing" in the git nebula) is definitely in the
TODO list.

-- 
·O·  Pierre Habouzit
··O                                                madcoder@debian.org
OOO                                                http://www.madism.org

Another aspect that I have not seen discussed yet is the handling of
testcases that expose a bug, and which are usually integrated as
non-reg tests afterwards.

If we're going the way of using git-note's, we can easily link the
commit which adds the testcase to the non-reg testsuite.

But we could be interested in running the testcase against an
arbitrary commit, thus allowing some sort of automated bisect (modulo
applicability of the testcase to other commits, which may cause false
negatives, but also false positives, and thus may need to be
hand-driven or at least hand-reviewed).

For this, however, would it be sufficient to have the testcase
committed somewhere in the history tree ?  Our hypothetic automatic
bisect tool could cherry-pick that commit and run the testcase, but at
would place an arbitrary restriction of "one testcase per commit",
which would probably a bad idea.

And having the annotation point to the blob does not seem a good idea
either, at least because currently in git tools we have test scripts
that excercise several testcases each.

Even an annotation to both a commit and a blob changed by that commit
would not be sufficiently accurate, when several testcases are added
to the same script in a single commit.


Another options would be to embed the testcase in an annotation: I
can't really see that being used, since that would duplicate the
testcase.  That makes me wonder whether it would not be required to
have testcases described in seperate files ?

Best regards,
-- 
Yann.

On Tue, Jun 05, 2007 at 12:03:08AM +0200, Yann Dirson wrote:

Another aspect that I have not seen discussed yet is the handling of
testcases that expose a bug, and which are usually integrated as
non-reg tests afterwards.

  Well IMHO testcases are to be dealt with from the build sytem, and
versionned in the SCM. You may want to have some kind of helper doing
the bisect automatically for you (though, speaking of experience, it's
not always _that_ simple because of broken commits that do not build
e.g.) But I hardly see how it can be realted to a BTS anyways. I mean,
bugs will likely have test cases to exhibit the bug, but those should be
put in the test-case part of the repository. I mean well, maybe there is
fancy things to do, but I don't think it should be the role of a BTS at
all.

-- 
·O·  Pierre Habouzit
··O                                                madcoder@debian.org
OOO                                                http://www.madism.org