Re: [RFC] git integrated bugtracking

On Sun, 2007-06-10 at 03:50, Jan Hudec wrote:

It would be really useful to have a tool, that could link a bug report to
a test case demonstrating it and reporting whenever output of that test case
changes. This would make it much easier for developer to see which bugs he
might have fixed when doing a refactoring.

I think the inverse is interesting too.  That is, applying
keywords to tests and then making the test database
keywords searchable.

I used to work for a company that had a huge collection
of test cases for its product.  It would take days to run
a validation run for some of the simplest of changes that
were introduced during development.

Often what I wanted, and what was needed, was a simple
sanity check that verified that the proposed change was
not brain-dead from the start.

Over time, I found that certain tests proved to be good
at detecting failures in particular portions of the code.
Often, that test was not necessarily directly related to
the concept it was _supposed_ to be designed to test, but
incidentally was good at some _other_ concept too.

The developer who made the change, and ran the tests,
then was able to state "Test t42 from suite Frotz is good
at detecting changes to the <SomeModule>."

The trick is to now associate with that test the keyword
"SomeModule" so that in the future, another developer
could ask:  "Say, I just modified <SomeModule>, are there
any tests that are good at proving my changes sound?"
A test driver could then focus some test cycles quickly.

Now, for git, it's not likely a problem to just run
through all of its tests.  At ${PriorCompany} it was
essential to pare down the test suites to a manageable
size, yet still have assurance that your coverage was
reasonably good.

jdl