Re: [Ksummit-discuss] [MAINTAINERS SUMMIT] & [TECH TOPIC] Improve regression tracking

On Wed, 2017-07-05 at 12:04 -0400, Steven Rostedt wrote:

On Wed, 05 Jul 2017 08:36:55 -0700
James Bottomley [off-list ref] wrote:

quoted

quoted

Are we tracking regressions or just simply bugs?  

A lot of device driver regressions are bugs that previously existed
in the code but which didn't manifest until something else
happened.  A huge number of locking and timing issues are like
this.  The irony is that a lot of them go from race always being
won (so bug never noticed) to race being lost often enough to make
something unusable.  To a user that ends up being a kernel
regression because it's a bug in the current kernel which they
didn't see previously which makes it unusable for them.

I've got to vote with my users here: that's a regression not a
"feature".

Let's take a step back. What exactly is the problem?

You mean what question was I answering?  It was your "is your problem a
regression?" one.

The regressions that we want to track? Why are they not fixed? Is it
because they are hard to reproduce? If so, how do we know they are a
regression or just some hard to hit bug? If it's hard to hit, how do
we know we fixed it?

Usually for the exposed races we develop a theoretical model which
tells us what the problem is and also the solution.

What exactly are the questions we want solved.

In the context of this subthread?  Tracking and fixing of regressions
meaning behaviour that damages or destroys usability of version k+1
that wasn't present in version k.

Granted, I used this thread to push more use of kselftests, and I
don't see any SCSI tests there at all!

It would be an interesting question for another thread to consider
whether that's a problem or not.

James