Re: [Ksummit-discuss] [MAINTAINERS SUMMIT] & [TECH TOPIC] Improve regression tracking

On Thu, Jul 06, 2017 at 07:48:05AM -0700, James Bottomley wrote:

On Thu, 2017-07-06 at 10:28 +0100, Mark Brown wrote:

quoted

I think before anything like that is viable we need to show a
concerted and visible interest in actually running the tests we
already have and paying attention to the results - if people can see
that they're just checking a checkbox that will often result in low
quality tests which can do more harm than good.

it depends what you mean by "we".  I used to run a battery of tests
over every SCSI commit.  It was time consuming and slowed down the

We as a community, I think something viable needs to be central services
like kernelci that's automated and allows multiple people to be involved
with the analysis.  Hand running tests at scale just doesn't.

The corollary I take away from this is that the less intrusive the test
infrastructure is (at least to my process) the happier I am.  The 0day
quantum leap for me was going from testing my tree and telling me of
problems after I've added the patch to testing patches posted to the
mailing list, which tells me of problems *before* the commit gets added
to the tree.

I think we'd get a long way just by looking at what's ending up in -next
- it's not as good as detecting things before they go in but it's
workable if people keep on top of things.