Re: [Ksummit-discuss] [MAINTAINERS SUMMIT] & [TECH TOPIC] Improve regression tracking

On Wednesday 05 Jul 2017 12:10:00 Steven Rostedt wrote:

On Wed, 5 Jul 2017 11:08:48 -0400 Carlos O'Donell wrote:

quoted

On 07/05/2017 10:33 AM, Steven Rostedt wrote:

quoted

No test should be written for a single specific hardware. It should be a
general functionality that different hardware can execute.

Why? We test all sorts of hardware in userspace and we see value in that
testing.

One reason is for bit rot. I'm not totally against it. But I envision
that if we have hundreds of tests for very specific pieces of hardware,
it's value will diminish over time. Unless we can get a good
infrastructure written where the hardware info is more of a data sheet
then a single test itself.

That's all nice, but when the hardware is complex and not fully abstracted 
behind a kernel API, tests are bound to be hardware-specific. Of course, a bug 
or regression observed only with a specific device, but triggered through the 
usage of abstract APIs only, can lead to a test case written for that device 
but runnable with any device in the same category. In that case the test case 
should certainly be added to a test suite for the corresponding API/subsystem, 
not to an hidden test suite for a particular device.

-- 
Regards,

Laurent Pinchart