Re: linux-next: origin tree build failure

On Fri, 2009-06-12 at 15:44 +0200, Ingo Molnar wrote:

This is certainly doable for agreeable features - which is the bulk 
- and it is being done.

But this is a catch-22 for _controversial_ new features - which 
perfcounters clearly was, in case you turned off your lkml 
subscription ;-)

I didn't :-) My point here is that Linus can make a decision with an
email -before- merging so that -next gets a chance, at least for a
couple of days, to do the integration testing once the controversy has
been sorted by his highness.

And if you hit that build breakage during bisection you can do:

   git cherry-pick e14112d

Right, I can, you can, but can random tester who wants to track down
what his problem is ? I'm not sure...

Also, you seem to brush off the notion that far more bugs slip 
through linux-next than get caught by it.

Less than without linux-next. We aren't perfect and no process will
solve everything. But this could have been easily avoided.

So if you think linux-next matters in terms of _regression_ testing, 
the numbers dont seem to support that notion. This particular 
incident does support that notion though, granted - but it's taken 
out of context IMHO:

In terms of test coverage, at least for our trees, less than 1% of 
the bugs we handle get reported in a linux-next context - and most 
of the bugs that get reported (against say the scheduler tree) are 
related to rare architectures.

But most obvious bugs will have been caught way before that, which
leaves the hard to catch ones or the configuration-specific ones. Those
will pass linux-next, I agree. But that isn't my point and that isn't
what linux-next will catch.  What is will catch is that kind of really
simple mechanical problems, such as build breakage for other archs.

If perfcounters had been 1 or 2 days in -next before being merged, we
would have avoided that problem and made everybody's bisecting life
easier.

In fact, i checked, there were _zero_ x86 bugs reported against 
linux-next and solved against it between v2.6.30-rc1 and v2.6.30:

No but Stephen caught a bunch of mechanical compile fails due to
integration problems.

   git log --grep=next -i v2.6.30-rc1..v2.6.30 arch/x86/

Doing it over the full cycle shows one commit altogether - a Xen 
build failure. In fact, i just checked the whole stabilization cycle 
for the whole kernel (v2.6.30-rc1..v2.6.30-final), and there were 
only 5 linux-next originated patches, most of them build failures.

I did this by looking at all occurances of 'next', in all commit 
logs:

   git log --grep=next -i v2.6.30-rc1..v2.6.30

and then manually checking the context of all 'next' matches and 
counting the linux-next related commits.

So lets be generous and say that because some people dont put the 
bug report originator into the changelog it was four times as many, 
20 - but that's still dwarved by the sheer amount of post-rc1 
changes: thousands of changes and hundreds of regressions.

linux-next is mostly useful (to me at least) not for the 
cross-builds it does, but in terms of mapping out upcoming conflicts 
- which also drives early detection of problematic patches and 
problematic conflicts.

Yes, it does. The problem is that it helps -you- that way, but won't
help -us- vs. that kind of mechanical problems unless -you- also play
the game and get your stuff in there for a little while before merging
it :-)

Cheers,
Ben.