Re: ❌ FAIL: Test report for kernel 5.5.0-rc6-40f39e8.cki (arm-next)

----- Original Message -----

From: "Ard Biesheuvel" <redacted>
To: "Will Deacon" <will@kernel.org>
Cc: "Jianwen Ji" <redacted>, "Veronika Kabatova" <redacted>, "Hangbin Liu" <redacted>,
"Catalin Marinas" [off-list ref], "Jianlin Shi" [off-list ref], "CKI Project"
[off-list ref], "linux-arm-kernel" [off-list ref]
Sent: Thursday, January 16, 2020 3:56:05 PM
Subject: Re: ❌ FAIL: Test report for kernel 5.5.0-rc6-40f39e8.cki (arm-next)

On Thu, 16 Jan 2020 at 15:33, Will Deacon [off-list ref] wrote:

quoted

Hi Veronika,

On Thu, Jan 16, 2020 at 09:14:49AM -0500, Veronika Kabatova wrote:

quoted

quoted

On Thu, 16 Jan 2020 at 14:20, CKI Project < cki-project@redhat.com >
wrote:

quoted

One or more kernel tests failed:

quoted

quoted

aarch64:

quoted

❌ Networking tunnel: gre basic

quoted

❌ Networking tunnel: vxlan basic

May I kindly suggest that you stop cc'ing the mailing list on
automated,
unmoderated CI results?

quoted

It's great that you run a CI system, but you should really have the
minimal
courtesy to double check failure reports before sending them out into
the
world.

we are sorry about that. The testing and emails to the list were
requested
by Will a while ago. If he or Catalin change their minds about them we
will
stop sending them, in the meanwhile you can always set up a filter.

The results are usually very useful (thanks!), and I think it's good to
have them on the list so that people can see them and reply to them, if
necessary. I don't /think/ anybody is disagreeing with that.

We're glad to hear that!

quoted

However, in this specific case, the results aren't very useful because the
infrastructure is playing up, so I think that's what Ard is getting at when
he says "unmoderated".

Apologies for the kneejerk reaction, but there are quite a number of
bots helpfully cc'ing the world on their reports these days, and the
fact that these particular ones are known to be false positives
triggered my unfiltered response.

No worries!

We're working with KernelCI to streamline the upstream testing efforts.
As everything is moving forward, you should see less bots on the list
(at least from people who are also working with the project).

These particular problems we ran into were harder to debug without extra
information (and not caused by our code but a project we depend on) so
we weren't able to filter them out automatically.

quoted

quoted

We are not paid by ARM so the testing is a "strive for the best" effort,
as we have a bunch of ARM machines we're willing to lend to community for
extra coverage. We always cc the test maintainers and also us to take a
look at the issues if they do occur. As Rachel mentioned, we already put
a workaround in place to prevent these issues from happening further so
hopefully you won't get more emails because of these Beaker bugs.

Thanks. If you get a chance, then it would be good to stop the test emails
going out to the mailing list while the infrastructure is known to be
broken, otherwise it can distract people into investigating failures that
aren't actually caused by changes to the kernel.

I agree.

We do have a few things in mind to reduce the false positives (which would
also filter out these ones) and pipeline stabilization in general. This
includes a switch to stop the automatic reports when we detect a reoccurring
infra problem and can't easily work around it.


Thanks,
Veronika

quoted

Cheers,

Will

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel