----- Original Message -----

From: "Veronika Kabatova" <redacted>
To: "Will Deacon" <will@kernel.org>
Cc: "catalin marinas" <catalin.marinas@arm.com>, linux-arm-kernel@lists.infradead.org, "CKI Project"
[off-list ref]
Sent: Thursday, February 11, 2021 1:25:34 PM
Subject: Re: ❌ FAIL:?Test?report?for?kernel?5.11.0-rc7 (arm-next)



----- Original Message -----

quoted

From: "Will Deacon" <will@kernel.org>
To: "Veronika Kabatova" <redacted>
Cc: "catalin marinas" <catalin.marinas@arm.com>,
linux-arm-kernel@lists.infradead.org, "CKI Project"
[off-list ref]
Sent: Thursday, February 11, 2021 12:50:50 PM
Subject: Re: ❌ FAIL:?Test?report?for?kernel?5.11.0-rc7 (arm-next)

On Thu, Feb 11, 2021 at 05:46:02AM -0500, Veronika Kabatova wrote:

quoted

quoted

On Wed, Feb 10, 2021 at 08:17:53PM +0000, Will Deacon wrote:

quoted

On Wed, Feb 10, 2021 at 02:31:45PM -0500, Veronika Kabatova wrote:

quoted

quoted

quoted

quoted

quoted

The machine in question can on course be somewhat flaky
(hard
to
eliminate
that possibility completely), but I checked our historical
data
and it
didn't fail to boot a single time other than with these two
new
kernels.

So the first thing we should probably try is whether vanilla
-rc7
fails
on
the machine causing us problems. If it does, then the arm64
queue
for
5.12
is out of the equation, if not then we can try a targetted
bisection.

Would you be able to try v5.11-rc7 please?

Can do.

Brill, thanks.

quoted

Someone snatched up the machine in the meanwhile and appears to
have
it
reserved till next Monday :/ I sincerely hope that they'll
release it
sooner
and have queued up the test job with high priority. Will let
you
know
right
as I get the results.

Just tell them the machine is broken and they really don't want
to
use
it
for anything important ;)

Our wishes were granted by the lab fairy and the machine was
returned
rather quickly :)

The 5.11-rc7 kernel boots.

Fantastic! Then it's something in the arm64 for-next/core tree that
was
added since then. The diff isn't huge and one change stands out for
me,
so
let me try reverting that and I'll update the branch...

Ok, I updated for-kernelci so that it contains the arm64 for-next/core
branch merged into -rc7, but with a couple of patches reverted on top.

HEAD is	e56137cc7606 ("Revert "arm64/mm: Fix pfn_valid() for
ZONE_DEVICE
based memory""). Please can you try that?

I didn't actually have to, the autopick picked the machine for the
testing
and it happily booted :) \o/

Phew, so I'll drop those from linux-next as well. Do you know if "earlycon"
works on the problematic machine? If possible, it would be helpful to try
booting the bad kernel with earlycon on the cmdline to see if it manages to
say anything in its dying breath.

No idea, we don't have the option enabled and I can't find any information
about it. I submitted a new job with the option and let's see whether it
works or not. The machine is reserved again so there may be some delays.

I finally got access to the machine and don't have any good news. The hang
is from before early printk takes effect. The output is identical to what
I sent before, with nothing resembling kernel lines in there.


Veronika

quoted

quoted

Apparently IT cut off our email sending access because we're sending too
many emails so that's why you don't have the report yet. When we work
around it I'll make sure to retrigger the sending so you have it.

That's nice of them...

Update for the drama loving folks: IT is actually innocent in this.
Someone deployed a misconfigured application in the same cluster as our
reporting system is in, and this application is sending out emails
every 3 seconds. We're collateral damage. Working on resolution, but
right now it seems that the situation is under control.


Veronika

quoted

Will

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel