arm/arm64 HARDLOCKUP Detector and PERF NMI support

3 messages, 3 authors, 2016-01-11 · open the first message on its own page

arm/arm64 HARDLOCKUP Detector and PERF NMI support

From: mark.rutland@arm.com (Mark Rutland)
Date: 2016-01-11 10:26:58

Hi,

On Sat, Jan 09, 2016 at 04:13:48PM -0800, Mario Smarduch wrote:
I'm trying to get HARDLOCKUP detector to work on arm (on foundation
model and Juno, but armv7 should have the same issue). Turns out there is no
PERF NMI support (HAVE_PERF_EVENTS_NMI), which I guess would be PMU NMI - I
tested overflow on cycle counter and PMU interrupts can be masked. HARDLOCKUP
assumes perf interrupts are non-maskable and uses the PMU to program an NMI to
detect hard lockup and that in turn drives softlockup detection.

Is there something, someplace in ARM manuals that addresses this, searched the
manuals but couldn't find anything?

Or other approaches that I'm not aware off or have floated on the mailing list.
Russell and Daniel did some work [1,2] to use FIQs as NMIs on arm, for
backtracing and PMU overflow.

For arm64 the kernel always runs on the non-secure side, so we don't
expect to have access to FIQ. It's possible to implement NMIs using
priorities (which it looks like Daniel was looking into [3]).

Thanks,
Mark.

[1] http://thread.gmane.org/gmane.linux.ports.arm.kernel/353795/
[2] http://www.linaro.org/blog/core-dump/debugging-arm-kernels-using-nmifiq/
[3] https://git.linaro.org/people/daniel.thompson/linux.git/shortlog/refs/heads/dev/arm64_nmi-v5

arm/arm64 HARDLOCKUP Detector and PERF NMI support

From: Daniel Thompson <hidden>
Date: 2016-01-11 11:25:49

On 11/01/16 10:26, Mark Rutland wrote:
Hi,

On Sat, Jan 09, 2016 at 04:13:48PM -0800, Mario Smarduch wrote:
quoted
I'm trying to get HARDLOCKUP detector to work on arm (on foundation
model and Juno, but armv7 should have the same issue). Turns out there is no
PERF NMI support (HAVE_PERF_EVENTS_NMI), which I guess would be PMU NMI - I
tested overflow on cycle counter and PMU interrupts can be masked. HARDLOCKUP
assumes perf interrupts are non-maskable and uses the PMU to program an NMI to
detect hard lockup and that in turn drives softlockup detection.

Is there something, someplace in ARM manuals that addresses this, searched the
manuals but couldn't find anything?

Or other approaches that I'm not aware off or have floated on the mailing list.
Russell and Daniel did some work [1,2] to use FIQs as NMIs on arm, for
backtracing and PMU overflow.
 >
For arm64 the kernel always runs on the non-secure side, so we don't
expect to have access to FIQ. It's possible to implement NMIs using
priorities (which it looks like Daniel was looking into [3]).
Thanks Mark.

The blog post is still pretty much describes my ambitions and the 
benefits these features can bring. It also has a link to a fairly 
elderly kernel that contains all the features fully integrated.

There are more patchsets for both ARM (if/f FIQ is available) and ARM64 
(if/f device has GICv3). However these only provide NMI backtrace.

arm:   http://thread.gmane.org/gmane.linux.kernel/2112583

arm64: http://thread.gmane.org/gmane.linux.kernel/2037735 (Mark's
        link to my git repo is more up to date than this patchset)

To get the hard lockup and profiling support upstream then my old code 
to integrate NMIs into the irq subsystem (which was just a cut down 
clone of the existing irq infrastructure) needs to be redesigned. 
However I decided to put the rewrite on ice until I get the foundational 
patches right.


Daniel.

arm/arm64 HARDLOCKUP Detector and PERF NMI support

From: Mario Smarduch <hidden>
Date: 2016-01-11 20:06:04


On 1/11/2016 3:25 AM, Daniel Thompson wrote:
On 11/01/16 10:26, Mark Rutland wrote:
quoted
Hi,

On Sat, Jan 09, 2016 at 04:13:48PM -0800, Mario Smarduch wrote:
quoted
I'm trying to get HARDLOCKUP detector to work on arm (on foundation
model and Juno, but armv7 should have the same issue). Turns out there is no
PERF NMI support (HAVE_PERF_EVENTS_NMI), which I guess would be PMU NMI - I
tested overflow on cycle counter and PMU interrupts can be masked. HARDLOCKUP
assumes perf interrupts are non-maskable and uses the PMU to program an NMI to
detect hard lockup and that in turn drives softlockup detection.

Is there something, someplace in ARM manuals that addresses this, searched the
manuals but couldn't find anything?

Or other approaches that I'm not aware off or have floated on the mailing list.
Russell and Daniel did some work [1,2] to use FIQs as NMIs on arm, for
backtracing and PMU overflow.
 >
quoted
For arm64 the kernel always runs on the non-secure side, so we don't
expect to have access to FIQ. It's possible to implement NMIs using
priorities (which it looks like Daniel was looking into [3]).
Thanks Mark.

The blog post is still pretty much describes my ambitions and the 
benefits these features can bring. It also has a link to a fairly 
elderly kernel that contains all the features fully integrated.

There are more patchsets for both ARM (if/f FIQ is available) and ARM64 
(if/f device has GICv3). However these only provide NMI backtrace.

arm:   http://thread.gmane.org/gmane.linux.kernel/2112583

arm64: http://thread.gmane.org/gmane.linux.kernel/2037735 (Mark's
        link to my git repo is more up to date than this patchset)

To get the hard lockup and profiling support upstream then my old code 
to integrate NMIs into the irq subsystem (which was just a cut down 
clone of the existing irq infrastructure) needs to be redesigned. 
However I decided to put the rewrite on ice until I get the foundational 
patches right.


Daniel.
Daniel, Mark - thanks for the info. I'll research the work done so far and
probably comeback later.

- Mario
Keyboard shortcuts
hback out one level
jnext message in thread
kprevious message in thread
ldrill in
Escclose help / fold thread tree
?toggle this help