Re: [PATCH v3 18/21] powerpc: Add option to use jump label for cpu_has_feature()

From
Nicholas Piggin <npiggin@gmail.com>
Date
2016-07-28 07:52:03
Reply to
Michael Ellerman, 2016-07-27
On Thu, 28 Jul 2016 00:18:14 +1000
Michael Ellerman [off-list ref] wrote:
quoted hunk ↗ jump to hunk
From: Kevin Hao <redacted>

We do binary patching of asm code using CPU features, which is a
one-time operation, done during early boot. However checks of CPU
features in C code are currently done at run time, even though the set
of CPU features can never change after boot.

We can optimise this by using jump labels to implement
cpu_has_feature(), meaning checks in C code are binary patched into a
single nop or branch.

For a C sequence along the lines of:

    if (cpu_has_feature(FOO))
         return 2;

The generated code before is roughly:

    ld      r9,-27640(r2)
    ld      r9,0(r9)
    lwz     r9,32(r9)
    cmpwi   cr7,r9,0
    bge     cr7, 1f
    li      r3,2
    blr
1:  ...

After (true):
    nop
    li      r3,2
    blr

After (false):
    b	1f
    li      r3,2
    blr
1:  ...

Signed-off-by: Kevin Hao <redacted>
Signed-off-by: Aneesh Kumar K.V <redacted>
Signed-off-by: Michael Ellerman <mpe@ellerman.id.au>
---
 arch/powerpc/include/asm/cpu_has_feature.h | 22
++++++++++++++++++++++ arch/powerpc/include/asm/cputable.h        |
6 ++++++ arch/powerpc/kernel/cputable.c             | 20
++++++++++++++++++++ arch/powerpc/lib/feature-fixups.c          |  1 +
 4 files changed, 49 insertions(+)

v3: Rename MAX_CPU_FEATURES as we already have a #define with that
name. Define NUM_CPU_FTR_KEYS as a constant.
    Rename the array to cpu_feature_keys.
    Use the kconfig we added to guard it.
    Rewrite the change log.
diff --git a/arch/powerpc/include/asm/cpu_has_feature.h
b/arch/powerpc/include/asm/cpu_has_feature.h index
ad296b2f1d84..18e60e61bea9 100644 ---
a/arch/powerpc/include/asm/cpu_has_feature.h +++
b/arch/powerpc/include/asm/cpu_has_feature.h @@ -11,10 +11,32 @@
static inline bool __cpu_has_feature(unsigned long feature)
(CPU_FTRS_POSSIBLE & cur_cpu_spec->cpu_features & feature)); }
 
+#ifdef CONFIG_JUMP_LABEL_FEATURE_CHECKS
+#include <linux/jump_label.h>
+
+#define NUM_CPU_FTR_KEYS	64
+
+extern struct static_key_true cpu_feature_keys[NUM_CPU_FTR_KEYS];
+
+static __always_inline bool cpu_has_feature(unsigned long feature)
+{
+	int i;
+
+	if (CPU_FTRS_ALWAYS & feature)
+		return true;
+
+	if (!(CPU_FTRS_POSSIBLE & feature))
+		return false;
+
+	i = __builtin_ctzl(feature);
+	return static_branch_likely(&cpu_feature_keys[i]);
Just a reminder to add a BUILD_BUG_ON(!__builtin_constant_p(feature));
for this.
Thread (40 messages) 40 messages, 5 authors, last reply 2016-08-08
Keyboard shortcuts
hback out one level
jnext message in thread / next series-diff position
kprevious message in thread / previous series-diff position
ldrill in
Escclose help / fold thread tree
?toggle this help