[PATCH] powerpc/kasan: require memintrinsic prefix support for KASAN
COLD17d
From: Mukesh Kumar Chaurasiya (IBM) <hidden>
Date: 2026-09-08 06:50:02
Also in:
linux-hardening, linux-riscv, lkml
Subsystem:
kasan, linux for powerpc (32-bit and 64-bit), the rest · Maintainers:
Andrey Ryabinin, Madhavan Srinivasan, Linus Torvalds
powerpc unconditionally selects GENERIC_ENTRY. The GENERIC_ENTRY
infrastructure relies on the compiler emitting __asan_mem*() calls at
instrumented mem*() sites rather than plain memset/memcpy/memmove, so
that entry/exit paths calling those functions are not instrumented.
This assumption is encoded in two places:
mm/kasan/shadow.c:
#if !defined(CONFIG_CC_HAS_KASAN_MEMINTRINSIC_PREFIX) &&
!defined(CONFIG_GENERIC_ENTRY)
include/linux/fortify-string.h:
#if !defined(CONFIG_CC_HAS_KASAN_MEMINTRINSIC_PREFIX) &&
!defined(CONFIG_GENERIC_ENTRY)
When GENERIC_ENTRY is set, both guards suppress the C wrappers for
memset/memcpy/memmove and the __underlying_mem*() redirections. This
is only safe when the compiler supports the prefixed __asan_mem*()
intrinsics. On older toolchains (e.g. GCC 9) that lack this support,
plain mem*() calls from instrumented code fall through to the raw
assembly implementations in mem_64.S / copy_32.S, completely bypassing
the KASAN shadow check.
Other arches with GENERIC_ENTRY (x86, s390, loongarch, riscv) do not
hit this because their CI toolchains are always new enough to support
the prefix flag.
Background: the !GENERIC_ENTRY guard was introduced by commit 69d4c0d32186
("entry, kasan, x86: Disallow overriding mem*() functions", Peter Zijlstra,
Jan 2023). The root problem is that the KASAN C wrappers override the
linker symbol memset/memcpy/memmove globally, so any call from noinstr or
__no_sanitize_address code (e.g. irqentry_enter/irqentry_exit) would still
reach the KASAN shadow-check wrapper -- at a point where KASAN invariants
may not hold. The compiler prefix approach (Marco Elver, Feb 2023,
commit 51287dcb00cc) solves this by having the compiler emit __asan_memset
at instrumented call sites and bare memset inside __no_sanitize_address
functions, splitting the decision at code-generation time rather than at
link time.
A manual C-level override cannot replicate this split: a single linker
symbol cannot be made to resolve differently depending on the caller.
x86 also placed its raw memset/memcpy/memmove implementations in
.noinstr.text (same commit, 69d4c0d32186), which is the other half of
the fix: noinstr callers hit the raw assembly directly, safely bypassing
KASAN. PowerPC has not done this. Placing mem_64.S / memcpy_64.S /
copy_32.S implementations in .noinstr.text would be the complementary
long-term fix that could re-enable KASAN on older toolchains, but it
requires care around linker stub overflow on large PPC64 kernels (the
same reason powerpc uses NOKPROBE_SYMBOL rather than noinstr for its
interrupt handlers -- see the comment in asm/interrupt.h). That work
is left as a follow-up.
For now, introduce PPC_CC_HAS_KASAN_MEMINTRINSIC_PREFIX, an arch-local
compiler probe that mirrors the same check as CC_HAS_KASAN_MEMINTRINSIC_PREFIX
in lib/Kconfig.kasan but lives outside the 'if KASAN' block to avoid a
recursive dependency (CC_HAS_KASAN_MEMINTRINSIC_PREFIX depends on KASAN
which depends on HAVE_ARCH_KASAN). Gate the three HAVE_ARCH_KASAN selects
on this new symbol so that KASAN is not offered as a config option on
toolchains that cannot support it correctly with GENERIC_ENTRY.
Since KASAN on powerpc now unconditionally implies
CC_HAS_KASAN_MEMINTRINSIC_PREFIX, the old !CC_HAS_KASAN_MEMINTRINSIC_PREFIX
code paths in asm/kasan.h and asm/string.h are dead. Clean them up:
- asm/kasan.h: remove the dual-entry-point variant of _GLOBAL_KASAN /
_GLOBAL_TOC_KASAN / EXPORT_SYMBOL_KASAN that emitted both memset and
__memset as entry points to the same assembly. These aliases were only
needed so the C KASAN wrappers in shadow.c could call __memset() to
reach raw memory ops; with the compiler prefix approach the wrappers are
not used at all for mem* on powerpc.
- asm/string.h: remove the separate __memset/__memcpy/__memmove symbol
declarations and the memset/memcpy/memmove macro redirections for
uninstrumented files that were needed on old toolchains. Simplify the
CONFIG_KASAN block to just the three #define aliases (which are still
used by shadow.c as raw backends).
- cputable.c, prom_init.c: update stale comments that said GCC replaces
memcpy() with __memcpy() under KASAN; with the prefix flag it emits
__asan_memcpy() instead.
Reported-by: Venkat Rao Bagalkote <redacted>
Closes: https://lore.kernel.org/all/96dae110-f79b-4e54-8a9b-514369019b5d@linux.ibm.com (local)
Signed-off-by: Mukesh Kumar Chaurasiya (IBM) <redacted>
---
arch/powerpc/Kconfig | 10 +++++++---
arch/powerpc/include/asm/kasan.h | 19 ++++++++-----------
arch/powerpc/include/asm/string.h | 25 +++++--------------------
arch/powerpc/kernel/cputable.c | 6 +++---
arch/powerpc/kernel/prom_init.c | 4 ++--
5 files changed, 25 insertions(+), 39 deletions(-)
diff --git a/arch/powerpc/Kconfig b/arch/powerpc/Kconfig
index 2580e27e4328..b27ed9739eea 100644
--- a/arch/powerpc/Kconfig
+++ b/arch/powerpc/Kconfig@@ -7,6 +7,10 @@ config CC_HAS_ELFV2 config CC_HAS_PREFIXED def_bool PPC64 && $(cc-option, -mcpu=power10 -mprefixed) +config PPC_CC_HAS_KASAN_MEMINTRINSIC_PREFIX + def_bool (CC_IS_CLANG && $(cc-option,-fsanitize=kernel-address -mllvm -asan-kernel-mem-intrinsic-prefix=1)) || \ + (CC_IS_GCC && $(cc-option,-fsanitize=kernel-address --param asan-kernel-mem-intrinsic-prefix=1)) + config CC_HAS_PCREL # Clang has a bug (https://github.com/llvm/llvm-project/issues/62372) # where pcrel code is not generated if -msoft-float, -mno-altivec, or
@@ -220,9 +224,9 @@ config PPC select HAVE_ARCH_HUGE_VMAP if PPC_RADIX_MMU || PPC_8xx select HAVE_ARCH_JUMP_LABEL select HAVE_ARCH_JUMP_LABEL_RELATIVE - select HAVE_ARCH_KASAN if PPC32 && PAGE_SHIFT <= 14 - select HAVE_ARCH_KASAN if PPC_RADIX_MMU - select HAVE_ARCH_KASAN if PPC_BOOK3E_64 + select HAVE_ARCH_KASAN if PPC32 && PAGE_SHIFT <= 14 && PPC_CC_HAS_KASAN_MEMINTRINSIC_PREFIX + select HAVE_ARCH_KASAN if PPC_RADIX_MMU && PPC_CC_HAS_KASAN_MEMINTRINSIC_PREFIX + select HAVE_ARCH_KASAN if PPC_BOOK3E_64 && PPC_CC_HAS_KASAN_MEMINTRINSIC_PREFIX select HAVE_ARCH_KASAN_VMALLOC if HAVE_ARCH_KASAN select HAVE_ARCH_KCSAN select HAVE_ARCH_KFENCE if ARCH_SUPPORTS_DEBUG_PAGEALLOC
diff --git a/arch/powerpc/include/asm/kasan.h b/arch/powerpc/include/asm/kasan.h
index a690e7da53c2..ffada5f51b2b 100644
--- a/arch/powerpc/include/asm/kasan.h
+++ b/arch/powerpc/include/asm/kasan.h@@ -2,20 +2,17 @@ #ifndef __ASM_KASAN_H #define __ASM_KASAN_H -#if defined(CONFIG_KASAN) && !defined(CONFIG_CC_HAS_KASAN_MEMINTRINSIC_PREFIX) -#define _GLOBAL_KASAN(fn) \ - _GLOBAL(fn); \ - _GLOBAL(__##fn) -#define _GLOBAL_TOC_KASAN(fn) \ - _GLOBAL_TOC(fn); \ - _GLOBAL_TOC(__##fn) -#define EXPORT_SYMBOL_KASAN(fn) \ - EXPORT_SYMBOL(__##fn) -#else /* CONFIG_KASAN && !CONFIG_CC_HAS_KASAN_MEMINTRINSIC_PREFIX */ +/* + * powerpc requires CC_HAS_KASAN_MEMINTRINSIC_PREFIX whenever KASAN is + * enabled (see PPC_CC_HAS_KASAN_MEMINTRINSIC_PREFIX in arch/powerpc/Kconfig), + * so the compiler always emits __asan_mem*() at instrumented call sites and + * bare mem*() inside __no_sanitize_address / noinstr code. The old dual + * entry-point trick (_GLOBAL_KASAN emitting both memset and __memset) is + * therefore never needed. + */ #define _GLOBAL_KASAN(fn) _GLOBAL(fn) #define _GLOBAL_TOC_KASAN(fn) _GLOBAL_TOC(fn) #define EXPORT_SYMBOL_KASAN(fn) -#endif /* CONFIG_KASAN && !CONFIG_CC_HAS_KASAN_MEMINTRINSIC_PREFIX */ #ifndef __ASSEMBLER__
diff --git a/arch/powerpc/include/asm/string.h b/arch/powerpc/include/asm/string.h
index 1981bd4036b5..743126e3dcc1 100644
--- a/arch/powerpc/include/asm/string.h
+++ b/arch/powerpc/include/asm/string.h@@ -29,29 +29,14 @@ extern void * memchr(const void *,int,__kernel_size_t); void memcpy_flushcache(void *dest, const void *src, size_t size); #ifdef CONFIG_KASAN -/* __mem variants are used by KASAN to implement instrumented meminstrinsics. */ -#ifdef CONFIG_CC_HAS_KASAN_MEMINTRINSIC_PREFIX +/* + * powerpc requires CC_HAS_KASAN_MEMINTRINSIC_PREFIX whenever KASAN is + * enabled, so the compiler emits __asan_mem*() at instrumented sites. + * The raw mem* symbols are always safe to call directly. + */ #define __memset memset #define __memcpy memcpy #define __memmove memmove -#else /* CONFIG_CC_HAS_KASAN_MEMINTRINSIC_PREFIX */ -void *__memset(void *s, int c, __kernel_size_t count); -void *__memcpy(void *to, const void *from, __kernel_size_t n); -void *__memmove(void *to, const void *from, __kernel_size_t n); -#ifndef __SANITIZE_ADDRESS__ -/* - * For files that are not instrumented (e.g. mm/slub.c) we - * should use not instrumented version of mem* functions. - */ -#define memcpy(dst, src, len) __memcpy(dst, src, len) -#define memmove(dst, src, len) __memmove(dst, src, len) -#define memset(s, c, n) __memset(s, c, n) - -#ifndef __NO_FORTIFY -#define __NO_FORTIFY /* FORTIFY_SOURCE uses __builtin_memcpy, etc. */ -#endif -#endif /* !__SANITIZE_ADDRESS__ */ -#endif /* CONFIG_CC_HAS_KASAN_MEMINTRINSIC_PREFIX */ #endif /* CONFIG_KASAN */ #ifdef CONFIG_PPC64
diff --git a/arch/powerpc/kernel/cputable.c b/arch/powerpc/kernel/cputable.c
index 6f6801da9dc1..44115f904c2c 100644
--- a/arch/powerpc/kernel/cputable.c
+++ b/arch/powerpc/kernel/cputable.c@@ -36,8 +36,8 @@ void __init set_cur_cpu_spec(struct cpu_spec *s) t = PTRRELOC(t); /* - * use memcpy() instead of *t = *s so that GCC replaces it - * by __memcpy() when KASAN is active + * use memcpy() instead of *t = *s so that the compiler replaces it + * by __asan_memcpy() when KASAN is active */ memcpy(t, s, sizeof(*t));
@@ -55,7 +55,7 @@ static struct cpu_spec * __init setup_cpu_spec(unsigned long offset, /* * Copy everything, then do fixups. Use memcpy() instead of *t = *s - * so that GCC replaces it by __memcpy() when KASAN is active + * so that the compiler replaces it by __asan_memcpy() when KASAN is active */ memcpy(t, s, sizeof(*t));
diff --git a/arch/powerpc/kernel/prom_init.c b/arch/powerpc/kernel/prom_init.c
index eb9f556b0937..b763720c68b6 100644
--- a/arch/powerpc/kernel/prom_init.c
+++ b/arch/powerpc/kernel/prom_init.c@@ -1365,8 +1365,8 @@ static void __init prom_check_platform_support(void) /* * First copy the architecture vec template * - * use memcpy() instead of *vec = *vec_template so that GCC replaces it - * by __memcpy() when KASAN is active + * use memcpy() instead of *vec = *vec_template so that the compiler + * replaces it by __asan_memcpy() when KASAN is active */ memcpy(&ibm_architecture_vec, &ibm_architecture_vec_template, sizeof(ibm_architecture_vec));
--
2.55.0