Re: [PATCH] mm/rmap: Convert from atomic_t to refcount_t on anon_vma->refcount
From: Kees Cook <hidden>
Date: 2021-08-20 17:26:15
Also in:
lkml
On Fri, Aug 20, 2021 at 11:03:00AM +0200, Peter Zijlstra wrote:
quoted hunk ↗ jump to hunk
On Fri, Aug 20, 2021 at 09:24:58AM +0100, Will Deacon wrote:quoted
quoted
gcc-10.2.1, x86_64-defconfig kernel/event/core.o-inline-ud1: 96454 kernel/event/core.o-outofline-ud1: 96604 kernel/event/core.o-outofline-call: 97072kernel/event/core.o-outofline-saturate-ud2: 96954 kernel/event/core.o: 97248quoted
Is that with the saturation moved to the UD handler as well?Yep, that's the full function call replaced with an exception.quoted
I think it would be good to keep that as close to the point at which we detect the problem as we can, so perhaps we can inline that part and leave the diagnostics to the exception handler?That's simpler execption code too, we can abuse the existing WARN/UD2 stuff. --- arch/x86/include/asm/refcount.h | 31 +++++++++++++++++++++++++++++++ include/asm-generic/bug.h | 4 ++++ include/linux/refcount.h | 15 +++++++++++---- lib/bug.c | 13 ++++++++++++- lib/refcount.c | 7 ++----- 5 files changed, 60 insertions(+), 10 deletions(-)diff --git a/arch/x86/include/asm/refcount.h b/arch/x86/include/asm/refcount.h new file mode 100644 index 000000000000..bed52b95d24c --- /dev/null +++ b/arch/x86/include/asm/refcount.h@@ -0,0 +1,31 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef _ASM_X86_REFCOUNT_H +#define _ASM_X86_REFCOUNT_H + +#define refcount_warn_saturate refcount_warn_saturate +static __always_inline void refcount_warn_saturate(refcount_t *r, const enum refcount_saturation_type t) +{ + refcount_set(r, REFCOUNT_SATURATED); + __WARN_FLAGS(BUGFLAG_ONCE|BUGFLAG_REFCOUNT|BUGFLAG_REFCOUNT_TYPE(t)); +}
Instead of using up warn flags, what was done in the past was to use an explicit EXTABLE in a cold text section: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/arch/x86/include/asm/asm.h?h=v4.15#n80 https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/arch/x86/mm/extable.c?h=v4.15#n45 https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/arch/x86/include/asm/refcount.h?h=v4.15
+
+#define refcount_dec_and_test refcount_dec_and_test
+static inline bool refcount_dec_and_test(refcount_t *r)
+{
+ asm_volatile_goto (LOCK_PREFIX "decl %[var]\n\t"
+ "jz %l[cc_zero]\n\t"
+ "jl %l[cc_error]"
+ : : [var] "m" (r->refs.counter)
+ : "memory" : cc_zero, cc_error);
+
+ return false;
+
+cc_zero:
+ return true;
+
+cc_error:
+ refcount_warn_saturate(r, REFCOUNT_SUB_UAF);
+ return false;
+}This looks basically the same as what we had before (i.e. the earlier REFCOUNT_CHECK_LE_ZERO within GEN_UNARY_SUFFIXED_RMWcc). -- Kees Cook