[tip:locking/core] locking/refcount: Create unchecked atomic_t implementation
From: tip-bot for Kees Cook <hidden>
Date: 2017-06-28 17:08:13
Also in:
lkml
Subsystem:
atomic infrastructure, library code, the rest · Maintainers:
Will Deacon, Peter Zijlstra, Boqun Feng, Andrew Morton, Linus Torvalds
Commit-ID: fd25d19f6b8da315332bb75936605fb45d3ea981 Gitweb: http://git.kernel.org/tip/fd25d19f6b8da315332bb75936605fb45d3ea981 Author: Kees Cook [off-list ref] AuthorDate: Wed, 21 Jun 2017 13:00:26 -0700 Committer: Ingo Molnar [off-list ref] CommitDate: Wed, 28 Jun 2017 18:54:46 +0200 locking/refcount: Create unchecked atomic_t implementation Many subsystems will not use refcount_t unless there is a way to build the kernel so that there is no regression in speed compared to atomic_t. This adds CONFIG_REFCOUNT_FULL to enable the full refcount_t implementation which has the validation but is slightly slower. When not enabled, refcount_t uses the basic unchecked atomic_t routines, which results in no code changes compared to just using atomic_t directly. Signed-off-by: Kees Cook <redacted> Acked-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org> Cc: Alexey Dobriyan <redacted> Cc: Andrew Morton <akpm@linux-foundation.org> Cc: Arnd Bergmann <arnd@arndb.de> Cc: Christoph Hellwig <hch@infradead.org> Cc: David S. Miller <davem@davemloft.net> Cc: David Windsor <redacted> Cc: Davidlohr Bueso <dave@stgolabs.net> Cc: Elena Reshetova <elena.reshetova@intel.com> Cc: Eric Biggers <redacted> Cc: Eric W. Biederman <redacted> Cc: Hans Liljestrand <redacted> Cc: James Bottomley <James.Bottomley@hansenpartnership.com> Cc: Jann Horn <jannh@google.com> Cc: Josh Poimboeuf <redacted> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Manfred Spraul <redacted> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Rik van Riel <redacted> Cc: Serge E. Hallyn <serge@hallyn.com> Cc: Thomas Gleixner <redacted> Cc: arozansk@redhat.com Cc: axboe@kernel.dk Cc: linux-arch <redacted> Link: http://lkml.kernel.org/r/20170621200026.GA115679@beast Signed-off-by: Ingo Molnar <mingo@kernel.org> --- arch/Kconfig | 9 +++++++++ include/linux/refcount.h | 42 ++++++++++++++++++++++++++++++++++++++++++ lib/refcount.c | 3 +++ 3 files changed, 54 insertions(+)
diff --git a/arch/Kconfig b/arch/Kconfig
index 6c00e5b..f76b214 100644
--- a/arch/Kconfig
+++ b/arch/Kconfig@@ -867,4 +867,13 @@ config STRICT_MODULE_RWX config ARCH_WANT_RELAX_ORDER bool +config REFCOUNT_FULL + bool "Perform full reference count validation at the expense of speed" + help + Enabling this switches the refcounting infrastructure from a fast + unchecked atomic_t implementation to a fully state checked + implementation, which can be (slightly) slower but provides protections + against various use-after-free conditions that can be used in + security flaw exploits. + source "kernel/gcov/Kconfig"
diff --git a/include/linux/refcount.h b/include/linux/refcount.h
index b34aa64..bb71f28 100644
--- a/include/linux/refcount.h
+++ b/include/linux/refcount.h@@ -41,6 +41,7 @@ static inline unsigned int refcount_read(const refcount_t *r) return atomic_read(&r->refs); } +#ifdef CONFIG_REFCOUNT_FULL extern __must_check bool refcount_add_not_zero(unsigned int i, refcount_t *r); extern void refcount_add(unsigned int i, refcount_t *r);
@@ -52,6 +53,47 @@ extern void refcount_sub(unsigned int i, refcount_t *r); extern __must_check bool refcount_dec_and_test(refcount_t *r); extern void refcount_dec(refcount_t *r); +#else +static inline __must_check bool refcount_add_not_zero(unsigned int i, refcount_t *r) +{ + return atomic_add_unless(&r->refs, i, 0); +} + +static inline void refcount_add(unsigned int i, refcount_t *r) +{ + atomic_add(i, &r->refs); +} + +static inline __must_check bool refcount_inc_not_zero(refcount_t *r) +{ + return atomic_add_unless(&r->refs, 1, 0); +} + +static inline void refcount_inc(refcount_t *r) +{ + atomic_inc(&r->refs); +} + +static inline __must_check bool refcount_sub_and_test(unsigned int i, refcount_t *r) +{ + return atomic_sub_and_test(i, &r->refs); +} + +static inline void refcount_sub(unsigned int i, refcount_t *r) +{ + atomic_sub(i, &r->refs); +} + +static inline __must_check bool refcount_dec_and_test(refcount_t *r) +{ + return atomic_dec_and_test(&r->refs); +} + +static inline void refcount_dec(refcount_t *r) +{ + atomic_dec(&r->refs); +} +#endif /* CONFIG_REFCOUNT_FULL */ extern __must_check bool refcount_dec_if_one(refcount_t *r); extern __must_check bool refcount_dec_not_one(refcount_t *r);
diff --git a/lib/refcount.c b/lib/refcount.c
index 9f90678..5d0582a 100644
--- a/lib/refcount.c
+++ b/lib/refcount.c@@ -37,6 +37,8 @@ #include <linux/refcount.h> #include <linux/bug.h> +#ifdef CONFIG_REFCOUNT_FULL + /** * refcount_add_not_zero - add a value to a refcount unless it is 0 * @i: the value to add to the refcount
@@ -225,6 +227,7 @@ void refcount_dec(refcount_t *r) WARN_ONCE(refcount_dec_and_test(r), "refcount_t: decrement hit 0; leaking memory.\n"); } EXPORT_SYMBOL(refcount_dec); +#endif /* CONFIG_REFCOUNT_FULL */ /** * refcount_dec_if_one - decrement a refcount if it is 1