Re: [RFC PATCH] iov: Bypass usercopy hardening for kernel iterators
From: Kees Cook <kees@kernel.org>
Date: 2026-03-25 21:27:11
Also in:
linux-block, linux-fsdevel, linux-hardening
On Tue, Mar 03, 2026 at 11:29:32AM -0500, Chuck Lever wrote:
From: Chuck Lever <chuck.lever@oracle.com> Profiling NFSD under an iozone workload showed that hardened usercopy checks consume roughly 1.3% of CPU in the TCP receive path. The runtime check in check_object_size() validates that copy buffers reside in expected slab regions, which is meaningful when data crosses the user/kernel boundary but adds no value when both source and destination are kernel addresses. Split check_copy_size() so that copy_to_iter() can bypass the runtime check_object_size() call for kernel-only iterators (ITER_BVEC, ITER_KVEC). Existing callers of check_copy_size() are unaffected; user-backed iterators still receive the full usercopy validation. This benefits all kernel consumers of copy_to_iter(), including the TCP receive path used by the NFS client and server, NVMe-TCP, and any other subsystem that uses ITER_BVEC or ITER_KVEC receive buffers.
So, I'm not a big fan of this just because the whole point is to catch unexpected conditions, but there is a reasonable point to be made that this case shouldn't be covered by kernel/kernel copies.
quoted hunk ↗ jump to hunk
Signed-off-by: Chuck Lever <chuck.lever@oracle.com> --- include/linux/ucopysize.h | 10 +++++++++- include/linux/uio.h | 9 +++++++-- 2 files changed, 16 insertions(+), 3 deletions(-)diff --git a/include/linux/ucopysize.h b/include/linux/ucopysize.h index 41c2d9720466..b3eacb4869a8 100644 --- a/include/linux/ucopysize.h +++ b/include/linux/ucopysize.h@@ -42,7 +42,7 @@ static inline void copy_overflow(int size, unsigned long count) } static __always_inline __must_check bool -check_copy_size(const void *addr, size_t bytes, bool is_source) +check_copy_size_nosec(const void *addr, size_t bytes, bool is_source)
"nosec" is kind of ambiguous. Since this is doing the compile-time checks, how about naming this __compiletime_check_copy_size() or so?
quoted hunk ↗ jump to hunk
{ int sz = __builtin_object_size(addr, 0); if (unlikely(sz >= 0 && sz < bytes)) {@@ -56,6 +56,14 @@ check_copy_size(const void *addr, size_t bytes, bool is_source) } if (WARN_ON_ONCE(bytes > INT_MAX)) return false; + return true; +} + +static __always_inline __must_check bool +check_copy_size(const void *addr, size_t bytes, bool is_source) +{ + if (!check_copy_size_nosec(addr, bytes, is_source)) + return false; check_object_size(addr, bytes, is_source); return true; }diff --git a/include/linux/uio.h b/include/linux/uio.h index a9bc5b3067e3..f860529abfbe 100644 --- a/include/linux/uio.h +++ b/include/linux/uio.h@@ -216,8 +216,13 @@ size_t copy_page_to_iter_nofault(struct page *page, unsigned offset, static __always_inline __must_check size_t copy_to_iter(const void *addr, size_t bytes, struct iov_iter *i) { - if (check_copy_size(addr, bytes, true)) - return _copy_to_iter(addr, bytes, i); + if (user_backed_iter(i)) { + if (check_copy_size(addr, bytes, true)) + return _copy_to_iter(addr, bytes, i); + } else { + if (check_copy_size_nosec(addr, bytes, true)) + return _copy_to_iter(addr, bytes, i); + } return 0; }
This seems reasonable with the renaming, though I might come back some day and ask that this get a boot param or something (we have a big hammer boot param for usercopy checking already, but I like this more focused check). -- Kees Cook