[PATCH net-next 08/12] net: Copy slab data for sendmsg(MSG_SPLICE_PAGES)
From: David Howells <dhowells@redhat.com>
Date: 2023-05-24 15:36:21
Also in:
linux-mm, lkml
Subsystem:
networking [general], the rest · Maintainers:
"David S. Miller", Eric Dumazet, Jakub Kicinski, Paolo Abeni, Linus Torvalds
If sendmsg() is passed MSG_SPLICE_PAGES and is given a buffer that contains some data that's resident in the slab, copy/coalesce it rather than returning EIO. Signed-off-by: David Howells <dhowells@redhat.com> cc: Eric Dumazet <edumazet@google.com> cc: "David S. Miller" <davem@davemloft.net> cc: David Ahern <dsahern@kernel.org> cc: Jakub Kicinski <kuba@kernel.org> cc: Paolo Abeni <pabeni@redhat.com> cc: Jens Axboe <axboe@kernel.dk> cc: Matthew Wilcox <willy@infradead.org> cc: netdev@vger.kernel.org --- include/linux/skbuff.h | 3 +++ net/core/skbuff.c | 33 ++++++++++++++++++++++++++++++--- 2 files changed, 33 insertions(+), 3 deletions(-)
diff --git a/include/linux/skbuff.h b/include/linux/skbuff.h
index e11a765fe7fa..11d98990f5f1 100644
--- a/include/linux/skbuff.h
+++ b/include/linux/skbuff.h@@ -5084,6 +5084,9 @@ static inline void skb_mark_for_recycle(struct sk_buff *skb) #endif } +ssize_t skb_splice_from_iter(struct sk_buff *skb, struct iov_iter *iter, + ssize_t maxsize, gfp_t gfp); + ssize_t skb_splice_from_iter(struct sk_buff *skb, struct iov_iter *iter, ssize_t maxsize, gfp_t gfp);
diff --git a/net/core/skbuff.c b/net/core/skbuff.c
index c2840b0dcad9..a16499b9942b 100644
--- a/net/core/skbuff.c
+++ b/net/core/skbuff.c@@ -6927,17 +6927,44 @@ ssize_t skb_splice_from_iter(struct sk_buff *skb, struct iov_iter *iter, break; } + if (space == 0 && + !skb_can_coalesce(skb, skb_shinfo(skb)->nr_frags, + pages[0], off)) { + iov_iter_revert(iter, len); + break; + } + i = 0; do { struct page *page = pages[i++]; size_t part = min_t(size_t, PAGE_SIZE - off, len); - - ret = -EIO; - if (WARN_ON_ONCE(!sendpage_ok(page))) + bool put = false; + + if (PageSlab(page)) { + const void *p; + void *q; + + p = kmap_local_page(page); + q = page_frag_memdup(NULL, p + off, part, gfp, + ULONG_MAX); + kunmap_local(p); + if (!q) { + iov_iter_revert(iter, len); + ret = -ENOMEM; + goto out; + } + page = virt_to_page(q); + off = offset_in_page(q); + put = true; + } else if (WARN_ON_ONCE(!sendpage_ok(page))) { + ret = -EIO; goto out; + } ret = skb_append_pagefrags(skb, page, off, part, frag_limit); + if (put) + put_page(page); if (ret < 0) { iov_iter_revert(iter, len); goto out;