Re: [PATCH net-next] scm: optimize put_cmsg()
From: Soheil Hassas Yeganeh <hidden>
Date: 2021-04-15 18:10:47
On Thu, Apr 15, 2021 at 1:38 PM Eric Dumazet [off-list ref] wrote:
From: Eric Dumazet <edumazet@google.com> Calling two copy_to_user() for very small regions has very high overhead. Switch to inlined unsafe_put_user() to save one stac/clac sequence, and avoid copy_to_user(). Signed-off-by: Eric Dumazet <edumazet@google.com> Cc: Soheil Hassas Yeganeh <redacted>
Acked-by: Soheil Hassas Yeganeh <redacted> Very nice! Thank you, Eric!
quoted hunk ↗ jump to hunk
--- net/core/scm.c | 21 ++++++++++++++------- 1 file changed, 14 insertions(+), 7 deletions(-)diff --git a/net/core/scm.c b/net/core/scm.c index 8156d4fb8a3966122fdfcfd0ebc9e5520aa7b67c..bd96c922041d22a2f3b7ee73e4b3183316f9b616 100644 --- a/net/core/scm.c +++ b/net/core/scm.c@@ -228,14 +228,16 @@ int put_cmsg(struct msghdr * msg, int level, int type, int len, void *data) if (msg->msg_control_is_user) { struct cmsghdr __user *cm = msg->msg_control_user; - struct cmsghdr cmhdr; - cmhdr.cmsg_level = level; - cmhdr.cmsg_type = type; - cmhdr.cmsg_len = cmlen; - if (copy_to_user(cm, &cmhdr, sizeof cmhdr) || - copy_to_user(CMSG_USER_DATA(cm), data, cmlen - sizeof(*cm))) - return -EFAULT; + if (!user_write_access_begin(cm, cmlen)) + goto efault; + + unsafe_put_user(len, &cm->cmsg_len, efault_end); + unsafe_put_user(level, &cm->cmsg_level, efault_end); + unsafe_put_user(type, &cm->cmsg_type, efault_end); + unsafe_copy_to_user(CMSG_USER_DATA(cm), data, + cmlen - sizeof(*cm), efault_end); + user_write_access_end(); } else { struct cmsghdr *cm = msg->msg_control;@@ -249,6 +251,11 @@ int put_cmsg(struct msghdr * msg, int level, int type, int len, void *data) msg->msg_control += cmlen; msg->msg_controllen -= cmlen; return 0; + +efault_end: + user_write_access_end(); +efault: + return -EFAULT; } EXPORT_SYMBOL(put_cmsg); --2.31.1.368.gbe11c130af-goog