Re: [PATCH net-next v10 08/16] tls: Inline do_tcp_sendpages()
From: Tariq Toukan <hidden>
Date: 2023-06-07 14:17:42
Also in:
linux-fsdevel, linux-mm, lkml
On 22/05/2023 15:11, David Howells wrote:
do_tcp_sendpages() is now just a small wrapper around tcp_sendmsg_locked(), so inline it, allowing do_tcp_sendpages() to be removed. This is part of replacing ->sendpage() with a call to sendmsg() with MSG_SPLICE_PAGES set. Signed-off-by: David Howells <dhowells@redhat.com> cc: Boris Pismenny <borisp@nvidia.com> cc: John Fastabend <john.fastabend@gmail.com> cc: Jakub Kicinski <kuba@kernel.org> cc: "David S. Miller" <davem@davemloft.net> cc: Eric Dumazet <edumazet@google.com> cc: Paolo Abeni <pabeni@redhat.com> cc: Jens Axboe <axboe@kernel.dk> cc: Matthew Wilcox <willy@infradead.org> cc: netdev@vger.kernel.org ---
Hi, My team spotted a new degradation in TLS TX device offload, bisected to this patch. From a quick look at the patch, it's not clear to me what's going wrong. Please let us know of any helpful information that we can provide to help in the debug. Regards, Tariq Reproduce Flow: client / server test using nginx and wrk (nothing special/custom about the apps used). client: /opt/mellanox/iproute2/sbin/ip link set dev eth3 up /opt/mellanox/iproute2/sbin/ip addr add 11.141.46.9/16 dev eth3 server: /opt/mellanox/iproute2/sbin/ip link set dev eth3 up /opt/mellanox/iproute2/sbin/ip addr add 11.141.46.10/16 dev eth3 client: /auto/sw/regression/sw_net_ver_tools/ktls/tools/x86_64/nginx_openssl_3_0_0 -p /usr/bin/drivertest_rpms/ktls/nginx/ /opt/mellanox/iproute2/sbin/ss -i src [11.141.46.9] server: /auto/sw/regression/sw_net_ver_tools/ktls/tools/x86_64/wrk_openssl_3_0_0 -b11.141.46.10 -t4 -c874 -d14 --timeout 5s https://11.141.46.9:20443/256000b.img client: dmesg /auto/sw/regression/sw_net_ver_tools/ktls/tools/x86_64/nginx_openssl_3_0_0 -p /usr/bin/drivertest_rpms/ktls/nginx/ -s stop [root@c-141-46-1-009 ~]# dmesg ------------[ cut here ]------------ WARNING: CPU: 1 PID: 977 at net/core/skbuff.c:6957 skb_splice_from_iter+0x102/0x300 Modules linked in: rpcrdma rdma_ucm ib_iser libiscsi scsi_transport_iscsi ib_umad rdma_cm ib_ipoib iw_cm ib_cm mlx5_ib ib_uverbs ib_core xt_conntrack xt_MASQUERADE nf_conntrack_netlink nfnetlink xt_addrtype iptable_nat nf_nat br_netfilter rpcsec_gss_krb5 auth_rpcgss oid_registry overlay mlx5_core zram zsmalloc fuse CPU: 1 PID: 977 Comm: nginx_openssl_3 Not tainted 6.4.0-rc3_for_upstream_min_debug_2023_06_01_23_04 #1 Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.13.0-0-gf21b5a4aeb02-prebuilt.qemu.org 04/01/2014 RIP: 0010:skb_splice_from_iter+0x102/0x300 Code: ef 48 8b 55 08 f6 c2 01 0f 85 54 01 00 00 8b 0d 98 cf 5f 01 48 89 ea 85 c9 0f 8f 4c 01 00 00 48 8b 12 80 e6 02 74 48 49 89 dd <0f> 0b 48 c7 c1 fb ff ff ff 45 01 65 70 45 01 65 74 45 01 a5 d0 00 RSP: 0018:ffff8881045abaa0 EFLAGS: 00010246 RAX: 0000000000000000 RBX: ffff88814370fe00 RCX: 0000000000000000 RDX: 0000000000000000 RSI: ffffea00051123c0 RDI: ffff88814370fe00 RBP: ffffea0005112400 R08: 0000000000000011 R09: 0000000000003ffd R10: 0000000000003ffd R11: 0000000000000008 R12: 0000000000002e6e R13: ffff88814370fe00 R14: ffff8881045abae8 R15: 000000000000118f FS: 00007f6e23043740(0000) GS:ffff88852c880000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 00000000009c6c00 CR3: 000000013b791001 CR4: 0000000000370ea0 DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 Call Trace: ? kmalloc_reserve+0x86/0xe0 tcp_sendmsg_locked+0x33e/0xd40 tls_push_sg+0xdd/0x230 tls_push_data+0x673/0x920 tls_device_sendmsg+0x6e/0xc0 sock_sendmsg+0x38/0x60 sock_write_iter+0x97/0x100 vfs_write+0x2df/0x380 ksys_write+0xa7/0xe0 do_syscall_64+0x3d/0x90 entry_SYSCALL_64_after_hwframe+0x46/0xb0 RIP: 0033:0x7f6e22f018b7 Code: 0f 00 f7 d8 64 89 02 48 c7 c0 ff ff ff ff eb b7 0f 1f 00 f3 0f 1e fa 64 8b 04 25 18 00 00 00 85 c0 75 10 b8 01 00 00 00 0f 05 <48> 3d 00 f0 ff ff 77 51 c3 48 83 ec 28 48 89 54 24 18 48 89 74 24 RSP: 002b:00007ffdb528a2f8 EFLAGS: 00000246 ORIG_RAX: 0000000000000001 RAX: ffffffffffffffda RBX: 0000000000004000 RCX: 00007f6e22f018b7 RDX: 0000000000004000 RSI: 00000000025cdef0 RDI: 0000000000000028 RBP: 00000000020103c0 R08: 00007ffdb5289a90 R09: 0000000000000001 R10: 0000000000000000 R11: 0000000000000246 R12: 00000000025cdef0 R13: 000000000204fca0 R14: 0000000000004000 R15: 0000000000004000 ---[ end trace 0000000000000000 ]---
quoted hunk ↗ jump to hunk
include/net/tls.h | 2 +- net/tls/tls_main.c | 24 +++++++++++++++--------- 2 files changed, 16 insertions(+), 10 deletions(-)diff --git a/include/net/tls.h b/include/net/tls.h index 6056ce5a2aa5..5791ca7a189c 100644 --- a/include/net/tls.h +++ b/include/net/tls.h@@ -258,7 +258,7 @@ struct tls_context { struct scatterlist *partially_sent_record; u16 partially_sent_offset; - bool in_tcp_sendpages; + bool splicing_pages; bool pending_open_record_frags; struct mutex tx_lock; /* protects partially_sent_* fields anddiff --git a/net/tls/tls_main.c b/net/tls/tls_main.c index f2e7302a4d96..3d45fdb5c4e9 100644 --- a/net/tls/tls_main.c +++ b/net/tls/tls_main.c@@ -125,7 +125,10 @@ int tls_push_sg(struct sock *sk, u16 first_offset, int flags) { - int sendpage_flags = flags | MSG_SENDPAGE_NOTLAST; + struct bio_vec bvec; + struct msghdr msg = { + .msg_flags = MSG_SENDPAGE_NOTLAST | MSG_SPLICE_PAGES | flags, + }; int ret = 0; struct page *p; size_t size;@@ -134,16 +137,19 @@ int tls_push_sg(struct sock *sk, size = sg->length - offset; offset += sg->offset; - ctx->in_tcp_sendpages = true; + ctx->splicing_pages = true; while (1) { if (sg_is_last(sg)) - sendpage_flags = flags; + msg.msg_flags = flags; /* is sending application-limited? */ tcp_rate_check_app_limited(sk); p = sg_page(sg); retry: - ret = do_tcp_sendpages(sk, p, offset, size, sendpage_flags); + bvec_set_page(&bvec, p, size, offset); + iov_iter_bvec(&msg.msg_iter, ITER_SOURCE, &bvec, 1, size); + + ret = tcp_sendmsg_locked(sk, &msg, size); if (ret != size) { if (ret > 0) {@@ -155,7 +161,7 @@ int tls_push_sg(struct sock *sk, offset -= sg->offset; ctx->partially_sent_offset = offset; ctx->partially_sent_record = (void *)sg; - ctx->in_tcp_sendpages = false; + ctx->splicing_pages = false; return ret; }@@ -169,7 +175,7 @@ int tls_push_sg(struct sock *sk, size = sg->length; } - ctx->in_tcp_sendpages = false; + ctx->splicing_pages = false; return 0; }@@ -247,11 +253,11 @@ static void tls_write_space(struct sock *sk) { struct tls_context *ctx = tls_get_ctx(sk); - /* If in_tcp_sendpages call lower protocol write space handler + /* If splicing_pages call lower protocol write space handler * to ensure we wake up any waiting operations there. For example - * if do_tcp_sendpages where to call sk_wait_event. + * if splicing pages where to call sk_wait_event. */ - if (ctx->in_tcp_sendpages) { + if (ctx->splicing_pages) { ctx->sk_write_space(sk); return; }