sock_def_readable() and sock_def_write_space() pass WF_SYNC to their
waitqueue wakeups, switch sock_def_error_report() to
wake_up_interruptible_sync_poll() so EPOLLERR waiters get the same hint.
ep_poll_callback() forwards it through to
try_to_wake_up() / select_task_rq_fair(), where wake_affine() can
prefer the waker CPU and skip a cross-CPU wakelist IPI when its
heuristics agree.
WF_SYNC matches the choice already made for readable/write_space. Some
error reports have the same producer/consumer shape: the waker has queued
an skb on sk->sk_error_queue, and the wakee is about to dequeue and copy
it out. Other reports publish sk_err/socket state directly, but still wake
a task likely to consume that socket state immediately.
Measured on a 176-core EPYC 9D64 host running a Meta production
workload, bpftrace on tracepoint:ipi:ipi_send_cpu with a kstack filter
attributed the sock_def_error_report -> ep_poll_callback ->
try_to_wake_up -> ttwu_queue_wakelist -> __smp_call_single_queue
chain to 16,326 IPIs/min. Switching to wake_up_interruptible_sync_poll()
will help reduce those IPIs.
Signed-off-by: Usama Arif <redacted>
---
net/core/sock.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/net/core/sock.c b/net/core/sock.c
index 8a59bfaa8096..c724f1442987 100644
--- a/net/core/sock.c
+++ b/net/core/sock.c
@@ -3639,7 +3639,7 @@ static void sock_def_error_report(struct sock *sk)
rcu_read_lock();
wq = rcu_dereference(sk->sk_wq);
if (skwq_has_sleeper(wq))
- wake_up_interruptible_poll(&wq->wait, EPOLLERR);
+ wake_up_interruptible_sync_poll(&wq->wait, EPOLLERR);
sk_wake_async_rcu(sk, SOCK_WAKE_IO, POLL_ERR);
rcu_read_unlock();
}
--
2.53.0-Meta