On Tue, Feb 28, 2023 at 5:44 PM Eric Dumazet [off-list ref] wrote:

On Fri, Feb 24, 2023 at 7:49 PM Florian Westphal [off-list ref] wrote:

quoted

There is a noticeable tcp performance regression (loopback or cross-netns),
seen with iperf3 -Z (sendfile mode) when generic retpolines are needed.

With SK_RECLAIM_THRESHOLD checks gone number of calls to enter/leave
memory pressure happen much more often. For TCP indirect calls are
used.

We can't remove the if-set-return short-circuit check in
tcp_enter_memory_pressure because there are callers other than
sk_enter_memory_pressure.  Doing a check in the sk wrapper too
reduces the indirect calls enough to recover some performance.

Before,
0.00-60.00  sec   322 GBytes  46.1 Gbits/sec                  receiver

After:
0.00-60.04  sec   359 GBytes  51.4 Gbits/sec                  receiver

BTW I was curious why Google was not seeing this, and it appears Brian Vasquez
forgot to upstream this change...

commit 5ea2f21d6c1078d2c563cb455ad5877b4ada94e1
Author: Brian Vazquez [off-list ref]
Date:   Thu Mar 3 19:09:49 2022 -0800

    PRODKERNEL: net-directcall: annotate tcp_leave_memory_pressure and
    tcp_getsockopt

    Switch to upstream infra to make rebase easier

    Tested: built and booted on lab machine

    Upstream-Plan: 150254871
    Effort: net-directcall

diff --git a/net/core/sock.c b/net/core/sock.c
index 05032b399c873984e5297898d647905ca9f21f2e..54cb989dc162f3982380ac12cf5a150214e209a2

100644

--- a/net/core/sock.c
+++ b/net/core/sock.c

@@ -2647,10 +2647,13 @@ static void sk_enter_memory_pressure(struct sock *sk)
        sk->sk_prot->enter_memory_pressure(sk);
 }

+INDIRECT_CALLABLE_DECLARE(void tcp_leave_memory_pressure(struct sock *sk));
+
 static void sk_leave_memory_pressure(struct sock *sk)
 {
        if (sk->sk_prot->leave_memory_pressure) {
-               sk->sk_prot->leave_memory_pressure(sk);
+               INDIRECT_CALL_1(sk->sk_prot->leave_memory_pressure,
+                               tcp_leave_memory_pressure, sk);
        } else {
                unsigned long *memory_pressure = sk->sk_prot->memory_pressure;

@@ -3439,6 +3442,10 @@ int sock_recv_errqueue(struct sock *sk, struct

msghdr *msg, int len,
 }
 EXPORT_SYMBOL(sock_recv_errqueue);

+INDIRECT_CALLABLE_DECLARE(int tcp_getsockopt(struct sock *sk, int level,
+                                            int optname, char __user *optval,
+                                            int __user *optlen));
+
 /*
  *     Get a socket option on an socket.
  *

@@ -3451,7 +3458,8 @@ int sock_common_getsockopt(struct socket *sock,

int level, int optname,
 {
        struct sock *sk = sock->sk;

-       return sk->sk_prot->getsockopt(sk, level, optname, optval, optlen);
+       return INDIRECT_CALL_1(sk->sk_prot->getsockopt, tcp_getsockopt,
+                               sk, level, optname, optval, optlen);
 }
 EXPORT_SYMBOL(sock_common_getsockopt);