Re: [PATCH net-next 0/2] tcp/dccp: refine source port selection
From: Jakub Sitnicki <jakub@cloudflare.com>
Date: 2024-01-03 14:19:16
On Thu, Dec 14, 2023 at 07:29 PM GMT, Eric Dumazet wrote:
This patch series leverages IP_LOCAL_PORT_RANGE option to no longer favor even source port selection at connect() time. This should lower time taken by connect() for hosts having many active connections to the same destination. Eric Dumazet (2): inet: returns a bool from inet_sk_get_local_port_range() tcp/dccp: change source port selection at connect() time include/net/ip.h | 2 +- net/ipv4/inet_connection_sock.c | 21 ++++++++++++++++----- net/ipv4/inet_hashtables.c | 27 ++++++++++++++++----------- 3 files changed, 33 insertions(+), 17 deletions(-)
This is great. Thank you.
# sysctl net.ipv4.ip_local_port_range
net.ipv4.ip_local_port_range = 32768 60999
# { sleep 3; stress-ng --sockmany 1 --sockmany-ops 20000; } & \/usr/share/bcc/tools/funclatency inet_hash_connect
[1] 240
Tracing 1 functions for "inet_hash_connect"... Hit Ctrl-C to end.
stress-ng: info: [243] defaulting to a 1 day, 0 secs run per stressor
stress-ng: info: [243] dispatching hogs: 1 sockmany
stress-ng: info: [243] skipped: 0
stress-ng: info: [243] passed: 1: sockmany (1)
stress-ng: info: [243] failed: 0
stress-ng: info: [243] metrics untrustworthy: 0
stress-ng: info: [243] successful run completed in 27.60 secs
^C
nsecs : count distribution
0 -> 1 : 0 | |
2 -> 3 : 0 | |
4 -> 7 : 0 | |
8 -> 15 : 0 | |
16 -> 31 : 0 | |
32 -> 63 : 0 | |
64 -> 127 : 0 | |
128 -> 255 : 0 | |
256 -> 511 : 0 | |
512 -> 1023 : 511 |** |
1024 -> 2047 : 8698 |****************************************|
2048 -> 4095 : 2870 |************* |
4096 -> 8191 : 1471 |****** |
8192 -> 16383 : 389 |* |
16384 -> 32767 : 114 | |
32768 -> 65535 : 43 | |
65536 -> 131071 : 15 | |
131072 -> 262143 : 0 | |
262144 -> 524287 : 1 | |
524288 -> 1048575 : 1 | |
1048576 -> 2097151 : 3 | |
2097152 -> 4194303 : 1609 |******* |
4194304 -> 8388607 : 4272 |******************* |
8388608 -> 16777215 : 4 | |
avg = 1314821 nsecs, total: 26297744706 nsecs, count: 20001
Detaching...
[1]+ Done { sleep 3; stress-ng --sockmany 1 --sockmany-ops 20000; }
# { sleep 3; LD_PRELOAD=./setsockopt_ip_local_port_range.so stress-ng --sockmany 1 --sockmany-ops 20000; } & \/usr/share/bcc/tools/funclatency inet_hash_connect
[1] 246
Tracing 1 functions for "inet_hash_connect"... Hit Ctrl-C to end.
stress-ng: info: [249] defaulting to a 1 day, 0 secs run per stressor
stress-ng: info: [249] dispatching hogs: 1 sockmany
stress-ng: info: [249] skipped: 0
stress-ng: info: [249] passed: 1: sockmany (1)
stress-ng: info: [249] failed: 0
stress-ng: info: [249] metrics untrustworthy: 0
stress-ng: info: [249] successful run completed in 1.01 secs
^C
nsecs : count distribution
0 -> 1 : 0 | |
2 -> 3 : 0 | |
4 -> 7 : 0 | |
8 -> 15 : 0 | |
16 -> 31 : 0 | |
32 -> 63 : 0 | |
64 -> 127 : 0 | |
128 -> 255 : 0 | |
256 -> 511 : 0 | |
512 -> 1023 : 2085 |****** |
1024 -> 2047 : 13401 |****************************************|
2048 -> 4095 : 3877 |*********** |
4096 -> 8191 : 561 |* |
8192 -> 16383 : 60 | |
16384 -> 32767 : 16 | |
32768 -> 65535 : 2 | |
avg = 1768 nsecs, total: 35376609 nsecs, count: 20002
Detaching...
[1]+ Done { sleep 3; LD_PRELOAD=./setsockopt_ip_local_port_range.so stress-ng --sockmany 1 --sockmany-ops 20000; }
# cat ./setsockopt_ip_local_port_range.c
#include <dlfcn.h>
#include <linux/in.h>
#include <sys/socket.h>
int socket(int domain, int type, int protocol)
{
int (*socket_fn)(int, int, int) = dlsym(RTLD_NEXT, "socket");
int fd;
fd = socket_fn(domain, type, protocol);
if (fd < 0)
return -1;
if (domain == AF_INET || domain == AF_INET6) {
setsockopt(fd, IPPROTO_IP, IP_LOCAL_PORT_RANGE,
&(__u32){ 0xffffU << 16 }, sizeof(__u32));
}
return fd;
}
#