Hi Alice,

On 07/07/2026 10:40, Alice Mikityanska wrote:

On Tue, Jul 7, 2026, at 11:06, Paolo Abeni wrote:

quoted

On 7/6/26 8:19 PM, Alice Mikityanska wrote:

(...)

quoted

quoted

+
+cleanup_tunnel() {
+	ip -netns "$CLIENT_NS" link del tun0
+	ip -netns "$SERVER_NS" link del tun1
+}
+
+cleanup() {
+	ip netns pids "$SERVER_NS" | xargs -r kill
+	ip netns pids "$CLIENT_NS" | xargs -r kill
+	ip netns del "$SERVER_NS"
+	ip netns del "$CLIENT_NS"
+	rm -rf "$WORKDIR"
+}
+
+do_test() {
+	# When tx csum offload is off, software GSO is performed before passing the
+	# packet to veth. Check BIG TCP packets inside the VXLAN tunnel to verify
+	# the software checksum path: if the checksum code is broken, these packets
+	# will be dropped.
+	if [ "$2" = on ]; then
+		CAPTURE_IFACE='link'
+	else
+		CAPTURE_IFACE='tun'
+	fi
+
+	ip netns exec "$SERVER_NS" tcpdump -nn -s 256 -i "${CAPTURE_IFACE}1" greater 65536 -w "$WORKDIR/server.pcap" 2> /dev/null &
+	TCPDUMP_SERVER_PID="$!"
+	ip netns exec "$CLIENT_NS" tcpdump -nn -s 256 -i "${CAPTURE_IFACE}0" greater 65536 -w "$WORKDIR/client.pcap" 2> /dev/null &
+	TCPDUMP_CLIENT_PID="$!"
+
+	# This filter doesn't capture all possible variants of SACK, but it's aimed
+	# at the typical one where SACK follows after [nop, nop, timestamp, nop,
+	# nop] (14 bytes after the 20-byte TCP header). IPv6 needs a separate match,
+	# because man tcpdump says:
+	# > Arithmetic expression against transport layer headers, like tcp[0], does
+	# > not work against IPv6 packets.  It only looks at IPv4 packets.
+	ip netns exec "$SERVER_NS" tcpdump -nn -s 256 -i "tun1" '(tcp[tcpflags] & (tcp-syn|tcp-ack) = tcp-ack and tcp[34:2] & 0xffc3 = 0x0502) or (ip6[6] = 0x06 and ip6[53] & 0x12 = 0x10 and ip6[74:2] & 0xffc3 = 0x0502)' -w "$WORKDIR/sack.pcap" 2> /dev/null &

The tcpdump commands might need to be used with ...

  --immediate-mode --packet-buffered

... but that's maybe not needed, see below.

quoted

quoted

+	TCPDUMP_SACK_PID="$!"
+
+	if [ "$1" = 4 ]; then
+		SERVER_IP="$SERVER_IP4_TUN"
+		echo "Running IPv4 traffic in the tunnel"
+	else
+		SERVER_IP="$SERVER_IP6_TUN"
+		echo "Running IPv6 traffic in the tunnel"
+	fi
+
+	sleep 1 # Give tcpdump a second to spin up.

This is possibly not needed, see below.

quoted

quoted

+	ip netns exec "$CLIENT_NS" netperf -t TCP_STREAM -l 5 -H "$SERVER_IP" -- \
+		-m 80000 > /dev/null
+	sleep 1 # Give tcpdump a second to process buffered packets.
+	kill "$TCPDUMP_SERVER_PID" "$TCPDUMP_CLIENT_PID" "$TCPDUMP_SACK_PID"
+	wait "$TCPDUMP_SERVER_PID" "$TCPDUMP_CLIENT_PID" "$TCPDUMP_SACK_PID"
+	PACKETS_SERVER=$(tcpdump --count -r "$WORKDIR/server.pcap" 2> /dev/null | cut -d ' ' -f 1)
+	PACKETS_CLIENT=$(tcpdump --count -r "$WORKDIR/client.pcap" 2> /dev/null | cut -d ' ' -f 1)
+	PACKETS_SACK=$(tcpdump --count -r "$WORKDIR/sack.pcap" 2> /dev/null | cut -d ' ' -f 1)
+
+	echo "Captured BIG TCP RX packets: $PACKETS_SERVER"
+	echo "Captured BIG TCP TX packets: $PACKETS_CLIENT"
+	echo "Captured TCP SACK packets: $PACKETS_SACK"
+	[ "$PACKETS_SERVER" -gt "$PACKETS_THRESHOLD" ] || return 1
+	[ "$PACKETS_CLIENT" -gt "$PACKETS_THRESHOLD" ] || return 1
+	[ "$PACKETS_SACK" -lt "$(( PACKETS_CLIENT / 2 ))" ] || return 1

KCI fails here, see:

https://netdev-ctrl.bots.linux.dev/logs/vmksft/net/results/722863/156-big-tcp-tunnels-sh/stdout

possibly the above parsing is a bit fragile/tcpdump version's dependent?!?

TL/DR: I've seen this when I ran out of space in /tmp before adding -s 256.

Hmm, the changes I made in this iteration were aimed at addressing version-
dependent behavior of tcpdump... I used to count the lines of its
output. Newer tcpdump prints two lines per encapsulated packet. Older
tcpdump can't parse BIG TCP in this test and prints one line per packet.
To make it more robust, I decided to store the pcap and count the
packets there (--count doesn't work with live capture when tcpdump is
interrupted). The pitfall is that now it takes a few hundred megabytes
in /tmp to store those pcaps.

Wow :)

If you need to keep tcpdump -- see below -- you can probably reduce even
more the packet size (-s 256), I don't know what's the minimum. And
purge the workdir in cleanup_tunnel().

Now, seeing empty stdout from tcpdump could be if the pcap file is empty
(doesn't contain the pcap header), because we ran out of space in /tmp
(first two tcpdumps took all space, the third didn't write the header -
I observed this behavior before I added -s 256). We don't see the
corresponding error messages, because tcpdump starts with 2> /dev/null
(otherwise it floods the screen with statistics). Could this be the
reason? How much space is allocated for /tmp in the CI runners?

I can probably address it by limiting the overall number of captured
packets with -c, but then I won't be able to compare $PACKETS_SACK
relative to $PACKETS_CLIENT.

If you only need to count some packets matching a filter, what about
using Netfilter rules with a counter, instead of using pcap files?

You can have counters not attached to rules altering packets:

  https://wiki.nftables.org/wiki-nftables/index.php/Counters

It might even be easier to match the SACK packets, with the 'tcp option
sack' filter from nft [1]. And for the size, I guess you can use "meta
length > 65536" [2], or checking other fields from IP/TCP headers.

If you want to keep your cBPF filter, you can also use nfbpf_compile for
the translation, and use that with 'iptables -m bpf --bytecode <...>',
but it might not be needed.

[1] https://www.mankier.com/8/nft#Payload_Expressions-Extension_Header_Expressions
[2] https://www.mankier.com/8/nft#Primary_Expressions-Meta_Expressions

Cheers,
Matt
-- 
Sponsored by the NGI0 Core fund.