[BUG] vrf: attaching a packet tap to a VRF device corrupts skb->csum for packets from L3 ingress devices (Udp6InCsumErrors, packets dropped)

Hi all,

Up front, so you know what you are dealing with: **I am not a kernel
programmer — I am a networking guy.** Please bear with me if I get the
terminology or the conventions wrong, and feel free to tell me bluntly if this
is not the right place or the right format. I also used an AI assistant to help
me navigate the kernel sources and narrow this down, so please treat my source
analysis below as a hypothesis rather than as an authoritative statement.

What I do have is a **small, deterministic, self-contained reproducer** that
triggers the problem 100% of the time on a stock kernel, plus a control test
that isolates the trigger condition. Those parts I am confident about, because I
ran them.

## Summary

Attaching a packet tap (tcpdump/dumpcap, i.e. anything that populates
`vrf_dev->ptype_all`) to a VRF device causes incoming UDP datagrams to be
dropped with `Udp6InCsumErrors`, when:

- the packet arrives on an ingress device with no `header_ops`
  (ARPHRD_NONE: tun, ip6tnl, gre, wireguard...) that is enslaved to
that VRF, and
- the skb is in `CHECKSUM_COMPLETE` state by the time it reaches
  `vrf_ip6_rcv()` (in my case because conntrack software-verified the checksum
  in PREROUTING).

Stop the capture and the packets are delivered again. Start it and they are
dropped again. Capturing on the enslaved ingress device instead of on the VRF
device is harmless.

IPv4 should be affected identically, since `vrf_ip_rcv()` uses the same helper.
I only tested IPv6.

## Real-world impact

I hit this in production: a SIP proxy bound into a VRF stopped receiving large
(fragmented) REGISTER messages over IPv6/UDP, but only while we were running a
packet capture on the VRF device to debug an unrelated issue. The capture we
took to diagnose the problem was the cause of the problem.

In the capture, the reassembled datagram as seen on the VRF device has
**source MAC == destination MAC == the VRF device MAC**, which is what put me
on the trail of `vrf_prepare_mac_header()`.

## Suspected cause

`drivers/net/vrf.c`, `vrf_prepare_mac_header()`:

__skb_push(skb, ETH_HLEN);
eth = (struct ethhdr *)skb->data;

skb_reset_mac_header(skb);
skb_reset_mac_len(skb);

ether_addr_copy(eth->h_dest, vrf_dev->dev_addr);
ether_addr_copy(eth->h_source, vrf_dev->dev_addr);
eth->h_proto = htons(proto);

skb->protocol = eth->h_proto;
skb->pkt_type = PACKET_HOST;

skb_postpush_rcsum(skb, skb->data, ETH_HLEN);

skb_pull_inline(skb, ETH_HLEN);

The synthetic 14-byte Ethernet header is folded into `skb->csum` by
`skb_postpush_rcsum()`, then removed from the packet by `skb_pull_inline()` —
but there is no matching `skb_postpull_rcsum()`. For a `CHECKSUM_COMPLETE` skb,
`skb->csum` is left permanently off by `csum_partial(fake_eth, ETH_HLEN)`, and
`udpv6_rcv()` subsequently rejects the datagram.

This code is only reached from `vrf_ip6_rcv()` / `vrf_ip_rcv()` inside

if (!list_empty(&vrf_dev->ptype_all)) {

which is exactly why the behaviour depends on whether a sniffer is attached.

`vrf_add_mac_header_if_unset()` short-circuits when the ingress device has a
real link layer (`dev_has_header(orig_dev)`), so a veth-enslaved port never
shows the problem — only L3 devices such as tun do.

Naively, the fix looks like it should be to make the accounting symmetric:
either drop the `skb_postpush_rcsum()` call, since the push/pull pair leaves the
packet unchanged, or add the matching `skb_postpull_rcsum()` before
`skb_pull_inline()`. I do not feel qualified to say which is correct, or whether
there are other callers/paths that rely on the current behaviour, so I am not
sending a patch.

## Why the packet is already reassembled when the VRF taps it

`l3mdev_ip6_rcv()` is called from `ip6_rcv_finish()`, i.e. after
`NF_INET_PRE_ROUTING`, so `nf_defrag_ipv6` (PREROUTING, prio -400) has already
reassembled the fragments. That is why the ingress device shows two fragments
while the VRF device shows one full datagram.

## Why the skb is CHECKSUM_COMPLETE

A tun device hands the stack a `CHECKSUM_NONE` skb, for which
`skb_postpush_rcsum()` is a no-op. In my setup the promotion is done by
conntrack: `nf_conntrack_udp_packet()` -> `nf_checksum()` -> `nf_ip6_checksum()`
-> `__skb_checksum_complete()`, which sets `CHECKSUM_COMPLETE` and stores
`skb->csum`. This also runs at PREROUTING, before the VRF rx handler, and only
when `net.netfilter.nf_conntrack_checksum=1` (the default).

Test 4 of the reproducer sets `net.netfilter.nf_conntrack_checksum=0` and the
problem disappears while the capture stays attached — which I believe confirms
that the corruption is in `skb->csum` and not in the packet data.

## Reproducer

Tested on `6.12.0-211.49.1.el10_2.x86_64` (Red Hat Enterprise Linux 10). It
needs root, `iproute2`, `nftables`, `tcpdump` and python3. No external network,
no second host — everything happens in one network namespace.

Topology:

[netns kbug]
  vrf0                 (table 100)
    └── tun0           IFF_TUN, ARPHRD_NONE, 2001:db8:2::1/64

`inject.py` opens `/dev/net/tun`, attaches to `tun0`, and writes two hand-built
IPv6 fragments of a single 1640-byte UDP datagram (correct UDP checksum) into
the fd. `receiver.py` is a UDP socket bound to `[2001:db8:2::1]:5060` with
`SO_BINDTODEVICE=vrf0`.

An nftables `ct state` rule in PREROUTING activates `nf_defrag_ipv6` and
conntrack:

add table inet ctstate
add chain inet ctstate prerouting { type filter hook prerouting
priority -150; policy accept; }
add rule inet ctstate prerouting ct state new,established,related,invalid accept

Results:

== Test 1 — no tap                                  PASS — delivered
(Udp6InCsumErrors +0)
== Test 2 — tcpdump on vrf0                         FAIL — dropped
(Udp6InCsumErrors +1)
== Test 3 — tcpdump on tun0 (ingress)               PASS — delivered
(Udp6InCsumErrors +0)
== Test 4 — tcpdump on vrf0, nf_conntrack_checksum=0 PASS — delivered
(Udp6InCsumErrors +0)

The full set of scripts (`setup.sh`, `inject.py`, `receiver.py`,
`reproduce.sh`, `teardown.sh`) is available here:

  https://gist.github.com/ssasso/c089227b116b0fa6a954a18d85639d67

They are short and have no dependencies beyond the above. I can inline them in a
follow-up mail if you prefer not to follow links.

## Workarounds I am using meanwhile

- Capture on the enslaved interface instead of on the VRF device.
- `sysctl -w net.netfilter.nf_conntrack_checksum=0` in the affected netns.

Happy to run any test, apply any debug patch, or collect any additional data you
need — just tell me exactly what to run.

Thanks for your time,
Stefano