This is a followup of the review of the teql arp-queue fix as reported
by Sashiko.
The issue:
The neighbour variant of fwd rewrites skb->dev to the configured output
device and calls neigh_xmit(), but leaves the skb's dst untouched. If that
device's neighbour for the configured nexthop is incomplete, the packet is
parked on the neighbour's arp_queue while the stale dst still describes the
device the packet was routed through. Draining the queue re-looks the
neighbour up from that dst, so the frame is built with the routed device's
link-layer header and then transmitted on the configured device. The packet
leaves with another device's addresses.
The Fix:
Release the dst before the device switch. Both inputs the neighbour
transmission needs, the device and the nexthop, are supplied by the rule,
so the queue drain keeps the parking neighbour and the frame carries the
configured device's addresses.
The release goes through skb_dst_drop_keep_metadata(), so encapsulation
parameters a route carries are handed over to a metadata dst instead of
being lost, and a metadata dst already on the packet is left in place. If
that copy cannot be allocated the packet is dropped, never sent with the
stale route.
What the configured device's transmit path still sees:
The released dst is the route of the device the packet arrived on, and the
packet leaves through another one, so consumers of that route on the way
out lose it:
- a tunnel with no configured remote takes its outer endpoint from the
route: ip_tunnel_xmit() reads the route's next hop, and ip6_tnl_xmit()
and sit derive it from the route as well. A steered packet whose route
carried no encapsulation parameters is left with no dst, so it is
dropped and counted as a transmit error instead of being encapsulated
towards an endpoint that belongs to another device's path.
- the route realm goes with it: a realm is a property of the route the
packet took, so nft meta rtclassid and nft rt classid stop matching on
that packet, and bpf_get_route_realm() returns 0.
- encapsulation parameters are not lost: they moved onto the metadata
dst, so a tunnel or an xfrm interface downstream still finds them.
Taking the outer endpoint or the realm from another device's route is the
same cross-device reuse this patch removes at the neighbour lookup: those
values were never the configured device's to use.
One consumer is older than this series. When a tunnel cannot take an
endpoint from the skb, ip_tunnel_xmit() and ip6_tnl_xmit() fall back to
skb_rtable(), a container_of() over whatever dst the skb carries, and a
metadata dst is not an rtable. That fallback is reachable without this
series: a tc-bpf program that calls bpf_skb_set_tunnel_key() with no
remote already hands an NBMA tunnel such a dst. The release here can leave
one behind as well, when the parameters a route carries are not usable for
the payload. The cast is repaired separately.
Conditions to recreate the bug:
Two network namespaces, A (10.10.0.1/24) and B (10.1.0.1/24) in one, their
veth peers in the other with only A's peer addressed (10.10.0.2/24) so B's
nexthop cannot be resolved by ARP. Resolve A's neighbour for 10.10.0.2 as
permanent, then load this ruleset on A:
table netdev t {
chain c {
type filter hook egress device va priority 0
policy accept
fwd ip to 10.1.0.2 device vb
}
}
Send one packet to 10.10.0.2. It is parked on B's incomplete neighbour
for 10.1.0.2; once that neighbour becomes valid the frame leaves B
carrying A's source address and A's neighbour as the destination.
Reported-by: Sashiko (nipa) <sashiko-bot@kernel.org>
Closes: https://netdev-ai.bots.linux.dev/sashiko/#/patchset/QDISC-6HS9.v3.r0.20261103062649-2@mojatatu.com
Link: https://lore.kernel.org/netdev/QDISC-6HS9.v3.r0.20261103062649-2@mojatatu.com/ (local)
Link: https://lore.kernel.org/netdev/avTfQjFQYR2NNdRg@chamomile/ (local)
Reviewed-by: Pablo Neira Ayuso <pablo@netfilter.org>
Reviewed-by: Victor Nogueira <redacted>
Signed-off-by: Jamal Hadi Salim <jhs@mojatatu.com>
---
v3 -> v4 (commit-log change only; the diff is unchanged from v3):
- Add the section on what the configured device's transmit path still
sees once the route is released: the NBMA tunnel drop with its
transmit-error count, the realm readers that stop matching, and the
encapsulation parameters that are preserved.
- Note that the skb_rtable() cast in ip_tunnel_xmit() and ip6_tnl_xmit()
is older than this series and is repaired separately. Considered
comments from Sashiko (nipa).
net/netfilter/nft_fwd_netdev.c | 6 ++++++
1 file changed, 6 insertions(+)
diff --git a/net/netfilter/nft_fwd_netdev.c b/net/netfilter/nft_fwd_netdev.c
index 7447da931a95..f486c7be385c 100644
--- a/net/netfilter/nft_fwd_netdev.c
+++ b/net/netfilter/nft_fwd_netdev.c
@@ -14,6 +14,7 @@
#include <net/netfilter/nf_tables.h>
#include <net/netfilter/nf_tables_offload.h>
#include <net/netfilter/nf_dup_netdev.h>
+#include <net/dst_metadata.h>
#include <net/netfilter/nf_recursion.h>
#include <net/neighbour.h>
#include <net/ip.h>
@@ -160,6 +161,11 @@ static void nft_fwd_neigh_eval(const struct nft_expr *expr,
goto out;
}
+ if (!skb_dst_drop_keep_metadata(skb)) {
+ verdict = NF_DROP;
+ goto out;
+ }
+
local_bh_disable();
if (nf_dev_xmit_recursion()) {
local_bh_enable();--
2.43.0