Thread (17 messages) flat view 17 messages, 3 authors, 4h ago
HOTtoday

Revision v1 of 2 in this series.

Revisions (2)
  1. rfc [diff vs current]
  2. v1 current

[PATCH net-next 00/14] skb extension for BPF metadata

From: Jakub Sitnicki <jakub@cloudflare.com>
Date: 2026-08-14 08:14:23
Also in: bpf

[I realize net-next closes soon. Posting only to collect more feedback.]

Please see the RFC cover letter for the overview [1].
I will focus here just on the latest developments.

1) Based on feedback from the RFC - the BPF skb extension is *no longer
scrubbed* on tunnel encap/decap or veth traversal. There is also *no
mechanism* to enable scrubbing as we don't seem to have a use case for it.

2) Since the RFC I've enabled access to BPF skb extension for the remaining
BPF program types which operate on skbs, namely netfilter, lwt family, and
- everyone's favorite - sk_skb.

Outside the patch set, two things happened:

3) At NetConf 2026 Kuniyuki presented effectively the same idea with a use
case in mind to attach metadata to packets on egress (IIRC). Hoping for
feedback if this meets your needs as well.

4) At BPF Summit 2026, Alexei asked - I'm paraphrasing: What percentage of
skbs will carry metadata in our workload? This determines if the cost of
attaching a tracing prog to consume_skb gets amortized. We've run
experiments and have some answers.

If bpf skb ext existed today, we would adopt it to attach metadata to
incoming TCP connections and ingress UDP packets to CDN to identify flows
that have been forwarded from one PoP to another.

Based on stats from a production node where we've been testing this patch
set, the fraction of skbs that would carry the metadata is <1% (~0.7%):
923m16$ sudo perf stat -a -r 10 \
            -e skb:consume_skb -e skb:kfree_skb \
            -e probe:skb_ext_add -- sleep 1

 Performance counter stats for 'system wide' (10 runs):

           334,256      skb:consume_skb                            ( +-  1.63% )
             1,525      skb:kfree_skb                              ( +-  4.09% )
             2,218      probe:skb_ext_add                          ( +- 18.58% )

           1.02305 +- 0.00243 seconds time elapsed  ( +-  0.24% )
I expect that even if we adopted bpf skb ext for other use cases we have in
mind, like labeling egress CDN packets with a customer identifier, we would
attach metadata only to around 5% of all skbs in flight.

We've run an experiment to evaluate this patch set - bpf skb extension -
against the alternative - using a BPF map keyed by &skb (LRU_HASH) to stash
metadata plus consume/kfree_skb tracepoint programs to clean up the
entries.

To do that we've modified the CDN component that labels the incoming
connections forwarded from another PoP:

- with bpf skb ext, BPF progs involved in processing consumed 7.5..10 CPU%,
- with BPF map + consume/kfree_skb tp, the overhead was between 15..20 CPU%

Not surprising considering the low fraction of skbs we attach metadata to.
(For graphs see slides 30 & 40 from our presentation at Netdev 0x1A [2].)

Note that we have not tried using RHASH as a BPF metadata stash, to see if
that could bridge the overhead gap, as that would involve deploying a
non-LTS kernel ATM.

Thanks,
-jkbs

[1] https://patch.msgid.link/20260714-bpf-meta-inside-skb-ext-v1-0-5871c07a8dd6@cloudflare.com
[2] https://github.com/jsitnicki/talks/blob/5d64c151f4dc0c38b2832fdcbce7284ec93fdca2/Netdev%200x1A%20-%20Thrice%20the%20charm%20-%20an%20skb%20extension%20for%20BPF%20metadata.pdf

Signed-off-by: Jakub Sitnicki <jakub@cloudflare.com>
---
Changes in v1:
- Don't scrub BPF skb extension. Remove F_NO_SCRUB flag. (Stan)
- Allow calling bpf_dynptr_from_skb_ext from NETFILTER, LWT_*, SK_SKB progs.
- Reorg tests into smaller commits. Add missing coverage.
- Link to RFC: https://patch.msgid.link/20260714-bpf-meta-inside-skb-ext-v1-0-5871c07a8dd6@cloudflare.com

---
Jakub Sitnicki (14):
      bpf: Introduce per-packet metadata storage for BPF programs
      bpf: Allow access to bpf_sock_ops_kern->skb
      bpf: Make BPF skb extension survive packet scrubbing
      selftests/bpf: Add tests for bpf_dynptr_from_skb_ext
      selftests/bpf: Test skb_ext on cloned skbs
      selftests/bpf: Test skb_ext survival across veth and GRE
      selftests/bpf: Test skb_ext read from cgroup_skb and sk_filter hooks
      selftests/bpf: Test skb_ext read from sock_ops and LSM hooks
      selftests/bpf: Test skb_ext read from kfree_skb tracepoint
      selftests/bpf: Test skb_ext read from netfilter hook
      selftests/bpf: Test skb_ext from LWT in, out, and xmit hooks
      selftests/bpf: Test skb_ext read from seg6local End.BPF hook
      selftests/bpf: Test skb_ext read from sk_skb stream verdict hook
      selftests/bpf: Use non-trivial test payload in xdp_context tests

 include/linux/bpf.h                                |  10 +
 include/linux/filter.h                             |  26 +
 include/linux/skbuff.h                             |   5 +
 include/uapi/linux/bpf.h                           |   5 +
 kernel/bpf/helpers.c                               |   7 +
 kernel/bpf/log.c                                   |   2 +
 kernel/bpf/verifier.c                              |  15 +-
 net/Kconfig                                        |  20 +
 net/core/filter.c                                  | 135 +++
 net/core/skbuff.c                                  |  81 +-
 net/ipv4/udp.c                                     |   9 +-
 tools/testing/selftests/bpf/config                 |   1 +
 .../selftests/bpf/prog_tests/socket_helpers.h      |   1 +
 .../bpf/prog_tests/xdp_context_test_run.c          | 988 ++++++++++++++++++++-
 tools/testing/selftests/bpf/progs/test_xdp_meta.c  | 489 +++++++++-
 15 files changed, 1772 insertions(+), 22 deletions(-)
Keyboard shortcuts
hback out one level
jnext message in thread
kprevious message in thread
ldrill in
Escclose help / fold thread tree
?toggle this help