Re: [RFC PATCH net-next 00/13] net: knod: in-kernel network offload device

On Sun, Jul 19, 2026 at 11:01 AM Taehee Yoo [off-list ref] wrote:

knod (in-kernel network offload device) drives a GPU directly from the
Linux kernel - with no userspace GPU runtime such as CUDA or ROCm, and
no userspace component in the data path - to accelerate packet
processing.

The kernel itself allocates the GPU's queues, JIT-compiles the
per-packet program to GPU machine code, and dispatches it; the NIC DMAs
received packets straight into GPU memory and the GPU returns a verdict.
The GPU is programmed like any other in-kernel offload, not through a
userspace framework, so existing XDP programs and IPsec SAs can be
offloaded to it transparently.

The design and motivation were presented at Linux Plumbers Conference
2025:

  https://lpc.events/event/19/contributions/2267/

Motivation
==========

Line-rate packet processing that does non-trivial per-packet work - an
XDP program doing L4 load balancing, or IPsec crypto - is bound by the
host CPU: each core handles one packet at a time, so scaling means
spending more cores. A GPU is the opposite shape - thousands of lanes
running the same small program over many packets at once (SIMT) - which
happens to match the per-packet-program model of XDP.

knod moves that per-packet compute off the host CPU and onto a GPU. The
NIC DMAs received packets directly into GPU memory, the GPU runs the
program across a batch of packets in parallel, and only the result
comes back: a verdict for every packet, plus the packet itself for the
ones destined to the host. The CPU no longer pays the per-packet
program cost, and throughput scales with GPU occupancy rather than core
count.

Crucially this happens entirely inside the kernel. GPU packet processing
today generally launches work from a userspace GPU runtime (CUDA and
friends) and keeps that runtime in the data path; knod instead builds
the GPU queues, compiles the program, and dispatches it from the kernel,
so it plugs into existing offload paths (XDP, xfrm) with nothing to
install or keep running in userspace.

TBH I found this motivation weak, especially since IIUC you're asking
for almost ~50K lines of code to be merged to the kernel. My thinking
is that (a) it's true that Native XDP takes up CPU cores, but
HW-offload XDP already exists and takes up no CPU. (b) with AI, GPUs
are very expensive and the work they're doing is critical, so you're
unlikely to buy a GPU and use it for knod-offloaded XDP; you'd
probably buy a cheaper smart-NIC? And in the cases where you do have a
GPU, it's likely your system's money-maker, and you probably want to
offload work from the GPU, not to your GPU.

But as far as I can tell there should be much more interesting
applications for what you're doing rather than offloading XDP. Like
wouldn't you with this feature be able to implement ML collectively
like all-to-all/all-reduce/all-gather as purely XDP programs offloaded
to the GPU? If you have that implemented and can positively compare
performance to RoCE or RDMA that would be a much more interesting use
case IMO.

Model
=====

knod binds two endpoints through a third object:

  - a NIC-side netdev, registered by the NIC driver;
  - an accelerator (the GPU), registered by a provider built into the
    GPU driver;
  - an offload device, created on attach, that connects them and owns
    the per-queue data-path state.

The accelerator ops are feature-agnostic, so different per-packet
programs plug in behind one interface. This series ships two features
to show the framework is not tied to a single use case (the accel-type
uAPI also reserves "dpu" for future non-GPU backends):

  - BPF:   an XDP program attached in offload mode is JIT-compiled from
           eBPF to an AMD GCN shader and executed on the GPU.
  - IPsec: RX ESP full-packet decrypt on the GPU (proof of concept,
           see below).

Data path
=========

For a NIC RX queue bound to an accelerator, a received packet flows as
follows:

  1. Attach reconfigures the NIC's page_pool so its pages come from GPU
     memory, exported as a dma-buf and plugged in through the devmem
     memory provider. The NIC therefore DMAs the packet straight into
     GPU memory - there is no host-side copy on RX.

  2. Instead of building an skb and entering the stack, the NIC's NAPI
     pushes a small descriptor (memory ref + offset + length) onto a
     per-queue lock-free SPSC ring shared with the GPU.

  3. A persistent GPU worker drains descriptors and runs the active
     feature's shader over a batch of packets in parallel - one
     workgroup per packet - then writes a per-packet verdict back onto
     the ring: PASS, DROP or TX.

  4. Back on the NIC NAPI, each verdict is applied:
       - DROP: the GPU page is recycled to the pool;
       - TX:   the packet is sent back out the NIC directly from GPU
               memory (XDP_TX), with no host round-trip;
       - PASS: the packet is copied out of GPU memory into a host
               delivery page by the GPU's DMA engine (async SDMA),
               wrapped as an skb, and handed to the normal stack.

So only PASS packets ever touch host memory, and even then the copy is
done by the GPU, not the CPU.

The control plane is a generic netlink family ("knod") for
attach/detach, accelerator/NIC inventory, and per-accelerator feature
selection, with notifications on bind/unbind.

Code layout
===========

knod separates into a subsystem-neutral core and a GPU-specific
provider; nothing in the core knows about GPUs. Roughly:

  - core + control plane + NIC drivers   net/, kernel/bpf/       (~4.4k)
  - GPU provider (queues, JIT, shaders)  drivers/gpu/drm/amd/    (~39k)

Since the changes are so large, I would break this work into many
series, the first which only supports a very minimal set of work
(which I would guess would be XDP_DROP and XDP_TX), and then follow up
with another series which adds XDP_PASS support, and then IPSec, etc.

-- 
Thanks,
Mina