On Tue, Nov 16, 2021 at 07:37:34AM +0000, Ciara Loftus wrote:

The common case for AF_XDP sockets (xsks) is creating a single xsk on a queue for sending and 
receiving frames as this is analogous to HW packet steering through RSS and other classification 
methods in the NIC. AF_XDP uses the xdp redirect infrastructure to direct packets to the socket. It 
was designed for the much more complicated case of DEVMAP xdp_redirects which directs traffic to 
another netdev and thus potentially another driver. In the xsk redirect case, by skipping the 
unnecessary parts of this common code we can significantly improve performance and pave the way 
for batching in the driver. This RFC proposes one such way to simplify the infrastructure which 
yields a 27% increase in throughput and a decrease in cycles per packet of 24 cycles [1]. The goal 
of this RFC is to start a discussion on how best to simplify the single-socket datapath while 
providing one method as an example.

Current approach:
1. XSK pointer: an xsk is created and a handle to the xsk is stored in the XSKMAP.
2. XDP program: bpf_redirect_map helper triggers the XSKMAP lookup which stores the result (handle 
to the xsk) and the map type (XSKMAP) in the percpu bpf_redirect_info struct. The XDP_REDIRECT 
action is returned.
3. XDP_REDIRECT handling called by the driver: the map type (XSKMAP) is read from the 
bpf_redirect_info which selects the xsk_map_redirect path. The xsk pointer is retrieved from the
bpf_redirect_info and the XDP descriptor is pushed to the xsk's Rx ring. The socket is added to a
list for flushing later.
4. xdp_do_flush: iterate through the lists of all maps that can be used for redirect (CPUMAP, 
DEVMAP and XSKMAP). When XSKMAP is flushed, go through all xsks that had any traffic redirected to 
them and bump the Rx ring head pointer(s).

For the end goal of submitting the descriptor to the Rx ring and bumping the head pointer of that 
ring, only some of these steps are needed. The rest is overhead. The bpf_redirect_map 
infrastructure is needed for all other redirect operations, but is not necessary when redirecting 
to a single AF_XDP socket. And similarly, flushing the list for every map type in step 4 is not 
necessary when only one socket needs to be flushed.

Proposed approach:
1. XSK pointer: an xsk is created and a handle to the xsk is stored both in the XSKMAP and also the 
netdev_rx_queue struct.
2. XDP program: new bpf_redirect_xsk helper returns XDP_REDIRECT_XSK.
3. XDP_REDIRECT_XSK handling called by the driver: the xsk pointer is retrieved from the 
netdev_rx_queue struct and the XDP descriptor is pushed to the xsk's Rx ring.
4. xsk_flush: fetch the handle from the netdev_rx_queue and flush the xsk.

This fast path is triggered on XDP_REDIRECT_XSK if:
  (i) AF_XDP socket SW Rx ring configured
 (ii) Exactly one xsk attached to the queue
If any of these conditions are not met, fall back to the same behavior as the original approach: 
xdp_redirect_map. This is handled under-the-hood in the new bpf_xdp_redirect_xsk helper so the user
does not need to be aware of these conditions.

I don't think the micro optimization for specific use case warrants addition of new apis.
Please optimize it without adding new actions and new helpers.