Re: [PATCH bpf-next] xsk: support AF_PACKET

On Fri, 28 May 2021 12:22:40 +0200
Daniel Borkmann [off-list ref] wrote:

On 5/28/21 12:00 PM, Magnus Karlsson wrote:

quoted

On Fri, May 28, 2021 at 11:52 AM Jesper Dangaard Brouer
[off-list ref] wrote:  

quoted

On Fri, 28 May 2021 17:02:01 +0800
Xuan Zhuo [off-list ref] wrote:  

quoted

On Fri, 28 May 2021 10:55:58 +0200, Toke Høiland-Jørgensen [off-list ref] wrote:  

quoted

Xuan Zhuo [off-list ref] writes:
 

quoted

In xsk mode, users cannot use AF_PACKET(tcpdump) to observe the current
rx/tx data packets. This feature is very important in many cases. So
this patch allows AF_PACKET to obtain xsk packages.  

You can use xdpdump to dump the packets from the XDP program before it
gets redirected into the XSK:
https://github.com/xdp-project/xdp-tools/tree/master/xdp-dump  

Wow, this is a good idea.  

Yes, it is rather cool (credit to Eelco).  Notice the extra info you
can capture from 'exit', like XDP return codes, if_index, rx_queue.

The tool uses the perf ring-buffer to send/copy data to userspace.
This is actually surprisingly fast, but I still think AF_XDP will be
faster (but it usually 'steals' the packet).

Another (crazy?) idea is to extend this (and xdpdump), is to leverage
Hangbin's recent XDP_REDIRECT extension e624d4ed4aa8 ("xdp: Extend
xdp_redirect_map with broadcast support").  We now have a
xdp_redirect_map flag BPF_F_BROADCAST, what if we create a
BPF_F_CLONE_PASS flag?

The semantic meaning of BPF_F_CLONE_PASS flag is to copy/clone the
packet for the specified map target index (e.g AF_XDP map), but
afterwards it does like veth/cpumap and creates an SKB from the
xdp_frame (see __xdp_build_skb_from_frame()) and send to netstack.
(Feel free to kick me if this doesn't make any sense)  

This would be a smooth way to implement clone support for AF_XDP. If
we had this and someone added AF_XDP support to libpcap, we could both
capture AF_XDP traffic with tcpdump (using this clone functionality in
the XDP program) and speed up tcpdump for dumping traffic destined for
regular sockets. Would that solve your use case Xuan? Note that I have
not looked into the BPF_F_CLONE_PASS code, so do not know at this
point what it would take to support this for XSKMAPs.  

There is no spoon... the BPF_F_CLONE_PASS code is an idea.

Recently also ended up with something similar for our XDP LB to record pcaps [0] ;)
My question is.. tcpdump doesn't really care where the packet data comes from,
so why not extending libpcap's Linux-related internals to either capture from
perf RB or BPF ringbuf 

Just want to first mention, that I do like adding a perf ring-buffer
(BPF ringbuf) interface to AF_PACKET.  But this is basically what
xdpdump already does.  The cool thing is that it is super flexible for
adding extra info like xdpdump does with XDP-return codes.

rather than AF_PACKET sockets? Cloning is slow, and if
you need to end up creating an skb which is then cloned once again inside AF_PACKET
it's even worse. Just relying and reading out, say, perf RB you don't need any
clones at all.

Well, this is exactly what we avoid with my idea of BPF_F_CLONE_PASS
when combined with AF_XDP. 

I should explain this idea better.  The trick is that AF_XDP have
preallocated all the packets it will every use (at setup time).   Thus,
the AF_XDP copy-mode does no allocations, which is why it is fast
(of-cause ZC mode is faster, but copy-mode AF_XDP is also VERY fast!).

(Details and step with AF_XDP code notes:)
When the xdp_do_redirect happens with ri->flags BPF_F_CLONE_PASS, then
the map specific enqueue (e.g. __xsk_map_redirect), will do a copy of
the xdp_buff (AF_XDP calls xsk_copy_xdp()) and for AF_XDP we don't need
to do a (real) allocation.  Instead of freeing the xdp_buff in xsk_rcv()
(see call to xdp_return_buff()) then we do the xdp_frame to SKB work.

-- 
Best regards,
  Jesper Dangaard Brouer
  MSc.CS, Principal Kernel Engineer at Red Hat
  LinkedIn: http://www.linkedin.com/in/brouer