Re: [RFC PATCH v2 02/11] netdev: implement netlink api to bind dma-buf to netdevice

On Mon, 21 Aug 2023 20:38:09 -0400 Willem de Bruijn wrote:

quoted

Are you talking about HW devices, or virt? I thought most HW made
in the last 10 years should be able to take down individual queues :o  

That's great. This is currently mostly encapsulated device-wide behind
ndo_close, with code looping over all rx rings, say.

Taking a look at one driver, bnxt, it indeed has a per-ring
communication exchange with the device, in hwrm_ring_free_send_msg
("/* Flush rings and disable interrupts */"), which is called before
the other normal steps: napi disable, dma unmap, posted mem free,
irq_release, napi delete and ring mem free.

This is what you meant? The issue I was unsure of was quiescing the
device immediately, i.e., that hwrm_ring_free_send_msg.

Yes, and I recall we had similar APIs at Netronome for the NFP.
I haven't see it in MS specs myself but I wouldn't be surprised if 
they required it..

There's a bit of an unknown in how well all of this actually works,
as the FW/HW paths were not exercised outside of RDMA and potentially
other proprietary stuff.

I guess this means that this could all be structured on a per-queue
basis rather than from ndo_close. Would be a significant change to
many drivers, I'd imagine.

Yes, it definitely is. The question is how much of it do we require
from Mina before merging the mem provider work. I'd really like to
avoid the "delegate all the work to the driver" approach that AF_XDP
has taken, which is what I'm afraid we'll end up with if we push too
hard for a full set of APIs from the start.