On Mon, Jul 07, 2025 at 02:55:11PM -0700, Mina Almasry wrote:

On Mon, Jul 7, 2025 at 2:35 PM Dragos Tatulea [off-list ref] wrote:

quoted

On Mon, Jul 07, 2025 at 11:44:19AM -0700, Mina Almasry wrote:

quoted

On Fri, Jul 4, 2025 at 6:11 AM Dragos Tatulea [off-list ref] wrote:

quoted

On Thu, Jul 03, 2025 at 01:58:50PM +0200, Parav Pandit wrote:

quoted

quoted

From: Jakub Kicinski <kuba@kernel.org>
Sent: 03 July 2025 02:23 AM

[...]

quoted

quoted

Maybe someone with closer understanding can chime in. If the kind of
subfunctions you describe are expected, and there's a generic way of
recognizing them -- automatically going to parent of parent would indeed be
cleaner and less error prone, as you suggest.

I am not sure when the parent of parent assumption would fail, but can be
a good start.

If netdev 8 bytes extension to store dma_dev is concern,
probably a netdev IFF_DMA_DEV_PARENT can be elegant to refer parent->parent?
So that there is no guess work in devmem layer.

That said, my understanding of devmem is limited, so I could be mistaken here.

In the long term, the devmem infrastructure likely needs to be
modernized to support queue-level DMA mapping.
This is useful because drivers like mlx5 already support
socket-direct netdev that span across two PCI devices.

Currently, devmem is limited to a single PCI device per netdev.
While the buffer pool could be per device, the actual DMA
mapping might need to be deferred until buffer posting
time to support such multi-device scenarios.

In an offline discussion, Dragos mentioned that io_uring already
operates at the queue level, may be some ideas can be picked up
from io_uring?

The problem for devmem is that the device based API is already set in
stone so not sure how we can change this. Maybe Mina can chime in.

I think what's being discussed here is pretty straight forward and
doesn't need UAPI changes, right? Or were you referring to another
API?

I was referring to the fact that devmem takes one big buffer, maps it
for a single device (in net_devmem_bind_dmabuf()) and then assigns it to
queues in net_devmem_bind_dmabuf_to_queue(). As the single buffer is
part of the API, I don't see how the mapping could be done in a per
queue way.

Oh, I see. devmem does support mapping a single buffer to multiple
queues in a single netlink API call, but there is nothing stopping the
user from mapping N buffers to N queues in N netlink API calls.

Oh, yes, of course. Why didn't I think of that...

Thanks,
Dragos