On 02/21, Mina Almasry wrote:

On Thu, Feb 20, 2025 at 11:13 AM Stanislav Fomichev
[off-list ref] wrote:

quoted

On 02/20, Mina Almasry wrote:

quoted

Drivers need to make sure not to pass netmem dma-addrs to the
dma-mapping API in order to support netmem TX.

Add helpers and netmem_dma_*() helpers that enables special handling of
netmem dma-addrs that drivers can use.

Document in netmem.rst what drivers need to do to support netmem TX.

Signed-off-by: Mina Almasry <redacted>

---

v4:
- New patch
---
 .../networking/net_cachelines/net_device.rst  |  1 +
 Documentation/networking/netdev-features.rst  |  5 +++++
 Documentation/networking/netmem.rst           | 14 +++++++++++--
 include/linux/netdevice.h                     |  2 ++
 include/net/netmem.h                          | 20 +++++++++++++++++++
 5 files changed, 40 insertions(+), 2 deletions(-)

diff --git a/Documentation/networking/net_cachelines/net_device.rst b/Documentation/networking/net_cachelines/net_device.rst
index 15e31ece675f..e3043b033647 100644
--- a/Documentation/networking/net_cachelines/net_device.rst
+++ b/Documentation/networking/net_cachelines/net_device.rst

@@ -10,6 +10,7 @@ Type                                Name                        fastpath_tx_acce
 =================================== =========================== =================== =================== ===================================================================================
 unsigned_long:32                    priv_flags                  read_mostly                             __dev_queue_xmit(tx)
 unsigned_long:1                     lltx                        read_mostly                             HARD_TX_LOCK,HARD_TX_TRYLOCK,HARD_TX_UNLOCK(tx)
+unsigned long:1                          netmem_tx:1;                read_mostly
 char                                name[16]
 struct netdev_name_node*            name_node
 struct dev_ifalias*                 ifalias

diff --git a/Documentation/networking/netdev-features.rst b/Documentation/networking/netdev-features.rst
index 5014f7cc1398..02bd7536fc0c 100644
--- a/Documentation/networking/netdev-features.rst
+++ b/Documentation/networking/netdev-features.rst

@@ -188,3 +188,8 @@ Redundancy) frames from one port to another in hardware.
 This should be set for devices which duplicate outgoing HSR (High-availability
 Seamless Redundancy) or PRP (Parallel Redundancy Protocol) tags automatically
 frames in hardware.
+
+* netmem-tx
+
+This should be set for devices which support netmem TX. See
+Documentation/networking/netmem.rst

diff --git a/Documentation/networking/netmem.rst b/Documentation/networking/netmem.rst
index 7de21ddb5412..43054d44c407 100644
--- a/Documentation/networking/netmem.rst
+++ b/Documentation/networking/netmem.rst

@@ -19,8 +19,8 @@ Benefits of Netmem :
 * Simplified Development: Drivers interact with a consistent API,
   regardless of the underlying memory implementation.

-Driver Requirements
-===================
+Driver RX Requirements
+======================

 1. The driver must support page_pool.

@@ -77,3 +77,13 @@ Driver Requirements
    that purpose, but be mindful that some netmem types might have longer
    circulation times, such as when userspace holds a reference in zerocopy
    scenarios.
+
+Driver TX Requirements
+======================
+
+1. Driver should use netmem_dma_unmap_page_attrs() in lieu of
+   dma_unmap_page[_attrs](), and netmem_dma_unmap_addr_set() in lieu of
+   dma_unmap_addr_set(). The netmem variants will handle netmems that should
+   not be dma-unmapped by the driver, such as dma-buf netmems.

Not all drivers use dma_unmap_addr_xxx APIs (looking at mlx5). Might
be worth mentioning that for the drivers managing the mappings
differently, care might be taken to not unmap netmems?

Yes now that I take a closer look, it's poorly worded to imply the
issue is limited to dma_unmap. I will reword to say that all
dma_map*() APIs must be avoided, and we have helpers for
dma_unmap_*(), and more helpers can be added if needed (similar to
wording in the Driver RX requirements).

quoted

quoted

+2. Driver should declare support by setting `netdev->netmem_tx = true`

diff --git a/include/linux/netdevice.h b/include/linux/netdevice.h
index fccc03cd2164..d8cfd5d69ddf 100644
--- a/include/linux/netdevice.h
+++ b/include/linux/netdevice.h

@@ -1753,6 +1753,7 @@ enum netdev_reg_state {
  *   @lltx:          device supports lockless Tx. Deprecated for real HW
  *                   drivers. Mainly used by logical interfaces, such as
  *                   bonding and tunnels
+ *   @netmem_tx:     device support netmem_tx.
  *
  *   @name:  This is the first field of the "visible" part of this structure
  *           (i.e. as seen by users in the "Space.c" file).  It is the name

@@ -2061,6 +2062,7 @@ struct net_device {
      struct_group(priv_flags_fast,
              unsigned long           priv_flags:32;
              unsigned long           lltx:1;
+             unsigned long           netmem_tx:1;
      );
      const struct net_device_ops *netdev_ops;
      const struct header_ops *header_ops;

diff --git a/include/net/netmem.h b/include/net/netmem.h
index a2148ffb203d..1fb39ad63290 100644
--- a/include/net/netmem.h
+++ b/include/net/netmem.h

@@ -8,6 +8,7 @@
 #ifndef _NET_NETMEM_H
 #define _NET_NETMEM_H

+#include <linux/dma-mapping.h>
 #include <linux/mm.h>
 #include <net/net_debug.h>

@@ -267,4 +268,23 @@ static inline unsigned long netmem_get_dma_addr(netmem_ref netmem)
 void get_netmem(netmem_ref netmem);
 void put_netmem(netmem_ref netmem);

[..]

quoted

+#define netmem_dma_unmap_addr_set(NETMEM, PTR, ADDR_NAME, VAL)   \
+     do {                                                     \
+             if (!netmem_is_net_iov(NETMEM))                  \
+                     dma_unmap_addr_set(PTR, ADDR_NAME, VAL); \
+             else                                             \
+                     dma_unmap_addr_set(PTR, ADDR_NAME, 0);   \
+     } while (0)

Any reason not do to static inline instaed?

Because the args passed to dma_unmap_addr_set are quite unique,
AFAICT. PTR is a pointer to any struct that has a field (anywhere)
inside of it called ADDR_NAME, then dma_unmap_addr_set does something
like:

PTR->ADDR_NAME = VAL;

A static inline needs well defined types, and I couldn't figure out
how to do that (or if it is possible), so a macro it is I guess.

Where I could, I went with static inline.

Ah dma_unmap_addr_set itself is an ugly define :-( Makes sense.