Re: [PATCH net-next] net/mlx4_en: avoid one cache line miss to ring doorbell

On Thu, Sep 30, 2021 at 5:10 PM Jakub Kicinski [off-list ref] wrote:

On Thu, 30 Sep 2021 12:40:31 -0700 Eric Dumazet wrote:

quoted

From: Eric Dumazet <redacted>

This patch caches doorbell address directly in struct mlx4_en_tx_ring.

This removes the need to bring in cpu caches whole struct mlx4_uar
in fast path.

Note that mlx4_uar is not guaranteed to be on a local node,
because mlx4_bf_alloc() uses a single free list (priv->bf_list)
regardless of its node parameter.

This kind of change does matter in presence of light/moderate traffic.
In high stress, this read-only line would be kept hot in caches.

Signed-off-by: Eric Dumazet <redacted>
Cc: Tariq Toukan <tariqt@nvidia.com>

quoted

      /* Following part should be mostly read */
+     void                    *doorbell_address;

We'll need to make sparse happy before applying:

drivers/net/ethernet/mellanox/mlx4/en_tx.c:133:32: warning: incorrect type in assignment (different address spaces)
drivers/net/ethernet/mellanox/mlx4/en_tx.c:133:32:    expected void *doorbell_address
drivers/net/ethernet/mellanox/mlx4/en_tx.c:133:32:    got void [noderef] __iomem *
drivers/net/ethernet/mellanox/mlx4/en_tx.c:757:56: warning: incorrect type in argument 2 (different address spaces)
drivers/net/ethernet/mellanox/mlx4/en_tx.c:757:56:    expected void [noderef] __iomem *
drivers/net/ethernet/mellanox/mlx4/en_tx.c:757:56:    got void *doorbell_address

Yes indeed, I'll send a V2 right away, thanks !