quoted

On 2/24/2021 11:18 AM, Ciara Loftus wrote:

quoted

This commit introduces support for preferred busy polling
to the AF_XDP PMD. This feature aims to improve single-core
performance for AF_XDP sockets under heavy load.

A new vdev arg is introduced called 'busy_budget' whose default
value is 64. busy_budget is the value supplied to the kernel
with the SO_BUSY_POLL_BUDGET socket option and represents the
busy-polling NAPI budget. To set the budget to a different value
eg. 256:

--vdev=net_af_xdp0,iface=eth0,busy_budget=256

Preferred busy polling is enabled by default provided a kernel with
version >= v5.11 is in use. To disable it, set the budget to zero.

The following settings are also strongly recommended to be used in
conjunction with this feature:

echo 2 | sudo tee /sys/class/net/eth0/napi_defer_hard_irqs
echo 200000 | sudo tee /sys/class/net/eth0/gro_flush_timeout

.. where eth0 is the interface being used by the PMD.

Signed-off-by: Ciara Loftus <redacted>
---
  doc/guides/nics/af_xdp.rst          | 38 ++++++++++++-
  drivers/net/af_xdp/compat.h         | 13 +++++
  drivers/net/af_xdp/rte_eth_af_xdp.c | 85

++++++++++++++++++++++++-----

quoted

  3 files changed, 121 insertions(+), 15 deletions(-)

Can you please update the release notes too to announce the feature?

Will do.

quoted

<...>

quoted

@@ -39,3 +39,16 @@ create_shared_socket(struct xsk_socket **xsk_ptr

__rte_unused,

quoted

  	return -1;
  }
  #endif
+
+#ifdef XDP_USE_NEED_WAKEUP
+static int
+syscall_needed(struct xsk_ring_prod *q, uint32_t busy_budget)
+{
+	return xsk_ring_prod__needs_wakeup(q) | busy_budget;
+}
+#else
+syscall_needed(struct xsk_ring_prod *q __rte_unused, uint32_t

busy_budget)

quoted

+{
+	return busy_budget;
+}

Is the return type missing in the definition?

Yes. Thanks for spotting this.

quoted

Also for the case when both 'XDP_USE_NEED_WAKEUP' &
'SO_PREFER_BUSY_POLL' this
function will always return '0', but current implementation doesn't know

this

quoted

in
the compile time and compiler can't optimize for it, do you think does it

make

quoted

sense to do this optimization?

It makes sense assuming the compile environment and run environment are
the same.
However you make a valid point below. If the environments are different,
we can't make this optimization because we can’t rely on the presence of the
flags alone to tell us if these features are supported. More below.

quoted

<...>

quoted

@@ -1628,8 +1670,22 @@ rte_pmd_af_xdp_probe(struct

rte_vdev_device

quoted

*dev)

quoted

  		return -EINVAL;
  	}

+#ifdef SO_PREFER_BUSY_POLL
+	busy_budget = busy_budget == -1 ?

ETH_AF_XDP_DFLT_BUSY_BUDGET :

quoted

+					busy_budget;
+	if (!busy_budget)
+		AF_XDP_LOG(ERR, "Preferred busy polling disabled\n");

Is this an error case? What do you think changing the log level to DEBUG or
INFO?

+1 for INFO

quoted

Also how these compile time flags will work if the compiled environment

and

quoted

run
environment kernel version are different and incompatible?

This is a valid point. Right now if XDP_USE_NEED_WAKEUP is defined we
assume the functionality is available in the kernel. If it's not, socket creation
will fail and we abort. Perhaps we should retry socket creation without the
flag if we get this failure. And record if support is available in a runtime
variable. I'll look at adding this as another patch to the v2 series.

Hi Ferruh,

I looked at this a little more. For the v2 I'll make sure busy poll can work in these environments with different compile-time and run-time kernels and use setsockopt() to detect support in the kernel.
Since it will require significant changes and validation I'll submit a separate series ensuring the same for the other existing flags (XDP_USE_NEED_WAKEUP / XDP_UMEM_UNALIGNED_CHUNK_FLAG / shared umem).

Thanks,
Ciara

quoted

Overall can it be possible to detect the support on runtime via

'setsockopt()'

quoted

without compile time macros and eliminate the compile time flags? Does it
make
sense?

I think this can be done. It should allow applications compiled on older
kernels without SO_PREFER_BUSY_POLL run on newer kernels with the
feature.
I will tackle this in the v2.

Thanks for your feedback!

Ciara