On 6/3/25 07:07, Gupta, Suraj wrote:

[Public]

quoted

-----Original Message-----
From: Sean Anderson <sean.anderson@linux.dev>
Sent: Saturday, May 31, 2025 2:15 AM
To: Gupta, Suraj <redacted>; andrew+netdev@lunn.ch;
davem@davemloft.net; edumazet@google.com; kuba@kernel.org;
pabeni@redhat.com; vkoul@kernel.org; Simek, Michal [off-list ref];
Pandey, Radhey Shyam [off-list ref]; horms@kernel.org
Cc: netdev@vger.kernel.org; linux-arm-kernel@lists.infradead.org; linux-
kernel@vger.kernel.org; git (AMD-Xilinx) [off-list ref]; Katakam, Harini
[off-list ref]
Subject: Re: [PATCH net-next] net: xilinx: axienet: Configure and report coalesce
parameters in DMAengine flow

Caution: This message originated from an External Source. Use proper caution
when opening attachments, clicking links, or responding.


On 5/30/25 06:18, Gupta, Suraj wrote:

quoted

[AMD Official Use Only - AMD Internal Distribution Only]

quoted

-----Original Message-----
From: Sean Anderson <sean.anderson@linux.dev>
Sent: Thursday, May 29, 2025 9:48 PM
To: Gupta, Suraj <redacted>; andrew+netdev@lunn.ch;
davem@davemloft.net; edumazet@google.com; kuba@kernel.org;
pabeni@redhat.com; vkoul@kernel.org; Simek, Michal
[off-list ref]; Pandey, Radhey Shyam
[off-list ref]; horms@kernel.org
Cc: netdev@vger.kernel.org; linux-arm-kernel@lists.infradead.org;
linux- kernel@vger.kernel.org; git (AMD-Xilinx) [off-list ref];
Katakam, Harini [off-list ref]
Subject: Re: [PATCH net-next] net: xilinx: axienet: Configure and
report coalesce parameters in DMAengine flow

Caution: This message originated from an External Source. Use proper
caution when opening attachments, clicking links, or responding.


On 5/28/25 08:00, Gupta, Suraj wrote:

quoted

[AMD Official Use Only - AMD Internal Distribution Only]

quoted

-----Original Message-----
From: Sean Anderson <sean.anderson@linux.dev>
Sent: Tuesday, May 27, 2025 9:47 PM
To: Gupta, Suraj <redacted>; andrew+netdev@lunn.ch;
davem@davemloft.net; edumazet@google.com; kuba@kernel.org;
pabeni@redhat.com; vkoul@kernel.org; Simek, Michal
[off-list ref]; Pandey, Radhey Shyam
[off-list ref]; horms@kernel.org
Cc: netdev@vger.kernel.org; linux-arm-kernel@lists.infradead.org;
linux- kernel@vger.kernel.org; git (AMD-Xilinx) [off-list ref];
Katakam, Harini [off-list ref]
Subject: Re: [PATCH net-next] net: xilinx: axienet: Configure and
report coalesce parameters in DMAengine flow

Caution: This message originated from an External Source. Use
proper caution when opening attachments, clicking links, or responding.


On 5/25/25 06:22, Suraj Gupta wrote:

quoted

Add support to configure / report interrupt coalesce count and
delay via ethtool in DMAEngine flow.
Netperf numbers are not good when using non-dmaengine default
values, so tuned coalesce count and delay and defined separate
default values in dmaengine flow.

Netperf numbers and CPU utilisation change in DMAengine flow
after introducing coalescing with default parameters:
coalesce parameters:
   Transfer type        Before(w/o coalescing)  After(with coalescing)
TCP Tx, CPU utilisation%      925, 27                 941, 22
TCP Rx, CPU utilisation%      607, 32                 741, 36
UDP Tx, CPU utilisation%      857, 31                 960, 28
UDP Rx, CPU utilisation%      762, 26                 783, 18

Above numbers are observed with 4x Cortex-a53.

How does this affect latency? I would expect these RX settings to
increase latency around 5-10x. I only use these settings with DIM
since it will disable coalescing during periods of light load for better latency.

(of course the way to fix this in general is RSS or some other
method involving multiple queues).

I took values before NAPI addition in legacy flow (rx_threshold:
24, rx_usec: 50) as

reference. But netperf numbers were low with them, so tried tuning
both and selected the pair which gives good numbers.

Yeah, but the reason is that you are trading latency for throughput.
There is only one queue, so when the interface is saturated you will
not get good latency anyway (since latency-sensitive packets will get
head-of-line blocked). But when activity is sparse you can good
latency if there is no coalescing. So I think coalescing should only
be used when there is a lot of traffic. Hence why I only adjusted the
settings once I implemented DIM. I think you should be able to
implement it by calling net_dim from axienet_dma_rx_cb, but it will not be as

efficient without NAPI.

quoted

quoted

Ok, got it. I'll keep default values used before NAPI in legacy flow (coalesce count:

24, delay: 50) for both Tx and Rx and remove perf comparisons.

Those settings are actually probably even worse for latency. I'd leave the settings at
0/0 (coalescing disabled) to match the existing behavior. I think the perf comparisons
are helpful, especially for people who know they are going to be throughput-limited.

My main point is that I think extending the dmaengine API to allow for DIM will have
practical benefits in reduced latency.

Sure, will implement DIM for both Tx and Rx in next version. However, I noticed it's implemented for Rx only in legacy flow. Is there any specific reason for that?

There's no latency issue with sending packets. It doesn't matter when we
process Tx completions as long as we refill the ring in time to send
more packets. So we can aggressively set the Tx coalescing for maximum
throughput.

--Sean