On 2023-08-17 08:30:25 [-0700], Jakub Kicinski wrote:

On Thu, 17 Aug 2023 15:16:12 +0200 Sebastian Andrzej Siewior wrote:

quoted

I've been looking at veth. In the xdp case it has its own NAPI instance.
In the non-xdp it uses backlog. This should be called from
ndo_start_xmit and user's write() so BH is off and interrupts are
enabled at this point and it should be kind of rate-limited. Couldn't we
bypass backlog in this case and deliver the packet directly to the
stack?

The backlog in veth eats measurable percentage points of RPS of real
workloads, and I think number of people looked at getting rid of it.
So worthy goal for sure, but may not be a trivial fix.

We could separate RPS from backlog but then we still process RPS after
backlog so not sure if this gains anything. Letting veth always use its
NAPI in this case would probably do that. Not sure if it helps…

To my knowledge the two main problems are:
 - we don't want to charge the sending application the processing for
   both "sides" of the connection and all the switching costs.

The packet is injected by the user and softirq is served once BH gets
back to 0. So it is served within the task's context and might be
accounted on softirq/ system (might as I think it needs to be observed
by the timer interrupt for the accounting).

 - we may get an AA deadlock if the packet ends up looping in any way.

Right, forgot about that one.

Or at least that's what I remember the problem being at 8am in the
morning :) Adding Daniel and Martin to CC, Paolo would also know this
better than me but I think he's AFK for the rest of the week.

Sebastian