On Thu, Aug 17, 2023 at 8:16 AM Sebastian Andrzej Siewior
[off-list ref] wrote:

On 2023-08-14 11:24:21 [-0700], Jakub Kicinski wrote:

quoted

On Mon, 14 Aug 2023 11:35:26 +0200 Sebastian Andrzej Siewior wrote:

quoted

The RPS code and "deferred skb free" both send IPI/ function call
to a remote CPU in which a softirq is raised. This leads to a warning on
PREEMPT_RT because raising softiqrs from function call led to undesired
behaviour in the past. I had duct tape in RT for the "deferred skb free"
and Wander Lairson Costa reported the RPS case.

Could you find a less invasive solution?
backlog is used by veth == most containerized environments.
This change has a very high risk of regression for a lot of people.

Looking at the cloudflare ppl here in the thread, I doubt they use
backlog but have proper NAPI so they might not need this.

Cloudflare does have backlog usage. On some veths we have to turn GRO
off to cope with multi-layer encapsulation, and there is also no XDP
attached on these interfaces, thus the backlog is used. There are also
other usage of backlog, tuntap, loopback and bpf-redirect ingress.
Frankly speaking, making a NAPI instance "threaded" itself is not a
concern. We have threaded NAPI running on some veth for quite a while,
and it performs pretty well. The concern, if any, would be the
maturity of new code. I am happy to help derisk with some lab tests
and dogfooding if generic agreement is reached to proceed with this
idea.

Yan

There is no threaded NAPI for backlog and RPS. This was suggested as the
mitigation for the highload/ DoS case. Can this become a problem or
- backlog is used only by old drivers so they can move to proper NAPI if
  it becomes a problem.
- RPS spreads the load across multiple CPUs so it unlikely to become a
  problem.

Making this either optional in general or mandatory for threaded
interrupts or PREEMPT_RT will probably not make the maintenance of this
code any simpler.

I've been looking at veth. In the xdp case it has its own NAPI instance.
In the non-xdp it uses backlog. This should be called from
ndo_start_xmit and user's write() so BH is off and interrupts are
enabled at this point and it should be kind of rate-limited. Couldn't we
bypass backlog in this case and deliver the packet directly to the
stack?

Sebastian

-- 

Yan