Re: [PATCH net-next v1 2/2] net/rds: Give each connection its own workqueue

On Tue, 2025-11-04 at 15:57 +0100, Paolo Abeni wrote:

On 10/29/25 6:46 PM, Allison Henderson wrote:

quoted

From: Allison Henderson <redacted>

RDS was written to require ordered workqueues for "cp->cp_wq":
Work is executed in the order scheduled, one item at a time.

If these workqueues are shared across connections,
then work executed on behalf of one connection blocks work
scheduled for a different and unrelated connection.

Luckily we don't need to share these workqueues.
While it obviously makes sense to limit the number of
workers (processes) that ought to be allocated on a system,
a workqueue that doesn't have a rescue worker attached,
has a tiny footprint compared to the connection as a whole:
A workqueue costs ~800 bytes, while an RDS/IB connection
totals ~5 MBytes.

Still a workqueue per connection feels overkill. Have you considered
moving to WQ_PERCPU for rds_wq? Why does not fit?

Thanks,

Paolo

Hi Paolo

I hadnt thought of WQ_PERCPU before, so I did some digging on it.  In our case though, we need FIFO behavior per-
connection, so if we switched to queues per cpu, we'd have to pin a CPU to a connection to get the right behavior.  And
then that brings back head of the line blocking since now all the items on that queue have to share that CPU even if the
other CPUs are idle.  So it wouldn't quite be a synonymous solution for what we're trying to do in this case.  I hope
that made sense?  Let me know what you think.

Thank you,
Allison