Re: [PATCH net-next 0/2] net/mlx5e: Batch RCU grace periods on channel open and close

Tariq Toukan [off-list ref] writes:

Hi,

This series by Carolina reduces the time mlx5e spends waiting for RCU
grace periods when bringing an interface up or down with many channels.

Today the channel open and close flows call synchronize_net() per
object. On close, RQ, ICOSQ, TX SQ and NAPI teardown each require a
grace period, so the number of waits scales with the number of channels.
TX SQ teardown adds a wait per traffic class, and XDP and QoS can add
more. On activation, mlx5e_activate_channels() triggers the ICOSQ of
every channel, and each trigger waits for its own grace period. With a
large channel count this makes link down and link up slow.

Patch 1 batches the waits on channel close. Channel deactivate and close
are split into pre_sync and post_sync phases: the ENABLED state is first
cleared on all queues of all channels, a single grace period is awaited,
and only then does the teardown continue. NAPI instances are likewise
unlinked with __netif_napi_del_locked(), followed by one
synchronize_net() before the channels are freed. The existing per object
synchronize_net() calls are kept for single queue paths. The change
touches the regular channels as well as the PTP, QoS and XSK queues.

Patch 2 does the same for the ICOSQ trigger on activation: all channel
ICOSQs are first marked as requiring locking, one grace period is
awaited, and then all of them are triggered. mlx5e_trigger_napi_icosq()
keeps its existing semantics for standalone callers.

Results with 248 channels and rcu_normal=1:
- ip link set eth2 down: 10.183s before, 0.715s after.
- ip link set eth2 up: 3.095s before, 1.794s after.

Nice! Thanks for tackling this :)

Don't know the driver well enough to comment on the details, but in
principle, emphatically:

Acked-by: Toke Høiland-Jørgensen <toke@kernel.org>