Re: [PATCH net-next v8 08/15] ibmveth: Add queue-aware RX buffer submit helper for MQ

From
Jakub Kicinski <kuba@kernel.org>
Date
2026-11-17 01:36:10
Also in
linuxppc-dev
Reply to
Mingming Cao, 2026-11-09
On Mon,  9 Nov 2026 15:55:49 -0800 Mingming Cao wrote:
This patch queue-indexes RX replenish and pool access ahead of MQ
enable:

  - hcall: ibmveth_add_logical_lan_buffers() (MQ queue hcall vs legacy)
  - concurrency: per-queue replenish_lock; replenish outcomes logged
    after unlock
  - poll/pool: rxq_* take queue_index; INVALID_MAP and correlator
    errors refactored
  - limits: IBMVETH_MAX_RX_PER_HCALL 12, IBMVETH_MAX_RX_REGULAR 8

Runtime stays single-queue until the MQ enable patch; correlator,
lock and replenish return paths apply on queue 0 immediately.

Replenish is the last open-path hypercall that still assumes queue 0.
Registration is already queue-aware, and multi-queue posts buffers
through H_ADD_LOGICAL_LAN_BUFFERS_QUEUE against adapter->queue_handle[],
but ibmveth_replenish_buffer_pool() still calls
h_add_logical_lan_buffer() or h_add_logical_lan_buffers() directly.

Add ibmveth_add_logical_lan_buffers() to route RX buffer pool
replenishment through H_ADD_LOGICAL_LAN_BUFFERS_QUEUE in multi-queue
mode, falling back to the legacy 8-buffer and single-buffer hypercalls
in single-queue mode.

Supporting that helper means the RX side can no longer assume queue 0.
The queue index is 0 everywhere until multi-queue RX is enabled, so
the queue plumbing below does not change behaviour yet. The correlator
and locking changes do take effect immediately, on the single queue.

Parameterise the RX accessors by queue. The eight ibmveth_rxq_*
helpers took only the adapter and hardcoded rx_queue[0]; they now take
a queue_index, as do ibmveth_remove_buffer_from_pool() and
ibmveth_rxq_get_buffer(). ibmveth_rxq_correlator() becomes
ibmveth_rxq_read_correlator(); poll still reads the correlator once
per slot, as net-next does, and now passes the queue index with it
to rxq_get_buffer() and rxq_harvest_buffer().

Add a per-queue replenish_lock to struct ibmveth_rx_q, initialised in
probe. Replenish is not the only writer of a pool's free_map: the
harvest and remove consumer runs from that queue's NAPI instance. At
this commit replenish also runs only from NAPI (netpoll's budget-0
poll skips it), but later patches (MQ open and the queue resize)
replenish from process context while that queue's NAPI may run, so
producer and consumer can run at once. Its users are NAPI and process
context, so it is taken with spin_lock_bh().

Give ibmveth_replenish_buffer_pool() a return value. It was void and
logged from inside what is now a critical section. It now returns one
of IBMVETH_REPLENISH_OK, _RESET_MAP, _RESET_MQ, _HCALL_FAIL or
_BATCH_FALLBACK, and ibmveth_replenish_task() does the logging and any
schedule_work() after dropping replenish_lock. Replenish map paths use
DMA_ATTR_NO_WARN; failures are counted, and the next poll's replenish
retries. As before this patch, a queue whose pools are left with
nothing posted gets no further poll and waits for a reopen. The
hcall-failure warning now logs against the netdev rather than the vio
device, the two batch-fallback info messages become one ratelimited
warning, and DMA_ATTR_NO_WARN silences the DMA layer's map-failure
warning. In multi-queue mode an H_FUNCTION from the buffer-add hcall
schedules a reset instead of the legacy single-buffer fallback;
that arm is unreachable until MQ RX is enabled.

An IBM_VETH_INVALID_MAP free_map entry no longer WARNs. It now drops
the partly filled batch (those skbs stay in the pool for reuse),
skips the remaining pools and schedules a reset, where the old code
posted the partial batch and moved on to the next pool.
replenish_task() also returns early for a queue index at or past the
live count, and ibmveth_poll() logs an IRQ enable failure before it
schedules the reset.

Update replenish_task_cycles under replenish_lock; the statistics patch
moves that counter per queue.

Snapshot the first BATCH_FALLBACK and HCALL_FAIL into their own
copies and log those after unlock, so a later pool cannot overwrite
the values those two messages print.

Make the correlator check per queue. ibmveth_rxq_correlator_valid()
takes the queue index and also requires the pool to be active with a
free_map. It no longer logs or schedules the reset, because
remove_buffer_from_pool() now calls it under replenish_lock:
ibmveth_rxq_bad_correlator() and ibmveth_rxq_no_skb() do the
ratelimited log and the reset, called by rxq_harvest_buffer() after
remove_buffer_from_pool() returns and by rxq_get_buffer() directly.
The ring still advances on every error. The KUnit tests pass the queue
index and set up free_map and replenish_lock.

Redefine IBMVETH_MAX_RX_PER_HCALL from 8 to 12, the argument-list
capacity of H_ADD_LOGICAL_LAN_BUFFERS_QUEUE, and add
IBMVETH_MAX_RX_REGULAR (8) for the legacy hypercall. Probe sets
rx_buffers_per_hcall to IBMVETH_MAX_RX_REGULAR, so the legacy batch
stays at 8 until multi-queue RX is enabled.

free_buffer_pool() keeps probe/sysfs geometry (active/size/threshold)
and zeroes pool->available so teardown cannot leave a stale count.
The further you get in the series the sloppier the commits.
Please trim this text, tell your LLM we can read the code :/
Thread (34 messages) 34 messages, 3 authors, last reply 19h ago
Keyboard shortcuts
hback out one level
jnext message in thread / next series-diff position
kprevious message in thread / previous series-diff position
ldrill in
Escclose help / fold thread tree
?toggle this help