Re: [RFC net 1/2] af_packet: retire TPACKET_V3 blocks without waiting for copiers

Jingxuan Xie wrote:

From: jingxuanxie <redacted>

TPACKET_V3 waits for a ring-wide read lock before retiring a block,
while holding sk_receive_queue.lock. A copier keeps the read lock
after reserving a frame and dropping the queue lock. On PREEMPT_RT,
preempting that copier can therefore delay another packet path or
the retirement timer until the copier runs again.

This was investigated after an approximately 355 ms block-retirement
wait in a real PREEMPT_RT deployment. A QEMU ARM64 SMP PREEMPT_RT
reproducer uses test-only instrumentation to delay one outgoing
copier after reservation, with a second sender forcing retirement.
The unmodified retirement logic blocks the second sender. The
instrumentation is not part of this series.

Track FREE, ACTIVE, SEALED and USER state per block in private
metadata. Reserve and complete copies under the receive queue lock,
retaining the original block reference across the unlocked copy.
Seal a full or expired block and try the immediately following block
without waiting. Publish sealed blocks in reservation order once
their copies have completed. Do not reuse a sealed block merely
because its mmap status still reads TP_STATUS_KERNEL.

Limit each locked publication batch to eight blocks and continue
ready backlogs on an unbound worker. Stop the timer and worker before
freeing metadata. Check readiness using the last published block,
and validate virtio headers before reserving a frame so an invalid
header cannot strand an unfinished reservation.

Tested against that reproducer with both full-block and timer
retirement: B completes while A remains delayed, blocks are not
published early, and sequence numbers, lengths and payload match.
An uninstrumented QEMU guest also passes psock_tpacket and
psock_fanout. The accompanying KUnit suite covers the state helpers.

Keep the mmap layout and userspace status protocol unchanged. Ring
exhaustion can still drop capture copies; a stalled copier no longer
has to block retirement or copying into another available block.

The block producer index must cover the full unsigned block count,
matching the independent publication index across large rings.

Assisted-by: LLM
Fixes: 632ca50f2cbd ("af_packet: TPACKET_V3: replace busy-wait loop")
Signed-off-by: Jingxuan Xie <redacted>
---
 net/packet/af_packet.c | 319 +++++++++++++++++++++++++----------------
 net/packet/internal.h  |  27 +++-

This is a huge change. Try to solve the problem with a minimal fix.

The patch is doing more in one patch than needed. Such as moving when
virtio_net_hdr_to_skb is called. Probably for good reason but then that is
a separate change.

The main purpose can hopefully be achieved with less code churn too.

Try to avoid re-acquiring sk_receive_queue.lock, a regression. Batching
adds complexity and is probably not needed. system_dfl_wq runs as
SCHED_NORMAL, so this may introduce a new priority inversion.

Also: whitespace corruption, a confidential ("top secret") footer.