libeth_xsk_tx_queue_bulk() takes the frags of a multi-buffer frame off
pool->xskb_list but leaves XDP_FLAGS_HAS_FRAGS set on the head. When
the head completes, libeth_xsk_buff_free_slow() frees whatever is on
the list at that point.
The XDPSQ can be completed by a bulk flush while the frags of a frame
are being queued, or by its cleanup timer while a partial Rx frame is
stashed. Those frags are then freed while in use, and the frame being
queued is sent truncated.
Clear the flag on the head once all of its frags are queued.
Fixes: b3ad8450b4dc ("libeth: xsk: add XSk XDP_TX sending helpers")
Signed-off-by: Maciej Fijalkowski <maciej.fijalkowski@intel.com>
---
include/net/libeth/xsk.h | 10 ++++++++--
1 file changed, 8 insertions(+), 2 deletions(-)
diff --git a/include/net/libeth/xsk.h b/include/net/libeth/xsk.h
index a452b7828ce4..249b1c740788 100644
--- a/include/net/libeth/xsk.h
+++ b/include/net/libeth/xsk.h
@@ -77,11 +77,17 @@ libeth_xsk_tx_queue_bulk(struct libeth_xdp_tx_bulk *bq,
if (!libeth_xsk_tx_queue_head(bq, xdp))
goto out;
- for (const struct libeth_xdp_buff *head = xdp; ; ) {
+ for (struct libeth_xdp_buff *head = xdp; ; ) {
xdp = container_of(xsk_buff_get_frag(&head->base),
typeof(*xdp), base);
- if (!xdp)
+ if (!xdp) {
+ /* Each frag is now freed via its own SQE. Freeing
+ * the head must not release what is on the pool's
+ * frag list at that time.
+ */
+ xdp_buff_clear_frags_flag(&head->base);
break;
+ }
if (unlikely(bq->count == LIBETH_XDP_TX_BULK) &&
unlikely(!flush_bulk(bq, LIBETH_XDP_TX_XSK))) {--
2.43.0