Re: [PATCH] wifi: ath12k: fix ring-buffer corruption

On Mon, May 26, 2025 at 01:35:02PM +0200, Johan Hovold wrote:

On Thu, May 22, 2025 at 05:11:21PM +0200, Remi Pommarel wrote:

quoted

On Fri, Mar 21, 2025 at 10:52:19AM +0100, Johan Hovold wrote:

quoted

Users of the Lenovo ThinkPad X13s have reported that Wi-Fi sometimes
breaks and the log fills up with errors like:

    ath11k_pci 0006:01:00.0: HTC Rx: insufficient length, got 1484, expected 1492
    ath11k_pci 0006:01:00.0: HTC Rx: insufficient length, got 1460, expected 1484

which based on a quick look at the ath11k driver seemed to indicate some
kind of ring-buffer corruption.

Miaoqing Pan tracked it down to the host seeing the updated destination
ring head pointer before the updated descriptor, and the error handling
for that in turn leaves the ring buffer in an inconsistent state.

While this has not yet been observed with ath12k, the ring-buffer
implementation is very similar to the ath11k one and it suffers from the
same bugs.

quoted

quoted

Note that the READ_ONCE() are only needed to avoid compiler mischief in
case the ring-buffer helpers are ever inlined.

quoted

quoted

@@ -343,11 +343,10 @@ static int ath12k_ce_completed_recv_next(struct ath12k_ce_pipe *pipe,
 		goto err;
 	}
 
+	/* Make sure descriptor is read after the head pointer. */
+	dma_rmb();
+

That does not seem to be the only place descriptor is read just after
the head pointer, ath12k_dp_rx_process{,err,reo_status,wbm_err} seem to
also suffer the same sickness.

Indeed, I only started with the corruption issues that users were
reporting (with ath11k) and was gonna follow up with further fixes once
the initial ones were merged (and when I could find more time).

quoted

Why not move the dma_rmb() in ath12k_hal_srng_access_begin() as below,
that would look to me as a good place to do it.

@@ -2133,6 +2133,9 @@ void ath12k_hal_srng_access_begin(struct

ath12k_base *ab, struct hal_srng *srng)
                        *(volatile u32 *)srng->u.src_ring.tp_addr;
        else
                srng->u.dst_ring.cached_hp = *srng->u.dst_ring.hp_addr;
+
+       /* Make sure descriptors are read after the head pointer. */
+       dma_rmb();
 }

This should ensure the issue does not happen anywhere not just for
ath12k_ce_recv_process_cb().

We only need the read barrier for dest rings so the barrier would go in
the else branch, but I prefer keeping it in the caller so that it is
more obvious when it is needed and so that we can skip the barrier when
the ring is empty (e.g. as done above).

Thanks for taking time to clarify this.

Yes I messed up doing the patch by hand sorry, internally I test with
the dma_rmb() in the else part. I tend to prefer having it in
ath12k_hal_srng_access_begin() as caller does not have to take care of
the barrier itself. Which for me seems a little bit risky if further
refactoring (or adding other ring processing) is done in the future;
the barrier could easily be forgotten don't you think ?

I've gone through and reviewed the remaining call sites now and will
send a follow-on fix for them.

quoted

Note that ath12k_hal_srng_dst_get_next_entry() does not need a barrier
as it uses cached_hp from ath12k_hal_srng_access_begin().

Yeah, it's only needed before accessing the descriptor fields.

quoted

quoted

@@ -1962,7 +1962,7 @@ u32 ath12k_hal_ce_dst_status_get_length(struct hal_ce_srng_dst_status_desc *desc
 {
 	u32 len;
 
-	len = le32_get_bits(desc->flags, HAL_CE_DST_STATUS_DESC_FLAGS_LEN);
+	len = le32_get_bits(READ_ONCE(desc->flags), HAL_CE_DST_STATUS_DESC_FLAGS_LEN);
 	desc->flags &= ~cpu_to_le32(HAL_CE_DST_STATUS_DESC_FLAGS_LEN);
 
 	return len;

@@ -2132,7 +2132,7 @@ void ath12k_hal_srng_access_begin(struct ath12k_base *ab, struct hal_srng *srng)
 		srng->u.src_ring.cached_tp =
 			*(volatile u32 *)srng->u.src_ring.tp_addr;
 	else
-		srng->u.dst_ring.cached_hp = *srng->u.dst_ring.hp_addr;
+		srng->u.dst_ring.cached_hp = READ_ONCE(*srng->u.dst_ring.hp_addr);

dma_rmb() acting also as a compiler barrier why the need for both
READ_ONCE() ?

Yeah, I was being overly cautious here and it should be fine with plain
accesses when reading the descriptor after the barrier, but the memory
model seems to require READ_ONCE() when fetching the head pointer.
Currently, hp_addr is marked as volatile so READ_ONCE() could be
dropped for that reason, but I'd rather keep it here explicitly (e.g. in
case someone decides to drop the volatile).

Yes actually after more thinking, the READ_ONCE for fetching hp does make
sense and is also in the patch I am currently testing.

Also for source rings don't we need a dma_wmb()/WRITE_ONCE before
modifying the tail pointer (see ath12k_hal_srng_access_end()) for quite
the same reason (updates of the descriptor have to be visible before
write to tail pointer) ?

Thanks

-- 
Remi