Subject: RE: [PATCH net] net: mana: Fix doorbell out of order violation and avoid
unnecessary doorbell rings

quoted

From: longli@linuxonhyperv.com <redacted>
Sent: Monday, August 5, 2024 4:38 PM
[...]
After napi_complete_done() is called, another NAPI may be running on
another CPU and ring the doorbell before the current CPU does. When

Can you please share more details about "another NAPI"? Is it about busy_poll?

quoted

combined with unnecessary rings when there is no need to ARM the CQ,
this triggers error paths in the hardware.

Fix this by always ring the doorbell in sequence and avoid unnecessary
rings.

I'm not sure what "error paths in the hardware" means. It's better to describe the
user-visible consequence.

Maybe this is clearer:

When there is no need to arm the CQ from NAPI's perspective, the driver must
not combine "too many" arming operations due to a MANA hardware
requirement:
the driver must ring the doorbell at least once within every 8 wraparounds of the
CQ, otherwise "XXX" would happen. //Dexuan: I don't know what the "XXX" is

Add a per-CQ counter cq->work_done_since_doorbell, and make sure the CQ is
armed within 4 wraparounds of the CQ. //Dexuan: why not 8 or 7?

I'm sending v2 to address the details in the comments.

quoted

+	if (w < cq->budget) {
+		mana_gd_ring_cq(gdma_queue, SET_ARM_BIT);
+		cq->work_done_since_doorbell = 0;
+		napi_complete_done(&cq->napi, w);
+	} else if (cq->work_done_since_doorbell >
+		   cq->gdma_cq->queue_size / COMP_ENTRY_SIZE * 4) {
+		/* MANA hardware requires at least one doorbell ring every 8

s/ring every 8/arming within every 8/ ?

quoted

+		 * wraparounds of CQ even there is no need to ARM. This
driver

s/ARM/arming/ ?
s/even/even if/ ?

Will fix this in v2.

Thanks,

Long