[net-next] enetc: fix locking for one-step timestamping packet transfer

The previous patch to support PTP Sync packet one-step timestamping
described one-step timestamping packet handling logic as below in
commit message:

- Trasmit packet immediately if no other one in transfer, or queue to
  skb queue if there is already one in transfer.
  The test_and_set_bit_lock() is used here to lock and check state.
- Start a work when complete transfer on hardware, to release the bit
  lock and to send one skb in skb queue if has.

There was not problem of the description, but there was a mistake in
implementation. The locking/test_and_set_bit_lock() should be put in
enetc_start_xmit() which may be called by worker, rather than in
enetc_xmit(). Otherwise, the worker calling enetc_start_xmit() after
bit lock released is not able to lock again for transfer.

Fixes: 7294380c5211 ("enetc: support PTP Sync packet one-step timestamping")
Signed-off-by: Yangbo Lu <yangbo.lu@nxp.com>
---
 drivers/net/ethernet/freescale/enetc/enetc.c | 18 +++++++++---------
 1 file changed, 9 insertions(+), 9 deletions(-)

diff --git a/drivers/net/ethernet/freescale/enetc/enetc.c b/drivers/net/ethernet/freescale/enetc/enetc.c
index 4a0adb0b8bd7..65f1772c5740 100644
--- a/drivers/net/ethernet/freescale/enetc/enetc.c
+++ b/drivers/net/ethernet/freescale/enetc/enetc.c

@@ -301,6 +301,15 @@ static netdev_tx_t enetc_start_xmit(struct sk_buff *skb,
 	struct enetc_bdr *tx_ring;
 	int count;
 
+	/* Queue one-step Sync packet if already locked */
+	if (skb->cb[0] & ENETC_F_TX_ONESTEP_SYNC_TSTAMP) {
+		if (test_and_set_bit_lock(ENETC_TX_ONESTEP_TSTAMP_IN_PROGRESS,
+					  &priv->flags)) {
+			skb_queue_tail(&priv->tx_skbs, skb);
+			return NETDEV_TX_OK;
+		}
+	}
+
 	tx_ring = priv->tx_ring[skb->queue_mapping];
 
 	if (unlikely(skb_shinfo(skb)->nr_frags > ENETC_MAX_SKB_FRAGS))

@@ -352,15 +361,6 @@ netdev_tx_t enetc_xmit(struct sk_buff *skb, struct net_device *ndev)
 			skb->cb[0] = ENETC_F_TX_TSTAMP;
 	}
 
-	/* Queue one-step Sync packet if already locked */
-	if (skb->cb[0] & ENETC_F_TX_ONESTEP_SYNC_TSTAMP) {
-		if (test_and_set_bit_lock(ENETC_TX_ONESTEP_TSTAMP_IN_PROGRESS,
-					  &priv->flags)) {
-			skb_queue_tail(&priv->tx_skbs, skb);
-			return NETDEV_TX_OK;
-		}
-	}
-
 	return enetc_start_xmit(skb, ndev);
 }
 

base-commit: 8ef7adc6beb2ef0bce83513dc9e4505e7b21e8c2
-- 
2.25.1

-----Original Message-----
From: Yangbo Lu <yangbo.lu@nxp.com>
Sent: Tuesday, April 13, 2021 6:48 AM

[...]

Subject: [net-next] enetc: fix locking for one-step timestamping packet
transfer

The previous patch to support PTP Sync packet one-step timestamping
described one-step timestamping packet handling logic as below in
commit message:

- Trasmit packet immediately if no other one in transfer, or queue to
 skb queue if there is already one in transfer.
 The test_and_set_bit_lock() is used here to lock and check state.
- Start a work when complete transfer on hardware, to release the bit
 lock and to send one skb in skb queue if has.

There was not problem of the description, but there was a mistake in
implementation. The locking/test_and_set_bit_lock() should be put in
enetc_start_xmit() which may be called by worker, rather than in
enetc_xmit(). Otherwise, the worker calling enetc_start_xmit() after
bit lock released is not able to lock again for transfer.

Fixes: 7294380c5211 ("enetc: support PTP Sync packet one-step
timestamping")
Signed-off-by: Yangbo Lu <yangbo.lu@nxp.com>
---

Reviewed-by: Claudiu Manoil <claudiu.manoil@nxp.com>

On Tue, 13 Apr 2021 11:48:17 +0800 Yangbo Lu wrote:

+	/* Queue one-step Sync packet if already locked */
+	if (skb->cb[0] & ENETC_F_TX_ONESTEP_SYNC_TSTAMP) {
+		if (test_and_set_bit_lock(ENETC_TX_ONESTEP_TSTAMP_IN_PROGRESS,
+					  &priv->flags)) {
+			skb_queue_tail(&priv->tx_skbs, skb);
+			return NETDEV_TX_OK;
+		}
+	}

Isn't this missing queue_work() as well?

Also as I mentioned I don't understand why you created a separate
workqueue instead of using the system workqueue via schedule_work().

Hi Jakub,

-----Original Message-----
From: Jakub Kicinski <kuba@kernel.org>
Sent: 2021年4月14日 1:11
To: Y.b. Lu <yangbo.lu@nxp.com>
Cc: netdev@vger.kernel.org; David S . Miller <davem@davemloft.net>; Richard
Cochran [off-list ref]; Claudiu Manoil
[off-list ref]; Vladimir Oltean [off-list ref];
Russell King [off-list ref]
Subject: Re: [net-next] enetc: fix locking for one-step timestamping packet
transfer

On Tue, 13 Apr 2021 11:48:17 +0800 Yangbo Lu wrote:

quoted

+	/* Queue one-step Sync packet if already locked */
+	if (skb->cb[0] & ENETC_F_TX_ONESTEP_SYNC_TSTAMP) {
+		if

(test_and_set_bit_lock(ENETC_TX_ONESTEP_TSTAMP_IN_PROGRESS,

quoted

+					  &priv->flags)) {
+			skb_queue_tail(&priv->tx_skbs, skb);
+			return NETDEV_TX_OK;
+		}
+	}

Isn't this missing queue_work() as well?

Also as I mentioned I don't understand why you created a separate workqueue
instead of using the system workqueue via schedule_work().

queue_work(system_wq, ) was put in clean_tx. I finally followed the logic you suggested :)

See my reply in v2 discussion
https://patchwork.kernel.org/project/netdevbpf/patch/20210408111350.3817-3-yangbo.lu@nxp.com/

And v3 patch merged (7294380c5211 enetc: support PTP Sync packet one-step timestamping)
https://patchwork.kernel.org/project/netdevbpf/patch/20210412090327.22330-3-yangbo.lu@nxp.com/

Thanks.

On Wed, 14 Apr 2021 06:18:57 +0000 Y.b. Lu wrote:

quoted

On Tue, 13 Apr 2021 11:48:17 +0800 Yangbo Lu wrote:  

quoted

+	/* Queue one-step Sync packet if already locked */
+	if (skb->cb[0] & ENETC_F_TX_ONESTEP_SYNC_TSTAMP) {
+		if  

(test_and_set_bit_lock(ENETC_TX_ONESTEP_TSTAMP_IN_PROGRESS,  

quoted

+					  &priv->flags)) {
+			skb_queue_tail(&priv->tx_skbs, skb);
+			return NETDEV_TX_OK;
+		}
+	}  

Isn't this missing queue_work() as well?

Also as I mentioned I don't understand why you created a separate workqueue
instead of using the system workqueue via schedule_work().  

queue_work(system_wq, ) was put in clean_tx. I finally followed the
logic you suggested :)

Ah, I didn't look close enough. I was expecting to see schedule_work(),
please consider sending a follow up, queue_work(system_wq, $work) is a
rare construct.

-----Original Message-----
From: Jakub Kicinski <kuba@kernel.org>
Sent: 2021年4月15日 0:36
To: Y.b. Lu <yangbo.lu@nxp.com>
Cc: netdev@vger.kernel.org; David S . Miller <davem@davemloft.net>; Richard
Cochran [off-list ref]; Claudiu Manoil
[off-list ref]; Vladimir Oltean [off-list ref];
Russell King [off-list ref]
Subject: Re: [net-next] enetc: fix locking for one-step timestamping packet
transfer

On Wed, 14 Apr 2021 06:18:57 +0000 Y.b. Lu wrote:

quoted

quoted

On Tue, 13 Apr 2021 11:48:17 +0800 Yangbo Lu wrote:

quoted

+	/* Queue one-step Sync packet if already locked */
+	if (skb->cb[0] & ENETC_F_TX_ONESTEP_SYNC_TSTAMP) {
+		if

(test_and_set_bit_lock(ENETC_TX_ONESTEP_TSTAMP_IN_PROGRESS,

quoted

+					  &priv->flags)) {
+			skb_queue_tail(&priv->tx_skbs, skb);
+			return NETDEV_TX_OK;
+		}
+	}

Isn't this missing queue_work() as well?

Also as I mentioned I don't understand why you created a separate
workqueue instead of using the system workqueue via schedule_work().

queue_work(system_wq, ) was put in clean_tx. I finally followed the
logic you suggested :)

Ah, I didn't look close enough. I was expecting to see schedule_work(), please
consider sending a follow up, queue_work(system_wq, $work) is a rare
construct.

Thank you Jakub. I sent another patch for this.
https://patchwork.kernel.org/project/netdevbpf/patch/20210415053455.10029-1-yangbo.lu@nxp.com/

Hi,

I think this patch was reviewed and no objection now, right? (I see status is " Changes Requested ".)
Thanks.

-----Original Message-----
From: Yangbo Lu <yangbo.lu@nxp.com>
Sent: 2021年4月13日 11:48
To: netdev@vger.kernel.org
Cc: Y.b. Lu <yangbo.lu@nxp.com>; David S . Miller <davem@davemloft.net>;
Richard Cochran [off-list ref]; Claudiu Manoil
[off-list ref]; Jakub Kicinski [off-list ref]; Vladimir
Oltean [off-list ref]; Russell King [off-list ref]
Subject: [net-next] enetc: fix locking for one-step timestamping packet transfer

The previous patch to support PTP Sync packet one-step timestamping
described one-step timestamping packet handling logic as below in commit
message:

- Trasmit packet immediately if no other one in transfer, or queue to
  skb queue if there is already one in transfer.
  The test_and_set_bit_lock() is used here to lock and check state.
- Start a work when complete transfer on hardware, to release the bit
  lock and to send one skb in skb queue if has.

There was not problem of the description, but there was a mistake in
implementation. The locking/test_and_set_bit_lock() should be put in
enetc_start_xmit() which may be called by worker, rather than in enetc_xmit().
Otherwise, the worker calling enetc_start_xmit() after bit lock released is not
able to lock again for transfer.

Fixes: 7294380c5211 ("enetc: support PTP Sync packet one-step
timestamping")
Signed-off-by: Yangbo Lu <yangbo.lu@nxp.com>

[...]