[PATCH net 0/2] amd-xgbe: PFC and KR-Training fixes

This patch series fixes the issues in kr-training and pfc

Patches:

0001 - There is difference in the TX Flow Control registers (TFCR)
between the revisions of the hardware. Update the driver to use the
TFCR based on the reported version of the hardware.

0002 - AN restart triggered during KR training not only aborts the KR
training process but also move the HW to unstable state. Add the
necessary changes to fix kr-taining.

Raju Rangoju (2):
  amd-xgbe: TX Flow Ctrl Registers are h/w ver dependent
  amd-xgbe: Delay AN timeout during KR training

 drivers/net/ethernet/amd/xgbe/xgbe-dev.c  | 23 ++++++++++++++--------
 drivers/net/ethernet/amd/xgbe/xgbe-mdio.c | 24 +++++++++++++++++++++++
 drivers/net/ethernet/amd/xgbe/xgbe.h      |  2 ++
 3 files changed, 41 insertions(+), 8 deletions(-)

-- 
2.25.1

There is difference in the TX Flow Control registers (TFCR) between the
revisions of the hardware. The older revisions of hardware used to have
single register per queue. Whereas, the newer revision of hardware (from
ver 30H onwards) have one register per priority.

Update the driver to use the TFCR based on the reported version of the
hardware.

Fixes: c5aa9e3b8156 ("amd-xgbe: Initial AMD 10GbE platform driver")
Co-developed-by: Ajith Nayak <redacted>
Signed-off-by: Ajith Nayak <redacted>
Signed-off-by: Raju Rangoju <Raju.Rangoju@amd.com>
Acked-by: Shyam Sundar S K <Shyam-sundar.S-k@amd.com>
---
 drivers/net/ethernet/amd/xgbe/xgbe-dev.c | 23 +++++++++++++++--------
 1 file changed, 15 insertions(+), 8 deletions(-)

diff --git a/drivers/net/ethernet/amd/xgbe/xgbe-dev.c b/drivers/net/ethernet/amd/xgbe/xgbe-dev.c
index 3936543a74d8..4030d619e84f 100644
--- a/drivers/net/ethernet/amd/xgbe/xgbe-dev.c
+++ b/drivers/net/ethernet/amd/xgbe/xgbe-dev.c

@@ -524,19 +524,28 @@ static void xgbe_disable_vxlan(struct xgbe_prv_data *pdata)
 	netif_dbg(pdata, drv, pdata->netdev, "VXLAN acceleration disabled\n");
 }
 
+static unsigned int xgbe_get_fc_queue_count(struct xgbe_prv_data *pdata)
+{
+	unsigned int max_q_count = XGMAC_MAX_FLOW_CONTROL_QUEUES;
+
+	/* From MAC ver 30H the TFCR is per priority, instead of per queue */
+	if (XGMAC_GET_BITS(pdata->hw_feat.version, MAC_VR, SNPSVER) >= 0x30)
+		return max_q_count;
+	else
+		return min_t(unsigned int, pdata->tx_q_count, max_q_count);
+}
+
 static int xgbe_disable_tx_flow_control(struct xgbe_prv_data *pdata)
 {
-	unsigned int max_q_count, q_count;
 	unsigned int reg, reg_val;
-	unsigned int i;
+	unsigned int i, q_count;
 
 	/* Clear MTL flow control */
 	for (i = 0; i < pdata->rx_q_count; i++)
 		XGMAC_MTL_IOWRITE_BITS(pdata, i, MTL_Q_RQOMR, EHFC, 0);
 
 	/* Clear MAC flow control */
-	max_q_count = XGMAC_MAX_FLOW_CONTROL_QUEUES;
-	q_count = min_t(unsigned int, pdata->tx_q_count, max_q_count);
+	q_count = xgbe_get_fc_queue_count(pdata);
 	reg = MAC_Q0TFCR;
 	for (i = 0; i < q_count; i++) {
 		reg_val = XGMAC_IOREAD(pdata, reg);

@@ -553,9 +562,8 @@ static int xgbe_enable_tx_flow_control(struct xgbe_prv_data *pdata)
 {
 	struct ieee_pfc *pfc = pdata->pfc;
 	struct ieee_ets *ets = pdata->ets;
-	unsigned int max_q_count, q_count;
 	unsigned int reg, reg_val;
-	unsigned int i;
+	unsigned int i, q_count;
 
 	/* Set MTL flow control */
 	for (i = 0; i < pdata->rx_q_count; i++) {

@@ -579,8 +587,7 @@ static int xgbe_enable_tx_flow_control(struct xgbe_prv_data *pdata)
 	}
 
 	/* Set MAC flow control */
-	max_q_count = XGMAC_MAX_FLOW_CONTROL_QUEUES;
-	q_count = min_t(unsigned int, pdata->tx_q_count, max_q_count);
+	q_count = xgbe_get_fc_queue_count(pdata);
 	reg = MAC_Q0TFCR;
 	for (i = 0; i < q_count; i++) {
 		reg_val = XGMAC_IOREAD(pdata, reg);

-- 
2.25.1

AN restart triggered during KR training not only aborts the KR training
process but also move the HW to unstable state. Driver has to wait upto
500ms or until the KR training is completed before restarting AN cycle.

Fixes: 7c12aa08779c ("amd-xgbe: Move the PHY support into amd-xgbe")
Co-developed-by: Sudheesh Mavila <redacted>
Signed-off-by: Sudheesh Mavila <redacted>
Signed-off-by: Raju Rangoju <Raju.Rangoju@amd.com>
Acked-by: Shyam Sundar S K <Shyam-sundar.S-k@amd.com>
---
 drivers/net/ethernet/amd/xgbe/xgbe-mdio.c | 24 +++++++++++++++++++++++
 drivers/net/ethernet/amd/xgbe/xgbe.h      |  2 ++
 2 files changed, 26 insertions(+)

diff --git a/drivers/net/ethernet/amd/xgbe/xgbe-mdio.c b/drivers/net/ethernet/amd/xgbe/xgbe-mdio.c
index 0c5c1b155683..43fdd111235a 100644
--- a/drivers/net/ethernet/amd/xgbe/xgbe-mdio.c
+++ b/drivers/net/ethernet/amd/xgbe/xgbe-mdio.c

@@ -496,6 +496,7 @@ static enum xgbe_an xgbe_an73_tx_training(struct xgbe_prv_data *pdata,
 	reg |= XGBE_KR_TRAINING_ENABLE;
 	reg |= XGBE_KR_TRAINING_START;
 	XMDIO_WRITE(pdata, MDIO_MMD_PMAPMD, MDIO_PMA_10GBR_PMD_CTRL, reg);
+	pdata->kr_start_time = jiffies;
 
 	netif_dbg(pdata, link, pdata->netdev,
 		  "KR training initiated\n");

@@ -632,6 +633,8 @@ static enum xgbe_an xgbe_an73_incompat_link(struct xgbe_prv_data *pdata)
 
 	xgbe_switch_mode(pdata);
 
+	pdata->an_result = XGBE_AN_READY;
+
 	xgbe_an_restart(pdata);
 
 	return XGBE_AN_INCOMPAT_LINK;

@@ -1275,9 +1278,30 @@ static bool xgbe_phy_aneg_done(struct xgbe_prv_data *pdata)
 static void xgbe_check_link_timeout(struct xgbe_prv_data *pdata)
 {
 	unsigned long link_timeout;
+	unsigned long kr_time;
+	int wait;
 
 	link_timeout = pdata->link_check + (XGBE_LINK_TIMEOUT * HZ);
 	if (time_after(jiffies, link_timeout)) {
+		if ((xgbe_cur_mode(pdata) == XGBE_MODE_KR) &&
+		    pdata->phy.autoneg == AUTONEG_ENABLE) {
+			/* AN restart should not happen while KR training is in progress.
+			 * The while loop ensures no AN restart during KR training,
+			 * waits up to 500ms and AN restart is triggered only if KR
+			 * training is failed.
+			 */
+			wait = XGBE_KR_TRAINING_WAIT_ITER;
+			while (wait--) {
+				kr_time = pdata->kr_start_time +
+					  msecs_to_jiffies(XGBE_AN_MS_TIMEOUT);
+				if (time_after(jiffies, kr_time))
+					break;
+				/* AN restart is not required, if AN result is COMPLETE */
+				if (pdata->an_result == XGBE_AN_COMPLETE)
+					return;
+				usleep_range(10000, 11000);
+			}
+		}
 		netif_dbg(pdata, link, pdata->netdev, "AN link timeout\n");
 		xgbe_phy_config_aneg(pdata);
 	}

diff --git a/drivers/net/ethernet/amd/xgbe/xgbe.h b/drivers/net/ethernet/amd/xgbe/xgbe.h
index 71f24cb47935..7a41367c437d 100644
--- a/drivers/net/ethernet/amd/xgbe/xgbe.h
+++ b/drivers/net/ethernet/amd/xgbe/xgbe.h

@@ -290,6 +290,7 @@
 /* Auto-negotiation */
 #define XGBE_AN_MS_TIMEOUT		500
 #define XGBE_LINK_TIMEOUT		5
+#define XGBE_KR_TRAINING_WAIT_ITER	50
 
 #define XGBE_SGMII_AN_LINK_STATUS	BIT(1)
 #define XGBE_SGMII_AN_LINK_SPEED	(BIT(2) | BIT(3))

@@ -1280,6 +1281,7 @@ struct xgbe_prv_data {
 	unsigned int parallel_detect;
 	unsigned int fec_ability;
 	unsigned long an_start;
+	unsigned long kr_start_time;
 	enum xgbe_an_mode an_mode;
 
 	/* I2C support */

-- 
2.25.1

On Wed, 11 Jan 2023 22:58:50 +0530 Raju Rangoju wrote:

0001 - There is difference in the TX Flow Control registers (TFCR)
between the revisions of the hardware. Update the driver to use the
TFCR based on the reported version of the hardware.

0002 - AN restart triggered during KR training not only aborts the KR
training process but also move the HW to unstable state. Add the
necessary changes to fix kr-taining.

Please err on the side of CCing people. Here the patches under Fixes
have Tom's sign off which makes our automation complain that he's not
CCed. No tag from him either.

On 1/12/2023 11:12 AM, Jakub Kicinski wrote:

On Wed, 11 Jan 2023 22:58:50 +0530 Raju Rangoju wrote:

quoted

0001 - There is difference in the TX Flow Control registers (TFCR)
between the revisions of the hardware. Update the driver to use the
TFCR based on the reported version of the hardware.

0002 - AN restart triggered during KR training not only aborts the KR
training process but also move the HW to unstable state. Add the
necessary changes to fix kr-taining.

Please err on the side of CCing people. Here the patches under Fixes
have Tom's sign off which makes our automation complain that he's not
CCed. No tag from him either.

I have put an Ack tag to the patches in the series, being the additional
maintainer for this driver. As Tom is busy working on other areas, I
shall be a single maintainer for this driver going forward.

I can submit a patch for the change to the MAINTAINERS file.

But would you mind pulling this series for now? Or would you like to see
the MAINTAINERS file getting updated first?

Thanks,
Shyam

On Thu, 12 Jan 2023 11:19:34 +0530 Shyam Sundar S K wrote:

I have put an Ack tag to the patches in the series, being the additional
maintainer for this driver. As Tom is busy working on other areas, I
shall be a single maintainer for this driver going forward.

I can submit a patch for the change to the MAINTAINERS file.

For Fixes of code that someone authored it'd be better if the person 
is CCed, as long as their email address still works. For net-next
material, if you're posting as a co-maintainer not CCing other
maintainers is no big deal.

But would you mind pulling this series for now? Or would you like to see
the MAINTAINERS file getting updated first?

Yes, no need to repost this one. Just something to keep in mind for 
the future.

On 1/13/2023 4:06 AM, Jakub Kicinski wrote:

On Thu, 12 Jan 2023 11:19:34 +0530 Shyam Sundar S K wrote:

quoted

I have put an Ack tag to the patches in the series, being the additional
maintainer for this driver. As Tom is busy working on other areas, I
shall be a single maintainer for this driver going forward.

I can submit a patch for the change to the MAINTAINERS file.

For Fixes of code that someone authored it'd be better if the person 
is CCed, as long as their email address still works. For net-next
material, if you're posting as a co-maintainer not CCing other
maintainers is no big deal.

Noted.

quoted

But would you mind pulling this series for now? Or would you like to see
the MAINTAINERS file getting updated first?

Yes, no need to repost this one. Just something to keep in mind for 
the future.

Thanks! Will take care of thie is future.

Thanks,
Shyam

On Wed, 11 Jan 2023 22:58:50 +0530 Raju Rangoju wrote:

This patch series fixes the issues in kr-training and pfc

Patches:

0001 - There is difference in the TX Flow Control registers (TFCR)
between the revisions of the hardware. Update the driver to use the
TFCR based on the reported version of the hardware.

0002 - AN restart triggered during KR training not only aborts the KR
training process but also move the HW to unstable state. Add the
necessary changes to fix kr-taining.

Applied, thanks!