[PATCH] atl1: disable 64bit DMA

Subsystems: networking drivers, the rest

STALE6986d

9 messages, 5 authors, 2007-06-27 · open the first message on its own page

[PATCH] atl1: disable 64bit DMA

From: Luca Tettamanti <kronos.it@gmail.com>
Date: 2007-06-25 21:19:17

Il Mon, Jun 25, 2007 at 07:42:44AM -0500, Jay Cliburn ha scritto: 
Jay L. T. Cornwall wrote:
quoted
Jay Cliburn wrote:
quoted
For reasons not yet clear to me, it appears the L1 driver has a bug or
the device itself has trouble with DMA in high memory.  This patch,
drafted by Luca Tettamanti, is being explored as a workaround.  I'd be
interested to know if it fixes your problem.
Yes, it certainly seems to. Now running with this patch and 4GB active,
I've transferred about 15GB with no problem so far. It usually oopses
after a GB or two.

I guess it's not an ideal solution, architecturally speaking, but it's a
good deal better than an unstable driver. If there's any other patches
you'd like me to test or traces to capture, I'm happy to help out.
Otherwise I'll run with this one for now since it does the job!
Okay Jay, thanks.

Luca, would you please submit your patch to Jeff Garzik and netdev?
Hi Jeff,
a couple of users reported hard lockups when using L1 NICs on machines
with 4GB or more of RAM. We're still waiting official confirmation from
the vendor, but it seems that L1 has problems doing DMA to/from high
memory (physical address above the 4GB limit). Passing 32bit DMA mask
cures the problem.

Signed-Off-By: Luca Tettamanti <kronos.it@gmail.com>

---
I think that the patch should be included in 2.6.22.

 drivers/net/atl1/atl1_main.c |   15 +++------------
 1 file changed, 3 insertions(+), 12 deletions(-)
diff --git a/drivers/net/atl1/atl1_main.c b/drivers/net/atl1/atl1_main.c
index 6862c11..a730f15 100644
--- a/drivers/net/atl1/atl1_main.c
+++ b/drivers/net/atl1/atl1_main.c
@@ -2097,21 +2097,16 @@ static int __devinit atl1_probe(struct pci_dev *pdev,
 	struct net_device *netdev;
 	struct atl1_adapter *adapter;
 	static int cards_found = 0;
-	bool pci_using_64 = true;
 	int err;
 
 	err = pci_enable_device(pdev);
 	if (err)
 		return err;
 
-	err = pci_set_dma_mask(pdev, DMA_64BIT_MASK);
+	err = pci_set_dma_mask(pdev, DMA_32BIT_MASK);
 	if (err) {
-		err = pci_set_dma_mask(pdev, DMA_32BIT_MASK);
-		if (err) {
-			dev_err(&pdev->dev, "no usable DMA configuration\n");
-			goto err_dma;
-		}
-		pci_using_64 = false;
+		dev_err(&pdev->dev, "no usable DMA configuration\n");
+		goto err_dma;
 	}
 	/* Mark all PCI regions associated with PCI device
 	 * pdev as being reserved by owner atl1_driver_name
@@ -2176,7 +2171,6 @@ static int __devinit atl1_probe(struct pci_dev *pdev,
 
 	netdev->ethtool_ops = &atl1_ethtool_ops;
 	adapter->bd_number = cards_found;
-	adapter->pci_using_64 = pci_using_64;
 
 	/* setup the private structure */
 	err = atl1_sw_init(adapter);
@@ -2193,9 +2187,6 @@ static int __devinit atl1_probe(struct pci_dev *pdev,
 	 */
 	/* netdev->features |= NETIF_F_TSO; */
 
-	if (pci_using_64)
-		netdev->features |= NETIF_F_HIGHDMA;
-
 	netdev->features |= NETIF_F_LLTX;
 
 	/*

Luca
-- 
Non ho ancora capito se il mio cane e` maschio o femmina:
quando fa la pipi` si chiude in bagno

Re: [PATCH] atl1: disable 64bit DMA

From: Chris Snook <hidden>
Date: 2007-06-25 21:36:54

Luca Tettamanti wrote:
quoted hunk
Il Mon, Jun 25, 2007 at 07:42:44AM -0500, Jay Cliburn ha scritto: 
quoted
Jay L. T. Cornwall wrote:
quoted
Jay Cliburn wrote:
quoted
For reasons not yet clear to me, it appears the L1 driver has a bug or
the device itself has trouble with DMA in high memory.  This patch,
drafted by Luca Tettamanti, is being explored as a workaround.  I'd be
interested to know if it fixes your problem.
Yes, it certainly seems to. Now running with this patch and 4GB active,
I've transferred about 15GB with no problem so far. It usually oopses
after a GB or two.

I guess it's not an ideal solution, architecturally speaking, but it's a
good deal better than an unstable driver. If there's any other patches
you'd like me to test or traces to capture, I'm happy to help out.
Otherwise I'll run with this one for now since it does the job!
Okay Jay, thanks.

Luca, would you please submit your patch to Jeff Garzik and netdev?
Hi Jeff,
a couple of users reported hard lockups when using L1 NICs on machines
with 4GB or more of RAM. We're still waiting official confirmation from
the vendor, but it seems that L1 has problems doing DMA to/from high
memory (physical address above the 4GB limit). Passing 32bit DMA mask
cures the problem.

Signed-Off-By: Luca Tettamanti <kronos.it@gmail.com>

---
I think that the patch should be included in 2.6.22.

 drivers/net/atl1/atl1_main.c |   15 +++------------
 1 file changed, 3 insertions(+), 12 deletions(-)
diff --git a/drivers/net/atl1/atl1_main.c b/drivers/net/atl1/atl1_main.c
index 6862c11..a730f15 100644
--- a/drivers/net/atl1/atl1_main.c
+++ b/drivers/net/atl1/atl1_main.c
@@ -2097,21 +2097,16 @@ static int __devinit atl1_probe(struct pci_dev *pdev,
 	struct net_device *netdev;
 	struct atl1_adapter *adapter;
 	static int cards_found = 0;
-	bool pci_using_64 = true;
 	int err;
 
 	err = pci_enable_device(pdev);
 	if (err)
 		return err;
 
-	err = pci_set_dma_mask(pdev, DMA_64BIT_MASK);
+	err = pci_set_dma_mask(pdev, DMA_32BIT_MASK);
 	if (err) {
-		err = pci_set_dma_mask(pdev, DMA_32BIT_MASK);
-		if (err) {
-			dev_err(&pdev->dev, "no usable DMA configuration\n");
-			goto err_dma;
-		}
-		pci_using_64 = false;
+		dev_err(&pdev->dev, "no usable DMA configuration\n");
+		goto err_dma;
 	}
 	/* Mark all PCI regions associated with PCI device
 	 * pdev as being reserved by owner atl1_driver_name
@@ -2176,7 +2171,6 @@ static int __devinit atl1_probe(struct pci_dev *pdev,
 
 	netdev->ethtool_ops = &atl1_ethtool_ops;
 	adapter->bd_number = cards_found;
-	adapter->pci_using_64 = pci_using_64;
 
 	/* setup the private structure */
 	err = atl1_sw_init(adapter);
@@ -2193,9 +2187,6 @@ static int __devinit atl1_probe(struct pci_dev *pdev,
 	 */
 	/* netdev->features |= NETIF_F_TSO; */
 
-	if (pci_using_64)
-		netdev->features |= NETIF_F_HIGHDMA;
-
 	netdev->features |= NETIF_F_LLTX;
 
 	/*

Luca
What boards have we seen this on?  It's quite possible this is:

a) an iommu-related problem specific to AMD or specific to Intel
b) a BIOS problem that atl1 happens to be a victim of

I'd rather not disable this unconditionally if we can get more 
information about why it's breaking.  Doing so might just end up 
covering up the most obvious manifestation of a larger problem.

	-- Chris

Re: [PATCH] atl1: disable 64bit DMA

From: Jay L. T. Cornwall <hidden>
Date: 2007-06-25 21:52:22

Chris Snook wrote:
What boards have we seen this on?  It's quite possible this is:
I can reproduce on an Asus P5K with a Core 2 Duo E6600.

lspci identifies the controller as:
  02:00.0 Ethernet controller: Attansic Technology Corp. L1 Gigabit
  Ethernet Adapter (rev b0)

dmesg notes the PCI-DMA mapping implementation:
  PCI-DMA: Using software bounce buffering for IO (SWIOTLB)

-- 
Jay L. T. Cornwall, http://www.esuna.co.uk/~jay/
PhD Student
Imperial College London

Re: [PATCH] atl1: disable 64bit DMA

From: Chris Snook <hidden>
Date: 2007-06-25 21:57:55

Jay L. T. Cornwall wrote:
Chris Snook wrote:
quoted
What boards have we seen this on?  It's quite possible this is:
I can reproduce on an Asus P5K with a Core 2 Duo E6600.

lspci identifies the controller as:
  02:00.0 Ethernet controller: Attansic Technology Corp. L1 Gigabit
  Ethernet Adapter (rev b0)

dmesg notes the PCI-DMA mapping implementation:
  PCI-DMA: Using software bounce buffering for IO (SWIOTLB)
I had a hunch this was on Intel.  I'd rather just disable this when swiotlb is 
in use, unless we get more complaints.  It's probably ultimately a BIOS quirk 
anyway.

	-- Chris

Re: [PATCH] atl1: disable 64bit DMA

From: Jeff Garzik <hidden>
Date: 2007-06-25 23:17:34

Jay Cliburn wrote:
On Mon, 25 Jun 2007 17:57:20 -0400
Chris Snook [off-list ref] wrote:
quoted
Jay L. T. Cornwall wrote:
quoted
Chris Snook wrote:
quoted
What boards have we seen this on?  It's quite possible this is:
I can reproduce on an Asus P5K with a Core 2 Duo E6600.

lspci identifies the controller as:
  02:00.0 Ethernet controller: Attansic Technology Corp. L1 Gigabit
  Ethernet Adapter (rev b0)

dmesg notes the PCI-DMA mapping implementation:
  PCI-DMA: Using software bounce buffering for IO (SWIOTLB)
I had a hunch this was on Intel.  I'd rather just disable this when
swiotlb is in use, unless we get more complaints.  It's probably
ultimately a BIOS quirk anyway.
So far we have reports from both camps:

Asus M2N8-VMX (AM2):	1 report of lockup
http://sourceforge.net/mailarchive/forum.php?thread_name=46780384.063603.26165%40m12-15.163.com&forum_name=atl1-devel

Asus P5K (LGA775):	2 reports of lockups
http://sourceforge.net/mailarchive/forum.php?thread_name=467E7E34.4010603%40gmail.com&forum_name=atl1-devel
http://lkml.org/lkml/2007/6/25/107

The common denominator in these reports is 4GB RAM.
Although its possible this device doesn't really support 64-bit, it's 
more likely that this is a platform problem of some sort, or a driver 
bug of some sort.  In the driver, maybe it has a problem when you 
-cross- a 4GB boundary, which is not uncommon.

	Jeff


Re: [PATCH] atl1: disable 64bit DMA

From: Chris Snook <hidden>
Date: 2007-06-25 23:40:40

Jeff Garzik wrote:
Jay Cliburn wrote:
quoted
On Mon, 25 Jun 2007 17:57:20 -0400
Chris Snook [off-list ref] wrote:
quoted
Jay L. T. Cornwall wrote:
quoted
Chris Snook wrote:
quoted
What boards have we seen this on?  It's quite possible this is:
I can reproduce on an Asus P5K with a Core 2 Duo E6600.

lspci identifies the controller as:
  02:00.0 Ethernet controller: Attansic Technology Corp. L1 Gigabit
  Ethernet Adapter (rev b0)

dmesg notes the PCI-DMA mapping implementation:
  PCI-DMA: Using software bounce buffering for IO (SWIOTLB)
I had a hunch this was on Intel.  I'd rather just disable this when
swiotlb is in use, unless we get more complaints.  It's probably
ultimately a BIOS quirk anyway.
So far we have reports from both camps:

Asus M2N8-VMX (AM2):    1 report of lockup
http://sourceforge.net/mailarchive/forum.php?thread_name=46780384.063603.26165%40m12-15.163.com&forum_name=atl1-devel 


Asus P5K (LGA775):    2 reports of lockups
http://sourceforge.net/mailarchive/forum.php?thread_name=467E7E34.4010603%40gmail.com&forum_name=atl1-devel 

http://lkml.org/lkml/2007/6/25/107

The common denominator in these reports is 4GB RAM.
Although its possible this device doesn't really support 64-bit, it's 
more likely that this is a platform problem of some sort, or a driver 
bug of some sort.  In the driver, maybe it has a problem when you 
-cross- a 4GB boundary, which is not uncommon.

    Jeff
I'm going on the record to say I don't trust the chipsets on these boards, and 
I'd like anyone having these problems to let us 
(atl1-devel@lists.sourceforge.net) know if they encounter similar problems with 
any other hardware.  That said, I'm not going to stand in the way of stability 
just because it *might* be someone else's fault.  I don't think limiting 
ourselves to dma32, at least while we track this down, is much of a loss on 
current hardware.

Acked-By: Chris Snook <redacted>

Re: [PATCH] atl1: disable 64bit DMA

From: Jay Cliburn <hidden>
Date: 2007-06-26 00:03:08

On Mon, 25 Jun 2007 17:57:20 -0400
Chris Snook [off-list ref] wrote:
Jay L. T. Cornwall wrote:
quoted
Chris Snook wrote:
quoted
What boards have we seen this on?  It's quite possible this is:
I can reproduce on an Asus P5K with a Core 2 Duo E6600.

lspci identifies the controller as:
  02:00.0 Ethernet controller: Attansic Technology Corp. L1 Gigabit
  Ethernet Adapter (rev b0)

dmesg notes the PCI-DMA mapping implementation:
  PCI-DMA: Using software bounce buffering for IO (SWIOTLB)
I had a hunch this was on Intel.  I'd rather just disable this when
swiotlb is in use, unless we get more complaints.  It's probably
ultimately a BIOS quirk anyway.
So far we have reports from both camps:

Asus M2N8-VMX (AM2):	1 report of lockup
http://sourceforge.net/mailarchive/forum.php?thread_name=46780384.063603.26165%40m12-15.163.com&forum_name=atl1-devel

Asus P5K (LGA775):	2 reports of lockups
http://sourceforge.net/mailarchive/forum.php?thread_name=467E7E34.4010603%40gmail.com&forum_name=atl1-devel
http://lkml.org/lkml/2007/6/25/107

The common denominator in these reports is 4GB RAM.

Re: [PATCH] atl1: disable 64bit DMA

From: Luca <kronos.it@gmail.com>
Date: 2007-06-26 21:12:55

On 6/26/07, Jeff Garzik [off-list ref] wrote:
Jay Cliburn wrote:
quoted
On Mon, 25 Jun 2007 17:57:20 -0400
Chris Snook [off-list ref] wrote:
quoted
Jay L. T. Cornwall wrote:
quoted
Chris Snook wrote:
quoted
What boards have we seen this on?  It's quite possible this is:
I can reproduce on an Asus P5K with a Core 2 Duo E6600.

lspci identifies the controller as:
  02:00.0 Ethernet controller: Attansic Technology Corp. L1 Gigabit
  Ethernet Adapter (rev b0)

dmesg notes the PCI-DMA mapping implementation:
  PCI-DMA: Using software bounce buffering for IO (SWIOTLB)
I had a hunch this was on Intel.  I'd rather just disable this when
swiotlb is in use, unless we get more complaints.  It's probably
ultimately a BIOS quirk anyway.
So far we have reports from both camps:

Asus M2N8-VMX (AM2):  1 report of lockup
http://sourceforge.net/mailarchive/forum.php?thread_name=46780384.063603.26165%40m12-15.163.com&forum_name=atl1-devel

Asus P5K (LGA775):    2 reports of lockups
http://sourceforge.net/mailarchive/forum.php?thread_name=467E7E34.4010603%40gmail.com&forum_name=atl1-devel
http://lkml.org/lkml/2007/6/25/107

The common denominator in these reports is 4GB RAM.
Although its possible this device doesn't really support 64-bit, it's
more likely that this is a platform problem of some sort, or a driver
bug of some sort.  In the driver, maybe it has a problem when you
-cross- a 4GB boundary, which is not uncommon.
I don't follow you :| What kind "common" mistakes should we check for
in the driver?

Luca

Re: [PATCH] atl1: disable 64bit DMA

From: Jay Cliburn <hidden>
Date: 2007-06-27 00:19:19

On Mon, 25 Jun 2007 23:18:55 +0200
Luca Tettamanti [off-list ref] wrote:
quoted hunk
Il Mon, Jun 25, 2007 at 07:42:44AM -0500, Jay Cliburn ha scritto: 
quoted
Jay L. T. Cornwall wrote:
quoted
Jay Cliburn wrote:
quoted
For reasons not yet clear to me, it appears the L1 driver has a
bug or the device itself has trouble with DMA in high memory.
This patch, drafted by Luca Tettamanti, is being explored as a
workaround.  I'd be interested to know if it fixes your problem.
Yes, it certainly seems to. Now running with this patch and 4GB
active, I've transferred about 15GB with no problem so far. It
usually oopses after a GB or two.

I guess it's not an ideal solution, architecturally speaking, but
it's a good deal better than an unstable driver. If there's any
other patches you'd like me to test or traces to capture, I'm
happy to help out. Otherwise I'll run with this one for now since
it does the job!
Okay Jay, thanks.

Luca, would you please submit your patch to Jeff Garzik and netdev?
Hi Jeff,
a couple of users reported hard lockups when using L1 NICs on machines
with 4GB or more of RAM. We're still waiting official confirmation
from the vendor, but it seems that L1 has problems doing DMA to/from
high memory (physical address above the 4GB limit). Passing 32bit DMA
mask cures the problem.

Signed-Off-By: Luca Tettamanti <kronos.it@gmail.com>

---
I think that the patch should be included in 2.6.22.

 drivers/net/atl1/atl1_main.c |   15 +++------------
 1 file changed, 3 insertions(+), 12 deletions(-)
diff --git a/drivers/net/atl1/atl1_main.c
b/drivers/net/atl1/atl1_main.c index 6862c11..a730f15 100644
--- a/drivers/net/atl1/atl1_main.c
+++ b/drivers/net/atl1/atl1_main.c
@@ -2097,21 +2097,16 @@ static int __devinit atl1_probe(struct
pci_dev *pdev, struct net_device *netdev;
 	struct atl1_adapter *adapter;
 	static int cards_found = 0;
-	bool pci_using_64 = true;
 	int err;
 
 	err = pci_enable_device(pdev);
 	if (err)
 		return err;
 
-	err = pci_set_dma_mask(pdev, DMA_64BIT_MASK);
+	err = pci_set_dma_mask(pdev, DMA_32BIT_MASK);
 	if (err) {
-		err = pci_set_dma_mask(pdev, DMA_32BIT_MASK);
-		if (err) {
-			dev_err(&pdev->dev, "no usable DMA
configuration\n");
-			goto err_dma;
-		}
-		pci_using_64 = false;
+		dev_err(&pdev->dev, "no usable DMA configuration\n");
+		goto err_dma;
 	}
 	/* Mark all PCI regions associated with PCI device
 	 * pdev as being reserved by owner atl1_driver_name
@@ -2176,7 +2171,6 @@ static int __devinit atl1_probe(struct pci_dev
*pdev, 
 	netdev->ethtool_ops = &atl1_ethtool_ops;
 	adapter->bd_number = cards_found;
-	adapter->pci_using_64 = pci_using_64;
 
 	/* setup the private structure */
 	err = atl1_sw_init(adapter);
@@ -2193,9 +2187,6 @@ static int __devinit atl1_probe(struct pci_dev
*pdev, */
 	/* netdev->features |= NETIF_F_TSO; */
 
-	if (pci_using_64)
-		netdev->features |= NETIF_F_HIGHDMA;
-
 	netdev->features |= NETIF_F_LLTX;
 
 	/*
Acked-by: Jay Cliburn <redacted>
Keyboard shortcuts
hback out one level
jnext message in thread
kprevious message in thread
ldrill in
Escclose help / fold thread tree
?toggle this help