ixp4xx eth broken in 3.7.0/3.8-rc5?

Mikael Pettersson writes:
 > When I try to update my ixp4xx machine to a 3.8-rc5 kernel it boots
 > Ok but the network (CONFIG_IXP4XX_ETH=y) fails to come up, with the
 > following in the kernel log:
 > 
 > net eth0: coherent DMA mask is unset
 > 
 > With a 3.7.0 kernel the situation is similar, except that the "net
 > eth0: coherent DMA mask is unset" message repeats itself endlessly
 > and I can't even get in on the serial console.
 > 
 > With the 3.6.0 kernel everything works perfectly.

A git bisect identified the following as the culprit:

 > From 1a4901177574083c35fafc24c4d151c2a7c7647c Mon Sep 17 00:00:00 2001
 > From: Xi Wang [off-list ref]
 > Date: Sat, 17 Nov 2012 20:25:09 +0000
 > Subject: [PATCH] ixp4xx_eth: avoid calling dma_pool_create() with NULL dev
 > 
 > Use &port->netdev->dev instead of NULL since dma_pool_create() doesn't
 > allow NULL dev.
 > 
 > Signed-off-by: Xi Wang [off-list ref]
 > Cc: Andrew Morton [off-list ref]
 > Signed-off-by: David S. Miller [off-list ref]
 > ---
 >  drivers/net/ethernet/xscale/ixp4xx_eth.c |    8 +++++---
 >  1 files changed, 5 insertions(+), 3 deletions(-)

Reverting this unbreaks ixp4xx_eth on my ixp4xx machine with kernels
3.7.0 and 3.8-rc5.

(Added cc: author and netdev.)

/Mikael

On 1/29/13 7:11 PM, Mikael Pettersson wrote:

A git bisect identified the following as the culprit:

  > From 1a4901177574083c35fafc24c4d151c2a7c7647c Mon Sep 17 00:00:00 2001
  > From: Xi Wang [off-list ref]
  > Date: Sat, 17 Nov 2012 20:25:09 +0000
  > Subject: [PATCH] ixp4xx_eth: avoid calling dma_pool_create() with NULL dev
  >
  > Use &port->netdev->dev instead of NULL since dma_pool_create() doesn't
  > allow NULL dev.
  >
  > Signed-off-by: Xi Wang [off-list ref]
  > Cc: Andrew Morton [off-list ref]
  > Signed-off-by: David S. Miller [off-list ref]
  > ---
  >  drivers/net/ethernet/xscale/ixp4xx_eth.c |    8 +++++---
  >  1 files changed, 5 insertions(+), 3 deletions(-)

Reverting this unbreaks ixp4xx_eth on my ixp4xx machine with kernels
3.7.0 and 3.8-rc5.

Thanks.  The problem was that in init_queues(), the dma_pool_create()
call requires a non-null dev.

  dma_pool = dma_pool_create(DRV_NAME, ??, ...);

What do you think would work here?

On Tue, 2013-01-29 at 19:32 -0500, Xi Wang wrote:

On 1/29/13 7:11 PM, Mikael Pettersson wrote:

quoted

A git bisect identified the following as the culprit:

  > From 1a4901177574083c35fafc24c4d151c2a7c7647c Mon Sep 17 00:00:00 2001
  > From: Xi Wang [off-list ref]
  > Date: Sat, 17 Nov 2012 20:25:09 +0000
  > Subject: [PATCH] ixp4xx_eth: avoid calling dma_pool_create() with NULL dev
  >
  > Use &port->netdev->dev instead of NULL since dma_pool_create() doesn't
  > allow NULL dev.
  >
  > Signed-off-by: Xi Wang [off-list ref]
  > Cc: Andrew Morton [off-list ref]
  > Signed-off-by: David S. Miller [off-list ref]
  > ---
  >  drivers/net/ethernet/xscale/ixp4xx_eth.c |    8 +++++---
  >  1 files changed, 5 insertions(+), 3 deletions(-)

Reverting this unbreaks ixp4xx_eth on my ixp4xx machine with kernels
3.7.0 and 3.8-rc5.

Thanks.  The problem was that in init_queues(), the dma_pool_create()
call requires a non-null dev.

  dma_pool = dma_pool_create(DRV_NAME, ??, ...);

What do you think would work here?

For DMA purposes I would expect the right device to be the parent bus
device, i.e. port->netdev->dev.parent.  The driver might also need to
set its DMA masks.

Ben.

-- 
Ben Hutchings, Staff Engineer, Solarflare
Not speaking for my employer; that's the marketing department's job.
They asked us to note that Solarflare product names are trademarked.

Ben Hutchings writes:
 > On Tue, 2013-01-29 at 19:32 -0500, Xi Wang wrote:
 > > On 1/29/13 7:11 PM, Mikael Pettersson wrote:
 > > > A git bisect identified the following as the culprit:
 > > > 
 > > >   > From 1a4901177574083c35fafc24c4d151c2a7c7647c Mon Sep 17 00:00:00 2001
 > > >   > From: Xi Wang [off-list ref]
 > > >   > Date: Sat, 17 Nov 2012 20:25:09 +0000
 > > >   > Subject: [PATCH] ixp4xx_eth: avoid calling dma_pool_create() with NULL dev
 > > >   >
 > > >   > Use &port->netdev->dev instead of NULL since dma_pool_create() doesn't
 > > >   > allow NULL dev.
 > > >   >
 > > >   > Signed-off-by: Xi Wang [off-list ref]
 > > >   > Cc: Andrew Morton [off-list ref]
 > > >   > Signed-off-by: David S. Miller [off-list ref]
 > > >   > ---
 > > >   >  drivers/net/ethernet/xscale/ixp4xx_eth.c |    8 +++++---
 > > >   >  1 files changed, 5 insertions(+), 3 deletions(-)
 > > > 
 > > > Reverting this unbreaks ixp4xx_eth on my ixp4xx machine with kernels
 > > > 3.7.0 and 3.8-rc5.
 > > 
 > > Thanks.  The problem was that in init_queues(), the dma_pool_create()
 > > call requires a non-null dev.
 > > 
 > >   dma_pool = dma_pool_create(DRV_NAME, ??, ...);

I had a look around in the 3.8-rc5 kernel source.

mm/dmapool.c:dma_pool_create() needs a non-NULL dev in the CONFIG_NUMA case,
because it calls dev_to_node(dev) which dereferences dev if CONFIG_NUMA.
But in the non-CONFIG_NUMA case there is no dereference of dev, and in fact
there is an explicit check for dev being NULL or not towards the end of
dma_pool_create().  So that explains why ixp4xx works without the change:
it's not NUMA.

A quick grep found three calls to dma_pool_create() with a NULL dev parameter,
two in drivers/usb/gadget/amd5536udc.c and one in arch/arm/mach-s3c64xx/dma.c.
These would all be broken with CONFIG_NUMA but Ok otherwise.

 > > 
 > > What do you think would work here?
 > 
 > For DMA purposes I would expect the right device to be the parent bus
 > device, i.e. port->netdev->dev.parent.  The driver might also need to
 > set its DMA masks.

At least on ixp4xx the DMA masks are normally set by the platform PCI init
code (it has a 64MB PCI DMA window limitation).

I can give the port->netdev->dev.parent variant a try later this evening.

/Mikael

Mikael Pettersson writes:
 > Ben Hutchings writes:
 >  > On Tue, 2013-01-29 at 19:32 -0500, Xi Wang wrote:
 >  > > On 1/29/13 7:11 PM, Mikael Pettersson wrote:
 >  > > > A git bisect identified the following as the culprit:
 >  > > > 
 >  > > >   > From 1a4901177574083c35fafc24c4d151c2a7c7647c Mon Sep 17 00:00:00 2001
 >  > > >   > From: Xi Wang [off-list ref]
 >  > > >   > Date: Sat, 17 Nov 2012 20:25:09 +0000
 >  > > >   > Subject: [PATCH] ixp4xx_eth: avoid calling dma_pool_create() with NULL dev
 >  > > >   >
 >  > > >   > Use &port->netdev->dev instead of NULL since dma_pool_create() doesn't
 >  > > >   > allow NULL dev.
 >  > > >   >
 >  > > >   > Signed-off-by: Xi Wang [off-list ref]
 >  > > >   > Cc: Andrew Morton [off-list ref]
 >  > > >   > Signed-off-by: David S. Miller [off-list ref]
 >  > > >   > ---
 >  > > >   >  drivers/net/ethernet/xscale/ixp4xx_eth.c |    8 +++++---
 >  > > >   >  1 files changed, 5 insertions(+), 3 deletions(-)
 >  > > > 
 >  > > > Reverting this unbreaks ixp4xx_eth on my ixp4xx machine with kernels
 >  > > > 3.7.0 and 3.8-rc5.
 >  > > 
 >  > > Thanks.  The problem was that in init_queues(), the dma_pool_create()
 >  > > call requires a non-null dev.
 >  > > 
 >  > >   dma_pool = dma_pool_create(DRV_NAME, ??, ...);
 > 
 > I had a look around in the 3.8-rc5 kernel source.
 > 
 > mm/dmapool.c:dma_pool_create() needs a non-NULL dev in the CONFIG_NUMA case,
 > because it calls dev_to_node(dev) which dereferences dev if CONFIG_NUMA.
 > But in the non-CONFIG_NUMA case there is no dereference of dev, and in fact
 > there is an explicit check for dev being NULL or not towards the end of
 > dma_pool_create().  So that explains why ixp4xx works without the change:
 > it's not NUMA.
 > 
 > A quick grep found three calls to dma_pool_create() with a NULL dev parameter,
 > two in drivers/usb/gadget/amd5536udc.c and one in arch/arm/mach-s3c64xx/dma.c.
 > These would all be broken with CONFIG_NUMA but Ok otherwise.
 > 
 >  > > 
 >  > > What do you think would work here?
 >  > 
 >  > For DMA purposes I would expect the right device to be the parent bus
 >  > device, i.e. port->netdev->dev.parent.  The driver might also need to
 >  > set its DMA masks.
 > 
 > At least on ixp4xx the DMA masks are normally set by the platform PCI init
 > code (it has a 64MB PCI DMA window limitation).
 > 
 > I can give the port->netdev->dev.parent variant a try later this evening.

No joy, s/&port->netdev->dev/port->netdev->dev.parent/ in init_queues' call
to dma_pool_create made no difference, it still fails.

/Mikael