net: macb: Align the dma and coherent dma masks

From: Marc St-Amand <redacted>

Single page and coherent memory blocks can use different DMA masks
when the macb accesses physical memory directly. The kernel is clever
enough to allocate pages that fit into the requested address width.

When using the ARM SMMU, the DMA mask must be the same for single
pages and big coherent memory blocks. Otherwise the translation
tables turn into one big mess.

  [   74.959909] macb ff0e0000.ethernet eth0: DMA bus error: HRESP not OK
  [   74.959989] arm-smmu fd800000.smmu: Unhandled context fault: fsr=0x402, iova=0x3165687460, fsynr=0x20001, cbfrsynra=0x877, cb=1
  [   75.173939] macb ff0e0000.ethernet eth0: DMA bus error: HRESP not OK
  [   75.173955] arm-smmu fd800000.smmu: Unhandled context fault: fsr=0x402, iova=0x3165687460, fsynr=0x20001, cbfrsynra=0x877, cb=1

Since using the same DMA mask does not hurt direct 1:1 physical
memory mappings, this commit always aligns DMA and coherent masks.

Signed-off-by: Marc St-Amand <redacted>
Signed-off-by: Harini Katakam <redacted>
---
 drivers/net/ethernet/cadence/macb_main.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/net/ethernet/cadence/macb_main.c b/drivers/net/ethernet/cadence/macb_main.c
index 1ce20bf52f72..4c231159b562 100644
--- a/drivers/net/ethernet/cadence/macb_main.c
+++ b/drivers/net/ethernet/cadence/macb_main.c

@@ -4765,7 +4765,7 @@ static int macb_probe(struct platform_device *pdev)
 
 #ifdef CONFIG_ARCH_DMA_ADDR_T_64BIT
 	if (GEM_BFEXT(DAW64, gem_readl(bp, DCFG6))) {
-		dma_set_mask(&pdev->dev, DMA_BIT_MASK(44));
+		dma_set_mask_and_coherent(&pdev->dev, DMA_BIT_MASK(44));
 		bp->hw_dma_cap |= HW_DMA_CAP_64B;
 	}
 #endif

-- 
2.17.1

On 09/02/2022 at 10:43, Harini Katakam wrote:

EXTERNAL EMAIL: Do not click links or open attachments unless you know the content is safe

From: Marc St-Amand <redacted>

Single page and coherent memory blocks can use different DMA masks
when the macb accesses physical memory directly. The kernel is clever
enough to allocate pages that fit into the requested address width.

When using the ARM SMMU, the DMA mask must be the same for single
pages and big coherent memory blocks. Otherwise the translation
tables turn into one big mess.

   [   74.959909] macb ff0e0000.ethernet eth0: DMA bus error: HRESP not OK
   [   74.959989] arm-smmu fd800000.smmu: Unhandled context fault: fsr=0x402, iova=0x3165687460, fsynr=0x20001, cbfrsynra=0x877, cb=1
   [   75.173939] macb ff0e0000.ethernet eth0: DMA bus error: HRESP not OK
   [   75.173955] arm-smmu fd800000.smmu: Unhandled context fault: fsr=0x402, iova=0x3165687460, fsynr=0x20001, cbfrsynra=0x877, cb=1

Since using the same DMA mask does not hurt direct 1:1 physical
memory mappings, this commit always aligns DMA and coherent masks.

Signed-off-by: Marc St-Amand <redacted>
Signed-off-by: Harini Katakam <redacted>

Ok:
Acked-by: Nicolas Ferre <nicolas.ferre@microchip.com>

Best regards,
  Nicolas

-- 
Nicolas Ferre

EXTERNAL EMAIL: Do not click links or open attachments unless you know the content is safe

From: Marc St-Amand <redacted>

Single page and coherent memory blocks can use different DMA masks
when the macb accesses physical memory directly. The kernel is clever
enough to allocate pages that fit into the requested address width.

When using the ARM SMMU, the DMA mask must be the same for single
pages and big coherent memory blocks. Otherwise the translation
tables turn into one big mess.

  [   74.959909] macb ff0e0000.ethernet eth0: DMA bus error: HRESP not OK
  [   74.959989] arm-smmu fd800000.smmu: Unhandled context fault: fsr=0x402, iova=0x3165687460, fsynr=0x20001, cbfrsynra=0x877, cb=1
  [   75.173939] macb ff0e0000.ethernet eth0: DMA bus error: HRESP not OK
  [   75.173955] arm-smmu fd800000.smmu: Unhandled context fault: fsr=0x402, iova=0x3165687460, fsynr=0x20001, cbfrsynra=0x877, cb=1

Since using the same DMA mask does not hurt direct 1:1 physical
memory mappings, this commit always aligns DMA and coherent masks.

Signed-off-by: Marc St-Amand <redacted>
Signed-off-by: Harini Katakam <redacted>

Tested-by: Conor Dooley <conor.dooley@microchip.com>

Fixes a DMA allocation failure for me on a non arm board, seen if
I only assigned DDR to 64 bit addresses. Without the change I was
getting the following page allocation error:

Starting network:
[    2.911830] ip: page allocation failure: order:2, mode:0xcc4(GFP_KERNEL|GFP_DMA32),nodemask=(null)
[    2.921256] CPU: 3 PID: 171 Comm: ip Not tainted 5.17.0-rc1-00640-g6cc001edd9ad #1
[    2.928915] Hardware name: Microchip PolarFire-SoC Icicle Kit (DT)
[    2.935147] Call Trace:
[    2.937626] [<ffffffff800047e0>] dump_backtrace+0x1c/0x24
[    2.943087] [<ffffffff807fcfce>] dump_stack_lvl+0x40/0x58
[    2.948547] [<ffffffff807fcffa>] dump_stack+0x14/0x1c
[    2.953649] [<ffffffff80104bfc>] warn_alloc+0xd6/0x14c
[    2.958849] [<ffffffff801053e0>] __alloc_pages_slowpath.constprop.0+0x76e/0x882
[    2.966242] [<ffffffff80105618>] __alloc_pages+0x124/0x174
[    2.971783] [<ffffffff80061440>] __dma_direct_alloc_pages+0x12c/0x28c
[    2.978286] [<ffffffff800616f2>] dma_direct_alloc+0x40/0x13e
[    2.983996] [<ffffffff80060bf2>] dma_alloc_attrs+0x78/0x86
[    2.989541] [<ffffffff805cdfb6>] macb_open+0x84/0x42c
[    2.994645] [<ffffffff806e2468>] __dev_open+0xb0/0x142
[    2.999845] [<ffffffff806e2884>] __dev_change_flags+0x180/0x1ec
[    3.005827] [<ffffffff806e290e>] dev_change_flags+0x1e/0x54
[    3.011461] [<ffffffff8076960e>] devinet_ioctl+0x1fc/0x612
[    3.017015] [<ffffffff8076b27e>] inet_ioctl+0x96/0xfa
[    3.022109] [<ffffffff806bf5e2>] sock_ioctl+0x256/0x29e
[    3.027396] [<ffffffff8012bc7c>] sys_ioctl+0x340/0x7f8
[    3.032595] [<ffffffff80003014>] ret_from_syscall+0x0/0x2
<snip>
[    3.176712] macb 20110000.ethernet eth0: Unable to allocate DMA memory (error -12)
ip: SIOCSIFFLAGS: Cannot allocate memory
FAIL

Hello:

This patch was applied to netdev/net.git (master)
by David S. Miller [off-list ref]:

On Wed, 9 Feb 2022 15:13:25 +0530 you wrote:

From: Marc St-Amand <redacted>

Single page and coherent memory blocks can use different DMA masks
when the macb accesses physical memory directly. The kernel is clever
enough to allocate pages that fit into the requested address width.

When using the ARM SMMU, the DMA mask must be the same for single
pages and big coherent memory blocks. Otherwise the translation
tables turn into one big mess.

[...]

Here is the summary with links:
  - net: macb: Align the dma and coherent dma masks
    https://git.kernel.org/netdev/net/c/37f7860602b5

You are awesome, thank you!
-- 
Deet-doot-dot, I am a bot.
https://korg.docs.kernel.org/patchwork/pwbot.html