[PATCH] stmmac: reduce queue lengths if MTU is very large

From: Alessandro Rubini <redacted>

In my use case (STA2X11 IO-Hub), the alloc_coherent, map_single and
friends map to a reserved area set up in swiotlb.  With the default
MTU each packet takes 2kB, but with jumbo frames it increases to 16kB
and consumes up my area (it takes 8MB instead of 1MB).

Instead of increasing the reserved (== wasted) preallocated area in
swiotlb, this patch rescales the user-selected queue lengths if the MTU
is modified at runtime to be larger than the default. User choices
are still obeyed (compiled default, module parameter, command line)

Signed-off-by: Alessandro Rubini <redacted>
Acked-by: Giancarlo Asnaghi <redacted>
Acked-by: Giuseppe Cavallaro <redacted>
---
 drivers/net/ethernet/stmicro/stmmac/stmmac_main.c |   19 ++++++++++++++++---
 1 files changed, 16 insertions(+), 3 deletions(-)

diff --git a/drivers/net/ethernet/stmicro/stmmac/stmmac_main.c b/drivers/net/ethernet/stmicro/stmmac/stmmac_main.c
index 72cd190..e8ebb6e 100644
--- a/drivers/net/ethernet/stmicro/stmmac/stmmac_main.c
+++ b/drivers/net/ethernet/stmicro/stmmac/stmmac_main.c

@@ -413,9 +413,8 @@ static void init_dma_desc_rings(struct net_device *dev)
 	int i;
 	struct stmmac_priv *priv = netdev_priv(dev);
 	struct sk_buff *skb;
-	unsigned int txsize = priv->dma_tx_size;
-	unsigned int rxsize = priv->dma_rx_size;
-	unsigned int bfsize;
+	unsigned int txsize, rxsize;
+	unsigned int bfsize = priv->dma_buf_sz;
 	int dis_ic = 0;
 	int des3_as_data_buf = 0;
 

@@ -428,6 +427,20 @@ static void init_dma_desc_rings(struct net_device *dev)
 	else
 		bfsize = stmmac_set_bfsize(dev->mtu, priv->dma_buf_sz);
 
+	/*
+	 * If we use jumbo frames, we end up consuming too much memory
+	 * in the coherent buffer, so reduce the queue lenght according
+	 * to the increase in frame size over the default.
+	 */
+	i = bfsize / BUF_SIZE_2KiB;
+	if (i > 1) {
+		pr_warning("%s: reducing queue length by %i\n", __func__, i);
+		priv->dma_tx_size /= i;
+		priv->dma_rx_size /= i;
+	}
+	txsize = priv->dma_tx_size;
+	rxsize = priv->dma_rx_size;
+
 #ifdef CONFIG_STMMAC_TIMER
 	/* Disable interrupts on completion for the reception if timer is on */
 	if (likely(priv->tm->enable))

-- 
1.7.4.4

From: Giuseppe CAVALLARO <redacted>
Date: Mon, 19 Dec 2011 06:45:30 +0100

From: Alessandro Rubini <redacted>

In my use case (STA2X11 IO-Hub), the alloc_coherent, map_single and
friends map to a reserved area set up in swiotlb.  With the default
MTU each packet takes 2kB, but with jumbo frames it increases to 16kB
and consumes up my area (it takes 8MB instead of 1MB).

Instead of increasing the reserved (== wasted) preallocated area in
swiotlb, this patch rescales the user-selected queue lengths if the MTU
is modified at runtime to be larger than the default. User choices
are still obeyed (compiled default, module parameter, command line)

Signed-off-by: Alessandro Rubini <redacted>
Acked-by: Giancarlo Asnaghi <redacted>
Acked-by: Giuseppe Cavallaro <redacted>

This change doesn't make sense.

If the user is only sending/receiving small frames, then this patch
will hurt them.

I'm not applying this, it just points out that the amount of reserved
space in your SWIOMMU setup might be insufficient for this situation
rather than something that should be tweaked in this driver.

This change doesn't make sense.

I disagree, but you are the boss here, so I accept that.

I'm not applying this, it just points out that the amount of reserved
space in your SWIOMMU setup might be insufficient for this situation
rather than something that should be tweaked in this driver.

Allocation of the queues is based on a parameter. If the parameter was
a size, instead of a lenght, all would be fine, wouldn't it? If you
agree, then this patch is turning the lenght into a size, in a way.

Now, how could we deal with the fact that the size explodes if the MTU
is increased?  I don't want to reserve and waste 16MB of RAM for a
single device: the device won't even use them for non-jumbo setups.  I
can use the parameter to shorten the queue in all cases, but this is
not good either.

What's your suggestion?

thanks
/alessandro

From: Alessandro Rubini <redacted>
Date: Tue, 20 Dec 2011 08:30:50 +0100

Allocation of the queues is based on a parameter. If the parameter was
a size, instead of a lenght, all would be fine, wouldn't it? If you
agree, then this patch is turning the lenght into a size, in a way.

That doesn't make any sense either, queues are sized by TX descriptor
slots not bytes.

Now, how could we deal with the fact that the size explodes if the MTU
is increased?

Maybe... set the initial parameter appropriately?  This is enormously
better than fixing it up after the fact.

Maybe I set the number of TX entries based upon my expected packet
load of small packets.  If I happen to set my MTU to 9000 that's
irrelevant to the setting.