[PATCH net-next] mlx4: change TX coalescing defaults

Subsystems: mellanox ethernet driver (mlx4_en), mellanox mlx4 core vpi driver, networking drivers, the rest

STALE5053d

6 messages, 4 authors, 2012-11-07 · open the first message on its own page

[PATCH net-next] mlx4: change TX coalescing defaults

From: Eric Dumazet <hidden>
Date: 2012-11-06 02:20:45

From: Eric Dumazet <edumazet@google.com>

mlx4 currently uses a too high tx coalescing setting, deferring
TX completion interrupts by up to 128 us.

With the recent skb_orphan() removal in commit 8112ec3b872,
performance of a single TCP flow is capped to ~4 Gbps, unless
we increase tcp_limit_output_bytes.

I suggest using 16 us instead of 128 us, allowing a finer control.

Performance of a single TCP flow is restored to previous levels,
while keeping TCP small queues fully enabled with default sysctl.

This patch is also a BQL prereq.

Reported-by: Vimalkumar <redacted>
Signed-off-by: Eric Dumazet <edumazet@google.com>
Cc: Yevgeny Petrilin <redacted>
Cc: Or Gerlitz <redacted>
---
By the way, is BQL addition planned for this driver ?

 drivers/net/ethernet/mellanox/mlx4/en_netdev.c |    2 +-
 drivers/net/ethernet/mellanox/mlx4/mlx4_en.h   |    2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/drivers/net/ethernet/mellanox/mlx4/en_netdev.c b/drivers/net/ethernet/mellanox/mlx4/en_netdev.c
index edd9cb8..2b23ca2 100644
--- a/drivers/net/ethernet/mellanox/mlx4/en_netdev.c
+++ b/drivers/net/ethernet/mellanox/mlx4/en_netdev.c

@@ -870,7 +870,7 @@ static void mlx4_en_set_default_moderation(struct mlx4_en_priv *priv)
 	/* If we haven't received a specific coalescing setting
 	 * (module param), we set the moderation parameters as follows:
 	 * - moder_cnt is set to the number of mtu sized packets to
-	 *   satisfy our coelsing target.
+	 *   satisfy our coalescing target.
 	 * - moder_time is set to a fixed value.
 	 */
 	priv->rx_frames = MLX4_EN_RX_COAL_TARGET;

diff --git a/drivers/net/ethernet/mellanox/mlx4/mlx4_en.h b/drivers/net/ethernet/mellanox/mlx4/mlx4_en.h
index 9d27e42..8a5e70d 100644
--- a/drivers/net/ethernet/mellanox/mlx4/mlx4_en.h
+++ b/drivers/net/ethernet/mellanox/mlx4/mlx4_en.h

@@ -126,7 +126,7 @@ enum {
 #define MLX4_EN_RX_COAL_TIME	0x10
 
 #define MLX4_EN_TX_COAL_PKTS	16
-#define MLX4_EN_TX_COAL_TIME	0x80
+#define MLX4_EN_TX_COAL_TIME	0x10
 
 #define MLX4_EN_RX_RATE_LOW		400000
 #define MLX4_EN_RX_COAL_TIME_LOW	0

Re: [PATCH net-next] mlx4: change TX coalescing defaults

From: Amir Mellanox <hidden>
Date: 2012-11-06 08:04:47

Eric Hi,

I will check and review your patch and return to you ASAP.

Regarding BQL, the driver has it over half a year now, In commit:
5b263f5374ea70daa61ed9f5cc5d8f4ac236efbd

Amir.

On Tue, Nov 6, 2012 at 4:20 AM, Eric Dumazet [off-list ref] wrote:

quoted hunk

From: Eric Dumazet <edumazet@google.com>

mlx4 currently uses a too high tx coalescing setting, deferring
TX completion interrupts by up to 128 us.

With the recent skb_orphan() removal in commit 8112ec3b872,
performance of a single TCP flow is capped to ~4 Gbps, unless
we increase tcp_limit_output_bytes.

I suggest using 16 us instead of 128 us, allowing a finer control.

Performance of a single TCP flow is restored to previous levels,
while keeping TCP small queues fully enabled with default sysctl.

This patch is also a BQL prereq.

Reported-by: Vimalkumar <redacted>
Signed-off-by: Eric Dumazet <edumazet@google.com>
Cc: Yevgeny Petrilin <redacted>
Cc: Or Gerlitz <redacted>
---
By the way, is BQL addition planned for this driver ?

 drivers/net/ethernet/mellanox/mlx4/en_netdev.c |    2 +-
 drivers/net/ethernet/mellanox/mlx4/mlx4_en.h   |    2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/drivers/net/ethernet/mellanox/mlx4/en_netdev.c b/drivers/net/ethernet/mellanox/mlx4/en_netdev.c
index edd9cb8..2b23ca2 100644
--- a/drivers/net/ethernet/mellanox/mlx4/en_netdev.c
+++ b/drivers/net/ethernet/mellanox/mlx4/en_netdev.c

@@ -870,7 +870,7 @@ static void mlx4_en_set_default_moderation(struct mlx4_en_priv *priv)
        /* If we haven't received a specific coalescing setting
         * (module param), we set the moderation parameters as follows:
         * - moder_cnt is set to the number of mtu sized packets to
-        *   satisfy our coelsing target.
+        *   satisfy our coalescing target.
         * - moder_time is set to a fixed value.
         */
        priv->rx_frames = MLX4_EN_RX_COAL_TARGET;

diff --git a/drivers/net/ethernet/mellanox/mlx4/mlx4_en.h b/drivers/net/ethernet/mellanox/mlx4/mlx4_en.h
index 9d27e42..8a5e70d 100644
--- a/drivers/net/ethernet/mellanox/mlx4/mlx4_en.h
+++ b/drivers/net/ethernet/mellanox/mlx4/mlx4_en.h

@@ -126,7 +126,7 @@ enum {
 #define MLX4_EN_RX_COAL_TIME   0x10

 #define MLX4_EN_TX_COAL_PKTS   16
-#define MLX4_EN_TX_COAL_TIME   0x80
+#define MLX4_EN_TX_COAL_TIME   0x10

 #define MLX4_EN_RX_RATE_LOW            400000
 #define MLX4_EN_RX_COAL_TIME_LOW       0


--

To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH net-next] mlx4: change TX coalescing defaults

From: Eric Dumazet <hidden>
Date: 2012-11-06 11:50:46

On Tue, 2012-11-06 at 10:04 +0200, Amir Mellanox wrote:

Eric Hi,

I will check and review your patch and return to you ASAP.

Regarding BQL, the driver has it over half a year now, In commit:
5b263f5374ea70daa61ed9f5cc5d8f4ac236efbd

Oh right, it seems I was not looking for right verbs or changelog
keywords ;)

Then I suspect my patch will help to keep inflight BQL counter lower and
reduce latencies.

Thanks

Re: [PATCH net-next] mlx4: change TX coalescing defaults

From: Amir Vadai <hidden>
Date: 2012-11-07 15:30:04

Acked.

BTW, a mail will be sent - I'm replacing Yevgeny Petrilin as mlx4_en maintainer.

(sorry for the previous non plain text mail)

Amir

On Tue, Nov 6, 2012 at 4:20 AM, Eric Dumazet [off-list ref] wrote:

quoted hunk

From: Eric Dumazet <edumazet@google.com>

mlx4 currently uses a too high tx coalescing setting, deferring
TX completion interrupts by up to 128 us.

With the recent skb_orphan() removal in commit 8112ec3b872,
performance of a single TCP flow is capped to ~4 Gbps, unless
we increase tcp_limit_output_bytes.

I suggest using 16 us instead of 128 us, allowing a finer control.

Performance of a single TCP flow is restored to previous levels,
while keeping TCP small queues fully enabled with default sysctl.

This patch is also a BQL prereq.

Reported-by: Vimalkumar <redacted>
Signed-off-by: Eric Dumazet <edumazet@google.com>
Cc: Yevgeny Petrilin <redacted>
Cc: Or Gerlitz <redacted>
---
By the way, is BQL addition planned for this driver ?

 drivers/net/ethernet/mellanox/mlx4/en_netdev.c |    2 +-
 drivers/net/ethernet/mellanox/mlx4/mlx4_en.h   |    2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/drivers/net/ethernet/mellanox/mlx4/en_netdev.c

b/drivers/net/ethernet/mellanox/mlx4/en_netdev.c
index edd9cb8..2b23ca2 100644

--- a/drivers/net/ethernet/mellanox/mlx4/en_netdev.c
+++ b/drivers/net/ethernet/mellanox/mlx4/en_netdev.c

@@ -870,7 +870,7 @@ static void mlx4_en_set_default_moderation(struct

mlx4_en_priv *priv)
        /* If we haven't received a specific coalescing setting
         * (module param), we set the moderation parameters as follows:
         * - moder_cnt is set to the number of mtu sized packets to
-        *   satisfy our coelsing target.
+        *   satisfy our coalescing target.
         * - moder_time is set to a fixed value.
         */
        priv->rx_frames = MLX4_EN_RX_COAL_TARGET;

diff --git a/drivers/net/ethernet/mellanox/mlx4/mlx4_en.h

b/drivers/net/ethernet/mellanox/mlx4/mlx4_en.h
index 9d27e42..8a5e70d 100644

--- a/drivers/net/ethernet/mellanox/mlx4/mlx4_en.h
+++ b/drivers/net/ethernet/mellanox/mlx4/mlx4_en.h

@@ -126,7 +126,7 @@ enum {
 #define MLX4_EN_RX_COAL_TIME   0x10

 #define MLX4_EN_TX_COAL_PKTS   16
-#define MLX4_EN_TX_COAL_TIME   0x80
+#define MLX4_EN_TX_COAL_TIME   0x10

 #define MLX4_EN_RX_RATE_LOW            400000
 #define MLX4_EN_RX_COAL_TIME_LOW       0


--

To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH net-next] mlx4: change TX coalescing defaults

From: Eric Dumazet <hidden>
Date: 2012-11-07 16:05:15

On Wed, 2012-11-07 at 17:30 +0200, Amir Vadai wrote:

Acked.

BTW, a mail will be sent - I'm replacing Yevgeny Petrilin as mlx4_en maintainer.

OK, thanks.

BTW proper way to Ack a patch so that its automatically taken into
account by tools is to use :

Acked-by: Amir Vadai <redacted>

Re: [PATCH net-next] mlx4: change TX coalescing defaults

From: David Miller <davem@davemloft.net>
Date: 2012-11-07 20:31:17

From: Eric Dumazet <redacted>
Date: Mon, 05 Nov 2012 18:20:42 -0800

From: Eric Dumazet <edumazet@google.com>

mlx4 currently uses a too high tx coalescing setting, deferring
TX completion interrupts by up to 128 us.

With the recent skb_orphan() removal in commit 8112ec3b872,
performance of a single TCP flow is capped to ~4 Gbps, unless
we increase tcp_limit_output_bytes.

I suggest using 16 us instead of 128 us, allowing a finer control.

Performance of a single TCP flow is restored to previous levels,
while keeping TCP small queues fully enabled with default sysctl.

This patch is also a BQL prereq.

Reported-by: Vimalkumar <redacted>
Signed-off-by: Eric Dumazet <edumazet@google.com>

Applied, thanks Eric.

`h`	back out one level
`j`	next message in thread
`k`	previous message in thread
`l`	drill in
`Esc`	close help / fold thread tree
`?`	toggle this help