[PATCH net-next 0/2] allow setting gso_maximum values

This is another way of addressing the GSO maximum performance issues for
containers on Azure. What happens is that the underlying infrastructure uses
a overlay network such that GSO packets over 64K - vlan header end up cause
either guest or host to have do expensive software copy and fragmentation.

The netvsc driver reports GSO maximum settings correctly, the issue
is that containers on veth devices still have the larger settings.
One solution that was examined was propogating the values back
through the bridge device, but this does not work for cases where
virtual container network is done on L3.

This patch set punts the problem to the orchestration layer that sets
up the container network. It also enables other virtual devices
to have configurable settings for GSO maximum.

Stephen Hemminger (2):
  rtnetlink: allow GSO maximums to be passed to device
  veth: allow configuring GSO maximums

 drivers/net/veth.c   | 20 ++++++++++++++++++++
 net/core/rtnetlink.c |  2 ++
 2 files changed, 22 insertions(+)

-- 
2.11.0

Allow GSO maximum segments and size as netlink parameters on input.

Signed-off-by: Stephen Hemminger <redacted>
---
 net/core/rtnetlink.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/net/core/rtnetlink.c b/net/core/rtnetlink.c
index dabba2a91fc8..8138194c5f81 100644
--- a/net/core/rtnetlink.c
+++ b/net/core/rtnetlink.c

@@ -1569,6 +1569,8 @@ static const struct nla_policy ifla_policy[IFLA_MAX+1] = {
 	[IFLA_PROMISCUITY]	= { .type = NLA_U32 },
 	[IFLA_NUM_TX_QUEUES]	= { .type = NLA_U32 },
 	[IFLA_NUM_RX_QUEUES]	= { .type = NLA_U32 },
+	[IFLA_GSO_MAX_SEGS]	= { .type = NLA_U32 },
+	[IFLA_GSO_MAX_SIZE]	= { .type = NLA_U32 },
 	[IFLA_PHYS_PORT_ID]	= { .type = NLA_BINARY, .len = MAX_PHYS_ITEM_ID_LEN },
 	[IFLA_CARRIER_CHANGES]	= { .type = NLA_U32 },  /* ignored */
 	[IFLA_PHYS_SWITCH_ID]	= { .type = NLA_BINARY, .len = MAX_PHYS_ITEM_ID_LEN },

-- 
2.11.0

Veth's can be used in environments (like Azure) where the underlying
network device is impacted by large GSO packets. This patch allows
gso maximum values to be passed in when creating the device via
netlink.

In theory, other pseudo devices could also use netlink attributes
to set GSO maximums but for now veth is what has been observed
to be an issue.

Signed-off-by: Stephen Hemminger <redacted>
---
 drivers/net/veth.c | 20 ++++++++++++++++++++
 1 file changed, 20 insertions(+)

diff --git a/drivers/net/veth.c b/drivers/net/veth.c
index f5438d0978ca..510c058ba227 100644
--- a/drivers/net/veth.c
+++ b/drivers/net/veth.c

@@ -410,6 +410,26 @@ static int veth_newlink(struct net *src_net, struct net_device *dev,
 	if (ifmp && (dev->ifindex != 0))
 		peer->ifindex = ifmp->ifi_index;
 
+	if (tbp[IFLA_GSO_MAX_SIZE]) {
+		u32 max_size = nla_get_u32(tbp[IFLA_GSO_MAX_SIZE]);
+
+		if (max_size > GSO_MAX_SIZE)
+			return -EINVAL;
+
+		peer->gso_max_size = max_size;
+		dev->gso_max_size = max_size;
+	}
+
+	if (tbp[IFLA_GSO_MAX_SEGS]) {
+		u32 max_segs = nla_get_u32(tbp[IFLA_GSO_MAX_SEGS]);
+
+		if (max_segs > GSO_MAX_SEGS)
+			return -EINVAL;
+
+		peer->gso_max_segs = max_segs;
+		dev->gso_max_segs = max_segs;
+	}
+
 	err = register_netdevice(peer);
 	put_net(net);
 	net = NULL;

-- 
2.11.0

On Fri,  1 Dec 2017 12:11:56 -0800
Stephen Hemminger [off-list ref] wrote:

This is another way of addressing the GSO maximum performance issues for
containers on Azure. What happens is that the underlying infrastructure uses
a overlay network such that GSO packets over 64K - vlan header end up cause
either guest or host to have do expensive software copy and fragmentation.

The netvsc driver reports GSO maximum settings correctly, the issue
is that containers on veth devices still have the larger settings.
One solution that was examined was propogating the values back
through the bridge device, but this does not work for cases where
virtual container network is done on L3.

This patch set punts the problem to the orchestration layer that sets
up the container network. It also enables other virtual devices
to have configurable settings for GSO maximum.

Stephen Hemminger (2):
  rtnetlink: allow GSO maximums to be passed to device
  veth: allow configuring GSO maximums

 drivers/net/veth.c   | 20 ++++++++++++++++++++
 net/core/rtnetlink.c |  2 ++
 2 files changed, 22 insertions(+)

I would like a confirmation from Intel that is doing Docker testing
that this works for them before merging.

On Fri, Dec 01, 2017 at 03:30:01PM -0800, Stephen Hemminger wrote:

On Fri,  1 Dec 2017 12:11:56 -0800
Stephen Hemminger [off-list ref] wrote:

quoted

This is another way of addressing the GSO maximum performance issues for
containers on Azure. What happens is that the underlying infrastructure uses
a overlay network such that GSO packets over 64K - vlan header end up cause
either guest or host to have do expensive software copy and fragmentation.

The netvsc driver reports GSO maximum settings correctly, the issue
is that containers on veth devices still have the larger settings.
One solution that was examined was propogating the values back
through the bridge device, but this does not work for cases where
virtual container network is done on L3.

This patch set punts the problem to the orchestration layer that sets
up the container network. It also enables other virtual devices
to have configurable settings for GSO maximum.

Stephen Hemminger (2):
  rtnetlink: allow GSO maximums to be passed to device
  veth: allow configuring GSO maximums

 drivers/net/veth.c   | 20 ++++++++++++++++++++
 net/core/rtnetlink.c |  2 ++
 2 files changed, 22 insertions(+)

I would like a confirmation from Intel that is doing Docker testing
that this works for them before merging.

This change and its iproute2 counterpart allow creating veth pairs with
specific gso_max{size,segs}. Thanks.

However, the docker code that sets up veth pairis is go-compiled in
their libnetwork. End-users won't be able to tweak gso settings at veth
creation. In this case, we would need to add ioctl (ip/iplink.c:do_set)
support to allow changes after veth is created.

On 12/1/17 1:11 PM, Stephen Hemminger wrote:

Given the role of veth it seems odd to only allow these to be set at
create time. I think Solio is saying the same with respect to the Docker
use case.

From: Stephen Hemminger <stephen@networkplumber.org>
Date: Fri, 1 Dec 2017 15:30:01 -0800

On Fri,  1 Dec 2017 12:11:56 -0800
Stephen Hemminger [off-list ref] wrote:

quoted

This is another way of addressing the GSO maximum performance issues for
containers on Azure. What happens is that the underlying infrastructure uses
a overlay network such that GSO packets over 64K - vlan header end up cause
either guest or host to have do expensive software copy and fragmentation.

The netvsc driver reports GSO maximum settings correctly, the issue
is that containers on veth devices still have the larger settings.
One solution that was examined was propogating the values back
through the bridge device, but this does not work for cases where
virtual container network is done on L3.

This patch set punts the problem to the orchestration layer that sets
up the container network. It also enables other virtual devices
to have configurable settings for GSO maximum.

Stephen Hemminger (2):
  rtnetlink: allow GSO maximums to be passed to device
  veth: allow configuring GSO maximums

 drivers/net/veth.c   | 20 ++++++++++++++++++++
 net/core/rtnetlink.c |  2 ++
 2 files changed, 22 insertions(+)

I would like a confirmation from Intel that is doing Docker testing
that this works for them before merging.

Like David Ahern, I think you should allow this net netlink setting
during changelink as well as newlink.

Thanks.

On Fri, Dec 1, 2017 at 12:11 PM, Stephen Hemminger
[off-list ref] wrote:

Veth's can be used in environments (like Azure) where the underlying
network device is impacted by large GSO packets. This patch allows
gso maximum values to be passed in when creating the device via
netlink.

In theory, other pseudo devices could also use netlink attributes
to set GSO maximums but for now veth is what has been observed
to be an issue.

It looks odd that you only allow setting them but not dumping them.

On Mon, 2017-12-04 at 16:03 -0800, Cong Wang wrote:

On Fri, Dec 1, 2017 at 12:11 PM, Stephen Hemminger
[off-list ref] wrote:

quoted

Veth's can be used in environments (like Azure) where the
underlying
network device is impacted by large GSO packets. This patch allows
gso maximum values to be passed in when creating the device via
netlink.

In theory, other pseudo devices could also use netlink attributes
to set GSO maximums but for now veth is what has been observed
to be an issue.

It looks odd that you only allow setting them but not dumping them.

Dump is already supported/provided by core stack

# ip -d link sh dev wlan0
2: wlan0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq state UP
mode DORMANT group default qlen 1000
    link/ether xx:xx:xx:xx:xx:xx brd ff:ff:ff:ff:ff:ff promiscuity 0
addrgenmode none numtxqueues 4 numrxqueues 1 gso_max_size 65536
gso_max_segs 65535