[PATCH net-next v3 0/2] veth and GSO maximums

This is the more general way to solving the issue of GSO limits
not being set correctly for containers on Azure. If a GSO packet
is sent to host that exceeds the limit (reported by NDIS), then
the host is forced to do segmentation in software which has noticeable
performance impact.

The core rtnetlink infrastructure already has the messages and
infrastructure to allow changing gso limits. With an updated iproute2
the following already works:
  # ip li set dev dummy0 gso_max_size 30000

These patches are about making it easier with veth.

Stephen Hemminger (2):
  rtnetlink: allow GSO maximums to be set on device creation
  veth: set peer GSO values

 drivers/net/veth.c   |  3 +++
 net/core/rtnetlink.c | 34 ++++++++++++++++++++++++++++++++++
 2 files changed, 37 insertions(+)

-- 
2.11.0

From: Stephen Hemminger <stephen@networkplumber.org>

Netlink device already allows changing GSO sizes with
ip set command. The part that is missing is allowing overriding
GSO settings on device creation.

Signed-off-by: Stephen Hemminger <redacted>

allow set gso on create
---
 net/core/rtnetlink.c | 34 ++++++++++++++++++++++++++++++++++
 1 file changed, 34 insertions(+)

diff --git a/net/core/rtnetlink.c b/net/core/rtnetlink.c
index a4faefd65006..412ebf0b09c6 100644
--- a/net/core/rtnetlink.c
+++ b/net/core/rtnetlink.c

@@ -1637,6 +1637,8 @@ static const struct nla_policy ifla_policy[IFLA_MAX+1] = {
 	[IFLA_PROMISCUITY]	= { .type = NLA_U32 },
 	[IFLA_NUM_TX_QUEUES]	= { .type = NLA_U32 },
 	[IFLA_NUM_RX_QUEUES]	= { .type = NLA_U32 },
+	[IFLA_GSO_MAX_SEGS]	= { .type = NLA_U32 },
+	[IFLA_GSO_MAX_SIZE]	= { .type = NLA_U32 },
 	[IFLA_PHYS_PORT_ID]	= { .type = NLA_BINARY, .len = MAX_PHYS_ITEM_ID_LEN },
 	[IFLA_CARRIER_CHANGES]	= { .type = NLA_U32 },  /* ignored */
 	[IFLA_PHYS_SWITCH_ID]	= { .type = NLA_BINARY, .len = MAX_PHYS_ITEM_ID_LEN },

@@ -2287,6 +2289,34 @@ static int do_setlink(const struct sk_buff *skb,
 		}
 	}
 
+	if (tb[IFLA_GSO_MAX_SIZE]) {
+		u32 max_size = nla_get_u32(tb[IFLA_GSO_MAX_SIZE]);
+
+		if (max_size > GSO_MAX_SIZE) {
+			err = -EINVAL;
+			goto errout;
+		}
+
+		if (dev->gso_max_size ^ max_size) {
+			netif_set_gso_max_size(dev, max_size);
+			status |= DO_SETLINK_MODIFIED;
+		}
+	}
+
+	if (tb[IFLA_GSO_MAX_SEGS]) {
+		u32 max_segs = nla_get_u32(tb[IFLA_GSO_MAX_SEGS]);
+
+		if (max_segs > GSO_MAX_SEGS) {
+			err = -EINVAL;
+			goto errout;
+		}
+
+		if (dev->gso_max_segs ^ max_segs) {
+			dev->gso_max_segs = max_segs;
+			status |= DO_SETLINK_MODIFIED;
+		}
+	}
+
 	if (tb[IFLA_OPERSTATE])
 		set_operstate(dev, nla_get_u8(tb[IFLA_OPERSTATE]));
 

@@ -2651,6 +2681,10 @@ struct net_device *rtnl_create_link(struct net *net,
 		dev->link_mode = nla_get_u8(tb[IFLA_LINKMODE]);
 	if (tb[IFLA_GROUP])
 		dev_set_group(dev, nla_get_u32(tb[IFLA_GROUP]));
+	if (tb[IFLA_GSO_MAX_SIZE])
+		netif_set_gso_max_size(dev, nla_get_u32(tb[IFLA_GSO_MAX_SIZE]));
+	if (tb[IFLA_GSO_MAX_SEGS])
+		dev->gso_max_size = nla_get_u32(tb[IFLA_GSO_MAX_SEGS]);
 
 	return dev;
 }

-- 
2.11.0

When new veth is created, and GSO values have been configured
on one device, clone those values to the peer.

For example:
   # ip link add dev vm1 gso_max_size 65530 type veth peer name vm2

This should create vm1 <--> vm2 with both having GSO maximum
size set to 65530.

Signed-off-by: Stephen Hemminger <redacted>
---
 drivers/net/veth.c | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/drivers/net/veth.c b/drivers/net/veth.c
index f5438d0978ca..a69ad39ee57e 100644
--- a/drivers/net/veth.c
+++ b/drivers/net/veth.c

@@ -410,6 +410,9 @@ static int veth_newlink(struct net *src_net, struct net_device *dev,
 	if (ifmp && (dev->ifindex != 0))
 		peer->ifindex = ifmp->ifi_index;
 
+	peer->gso_max_size = dev->gso_max_size;
+	peer->gso_max_segs = dev->gso_max_segs;
+
 	err = register_netdevice(peer);
 	put_net(net);
 	net = NULL;

-- 
2.11.0

From: Stephen Hemminger <stephen@networkplumber.org>
Date: Thu,  7 Dec 2017 15:40:18 -0800

This is the more general way to solving the issue of GSO limits
not being set correctly for containers on Azure. If a GSO packet
is sent to host that exceeds the limit (reported by NDIS), then
the host is forced to do segmentation in software which has noticeable
performance impact.

The core rtnetlink infrastructure already has the messages and
infrastructure to allow changing gso limits. With an updated iproute2
the following already works:
  # ip li set dev dummy0 gso_max_size 30000

These patches are about making it easier with veth.

Ok, this is definitely a step forward.

Series applied, thanks Stephen.

On Fri, Dec 08, 2017 at 02:23:23PM -0500, David Miller wrote:

From: Stephen Hemminger <stephen@networkplumber.org>
Date: Thu,  7 Dec 2017 15:40:18 -0800

quoted

This is the more general way to solving the issue of GSO limits
not being set correctly for containers on Azure. If a GSO packet
is sent to host that exceeds the limit (reported by NDIS), then
the host is forced to do segmentation in software which has noticeable
performance impact.

The core rtnetlink infrastructure already has the messages and
infrastructure to allow changing gso limits. With an updated iproute2
the following already works:
  # ip li set dev dummy0 gso_max_size 30000

These patches are about making it easier with veth.

Ok, this is definitely a step forward.

Series applied, thanks Stephen.

Thanks.
Still not seeing the iproute2 patch though, either master or net-next.

On Fri, 8 Dec 2017 15:50:25 -0800
Solio Sarabia [off-list ref] wrote:

On Fri, Dec 08, 2017 at 02:23:23PM -0500, David Miller wrote:

quoted

From: Stephen Hemminger <stephen@networkplumber.org>
Date: Thu,  7 Dec 2017 15:40:18 -0800
  

quoted

This is the more general way to solving the issue of GSO limits
not being set correctly for containers on Azure. If a GSO packet
is sent to host that exceeds the limit (reported by NDIS), then
the host is forced to do segmentation in software which has noticeable
performance impact.

The core rtnetlink infrastructure already has the messages and
infrastructure to allow changing gso limits. With an updated iproute2
the following already works:
  # ip li set dev dummy0 gso_max_size 30000

These patches are about making it easier with veth.  

Ok, this is definitely a step forward.

Series applied, thanks Stephen.  

Thanks.
Still not seeing the iproute2 patch though, either master or net-next.

I held off on iproute2 until kernel change is accepted upstream.

On 12/7/17 4:40 PM, Stephen Hemminger wrote:

What if gso changes are made after device create? They are not
propagated to the peer device like they are on link create.

On Sat, Dec 09, 2017 at 09:41:25AM -0700, David Ahern wrote:

On 12/7/17 4:40 PM, Stephen Hemminger wrote:

quoted

diff --git a/drivers/net/veth.c b/drivers/net/veth.c
index f5438d0978ca..a69ad39ee57e 100644
--- a/drivers/net/veth.c
+++ b/drivers/net/veth.c

@@ -410,6 +410,9 @@ static int veth_newlink(struct net *src_net, struct net_device *dev,
 	if (ifmp && (dev->ifindex != 0))
 		peer->ifindex = ifmp->ifi_index;
 
+	peer->gso_max_size = dev->gso_max_size;
+	peer->gso_max_segs = dev->gso_max_segs;
+
 	err = register_netdevice(peer);
 	put_net(net);
 	net = NULL;

What if gso changes are made after device create? They are not
propagated to the peer device like they are on link create.

This would be a nice addition after ongoing patches are merged,
since veth usually lives in another netns. For docker, it requires
a couple of extra commands to expose the peer's netns.