[RFC] Ethernet drivers to add padding on egress

Hi networking driver maintainers,

I suggest that the TX functions of Ethernet interface drivers accept packets with less than 60 byte payload, and transmit them on the medium as valid Ethernet frames, i.e. by padding the packets up to the minimum Ethernet packet size of 64 bytes incl. Ethernet FCS, instead of discarding them.

This feature makes it easier for application developers who are using DPDK as the lower layer in an IP stack, where lots of packets have less than 60 bytes Ethernet payload, e.g. TCP SYN and TCP ACK packets.

This feature also makes it easier for application developers who are using DPDK library functions that split, merge or otherwise transform packets into packets of other sizes, e.g. Generic Segmentation Offload, IP Fragmentation and various tunnel encapsulation/decapsulation functions.

Currently (without this feature), it is required by the application to check if packets originating from the IP stack or having passed through a split/merge/transform function are about to egress on an Ethernet interface, and in that case, if some of the packets are less than 60 bytes (excl. Ethernet FCS), add padding before passing them on to the driver's TX function.

E.g. when using Generic Segmentation Offload, a packet carrying 1461 byte TCP payload (excl. 54 bytes Ethernet+IP+TCP headers) will be split into two packets of respectively 1514 byte (incl. 54 bytes Ethernet+IP+TCP headers) and 55 bytes (incl. 54 bytes Ethernet+IP+TCP headers), and the latter must be padded before it is transmitted on an Ethernet interface.


In my opinion, it should be a requirement that the Ethernet interface drivers ensure correct padding when egressing the packet on the medium. Alternatively, it can be an optional feature, which could be exposed as an TX Capabilities flag of the driver.

What do you think?


I do not suggest any changes regarding ingress - runts (undersize Ethernet packets) received from the medium are invalid, and should be discarded and counted as errors.


My RFC was triggered by this discussion:
https://mails.dpdk.org/archives/dev/2018-November/119135.html

PS: I acknowledge Keith's comment that I am pushing for a feature with wide ranging consequences - modifying all PMDs and possibly costing some performance - based on one assumption.


Med venlig hilsen / kind regards

Morten Brørup
CTO


SmartShare Systems A/S
Tonsbakken 16-18
DK-2740 Skovlunde
Denmark

Office      +45 70 20 00 93
Direct      +45 89 93 50 22
Mobile     +45 25 40 82 12

mb@smartsharesystems.com
www.smartsharesystems.com

Thursday, November 15, 2018 6:57 PM, Morten Brørup:

Subject: [RFC] Ethernet drivers to add padding on egress

Hi networking driver maintainers,

I suggest that the TX functions of Ethernet interface drivers accept packets
with less than 60 byte payload, and transmit them on the medium as valid
Ethernet frames, i.e. by padding the packets up to the minimum Ethernet
packet size of 64 bytes incl. Ethernet FCS, instead of discarding them.

This feature makes it easier for application developers who are using DPDK as
the lower layer in an IP stack, where lots of packets have less than 60 bytes
Ethernet payload, e.g. TCP SYN and TCP ACK packets.

This feature also makes it easier for application developers who are using
DPDK library functions that split, merge or otherwise transform packets into
packets of other sizes, e.g. Generic Segmentation Offload, IP Fragmentation
and various tunnel encapsulation/decapsulation functions.

Currently (without this feature), it is required by the application to check if
packets originating from the IP stack or having passed through a
split/merge/transform function are about to egress on an Ethernet interface,
and in that case, if some of the packets are less than 60 bytes (excl. Ethernet
FCS), add padding before passing them on to the driver's TX function.

E.g. when using Generic Segmentation Offload, a packet carrying 1461 byte
TCP payload (excl. 54 bytes Ethernet+IP+TCP headers) will be split into two
packets of respectively 1514 byte (incl. 54 bytes Ethernet+IP+TCP headers)
and 55 bytes (incl. 54 bytes Ethernet+IP+TCP headers), and the latter must
be padded before it is transmitted on an Ethernet interface.


In my opinion, it should be a requirement that the Ethernet interface drivers
ensure correct padding when egressing the packet on the medium.
Alternatively, it can be an optional feature, which could be exposed as an TX
Capabilities flag of the driver.

What do you think?

I think at the first stage it should be a Tx offload capability - the ability to pad (maybe in HW) the packets and avoid the cost of padding in SW.
PMD vendors who wants to make an easier life for their customers can implement it in SW, however the gain here is only with simplicity of code for application. Performance wise it wouldn't matter. 

When the majority/all PMDs will have this feature we can discuss on making it a standard for each PMD (like the CRC strip we have today). 

I do not suggest any changes regarding ingress - runts (undersize Ethernet
packets) received from the medium are invalid, and should be discarded and
counted as errors.


My RFC was triggered by this discussion:
https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fma
ils.dpdk.org%2Farchives%2Fdev%2F2018-
November%2F119135.html&data=02%7C01%7Cshahafs%40mellanox.co
m%7C395ae1ac33674144116508d64b1b59f0%7Ca652971c7d2e4d9ba6a4d1492
56f461b%7C0%7C0%7C636778978183517821&sdata=xWcA5bhbhhqhZkg
CYuwUP4qjLVlCLMjZrJaitQhAZ8c%3D&reserved=0

PS: I acknowledge Keith's comment that I am pushing for a feature with wide
ranging consequences - modifying all PMDs and possibly costing some
performance - based on one assumption.


Med venlig hilsen / kind regards

Morten Brørup
CTO


SmartShare Systems A/S
Tonsbakken 16-18
DK-2740 Skovlunde
Denmark

Office      +45 70 20 00 93
Direct      +45 89 93 50 22
Mobile     +45 25 40 82 12

mb@smartsharesystems.com
https://emea01.safelinks.protection.outlook.com/?url=www.smartsharesyst
ems.com&data=02%7C01%7Cshahafs%40mellanox.com%7C395ae1ac33
674144116508d64b1b59f0%7Ca652971c7d2e4d9ba6a4d149256f461b%7C0%7C
0%7C636778978183517821&sdata=gbdMkjhODS74P0Mtm%2FNEyNQ5h
gFZI0xRgvNdA5M2ydM%3D&reserved=0

On Thu, 15 Nov 2018 17:56:48 +0100
Morten Brørup [off-list ref] wrote:

Hi networking driver maintainers,

I suggest that the TX functions of Ethernet interface drivers accept packets with less than 60 byte payload, and transmit them on the medium as valid Ethernet frames, i.e. by padding the packets up to the minimum Ethernet packet size of 64 bytes incl. Ethernet FCS, instead of discarding them.

This feature makes it easier for application developers who are using DPDK as the lower layer in an IP stack, where lots of packets have less than 60 bytes Ethernet payload, e.g. TCP SYN and TCP ACK packets.

This feature also makes it easier for application developers who are using DPDK library functions that split, merge or otherwise transform packets into packets of other sizes, e.g. Generic Segmentation Offload, IP Fragmentation and various tunnel encapsulation/decapsulation functions.

Currently (without this feature), it is required by the application to check if packets originating from the IP stack or having passed through a split/merge/transform function are about to egress on an Ethernet interface, and in that case, if some of the packets are less than 60 bytes (excl. Ethernet FCS), add padding before passing them on to the driver's TX function.

E.g. when using Generic Segmentation Offload, a packet carrying 1461 byte TCP payload (excl. 54 bytes Ethernet+IP+TCP headers) will be split into two packets of respectively 1514 byte (incl. 54 bytes Ethernet+IP+TCP headers) and 55 bytes (incl. 54 bytes Ethernet+IP+TCP headers), and the latter must be padded before it is transmitted on an Ethernet interface.


In my opinion, it should be a requirement that the Ethernet interface drivers ensure correct padding when egressing the packet on the medium. Alternatively, it can be an optional feature, which could be exposed as an TX Capabilities flag of the driver.

What do you think?


I do not suggest any changes regarding ingress - runts (undersize Ethernet packets) received from the medium are invalid, and should be discarded and counted as errors.

Devices drivers should work like Linux and BSD!
They must pad packets to meet any restrictions by the underlying technology.

Please don't add another capability flag or other way for applications to see this.
DPDK already has way to many hardware capability flags.
It should just happen.

Monday, November 19, 2018 6:10 PM, Stephen Hemminger:

Subject: Re: [dpdk-dev] [RFC] Ethernet drivers to add padding on egress

On Thu, 15 Nov 2018 17:56:48 +0100
Morten Brørup [off-list ref] wrote:

quoted

Hi networking driver maintainers,

I suggest that the TX functions of Ethernet interface drivers accept packets

with less than 60 byte payload, and transmit them on the medium as valid
Ethernet frames, i.e. by padding the packets up to the minimum Ethernet
packet size of 64 bytes incl. Ethernet FCS, instead of discarding them.

quoted

This feature makes it easier for application developers who are using DPDK

as the lower layer in an IP stack, where lots of packets have less than 60
bytes Ethernet payload, e.g. TCP SYN and TCP ACK packets.

quoted

This feature also makes it easier for application developers who are using

DPDK library functions that split, merge or otherwise transform packets into
packets of other sizes, e.g. Generic Segmentation Offload, IP Fragmentation
and various tunnel encapsulation/decapsulation functions.

quoted

Currently (without this feature), it is required by the application to check if

packets originating from the IP stack or having passed through a
split/merge/transform function are about to egress on an Ethernet interface,
and in that case, if some of the packets are less than 60 bytes (excl. Ethernet
FCS), add padding before passing them on to the driver's TX function.

quoted

E.g. when using Generic Segmentation Offload, a packet carrying 1461 byte

TCP payload (excl. 54 bytes Ethernet+IP+TCP headers) will be split into two
packets of respectively 1514 byte (incl. 54 bytes Ethernet+IP+TCP headers)
and 55 bytes (incl. 54 bytes Ethernet+IP+TCP headers), and the latter must
be padded before it is transmitted on an Ethernet interface.

quoted

In my opinion, it should be a requirement that the Ethernet interface

drivers ensure correct padding when egressing the packet on the medium.
Alternatively, it can be an optional feature, which could be exposed as an TX
Capabilities flag of the driver.

quoted

What do you think?


I do not suggest any changes regarding ingress - runts (undersize Ethernet

packets) received from the medium are invalid, and should be discarded and
counted as errors.

quoted

Devices drivers should work like Linux and BSD!
They must pad packets to meet any restrictions by the underlying
technology.

Please don't add another capability flag or other way for applications to see
this.
DPDK already has way to many hardware capability flags.
It should just happen.

I would like to change my previous comment, I fully agree w/ Stephan here. 

On Mon, 19 Nov 2018 08:02:02 +0000
Shahaf Shuler [off-list ref] wrote:

Thursday, November 15, 2018 6:57 PM, Morten Brørup:

quoted

Subject: [RFC] Ethernet drivers to add padding on egress

Hi networking driver maintainers,

I suggest that the TX functions of Ethernet interface drivers accept packets
with less than 60 byte payload, and transmit them on the medium as valid
Ethernet frames, i.e. by padding the packets up to the minimum Ethernet
packet size of 64 bytes incl. Ethernet FCS, instead of discarding them.

This feature makes it easier for application developers who are using DPDK as
the lower layer in an IP stack, where lots of packets have less than 60 bytes
Ethernet payload, e.g. TCP SYN and TCP ACK packets.

This feature also makes it easier for application developers who are using
DPDK library functions that split, merge or otherwise transform packets into
packets of other sizes, e.g. Generic Segmentation Offload, IP Fragmentation
and various tunnel encapsulation/decapsulation functions.

Currently (without this feature), it is required by the application to check if
packets originating from the IP stack or having passed through a
split/merge/transform function are about to egress on an Ethernet interface,
and in that case, if some of the packets are less than 60 bytes (excl. Ethernet
FCS), add padding before passing them on to the driver's TX function.

E.g. when using Generic Segmentation Offload, a packet carrying 1461 byte
TCP payload (excl. 54 bytes Ethernet+IP+TCP headers) will be split into two
packets of respectively 1514 byte (incl. 54 bytes Ethernet+IP+TCP headers)
and 55 bytes (incl. 54 bytes Ethernet+IP+TCP headers), and the latter must
be padded before it is transmitted on an Ethernet interface.


In my opinion, it should be a requirement that the Ethernet interface drivers
ensure correct padding when egressing the packet on the medium.
Alternatively, it can be an optional feature, which could be exposed as an TX
Capabilities flag of the driver.

What do you think?  

I think at the first stage it should be a Tx offload capability - the ability to pad (maybe in HW) the packets and avoid the cost of padding in SW.
PMD vendors who wants to make an easier life for their customers can implement it in SW, however the gain here is only with simplicity of code for application. Performance wise it wouldn't matter. 

When the majority/all PMDs will have this feature we can discuss on making it a standard for each PMD (like the CRC strip we have today).

Yet another tx offload flag may look good as a vendor but doesn't add anything useful and hurts useablity.
Every driver should take any size Ethernet packet and pad in hardware (or software) based on what it knows the NIC hardware can do.
For virtual devices where there is no minimum length is required, then nothing needs to be done.

Packets < Ether header are obvious errors and should increment tx_output errors.