Re: [RFC PATCH 0/2] Coalesce MMIO writes for transmits

On Wed, 11 Jul 2012 17:25:58 -0700
Alexander Duyck [off-list ref] wrote:

This patch set is meant to address recent issues I found with ixgbe
performance being bound by Tx tail writes.  With these changes in place
and the dispatch_limit set to 1 or more I see a significant increase in
performance.

In the case of one of my systems I saw the routing rate for 7 queues jump
from 10.5 to 11.7Mpps.  The overall increase I have seen on most systems is
something on the order of about 15%.  In the case of pktgen I have also
seen a noticeable increase as the previous limit for transmits was
~12.5Mpps, but with this patch set in place and the dispatch_limit enabled
the value increases to ~14.2Mpps.

I expected there to be an increase in latency, however so far I have not
ran into that.  I have tried running NPtcp tests for latency and seen no
difference in the coalesced and non-coalesced transaction times.  I welcome
any suggestions for tests I might run that might expose any latency issues
as a result of this patch.

---

Alexander Duyck (2):
      ixgbe: Add functionality for delaying the MMIO write for Tx
      net: Add new network device function to allow for MMIO batching


 drivers/net/ethernet/intel/ixgbe/ixgbe_main.c |   22 +++++++-
 include/linux/netdevice.h                     |   57 +++++++++++++++++++++
 net/core/dev.c                                |   67 +++++++++++++++++++++++++
 net/core/net-sysfs.c                          |   36 +++++++++++++
 4 files changed, 180 insertions(+), 2 deletions(-)

This is a good idea. I was thinking of adding a multi-skb operation
to netdevice_ops to allow this. Something like ndo_start_xmit_pkts but
the problem is how to deal with the boundary case where there is only
a limited number of slots in the ring.  Using a "that's all folks"
operation seems better.