On 11/12/2015 06:16 PM, Jason Wang wrote:

Hi all:

This series tries to add basic busy polling for vhost net. The idea is
simple: at the end of tx/rx processing, busy polling for new tx added
descriptor and rx receive socket for a while. The maximum number of
time (in us) could be spent on busy polling was specified ioctl.

Test were done through:

- 50 us as busy loop timeout
- Netperf 2.6
- Two machines with back to back connected ixgbe
- Guest with 1 vcpu and 1 queue

Results:
- For stream workload, ioexits were reduced dramatically in medium
  size (1024-2048) of tx (at most -39%) and almost all rx (at most
  -79%) as a result of polling. This compensate for the possible
  wasted cpu cycles more or less. That porbably why we can still see
  some increasing in the normalized throughput in some cases.
- Throughput of tx were increased (at most 105%) expect for the huge
  write (16384). And we can send more packets in the case (+tpkts were
  increased).
- Very minor rx regression in some cases.
- Improvemnt on TCP_RR (at most 16%).

Forget to mention, the following test results by order are:

1) Guest TX
2) Guest RX
3) TCP_RR

size/session/+thu%/+normalize%/+tpkts%/+rpkts%/+ioexits%/
   64/     1/   +9%/  -17%/   +5%/  +10%/   -2%
   64/     2/   +8%/  -18%/   +6%/  +10%/   -1%
   64/     4/   +4%/  -21%/   +6%/  +10%/   -1%
   64/     8/   +9%/  -17%/   +6%/   +9%/   -2%
  256/     1/  +20%/   -1%/  +15%/  +11%/   -9%
  256/     2/  +15%/   -6%/  +15%/   +8%/   -8%
  256/     4/  +17%/   -4%/  +16%/   +8%/   -8%
  256/     8/  -61%/  -69%/  +16%/  +10%/  -10%
  512/     1/  +15%/   -3%/  +19%/  +18%/  -11%
  512/     2/  +19%/    0%/  +19%/  +13%/  -10%
  512/     4/  +18%/   -2%/  +18%/  +15%/  -10%
  512/     8/  +17%/   -1%/  +18%/  +15%/  -11%
 1024/     1/  +25%/   +4%/  +27%/  +16%/  -21%
 1024/     2/  +28%/   +8%/  +25%/  +15%/  -22%
 1024/     4/  +25%/   +5%/  +25%/  +14%/  -21%
 1024/     8/  +27%/   +7%/  +25%/  +16%/  -21%
 2048/     1/  +32%/  +12%/  +31%/  +22%/  -38%
 2048/     2/  +33%/  +12%/  +30%/  +23%/  -36%
 2048/     4/  +31%/  +10%/  +31%/  +24%/  -37%
 2048/     8/ +105%/  +75%/  +33%/  +23%/  -39%
16384/     1/    0%/  -14%/   +2%/    0%/  +19%
16384/     2/    0%/  -13%/  +19%/  -13%/  +17%
16384/     4/    0%/  -12%/   +3%/    0%/   +2%
16384/     8/    0%/  -11%/   -2%/   +1%/   +1%
size/session/+thu%/+normalize%/+tpkts%/+rpkts%/+ioexits%/
   64/     1/   -7%/  -23%/   +4%/   +6%/  -74%
   64/     2/   -2%/  -12%/   +2%/   +2%/  -55%
   64/     4/   +2%/   -5%/  +10%/   -2%/  -43%
   64/     8/   -5%/   -5%/  +11%/  -34%/  -59%
  256/     1/   -6%/  -16%/   +9%/  +11%/  -60%
  256/     2/   +3%/   -4%/   +6%/   -3%/  -28%
  256/     4/    0%/   -5%/   -9%/   -9%/  -10%
  256/     8/   -3%/   -6%/  -12%/   -9%/  -40%
  512/     1/   -4%/  -17%/  -10%/  +21%/  -34%
  512/     2/    0%/   -9%/  -14%/   -3%/  -30%
  512/     4/    0%/   -4%/  -18%/  -12%/   -4%
  512/     8/   -1%/   -4%/   -1%/   -5%/   +4%
 1024/     1/    0%/  -16%/  +12%/  +11%/  -10%
 1024/     2/    0%/  -11%/    0%/   +5%/  -31%
 1024/     4/    0%/   -4%/   -7%/   +1%/  -22%
 1024/     8/   -5%/   -6%/  -17%/  -29%/  -79%
 2048/     1/    0%/  -16%/   +1%/   +9%/  -10%
 2048/     2/    0%/  -12%/   +7%/   +9%/  -26%
 2048/     4/    0%/   -7%/   -4%/   +3%/  -64%
 2048/     8/   -1%/   -5%/   -6%/   +4%/  -20%
16384/     1/    0%/  -12%/  +11%/   +7%/  -20%
16384/     2/    0%/   -7%/   +1%/   +5%/  -26%
16384/     4/    0%/   -5%/  +12%/  +22%/  -23%
16384/     8/    0%/   -1%/   -8%/   +5%/   -3%
size/session/+thu%/+normalize%/+tpkts%/+rpkts%/+ioexits%/
    1/     1/   +9%/  -29%/   +9%/   +9%/   +9%
    1/    25/   +6%/  -18%/   +6%/   +6%/   -1%
    1/    50/   +6%/  -19%/   +5%/   +5%/   -2%
    1/   100/   +5%/  -19%/   +4%/   +4%/   -3%
   64/     1/  +10%/  -28%/  +10%/  +10%/  +10%
   64/    25/   +8%/  -18%/   +7%/   +7%/   -2%
   64/    50/   +8%/  -17%/   +8%/   +8%/   -1%
   64/   100/   +8%/  -17%/   +8%/   +8%/   -1%
  256/     1/  +10%/  -28%/  +10%/  +10%/  +10%
  256/    25/  +15%/  -13%/  +15%/  +15%/    0%
  256/    50/  +16%/  -14%/  +18%/  +18%/   +2%
  256/   100/  +15%/  -13%/  +12%/  +12%/   -2%

Changes from V2:
- poll also at the end of rx handling
- factor out the polling logic and optimize the code a little bit
- add two ioctls to get and set the busy poll timeout
- test on ixgbe (which can give more stable and reproducable numbers)
  instead of mlx4.

Changes from V1:
- Add a comment for vhost_has_work() to explain why it could be
  lockless
- Add param description for busyloop_timeout
- Split out the busy polling logic into a new helper
- Check and exit the loop when there's a pending signal
- Disable preemption during busy looping to make sure lock_clock() was
  correctly used.

Jason Wang (3):
  vhost: introduce vhost_has_work()
  vhost: introduce vhost_vq_more_avail()
  vhost_net: basic polling support

 drivers/vhost/net.c        | 77 +++++++++++++++++++++++++++++++++++++++++++---
 drivers/vhost/vhost.c      | 48 +++++++++++++++++++++++------
 drivers/vhost/vhost.h      |  3 ++
 include/uapi/linux/vhost.h | 11 +++++++
 4 files changed, 125 insertions(+), 14 deletions(-)