Re: CPU scheduler to TXQ binding? (ixgbe vs. igb)

On Thu, 2014-09-18 at 17:59 +0200, Jesper Dangaard Brouer wrote:

On Thu, 18 Sep 2014 08:42:31 -0700
Eric Dumazet [off-list ref] wrote:

quoted

On Thu, 2014-09-18 at 06:41 -0700, Eric Dumazet wrote:

quoted

Last but not least, there is the fact that networking stacks use
mod_timer() to arm timers, and that by default, timer migration is on 
( cf /proc/sys/kernel/timer_migration )

I don't have this proc file on my system, as I didn't select CONFIG_SCHED_DEBUG.

Interesting... this timer_migration stuff seems a bit scary to me.

quoted

quoted

We probably should use mod_timer_pinned(), but I could not really see
any difference.

Hmm... actually its quite noticeable :

Interesting impact.

I'm looking for some 1G hardware without multiqueue, so I can get
around this measurement constraint.  And possibly turning it down to
100Mbit/s, so I can more easily measure the HoL blocking effect.

ethtool -L   eth0    rx 1 tx 1 

(Or similar if combined is used)

quoted

# ./super_netperf 500 --google-pacing-rate 3000000 -H lpaa24 -l 1000 &
...

Interesting option "--google-pacing-rate" ;-)

Its using upstream SO_MAX_PACING_RATE, nothing fancy ;)

quoted

# echo 1 >/proc/sys/kernel/timer_migration
# vmstat 5
procs -----------memory---------- ---swap-- -----io---- -system-- ----cpu----
 r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id wa
 2  0      0 261178336  15812 1001880    0    0     5     1  185  217  0  4 96  0
 0  0      0 261173456  15812 1001884    0    0     0     0 1548055 35472  0 15 85  0
 2  0      0 261174880  15812 1001888    0    0     0     0 1533309 35163  0 15 85  0
 3  0      0 261176768  15812 1001896    0    0     0     0 1533442 35694  0 15 85  0

[]

quoted

# echo 0 >/proc/sys/kernel/timer_migration
# vmstat 5
procs -----------memory---------- ---swap-- -----io---- -system-- ----cpu----
 r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id wa
 2  0      0 261172784  15812 1001936    0    0     5     1  165  228  0  5 95  0
 1  0      0 261175776  15812 1001940    0    0     0     0 1187446 32238  0 12 88  0
 2  0      0 261172752  15812 1001940    0    0     0     3 1166697 32060  0 12 88  0

Quite significant, both interrupts and especially CPU system usage drop.

Yep...