Final note:
I had captured latency measurements for
two of the three kernels. Just ran
2.6.18(rhel5) and the results are
stunning. The older kernel is much,
much better then the newer kernel.
Average latency is three times better
and the standard deviation is six
time better. As in 300% and 600%.
Latency here is the time it takes
a packet to travel from the kernel
(where it is timestamped) till it
reaches the final consumption point
in the application.
Makes me think that the old kernel
is better at keeping caches hot and
scheduling woken threads on the same
cores as the threads that triggered
them.
Le mercredi 05 octobre 2011 à 02:58 -0400, starlight@binnacle.cx a
écrit :
Final note:
I had captured latency measurements for
two of the three kernels. Just ran
2.6.18(rhel5) and the results are
stunning. The older kernel is much,
much better then the newer kernel.
Average latency is three times better
and the standard deviation is six
time better. As in 300% and 600%.
Latency here is the time it takes
a packet to travel from the kernel
(where it is timestamped) till it
reaches the final consumption point
in the application.
Makes me think that the old kernel
is better at keeping caches hot and
scheduling woken threads on the same
cores as the threads that triggered
them.
Note :
Your results are from a combination of a user application and kernel
default strategies.
On other combinations, results can be completely different.
A wakeup strategy is somewhat tricky :
- Should we affine or not.
- Should we queue the wakeup on a remote CPU, to keep scheduler data hot
in a single cpu cache.
- Should we use RPS/RFS to queue the packet to another CPU before even
handling it in our stack, to keep network data hot in a single cpu
cache. (check Documentation/networking/scaling.txt)
At least, with recent kernels, we have many available choices to tune a
workload.