Re: ksoftirqd 100% after disabling IPv4 route cache on high pps.

Thanks Eric!

Following is only to help someone else who may run into the problem
and searching for options:

the ksoftirqd load is generated by route lookup subrouting. since this
is DDOS traffic we are dealing (more than 10x of our normal PPS) and I
have a fixed number of destinations, I've tested that I can filter
them efficiently in mangle table PREROUTING chain, and use hashlimit
to limit pps to send to the routing subsystem, ksoftirq problem is
gone and no kernel upgraded is needed until the inevitable.

Cheers.

On Wed, Dec 5, 2012 at 4:32 PM, 叶雨飞 [off-list ref] wrote:

Thanks Eric!

Following is only to help someone else who may run into the problem and
searching for options:

the ksoftirqd load is generated by route lookup subrouting. since this is
DDOS traffic we are dealing (more than 10x of our normal PPS) and I have a
fixed number of destinations, I've tested that I can filter them efficiently
in mangle table PREROUTING chain, and use hashlimit to limit pps to send to
the routing subsystem, ksoftirq problem is gone and no kernel upgraded is
needed until the inevitable.

Cheers.


On Wed, Dec 5, 2012 at 3:48 PM, Eric Dumazet [off-list ref] wrote:

quoted

On Wed, 2012-12-05 at 13:22 -0800, 叶雨飞 wrote:

quoted

Hi Eric,

After disabling the route cache, I found a forwarding performance
problem (inevitable ?). Basically the kernel couldn't keep up under
about 100k pps and ksoftirqd is dominating the CPU. This problem went
away right away if I do

echo  1 > rt_cache_rebuild_count

and comes back as soon as i do echo -1 > rt_cache_rebuild_count.

I then tried to use RPS/RFS to share the load on to mulitple cpus
(since ksoftirqd is only using 1 core, clearly). but that has little
effect.

Is there some tweaks/patches you recommend?

Yes, upgrade your kernel to 3.6 or 3.7, this problem has been solved for
good. No more IP route cache and good performance.