Hi,
I'm testing htb performance on a gigabit router running 2.6.32 kernel.
Packet path: PC1 ---> Router LAN port ---> Router WAN port ---> PC2
pfifo_fast: 920Mbps
htb: 750Mbps, ~20% drops compared to pfifo_fast
htb tc commands as below,
# tc qdisc add dev eth10 root handle 20: htb default 1
# tc class add dev eth10 parent 20 classid 20:1 htb prio 2 rate
1024Mbit ceil 1024Mbit burst 1281408b cburst 1281408b
The performance drop seems caused by the complex htb enqueue/dequeue algorithm.
I had a quick look at CoDel code, seems it does not have so complex
data structure as HTB.
I'm going to backport CoDel. Is this a good choice?
Can I gain similar performance as pfifo_fast?
Thanks,
Lin Ming
From: Eric Dumazet <hidden> Date: 2012-09-18 09:45:16
On Tue, 2012-09-18 at 17:28 +0800, Lin Ming wrote:
Hi,
I'm testing htb performance on a gigabit router running 2.6.32 kernel.
Packet path: PC1 ---> Router LAN port ---> Router WAN port ---> PC2
pfifo_fast: 920Mbps
htb: 750Mbps, ~20% drops compared to pfifo_fast
htb tc commands as below,
# tc qdisc add dev eth10 root handle 20: htb default 1
# tc class add dev eth10 parent 20 classid 20:1 htb prio 2 rate
1024Mbit ceil 1024Mbit burst 1281408b cburst 1281408b
The performance drop seems caused by the complex htb enqueue/dequeue algorithm.
I had a quick look at CoDel code, seems it does not have so complex
data structure as HTB.
I'm going to backport CoDel. Is this a good choice?
Can I gain similar performance as pfifo_fast?
codel is quite different than HTB : It has no rate control, so its very
fast. (But it has no prio differentiation as pfifo_fast with its 3
bands)
So what are your exact needs ?
On Tue, Sep 18, 2012 at 5:45 PM, Eric Dumazet [off-list ref] wrote:
On Tue, 2012-09-18 at 17:28 +0800, Lin Ming wrote:
quoted
Hi,
I'm testing htb performance on a gigabit router running 2.6.32 kernel.
Packet path: PC1 ---> Router LAN port ---> Router WAN port ---> PC2
pfifo_fast: 920Mbps
htb: 750Mbps, ~20% drops compared to pfifo_fast
htb tc commands as below,
# tc qdisc add dev eth10 root handle 20: htb default 1
# tc class add dev eth10 parent 20 classid 20:1 htb prio 2 rate
1024Mbit ceil 1024Mbit burst 1281408b cburst 1281408b
The performance drop seems caused by the complex htb enqueue/dequeue algorithm.
I had a quick look at CoDel code, seems it does not have so complex
data structure as HTB.
I'm going to backport CoDel. Is this a good choice?
Can I gain similar performance as pfifo_fast?
codel is quite different than HTB : It has no rate control, so its very
fast. (But it has no prio differentiation as pfifo_fast with its 3
bands)
So what are your exact needs ?
I need traffic priority/traffic shaping/rate control ... actually all
QoS features on the router.
And if I just set the rate to gigabit(no other settings), for example,
# tc qdisc add dev eth10 root handle 20: htb default 1
# tc class add dev eth10 parent 20 classid 20:1 htb prio 2 rate
1024Mbit ceil 1024Mbit burst 1281408b cburst 1281408b
it should gain similar performance as pfifo_fast.
codel has no rate control. So seems I have to find way to optimize htb?
From: Eric Dumazet <hidden> Date: 2012-09-18 10:15:58
On Tue, 2012-09-18 at 17:56 +0800, Lin Ming wrote:
I need traffic priority/traffic shaping/rate control ... actually all
QoS features on the router.
And if I just set the rate to gigabit(no other settings), for example,
# tc qdisc add dev eth10 root handle 20: htb default 1
# tc class add dev eth10 parent 20 classid 20:1 htb prio 2 rate
1024Mbit ceil 1024Mbit burst 1281408b cburst 1281408b
it should gain similar performance as pfifo_fast.
codel has no rate control. So seems I have to find way to optimize htb?
Are you really cpu limited ? You might hit some clocks artifacts.
rate limiting to 1Gbps probably need high resolution timers.
HTB is not the only way to rate limit.
On Tue, Sep 18, 2012 at 6:15 PM, Eric Dumazet [off-list ref] wrote:
On Tue, 2012-09-18 at 17:56 +0800, Lin Ming wrote:
quoted
I need traffic priority/traffic shaping/rate control ... actually all
QoS features on the router.
And if I just set the rate to gigabit(no other settings), for example,
# tc qdisc add dev eth10 root handle 20: htb default 1
# tc class add dev eth10 parent 20 classid 20:1 htb prio 2 rate
1024Mbit ceil 1024Mbit burst 1281408b cburst 1281408b
it should gain similar performance as pfifo_fast.
codel has no rate control. So seems I have to find way to optimize htb?
Are you really cpu limited ? You might hit some clocks artifacts.
Did you mean the cpu speed? It's an ARMv5 processor.
BogoMIPS : 1196.03
rate limiting to 1Gbps probably need high resolution timers.
From: Rick Jones <hidden> Date: 2012-09-19 17:00:35
On 09/18/2012 06:26 PM, Lin Ming wrote:
On Tue, Sep 18, 2012 at 6:15 PM, Eric Dumazet [off-list ref] wrote:
quoted
Are you really cpu limited ? You might hit some clocks artifacts.
Did you mean the cpu speed? It's an ARMv5 processor.
BogoMIPS : 1196.03
At the risk of typing into Eric's keyboard, he was asking if you were
saturating the CPU - was it getting to 100% utilization, that sort of thing.
quoted
rate limiting to 1Gbps probably need high resolution timers.
High resolution timer is enabled.
When you are running your tests, what sort of CPU utilization do you see
on the CPU of your router with HTB on vs off. Some "quick and dirty"
netperf tests on an Centrino-based laptop suggested that as an end
system at least, HTB at 1 GbE (using your tc commands) increases service
demand (what netperf calculates as CPU consumed per unit of work
performed) ~15% for bulk transfer (netperf TCP_STREAM) and ~18% for
small packet request/response (TCP_RR).
rick jones
On Thu, Sep 20, 2012 at 1:00 AM, Rick Jones [off-list ref] wrote:
On 09/18/2012 06:26 PM, Lin Ming wrote:
quoted
On Tue, Sep 18, 2012 at 6:15 PM, Eric Dumazet [off-list ref]
wrote:
quoted
Are you really cpu limited ? You might hit some clocks artifacts.
Did you mean the cpu speed? It's an ARMv5 processor.
BogoMIPS : 1196.03
At the risk of typing into Eric's keyboard, he was asking if you were
saturating the CPU - was it getting to 100% utilization, that sort of thing.
Yes, ~98% utilization.
quoted
quoted
rate limiting to 1Gbps probably need high resolution timers.
High resolution timer is enabled.
When you are running your tests, what sort of CPU utilization do you see on
the CPU of your router with HTB on vs off. Some "quick and dirty" netperf
HTB off: ~85%
HTB on: ~98%
tests on an Centrino-based laptop suggested that as an end system at least,
HTB at 1 GbE (using your tc commands) increases service demand (what netperf
calculates as CPU consumed per unit of work performed) ~15% for bulk
transfer (netperf TCP_STREAM) and ~18% for small packet request/response
(TCP_RR).
rick jones