2011/10/27 Eric Dumazet [off-list ref]:

Le jeudi 27 octobre 2011 à 18:08 +0200, Eric Dumazet a écrit :

quoted

Le jeudi 27 octobre 2011 à 14:46 +0200, Karel Rericha a écrit :

quoted

Actually I am doing some reseach to replace our main shaping machine
with 60 000+ htb classes, which now saturates 12 core Xeon Westmere to
30% (there are five gigabit network ports on each interface affinited
to cores). AFAIK QFQ should be O(1) complexity so it would bring
saturation a requirements for number of cores down considerably (HTB
has O(log(N)) complexity).

I have test machine and about two months to decide if we will stay
with HTB or we will try something else. So it would be VERY helpful,
if you would search you memory instead your dead disk :-) and send me
some example of QFQ usage, if I can ask for a little of your time. I
promise to have results published here in return.

Thanks, Karel

That seems a good challenge to me ;)

First upgrade to a recent kernel with QFQ included.
Also upgrade iproute2 to a recent enough version as well.

Then you discover "tc  ... qfq help" is not that helpful :(

# tc qdisc add dev eth3 root qfq help
Usage: ... qfq

OK, its parameters are :

      qfq weight num1 [maxpkt BYTES]

You should not touch maxpkt, its default value being 2048

Oh well, I just tried the obvious and my (remote) machine doesnt answer
to me anymore...

Time for a bit of debugging I am afraid :(

Never mind, it was an user error :)

Here is what I used during my tests, I guess you can adapt your
scripts...

DEV=eth3
RATE="rate 40Mbit"
TNETS="10.2.2.0/25"
ALLOT="allot 20000"

tc qdisc del dev dummy0 root 2>/dev/null

tc qdisc add dev $DEV root handle 1: cbq avpkt 1000 rate 1000Mbit \
       bandwidth 1000Mbit
tc class add dev $DEV parent 1: classid 1:1 \
       est 1sec 8sec cbq allot 10000 mpu 64 \
       rate 1000Mbit prio 1 avpkt 1500 bounded

# output to test nets :  40 Mbit limit
tc class add dev $DEV parent 1:1 classid 1:11 \
       est 1sec 8sec cbq $ALLOT mpu 64      \
       $RATE prio 2 avpkt 1400 bounded

tc qdisc add dev $DEV parent 1:11 handle 11:  \
       est 1sec 8sec qfq

tc filter add dev $DEV protocol ip parent 11: handle 3 \
       flow hash keys rxhash divisor 8

for i in `seq 1 8`
do
 classid=11:$(printf %x $i)
 tc class add dev $DEV classid $classid qfq
 tc qdisc add dev $DEV parent $classid pfifo limit 30
done

for privnet in $TNETS
do
       tc filter add dev $DEV parent 1: protocol ip prio 100 u32 \
               match ip dst $privnet flowid 1:11
done

tc filter add dev $DEV parent 1: protocol ip prio 100 u32 \
       match ip protocol 0 0x00 flowid 1:1

iperf -u -c 10.2.2.1 -P 32 -l 50

Thanks for example Eric.

But it only added more confusion to me now :-) I was under impression
(and read somewhere) that QFQ is non work conserving scheduler so I
can use it more or less like HTB or HFSC to set bandwidth constraints
to flows. But from this example (and from sources/patches/papers I try
not to pretend I fully understand) it looks to me like some multiqueue
scheduler with arbitrary number of queues and ability to arbitrary
assign flows to this queues. So some sort of fair division of
available bandwidth to flows without arbitrary bandwidth caps to these
flows. I really dont see what is non work conserving here :-S

Please save my soul and enlighten me because I am at dead end now :-)