Re: [Lse-tech] Re: (RFC): SKB Initialization

On Sun, 25 Aug 2002, Mala Anand wrote:

Jamal wrote ..

quoted

Can you repeat your tests with the hotlist turned off (i.e set to 0)?

 Even if I turned the hot list, the slab allocator has a per cpu array of
objects. In this case it keeps by default 60 objects and hot list keeps
126 objects. So there may be a difference. I will try this.

Well, the hotlist is supposed to be one way to reduce the effect of
initialization.

 This skb init work is the result of my probing in to the slab cache work.
Read
my posting on slab cache:
http://marc.theaimsgroup.com/?l=linux-kernel&m=102773718023056&w=2
This work triggered the skb init patch. To quantify the effect of bouncing
the objects between cpus, I choose skb to measure. And it turns out that
the
limited cpu array is not the culprit in this case, it is how the objects
are
allocated in one cpu and freed in another cpu is what causing the bouncing
of objects between cpus.

Same conclusion as us then

quoted

Also Robert and i did a few tests and we did find skb recycling (based on
a patch from Robert a few years back) was infact giving perfomance
improvements of upto 15% over regular slab.
Did you test with that patch for the e1000 he pointed you at?
I repeated the tests (around June/July) with the tulip with input rates of
a few 100K packets/sec and noticed a improvement over regular NAPI by
about 10%. Theres one bug on the tulip which we are chasing that
might be related to tulips alignment requirements;

Yes I got the patch from Robert and I am planning on testing the patch. My
understanding is that skbs are recylced in other operating systems as well
to
improve performance. And it particularly helps in architectures where pci
mapping is expensive and when skbs are recycled, remapping is eliminated.

standard practise for eons on networking in rtoses at least.
Slab on Linux was supposed to get rid of this. Thats why Robert
hid his original patch.

I think the skbinit patch and recycling skbs are mutually exclusive.

I would say they are more orthogonal than mutually exclusive.
Although ou still need to prove that relocating the code actually helps in
real life. On paper it looks good.

Recycling
skbs will reduce the number of times we hit alloc_skb and __kfree_skb.

Thats what we see.

quoted

The idea of only freeing on the same CPU a skb allocated is free with
the e1000 NAPI driver style but not in the tulip NAPI  where a txmit
interupt might happen on a different CPU. The skb recycler patch only
recylces if allocation and freeing are happening on the same CPU;
otherwise we let the slab take the hit. On the tulip this happens about
50% of the time.

 So skbinit patch will help the other case.

It will. Note, however that on the e1000 style coding, the hit ration is
much higher -- almost 100% in theory at least.
Perhaps its time to convert the tulip ...

cheers,
jamal