Re: [RFC][PATCH 2/2] bgmac: pass received packet to the netif instead of copying it

On 2013-08-15 10:21 PM, Rafał Miłecki wrote:

2013/8/15 Felix Fietkau [off-list ref]:

quoted

On 2013-08-15 1:36 PM, Rafał Miłecki wrote:

quoted

2013/8/11 Rafał Miłecki [off-list ref]:

quoted

It makes more sense to allocate new (empty) skb and pass it to the
hardware. That way we avoid copying whole packet into new skb which
should result in better performance.

I did some testing of this patch using "perf" tool and iperf -s
running on the OpenWrt machine (with bgmac supported hardware).

So you can see that __copy_user_common usage has really decreased with
this patch!

Unfortunately it didn't result in better performance... no idea why :(

Running iperf on the router is not useful as an indicator of routing
performance. Please focus on tests where you only push traffic through
the router, not directly to it.

OK, so I started "iperf -s" on notebook plugged into WAN port, and
then played with "iperf -c" on notebook connected to LAN#2.

With some old 3.6.11 based OpenWrt build I got:
[ 4] 0.0-60.0 sec 690 MBytes 96.4 Mbits/sec

With very recent 3.10.4 based OpenWrt build:
[ 4] 0.0-60.0 sec 667 MBytes 93.2 Mbits/sec

After applying my patch on top of that 3.10.4:
[ 5] 0.0-60.0 sec 759 MBytes 106 Mbits/sec

And some dumps from "perf top":

3.10.4
6.75% [kernel] [k] __copy_user_common
6.73% [ip_tables] [k] ipt_do_table
4.33% [kernel] [k] arch_cpu_idle
3.96% [kernel] [k] arch_local_irq_restore
3.42% [bgmac] [k] 0x000007e0
3.35% [nf_conntrack] [k] nf_conntrack_proto_fini
2.72% [nf_conntrack] [k] nf_conntrack_in
2.50% [kernel] [k] __netif_receive_skb_core
2.42% [kernel] [k] r4k_dma_cache_inv
2.38% [kernel] [k] fib_table_lookup
2.20% [kernel] [k] dev_queue_xmit
2.11% [xt_conntrack] [k] 0x00000360
2.10% [kernel] [k] ip_route_input_noref
2.06% [nf_conntrack_ipv4] [k] need_ipv4_conntrack

3.10.4 + 0002-bgmac-pass-received-packet-to-the-netif-instead-of-c.patch
6.09% [ip_tables] [k] ipt_do_table
4.71% [kernel] [k] arch_cpu_idle
4.48% [bgmac] [k] 0x00000d7c
3.50% [nf_conntrack] [k] nf_conntrack_in
3.22% [kernel] [k] arch_local_irq_restore
3.16% [nf_conntrack] [k] nf_conntrack_proto_fini
2.88% [kernel] [k] __netif_receive_skb_core
2.78% [xt_conntrack] [k] 0x0000011c
2.69% [kernel] [k] r4k_dma_cache_inv
2.67% [iptable_nat] [k] 0x000002a0
2.36% [kernel] [k] ip_route_input_noref
2.27% [kernel] [k] ip_rcv
2.25% [nf_conntrack_ipv4] [k] need_ipv4_conntrack
2.23% [kernel] [k] nf_iterate

I've compiled bgmac into the kernel and it seems that the magic 0xd7c
was bgmac_poll.

I'm afraid this "perf top" output doesn't really tell us where to look
for optimizations :| I'll still try Felix ideas tomorrow, but I'm not
sure if they help, since there isn't __copy_user_common anymore in the
"perf top" output...

What's the CPU load while passing traffic without running perf?
Have you tested bridging performance?

- Felix
_______________________________________________
openwrt-devel mailing list
openwrt-devel@lists.openwrt.org
https://lists.openwrt.org/cgi-bin/mailman/listinfo/openwrt-devel