On Tue, Oct 30, 2012 at 01:23:52PM -0400, Konrad Rzeszutek Wilk wrote:

On Tue, Oct 30, 2012 at 12:53:09PM -0400, Konrad Rzeszutek Wilk wrote:

quoted

On Wed, Oct 24, 2012 at 06:43:20PM +0200, Eric Dumazet wrote:

quoted

On Wed, 2012-10-24 at 17:22 +0100, Ian Campbell wrote:

quoted

On Wed, 2012-10-24 at 16:21 +0100, Eric Dumazet wrote:

quoted

quoted

If you really have such problems, why locally generated TCP traffic
doesnt also have it ?

I think it does. The reason I noticed the original problem was that ssh
to the machine was virtually (no pun intended) unusable.

quoted

Your patch doesnt touch sk_page_frag_refill(), does it ?

That's right. It doesn't. When is (sk->sk_allocation & __GFP_WAIT) true?
Is it possible I'm just not hitting that case?

I hope not. GFP_KERNEL has __GFP_WAIT.

quoted

Is it possible that this only affects certain traffic patterns (I only
really tried ssh/scp and ping)? Or perhaps its just that the swiotlb is
only broken in one corner case and not the other.

Could you try a netperf -t TCP_STREAM ?

For fun I did a couple of tests - I setup two machines (one r8168, the other
e1000e) and tried to do netperf/netserver. Both of them are running a baremetal
kernel and one of them has 'iommu=soft swiotlb=force' to simulate the worst
case. This is using v3.7-rc3.

I also did a test with the patch at the top, with the same setup and ... it
does look like it fixes some issues, but not the underlaying one.

The same test, with net.core.netdev_frag_page_max_order=0, the e1000e->r8169
gets ~124, but then on subsequent runs it picks up to ~933. If I let the
machine stay a bit idle and then do this again, it does around ~124 again.

Thoughts?

Argh. Please disregard this test. I added an extra patch in the kernel
tree to track the SWIOTLB bounce usage and print it.. and the printk was
going out to the FB, which was not too fast - so the whole test was being
slowed down by FB drivers :-)

Will re-run this test without the offending patch.