On 2016/4/11 20:13, Eric Dumazet wrote:

On Mon, 2016-04-11 at 19:57 +0800, Yang Yingliang wrote:

quoted

On 2016/4/8 22:44, Eric Dumazet wrote:

quoted

On Fri, 2016-04-08 at 19:18 +0800, Yang Yingliang wrote:

quoted

I expand  tcp_adv_win_scale and tcp_rmem. It has no effect.

Try :

echo -2 >/proc/sys/net/ipv4/tcp_adv_win_scale

And restart your flows.

cat /proc/sys/net/ipv4/tcp_rmem
10240 2097152 10485760

What about leaving the default values ?

I tried, it did not work.

$ cat /proc/sys/net/ipv4/tcp_rmem
4096	87380	6291456

quoted

echo 102400 20971520 104857600 > /proc/sys/net/ipv4/tcp_rmem
echo -2 >/proc/sys/net/ipv4/tcp_adv_win_scale

It seems has not effect.

I have no idea what you did on the sender side to allow it to send more
than 1.5 MB then.

We are doing performance test. The sender send 256KB per-block with 128
threads to one socket. And the receiver uses 10Gb NIC to handle the
data on ARM64. The data flow is driver->ip layer->tcp layer->iscsi.

I added some debug messages and found handling backlog packets in 
__release_sock() cost about 11ms at most. This can cause backlog queue
overflow. The sk_data_ready is re-assigned, it may cost time in our
program. I will check it out.