[RFC PATCH v2 0/2] nb8800 suspend/resume support

STALE3286d

Revision v2 of 6 in this series.

6 messages, 3 authors, 2017-08-03 · open the first message on its own page

[RFC PATCH v2 0/2] nb8800 suspend/resume support

From: Måns Rullgård <hidden>
Date: 2017-08-02 16:10:06

Mason [off-list ref] writes:
On 02/08/2017 17:56, M?ns Rullg?rd wrote:
quoted
Mason writes:
quoted
From my perspective, the older method does not work on newer chips :-)
It does work on tango4.
Agreed.
quoted
What does the tango5 do if you flood it with packets faster than the
kernel can keep up with?  That would make it hit the end of the rx
chain, which is apparently what makes it miserable with the current dma
stop code.
The simplest way to test this would be sending tiny packets
as fast as possible, right? So ping -f on a GigE link should
fit the bill?
ping -f is limited to 100 packets per second.  Use something like iperf
in UDP mode instead.

-- 
M?ns Rullg?rd

[RFC PATCH v2 0/2] nb8800 suspend/resume support

From: David Laight <hidden>
Date: 2017-08-02 16:19:16

From: M?ns Rullg?rd
Sent: 02 August 2017 17:10
...
ping -f is limited to 100 packets per second.  Use something like iperf
in UDP mode instead.
Or break a MAC driver so it just saturates the network.

You might actually need bursts of traffic - otherwise the receiver
will be continuously out of rx buffers and no dma will be active.

	David

[RFC PATCH v2 0/2] nb8800 suspend/resume support

From: Mason <hidden>
Date: 2017-08-02 16:39:08

On 02/08/2017 18:10, M?ns Rullg?rd wrote:
Mason writes:
quoted
On 02/08/2017 17:56, M?ns Rullg?rd wrote:
quoted
What does the tango5 do if you flood it with packets faster than the
kernel can keep up with?  That would make it hit the end of the rx
chain, which is apparently what makes it miserable with the current dma
stop code.
The simplest way to test this would be sending tiny packets
as fast as possible, right? So ping -f on a GigE link should
fit the bill?
ping -f is limited to 100 packets per second.  Use something like iperf
in UDP mode instead.
ping -f can go 100 times faster than 100 pps:

# ping -f -q -c 150000 -s 300 172.27.64.45
PING 172.27.64.45 (172.27.64.45) 300(328) bytes of data.
--- 172.27.64.45 ping statistics ---
150000 packets transmitted, 150000 received, 0% packet loss, time 15035ms
rtt min/avg/max/mdev = 0.065/0.084/0.537/0.014 ms, ipg/ewma 0.100/0.087 ms


150,000 packets in 15 seconds = 10,000 pps

(172.27.64.45 is the tango5 board)

Ergo, dealing with 10,000 packets per second does not hose RX.

Regards.

[RFC PATCH v2 0/2] nb8800 suspend/resume support

From: Mason <hidden>
Date: 2017-08-02 17:31:19

On 02/08/2017 18:10, M?ns Rullg?rd wrote:
ping -f is limited to 100 packets per second.
Use something like iperf in UDP mode instead.
CLIENT: DESKTOP
# iperf3 -c 172.27.64.45 -u -b 950M
Connecting to host 172.27.64.45, port 5201
[  4] local 172.27.64.1 port 55533 connected to 172.27.64.45 port 5201
[ ID] Interval           Transfer     Bandwidth       Total Datagrams
[  4]   0.00-1.00   sec   102 MBytes   858 Mbits/sec  13091  
[  4]   1.00-2.00   sec   114 MBytes   953 Mbits/sec  14541  
[  4]   2.00-3.00   sec   114 MBytes   953 Mbits/sec  14542  
[  4]   3.00-4.00   sec   114 MBytes   953 Mbits/sec  14541  
[  4]   4.00-5.00   sec   114 MBytes   953 Mbits/sec  14542  
[  4]   5.00-6.00   sec   114 MBytes   953 Mbits/sec  14541  
[  4]   6.00-7.00   sec   114 MBytes   953 Mbits/sec  14535  
[  4]   7.00-8.00   sec   114 MBytes   953 Mbits/sec  14536  
[  4]   8.00-9.00   sec   114 MBytes   953 Mbits/sec  14543  
[  4]   9.00-10.00  sec   114 MBytes   953 Mbits/sec  14541  
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bandwidth       Jitter    Lost/Total Datagrams
[  4]   0.00-10.00  sec  1.10 GBytes   943 Mbits/sec  0.247 ms  132176/143788 (92%)  
[  4] Sent 143788 datagrams

iperf Done.


SERVER: TANGO BOARD
# iperf3 -s
Accepted connection from 172.27.64.1, port 44995
[  5] local 172.27.64.45 port 5201 connected to 172.27.64.1 port 55533
[ ID] Interval           Transfer     Bandwidth       Jitter    Lost/Total Datagrams
[  5]   0.00-1.00   sec  7.90 MBytes  66.2 Mbits/sec  0.218 ms  11365/12376 (92%)  
[  5]   1.00-2.00   sec  9.13 MBytes  76.6 Mbits/sec  0.230 ms  13381/14550 (92%)  
[  5]   2.00-3.00   sec  9.12 MBytes  76.5 Mbits/sec  0.290 ms  13372/14540 (92%)  
[  5]   3.00-4.00   sec  9.11 MBytes  76.4 Mbits/sec  0.292 ms  13369/14535 (92%)  
[  5]   4.00-5.00   sec  9.18 MBytes  77.0 Mbits/sec  0.178 ms  13374/14549 (92%)  
[  5]   5.00-6.00   sec  9.10 MBytes  76.3 Mbits/sec  0.228 ms  13367/14532 (92%)  
[  5]   6.00-7.00   sec  9.26 MBytes  77.7 Mbits/sec  0.607 ms  13356/14541 (92%)  
[  5]   7.00-8.00   sec  9.23 MBytes  77.4 Mbits/sec  0.507 ms  13364/14545 (92%)  
[  5]   8.00-9.00   sec  9.20 MBytes  77.1 Mbits/sec  0.215 ms  13351/14528 (92%)  
[  5]   9.00-10.00  sec  9.16 MBytes  76.9 Mbits/sec  0.188 ms  13356/14529 (92%)  
[  5]  10.00-10.04  sec   336 KBytes  72.2 Mbits/sec  0.247 ms  521/563 (93%)  
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bandwidth       Jitter    Lost/Total Datagrams
[  5]   0.00-10.04  sec  0.00 Bytes  0.00 bits/sec  0.247 ms  132176/143788 (92%)  


There is some packet loss, but the board doesn't lose connectivity.

With smaller packets...

# iperf3 -c 172.27.64.45 -u -b 950M -l 800
Connecting to host 172.27.64.45, port 5201
[  4] local 172.27.64.1 port 35197 connected to 172.27.64.45 port 5201
[ ID] Interval           Transfer     Bandwidth       Total Datagrams
[  4]   0.00-1.00   sec  90.6 MBytes   760 Mbits/sec  118724  
[  4]   1.00-2.00   sec   107 MBytes   894 Mbits/sec  139718  
[  4]   2.00-3.00   sec   106 MBytes   889 Mbits/sec  138918  
[  4]   3.00-4.00   sec   107 MBytes   895 Mbits/sec  139768  
[  4]   4.00-5.00   sec   106 MBytes   891 Mbits/sec  139275  
[  4]   5.00-6.00   sec   107 MBytes   895 Mbits/sec  139862  
[  4]   6.00-7.00   sec   107 MBytes   895 Mbits/sec  139825  
[  4]   7.00-8.00   sec   107 MBytes   895 Mbits/sec  139775  
[  4]   8.00-9.00   sec   107 MBytes   895 Mbits/sec  139849  
[  4]   9.00-10.00  sec   107 MBytes   895 Mbits/sec  139835  
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bandwidth       Jitter    Lost/Total Datagrams
[  4]   0.00-10.00  sec  1.02 GBytes   880 Mbits/sec  0.009 ms  564117/1375520 (41%)  
[  4] Sent 1375520 datagrams

iperf Done.


Accepted connection from 172.27.64.1, port 46151
[  5] local 172.27.64.45 port 5201 connected to 172.27.64.1 port 35197
[ ID] Interval           Transfer     Bandwidth       Jitter    Lost/Total Datagrams
[  5]   0.00-1.00   sec  60.8 MBytes   510 Mbits/sec  0.004 ms  33508/113252 (30%)  
iperf3: OUT OF ORDER - incoming packet = 147146 and received packet = 0 AND SP = 147497
iperf3: OUT OF ORDER - incoming packet = 146128 and received packet = 0 AND SP = 147690
iperf3: OUT OF ORDER - incoming packet = 146067 and received packet = 0 AND SP = 147863
iperf3: OUT OF ORDER - incoming packet = 147242 and received packet = 0 AND SP = 148039
iperf3: OUT OF ORDER - incoming packet = 163837 and received packet = 0 AND SP = 164363
[  5]   1.00-2.00   sec  61.0 MBytes   511 Mbits/sec  0.198 ms  59635/139649 (43%)  
iperf3: OUT OF ORDER - incoming packet = 286437 and received packet = 0 AND SP = 287226
iperf3: OUT OF ORDER - incoming packet = 302990 and received packet = 0 AND SP = 305710
[  5]   2.00-3.00   sec  61.5 MBytes   517 Mbits/sec  0.005 ms  58369/138944 (42%)  
iperf3: OUT OF ORDER - incoming packet = 427697 and received packet = 0 AND SP = 428089
iperf3: OUT OF ORDER - incoming packet = 441087 and received packet = 0 AND SP = 442963
iperf3: OUT OF ORDER - incoming packet = 453844 and received packet = 0 AND SP = 454238
[  5]   3.00-4.00   sec  61.7 MBytes   517 Mbits/sec  0.008 ms  58975/139795 (42%)  
[  5]   4.00-5.00   sec  61.3 MBytes   514 Mbits/sec  0.010 ms  58453/138795 (42%)  
[  5]   5.00-6.00   sec  61.2 MBytes   513 Mbits/sec  0.014 ms  60141/140338 (43%)  
[  5]   6.00-7.00   sec  61.3 MBytes   514 Mbits/sec  0.009 ms  59055/139384 (42%)  
iperf3: OUT OF ORDER - incoming packet = 993646 and received packet = 0 AND SP = 994370
[  5]   7.00-8.00   sec  61.0 MBytes   512 Mbits/sec  0.007 ms  60265/140210 (43%)  
iperf3: OUT OF ORDER - incoming packet = 1134516 and received packet = 0 AND SP = 1135942
[  5]   8.00-9.00   sec  62.6 MBytes   523 Mbits/sec  0.198 ms  57805/139795 (41%)  
iperf3: OUT OF ORDER - incoming packet = 1275612 and received packet = 0 AND SP = 1277716
[  5]   9.00-10.00  sec  64.0 MBytes   538 Mbits/sec  0.007 ms  56029/139888 (40%)  
[  5]  10.00-10.04  sec  2.74 MBytes   589 Mbits/sec  0.009 ms  1882/5470 (34%)  
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bandwidth       Jitter    Lost/Total Datagrams
[  5]   0.00-10.04  sec  0.00 Bytes  0.00 bits/sec  0.009 ms  564117/1375520 (41%)  
[SUM]  0.0-10.0 sec  13 datagrams received out-of-order


A lot more packet loss, but RX holds on.

Regards.

[RFC PATCH v2 0/2] nb8800 suspend/resume support

From: Mason <hidden>
Date: 2017-08-02 20:02:24

On 02/08/2017 19:31, Mason wrote:
# iperf3 -c 172.27.64.45 -u -b 950M
Connecting to host 172.27.64.45, port 5201
[  4] local 172.27.64.1 port 55533 connected to 172.27.64.45 port 5201
[ ID] Interval           Transfer     Bandwidth       Total Datagrams
[  4]   0.00-1.00   sec   102 MBytes   858 Mbits/sec  13091  
[  4]   1.00-2.00   sec   114 MBytes   953 Mbits/sec  14541  
114 MB in 14541 packets => 7840 bytes per packet
Is iperf3 sending jumbo frames??
In the nb8800 driver, RX_BUF_SIZE is only 1552,
how would it deal with jumbo frames... truncate?
# iperf3 -c 172.27.64.45 -u -b 950M -l 800
Connecting to host 172.27.64.45, port 5201
[  4] local 172.27.64.1 port 35197 connected to 172.27.64.45 port 5201
[ ID] Interval           Transfer     Bandwidth       Total Datagrams
[  4]   0.00-1.00   sec  90.6 MBytes   760 Mbits/sec  118724  
[  4]   1.00-2.00   sec   107 MBytes   894 Mbits/sec  139718  
107 MB in 139718 packets => 766 bytes per packet
800 - 8 (UDP) - 20 (IPv4) = 772 bytes per packet
I suppose that's close enough...
772 * 139718 = 107.86 MB

I need to run the test slightly slower, to prevent
packet loss at the sender.

Perhaps -b 0 or -b 800M

Regards.

[RFC PATCH v2 0/2] nb8800 suspend/resume support

From: Mason <hidden>
Date: 2017-08-03 08:34:31

On 02/08/2017 22:02, Mason wrote:
I need to run the test slightly slower, to prevent packet loss
at the sender.
# iperf3 -c 172.27.64.45 -u -b 0 -l 1000
Connecting to host 172.27.64.45, port 5201
[  4] local 172.27.64.1 port 42607 connected to 172.27.64.45 port 5201
[ ID] Interval           Transfer     Bandwidth       Total Datagrams
[  4]   0.00-1.00   sec   111 MBytes   931 Mbits/sec  116420  
[  4]   1.00-2.00   sec   111 MBytes   931 Mbits/sec  116390  
[  4]   2.00-3.00   sec   111 MBytes   930 Mbits/sec  116220  
[  4]   3.00-4.00   sec   111 MBytes   930 Mbits/sec  116310  
[  4]   4.00-5.00   sec   111 MBytes   931 Mbits/sec  116380  
[  4]   5.00-6.00   sec   111 MBytes   930 Mbits/sec  116280  
[  4]   6.00-7.00   sec   111 MBytes   931 Mbits/sec  116390  
[  4]   7.00-8.00   sec   111 MBytes   931 Mbits/sec  116370  
[  4]   8.00-9.00   sec   111 MBytes   931 Mbits/sec  116340  
[  4]   9.00-10.00  sec   111 MBytes   930 Mbits/sec  116310  
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bandwidth       Jitter    Lost/Total Datagrams
[  4]   0.00-10.00  sec  1.08 GBytes   931 Mbits/sec  0.009 ms  278644/1163363 (24%)  
[  4] Sent 1163363 datagrams

iperf Done.


# iperf3 -s
Accepted connection from 172.27.64.1, port 42966
[  5] local 172.27.64.45 port 5201 connected to 172.27.64.1 port 42607
[ ID] Interval           Transfer     Bandwidth       Jitter    Lost/Total Datagrams
[  5]   0.00-1.00   sec  81.1 MBytes   681 Mbits/sec  0.017 ms  26834/111909 (24%)  
[  5]   1.00-2.00   sec  84.2 MBytes   706 Mbits/sec  0.019 ms  28127/116384 (24%)  
[  5]   2.00-3.00   sec  84.2 MBytes   706 Mbits/sec  0.013 ms  27946/116204 (24%)  
[  5]   3.00-4.00   sec  84.5 MBytes   709 Mbits/sec  0.013 ms  27674/116311 (24%)  
[  5]   4.00-5.00   sec  84.6 MBytes   709 Mbits/sec  0.015 ms  27712/116387 (24%)  
[  5]   5.00-6.00   sec  84.5 MBytes   709 Mbits/sec  0.010 ms  27649/116265 (24%)  
[  5]   6.00-7.00   sec  84.3 MBytes   707 Mbits/sec  0.011 ms  27995/116382 (24%)  
[  5]   7.00-8.00   sec  84.3 MBytes   707 Mbits/sec  0.013 ms  27972/116387 (24%)  
[  5]   8.00-9.00   sec  84.3 MBytes   708 Mbits/sec  0.020 ms  27899/116343 (24%)  
[  5]   9.00-10.00  sec  84.4 MBytes   708 Mbits/sec  0.014 ms  27759/116305 (24%)  
[  5]  10.00-10.04  sec  3.25 MBytes   710 Mbits/sec  0.009 ms  1077/4486 (24%)  
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bandwidth       Jitter    Lost/Total Datagrams
[  5]   0.00-10.04  sec  0.00 Bytes  0.00 bits/sec  0.009 ms  278644/1163363 (24%)  


IIUC, sender (desktop system) sends datagrams as fast as possible.
Receiver (tango board) drops around 24% of all datagrams.
I think this invalidates the theory that exhausting RX descriptors
wedges RX DMA.

Regards.
Keyboard shortcuts
hback out one level
jnext message in thread
kprevious message in thread
ldrill in
Escclose help / fold thread tree
?toggle this help