RE: DPAA Ethernet problems with mainstream Linux kernels

-----Original Message-----
From: Linuxppc-dev [mailto:linuxppc-dev-
bounces+madalin.bucur=nxp.com@lists.ozlabs.org] On Behalf Of Jamie Krueger
Sent: Wednesday, January 10, 2018 5:57 PM
To: linuxppc-dev@lists.ozlabs.org
Subject: DPAA Ethernet problems with mainstream Linux kernels

Hello all @ linuxppc-dev,

I have been working with a team of people maintaining PowerPC
Linux for the new AmigaONE X5000/20 (a Freescale p5020 SoC based
machine).

We are trying to determine why the submitted Data Path Acceleration
Architecture (DPAA) Ethernet Driver is not fully functional with
the mainstream Linux kernels.

Hi Jamie,

We are testing the DPAA driver on several DS and RDB platforms and it
is working properly. The issues you encounter with it on the X5000/20
are likely caused by some issues specific to that particular platform.
The device tree that you mention, cyrus_p5020.eth.dts is not found in
the Linux kernel sources. The cyrus_p5020.dts file from the fsl ppc
device tree folder does not include the PHY information for the DPAA
interfaces. The problems that you experience may be caused by some
issues with the PHY configuration (i.e. internal delay). I suggest
that you connect the DPAA interface to a traffic analyzer or directly
to another device on which you can capture the incoming traffic and
check that the received frames are correct.

Madalin

On 01/12/2018 08:22 AM, Madalin-cristian Bucur wrote:

quoted

-----Original Message-----
From: Linuxppc-dev [mailto:linuxppc-dev-
bounces+madalin.bucur=nxp.com@lists.ozlabs.org] On Behalf Of Jamie Krueger
Sent: Wednesday, January 10, 2018 5:57 PM
To: linuxppc-dev@lists.ozlabs.org
Subject: DPAA Ethernet problems with mainstream Linux kernels

Hello all @ linuxppc-dev,

I have been working with a team of people maintaining PowerPC
Linux for the new AmigaONE X5000/20 (a Freescale p5020 SoC based
machine).

We are trying to determine why the submitted Data Path Acceleration
Architecture (DPAA) Ethernet Driver is not fully functional with
the mainstream Linux kernels.

Hi Jamie,

Hi Madalin,

We are testing the DPAA driver on several DS and RDB platforms and it
is working properly. The issues you encounter with it on the X5000/20
are likely caused by some issues specific to that particular platform.

It is good to hear that the DPAA driver is functioning correctly
on the reference platforms. I am positive you are correct that
the issue is the difference in implementation on the X5000/20
(Cyrus) motherboard, as compared to the reference boards.

Can you verify which Linux Kernel sources your tests are being
performed on? We have been testing using the mainstream
Linux sources up to linux-4.15-rc6 thus far.

The device tree that you mention, cyrus_p5020.eth.dts is not found in
the Linux kernel sources. The cyrus_p5020.dts file from the fsl ppc
device tree folder does not include the PHY information for the DPAA
interfaces. The problems that you experience may be caused by some
issues with the PHY configuration (i.e. internal delay).

The cyrus_p5020.eth.dts is a modified version of the cyrus_p5020.dts,
which of course was based off the original p5020ds.dts file. As you
noted, the current cyrus_p5020.dts file is incomplete, and does not
map the Ethernet connections properly.

The cyrus_p5020.eth.dts file, along with it's cyrus-pre.dtsi dependent
file, are an attempt to correctly define the Ethernet hardware, as it is
implemented on the X5000/20.

** I have attached both the cyrus_p5020.eth.dts and cyrus-pre.dtsi
      files with this email for comparison. Please let me know if you see
      any corrections that should be made to either file.

I am not sure what PHY hardware/configuration you are using on the
DS and RDB platforms, but I can confirm that AmigaONE X5000/20
(Cyrus Motherboard with p5020 SoC), has dTSEC 4 and dTSEC 5
wired to two Micrel KSZ9021RN Gigabit Ethernet PHYs, using the
RGMII protocol.

  I suggest
that you connect the DPAA interface to a traffic analyzer or directly
to another device on which you can capture the incoming traffic and
check that the received frames are correct.

I have started testing along that line, using Wireshark to view the
traffic on the X5000/20 itself, and from another machine connected
on the same subnet. So far (as indicated by some details of in my
initial email), I can see outgoing broadcast requests (for DHCP)
being sent out from the X5000/20, and these requests are correctly
constructed and visible outside the X5000/20.

However, no responses to the DHCP broadcasts appear to reach
to X5000/20's DPAA Ethernet. I will need to setup some further
tests to determine if the DHCP server saw the requests and responded
to them. (I assume the DHCP server is getting them, and responding,
as I can always get a successful DHCP response to the X5000/20
when using an add-on Ethernet PICe card on the same subnet).

I will setup some more direct machine-to-machine testing to
see what else I can glean from the network traffic.

Please have a look at the attached dts files, maybe there is something
obvious there we are not seeing.

Also, given that the X5000/20 uses Micrel KSZ9021RN PHYs in RGMII
mode, what changes to the DPAA hardware configuration should we
expect to see so that the DPAA is configured to talk to them?

Madalin

-- 

Best Regards,

Jamie Krueger
BITbyBIT Software Group LLC

quoted

Here is the results from my latest tests. They were performed using
the linux-4.10.17 ppc64, since that represents when the DPAA Ethernet
code was introduced.

Similar tests, with similar results, were also performed
using the latest Linux kernels:

linux-4.15-rc5
linux-4.15-rc6
linux-4.15-rc7

(Hence the reason for falling back to test the kernel right
   after the introduction of the DPAA Ethernet driver sources)

---

All Kernel builds had the DPAA Ethernet enabled in the kernel,
and are using the correct cyrus_p5020.eth.dtb device tree file
(for use on the X5000/20).

The results are quite similar for all kernels in regards to the DPAA
Ethernet.

All tested kernels setup the two Ethernet interfaces correctly
as eth0 and eth1, and pull the correct MAC addresses from U-Boot
environment variables ethaddr and eth1addr respectively.

So at this point Linux has what it believes is fully configured
hardware, waiting to have an IP Address/Netmask/Gateway
to be set and to bring the interface online.

However, all attempts to communicate with the outside world
do not make it out the physical (PHY) hardware - or do they?

** The following results were captured under linux-4.10.17 **

When I bring the interface up using a static address, in this case
192.168.1.21, I see the following (NOTE TX bytes says 154.0 KB,
while RX bytes says 0.0 B):

jamie@X5000-Linux:$ ifconfig
eth0      Link encap:Ethernet  HWaddr 00:80:10:11:11:11
            inet addr:192.168.1.21  Bcast:192.168.1.255 Mask:255.255.255.0
            inet6 addr: fe80::280:10ff:fe11:1111/64 Scope:Link
            UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
            RX packets:0 errors:0 dropped:0 overruns:0 frame:0
            TX packets:1428 errors:0 dropped:0 overruns:0 carrier:0
            collisions:0 txqueuelen:1000
            RX bytes:0 (0.0 B)  TX bytes:154066 (154.0 KB)
            Memory:fe4e6000-fe4e6fff

eth1      Link encap:Ethernet  HWaddr 00:80:10:22:22:22
            UP BROADCAST MULTICAST  MTU:1500  Metric:1
            RX packets:0 errors:0 dropped:0 overruns:0 frame:0
            TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
            collisions:0 txqueuelen:1000
            RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)
            Memory:fe4e8000-fe4e8fff

lo        Link encap:Local Loopback
            inet addr:127.0.0.1  Mask:255.0.0.0
            inet6 addr: ::1/128 Scope:Host
            UP LOOPBACK RUNNING  MTU:65536  Metric:1
            RX packets:1869 errors:0 dropped:0 overruns:0 frame:0
            TX packets:1869 errors:0 dropped:0 overruns:0 carrier:0
            collisions:0 txqueuelen:1000
            RX bytes:156932 (156.9 KB)  TX bytes:156932 (156.9 KB)

Checking the routing table, everything looks fine there:

jamie@X5000-Linux:$ netstat -r
Kernel IP routing table
Destination     Gateway         Genmask         Flags   MSS Window  irtt
Iface
default         192.168.1.1     0.0.0.0         UG        0 0          0
eth0
link-local      *               255.255.0.0     U         0 0          0
eth0
192.168.1.0     *               255.255.255.0   U         0 0          0
eth0

Attempting to PING the interface itself works:

jamie@X5000-Linux:$ ping 192.168.1.21
PING 192.168.1.21 (192.168.1.21) 56(84) bytes of data.
64 bytes from 192.168.1.21: icmp_seq=1 ttl=64 time=0.037 ms
64 bytes from 192.168.1.21: icmp_seq=2 ttl=64 time=0.045 ms
64 bytes from 192.168.1.21: icmp_seq=3 ttl=64 time=0.033 ms
^C

--- 192.168.1.21 ping statistics ---

3 packets transmitted, 3 received, 0% packet loss, time 2043ms

However, attempts to PING the gateway (192.168.1.1) fail as unreachable:

jamie@X5000-Linux:$ ping 192.168.1.1
PING 192.168.1.1 (192.168.1.1) 56(84) bytes of data.
  From 192.168.1.21 icmp_seq=1 Destination Host Unreachable
  From 192.168.1.21 icmp_seq=2 Destination Host Unreachable
  From 192.168.1.21 icmp_seq=3 Destination Host Unreachable
^C

--- 192.168.1.1 ping statistics ---

7 packets transmitted, 0 received, +3 errors, 100% packet loss, time
6077ms

In order to take a closer look at what is going on I installed Wireshark
both on my test X5000/20 Linux install (Ubuntu 16.04.3 LTS), and on
another Linux box connected to the same network switch (in this case
at IP address 192.168.1.210)

In this test I start the capture on eth0 (X5000/20) before it is put
online,
and attempt to bring it up using DHCP to obtain it's address.

What I found was that network traffic *was* being attempted over eth0.
Here is a plain text export of the transmit side (the X5000/20) that
was captured using Wireshark.

(There were more network packets being sent from the X5000/20,
   however, I am only showing DHCP traffic to save space in this post):

---
No.     Time           Source                Destination Protocol Length
Info
        2 0.042259843    0.0.0.0               255.255.255.255 DHCP
342    DHCP Discover - Transaction ID 0x655d91e8

Frame 2: 342 bytes on wire (2736 bits), 342 bytes captured (2736 bits)
on interface 0
Ethernet II, Src: Commodor_11:11:11 (00:80:10:11:11:11), Dst: Broadcast
(ff:ff:ff:ff:ff:ff)
Internet Protocol Version 4, Src: 0.0.0.0, Dst: 255.255.255.255
User Datagram Protocol, Src Port: 68, Dst Port: 67
Bootstrap Protocol (Discover)

No.     Time           Source                Destination Protocol Length
Info
       16 3.830001152    0.0.0.0               255.255.255.255 DHCP
342    DHCP Discover - Transaction ID 0x655d91e8

Frame 16: 342 bytes on wire (2736 bits), 342 bytes captured (2736 bits)
on interface 0
Ethernet II, Src: Commodor_11:11:11 (00:80:10:11:11:11), Dst: Broadcast
(ff:ff:ff:ff:ff:ff)
Internet Protocol Version 4, Src: 0.0.0.0, Dst: 255.255.255.255
User Datagram Protocol, Src Port: 68, Dst Port: 67
Bootstrap Protocol (Discover)

No.     Time           Source                Destination Protocol Length
Info
       21 9.308914533    0.0.0.0               255.255.255.255 DHCP
342    DHCP Discover - Transaction ID 0x655d91e8

Frame 21: 342 bytes on wire (2736 bits), 342 bytes captured (2736 bits)
on interface 0
Ethernet II, Src: Commodor_11:11:11 (00:80:10:11:11:11), Dst: Broadcast
(ff:ff:ff:ff:ff:ff)
Internet Protocol Version 4, Src: 0.0.0.0, Dst: 255.255.255.255
User Datagram Protocol, Src Port: 68, Dst Port: 67
Bootstrap Protocol (Discover)

No.     Time           Source                Destination Protocol Length
Info
       23 18.906405343   0.0.0.0               255.255.255.255 DHCP
342    DHCP Discover - Transaction ID 0x655d91e8

Frame 23: 342 bytes on wire (2736 bits), 342 bytes captured (2736 bits)
on interface 0
Ethernet II, Src: Commodor_11:11:11 (00:80:10:11:11:11), Dst: Broadcast
(ff:ff:ff:ff:ff:ff)
Internet Protocol Version 4, Src: 0.0.0.0, Dst: 255.255.255.255
User Datagram Protocol, Src Port: 68, Dst Port: 67
Bootstrap Protocol (Discover)

No.     Time           Source                Destination Protocol Length
Info
       25 36.390926450   0.0.0.0               255.255.255.255 DHCP
342    DHCP Discover - Transaction ID 0x655d91e8

Frame 25: 342 bytes on wire (2736 bits), 342 bytes captured (2736 bits)
on interface 0
Ethernet II, Src: Commodor_11:11:11 (00:80:10:11:11:11), Dst: Broadcast
(ff:ff:ff:ff:ff:ff)
Internet Protocol Version 4, Src: 0.0.0.0, Dst: 255.255.255.255
User Datagram Protocol, Src Port: 68, Dst Port: 67
Bootstrap Protocol (Discover)

No.     Time           Source                Destination Protocol Length
Info
       26 44.048328412   0.0.0.0               255.255.255.255 DHCP
342    DHCP Discover - Transaction ID 0x655d91e8

Frame 26: 342 bytes on wire (2736 bits), 342 bytes captured (2736 bits)
on interface 0
Ethernet II, Src: Commodor_11:11:11 (00:80:10:11:11:11), Dst: Broadcast
(ff:ff:ff:ff:ff:ff)
Internet Protocol Version 4, Src: 0.0.0.0, Dst: 255.255.255.255
User Datagram Protocol, Src Port: 68, Dst Port: 67
Bootstrap Protocol (Discover)

No.     Time           Source                Destination Protocol Length
Info
       30 44.889049203   0.0.0.0               255.255.255.255 DHCP
342    DHCP Discover - Transaction ID 0x421bade3

Frame 30: 342 bytes on wire (2736 bits), 342 bytes captured (2736 bits)
on interface 0
Ethernet II, Src: Commodor_11:11:11 (00:80:10:11:11:11), Dst: Broadcast
(ff:ff:ff:ff:ff:ff)
Internet Protocol Version 4, Src: 0.0.0.0, Dst: 255.255.255.255
User Datagram Protocol, Src Port: 68, Dst Port: 67
Bootstrap Protocol (Discover)

No.     Time           Source                Destination Protocol Length
Info
       44 48.254495304   0.0.0.0               255.255.255.255 DHCP
342    DHCP Discover - Transaction ID 0x421bade3

Frame 44: 342 bytes on wire (2736 bits), 342 bytes captured (2736 bits)
on interface 0
Ethernet II, Src: Commodor_11:11:11 (00:80:10:11:11:11), Dst: Broadcast
(ff:ff:ff:ff:ff:ff)
Internet Protocol Version 4, Src: 0.0.0.0, Dst: 255.255.255.255
User Datagram Protocol, Src Port: 68, Dst Port: 67
Bootstrap Protocol (Discover)

No.     Time           Source                Destination Protocol Length
Info
       49 54.299052732   0.0.0.0               255.255.255.255 DHCP
342    DHCP Discover - Transaction ID 0x421bade3

Frame 49: 342 bytes on wire (2736 bits), 342 bytes captured (2736 bits)
on interface 0
Ethernet II, Src: Commodor_11:11:11 (00:80:10:11:11:11), Dst: Broadcast
(ff:ff:ff:ff:ff:ff)
Internet Protocol Version 4, Src: 0.0.0.0, Dst: 255.255.255.255
User Datagram Protocol, Src Port: 68, Dst Port: 67
Bootstrap Protocol (Discover)

No.     Time           Source                Destination Protocol Length
Info
       51 62.672007482   0.0.0.0               255.255.255.255 DHCP
342    DHCP Discover - Transaction ID 0x421bade3

Frame 51: 342 bytes on wire (2736 bits), 342 bytes captured (2736 bits)
on interface 0
Ethernet II, Src: Commodor_11:11:11 (00:80:10:11:11:11), Dst: Broadcast
(ff:ff:ff:ff:ff:ff)
Internet Protocol Version 4, Src: 0.0.0.0, Dst: 255.255.255.255
User Datagram Protocol, Src Port: 68, Dst Port: 67
Bootstrap Protocol (Discover)

No.     Time           Source                Destination Protocol Length
Info
       52 77.485896202   0.0.0.0               255.255.255.255 DHCP
342    DHCP Discover - Transaction ID 0x421bade3

Frame 52: 342 bytes on wire (2736 bits), 342 bytes captured (2736 bits)
on interface 0
Ethernet II, Src: Commodor_11:11:11 (00:80:10:11:11:11), Dst: Broadcast
(ff:ff:ff:ff:ff:ff)
Internet Protocol Version 4, Src: 0.0.0.0, Dst: 255.255.255.255
User Datagram Protocol, Src Port: 68, Dst Port: 67
Bootstrap Protocol (Discover)

No.     Time           Source                Destination Protocol Length
Info
       56 89.895304152   0.0.0.0               255.255.255.255 DHCP
342    DHCP Discover - Transaction ID 0x5df47c84

Frame 56: 342 bytes on wire (2736 bits), 342 bytes captured (2736 bits)
on interface 0
Ethernet II, Src: Commodor_11:11:11 (00:80:10:11:11:11), Dst: Broadcast
(ff:ff:ff:ff:ff:ff)
Internet Protocol Version 4, Src: 0.0.0.0, Dst: 255.255.255.255
User Datagram Protocol, Src Port: 68, Dst Port: 67
Bootstrap Protocol (Discover)

No.     Time           Source                Destination Protocol Length
Info
       71 93.828837008   0.0.0.0               255.255.255.255 DHCP
342    DHCP Discover - Transaction ID 0x5df47c84

Frame 71: 342 bytes on wire (2736 bits), 342 bytes captured (2736 bits)
on interface 0
Ethernet II, Src: Commodor_11:11:11 (00:80:10:11:11:11), Dst: Broadcast
(ff:ff:ff:ff:ff:ff)
Internet Protocol Version 4, Src: 0.0.0.0, Dst: 255.255.255.255
User Datagram Protocol, Src Port: 68, Dst Port: 67
Bootstrap Protocol (Discover)

No.     Time           Source                Destination Protocol Length
Info
       74 97.948453158   0.0.0.0               255.255.255.255 DHCP
342    DHCP Discover - Transaction ID 0x5df47c84

Frame 74: 342 bytes on wire (2736 bits), 342 bytes captured (2736 bits)
on interface 0
Ethernet II, Src: Commodor_11:11:11 (00:80:10:11:11:11), Dst: Broadcast
(ff:ff:ff:ff:ff:ff)
Internet Protocol Version 4, Src: 0.0.0.0, Dst: 255.255.255.255
User Datagram Protocol, Src Port: 68, Dst Port: 67
Bootstrap Protocol (Discover)
---

Now, over on the external Linux machine (192.168.1.210), I setup a
Wireshark capture
which filtered for any traffic to/from the MAC address of the X5000/20's
eth0, in this case
shown as Commodor_11:11:11 (00:80:10:11:11:11) below:

This export shows only the DHCP traffic seen from outside the X5000/20,
and as you can see, a matching set of DHCP requests *do in fact* make
it to the outside network.

---
No.     Time           Source                Destination Protocol Length
Info
       39 5.671762509    0.0.0.0               255.255.255.255 DHCP
342    DHCP Discover - Transaction ID 0x655d91e8

Frame 39: 342 bytes on wire (2736 bits), 342 bytes captured (2736 bits)
on interface 0
Ethernet II, Src: Commodor_11:11:11 (00:80:10:11:11:11), Dst: Broadcast
(ff:ff:ff:ff:ff:ff)
      Destination: Broadcast (ff:ff:ff:ff:ff:ff)
      Source: Commodor_11:11:11 (00:80:10:11:11:11)
      Type: IPv4 (0x0800)
Internet Protocol Version 4, Src: 0.0.0.0, Dst: 255.255.255.255
User Datagram Protocol, Src Port: 68, Dst Port: 67
Bootstrap Protocol (Discover)

No.     Time           Source                Destination Protocol Length
Info
       73 9.451895404    0.0.0.0               255.255.255.255 DHCP
342    DHCP Discover - Transaction ID 0x655d91e8

Frame 73: 342 bytes on wire (2736 bits), 342 bytes captured (2736 bits)
on interface 0
Ethernet II, Src: Commodor_11:11:11 (00:80:10:11:11:11), Dst: Broadcast
(ff:ff:ff:ff:ff:ff)
      Destination: Broadcast (ff:ff:ff:ff:ff:ff)
      Source: Commodor_11:11:11 (00:80:10:11:11:11)
      Type: IPv4 (0x0800)
Internet Protocol Version 4, Src: 0.0.0.0, Dst: 255.255.255.255
User Datagram Protocol, Src Port: 68, Dst Port: 67
Bootstrap Protocol (Discover)

No.     Time           Source                Destination Protocol Length
Info
      154 14.919944480   0.0.0.0               255.255.255.255 DHCP
342    DHCP Discover - Transaction ID 0x655d91e8

Frame 154: 342 bytes on wire (2736 bits), 342 bytes captured (2736 bits)
on interface 0
Ethernet II, Src: Commodor_11:11:11 (00:80:10:11:11:11), Dst: Broadcast
(ff:ff:ff:ff:ff:ff)
      Destination: Broadcast (ff:ff:ff:ff:ff:ff)
      Source: Commodor_11:11:11 (00:80:10:11:11:11)
      Type: IPv4 (0x0800)
Internet Protocol Version 4, Src: 0.0.0.0, Dst: 255.255.255.255
User Datagram Protocol, Src Port: 68, Dst Port: 67
Bootstrap Protocol (Discover)

No.     Time           Source                Destination Protocol Length
Info
      269 24.498335996   0.0.0.0               255.255.255.255 DHCP
342    DHCP Discover - Transaction ID 0x655d91e8

Frame 269: 342 bytes on wire (2736 bits), 342 bytes captured (2736 bits)
on interface 0
Ethernet II, Src: Commodor_11:11:11 (00:80:10:11:11:11), Dst: Broadcast
(ff:ff:ff:ff:ff:ff)
      Destination: Broadcast (ff:ff:ff:ff:ff:ff)
      Source: Commodor_11:11:11 (00:80:10:11:11:11)
      Type: IPv4 (0x0800)
Internet Protocol Version 4, Src: 0.0.0.0, Dst: 255.255.255.255
User Datagram Protocol, Src Port: 68, Dst Port: 67
Bootstrap Protocol (Discover)

No.     Time           Source                Destination Protocol Length
Info
      630 41.948018648   0.0.0.0               255.255.255.255 DHCP
342    DHCP Discover - Transaction ID 0x655d91e8

Frame 630: 342 bytes on wire (2736 bits), 342 bytes captured (2736 bits)
on interface 0
Ethernet II, Src: Commodor_11:11:11 (00:80:10:11:11:11), Dst: Broadcast
(ff:ff:ff:ff:ff:ff)
      Destination: Broadcast (ff:ff:ff:ff:ff:ff)
      Source: Commodor_11:11:11 (00:80:10:11:11:11)
      Type: IPv4 (0x0800)
Internet Protocol Version 4, Src: 0.0.0.0, Dst: 255.255.255.255
User Datagram Protocol, Src Port: 68, Dst Port: 67
Bootstrap Protocol (Discover)

No.     Time           Source                Destination Protocol Length
Info
      701 49.590211264   0.0.0.0               255.255.255.255 DHCP
342    DHCP Discover - Transaction ID 0x655d91e8

Frame 701: 342 bytes on wire (2736 bits), 342 bytes captured (2736 bits)
on interface 0
Ethernet II, Src: Commodor_11:11:11 (00:80:10:11:11:11), Dst: Broadcast
(ff:ff:ff:ff:ff:ff)
      Destination: Broadcast (ff:ff:ff:ff:ff:ff)
      Source: Commodor_11:11:11 (00:80:10:11:11:11)
      Type: IPv4 (0x0800)
Internet Protocol Version 4, Src: 0.0.0.0, Dst: 255.255.255.255
User Datagram Protocol, Src Port: 68, Dst Port: 67
Bootstrap Protocol (Discover)

No.     Time           Source                Destination Protocol Length
Info
      706 50.429265938   0.0.0.0               255.255.255.255 DHCP
342    DHCP Discover - Transaction ID 0x421bade3

Frame 706: 342 bytes on wire (2736 bits), 342 bytes captured (2736 bits)
on interface 0
Ethernet II, Src: Commodor_11:11:11 (00:80:10:11:11:11), Dst: Broadcast
(ff:ff:ff:ff:ff:ff)
      Destination: Broadcast (ff:ff:ff:ff:ff:ff)
      Source: Commodor_11:11:11 (00:80:10:11:11:11)
      Type: IPv4 (0x0800)
Internet Protocol Version 4, Src: 0.0.0.0, Dst: 255.255.255.255
User Datagram Protocol, Src Port: 68, Dst Port: 67
Bootstrap Protocol (Discover)

No.     Time           Source                Destination Protocol Length
Info
      744 53.788035317   0.0.0.0               255.255.255.255 DHCP
342    DHCP Discover - Transaction ID 0x421bade3

Frame 744: 342 bytes on wire (2736 bits), 342 bytes captured (2736 bits)
on interface 0
Ethernet II, Src: Commodor_11:11:11 (00:80:10:11:11:11), Dst: Broadcast
(ff:ff:ff:ff:ff:ff)
      Destination: Broadcast (ff:ff:ff:ff:ff:ff)
      Source: Commodor_11:11:11 (00:80:10:11:11:11)
      Type: IPv4 (0x0800)
Internet Protocol Version 4, Src: 0.0.0.0, Dst: 255.255.255.255
User Datagram Protocol, Src Port: 68, Dst Port: 67
Bootstrap Protocol (Discover)

No.     Time           Source                Destination Protocol Length
Info
      797 59.820568614   0.0.0.0               255.255.255.255 DHCP
342    DHCP Discover - Transaction ID 0x421bade3

Frame 797: 342 bytes on wire (2736 bits), 342 bytes captured (2736 bits)
on interface 0
Ethernet II, Src: Commodor_11:11:11 (00:80:10:11:11:11), Dst: Broadcast
(ff:ff:ff:ff:ff:ff)
      Destination: Broadcast (ff:ff:ff:ff:ff:ff)
      Source: Commodor_11:11:11 (00:80:10:11:11:11)
      Type: IPv4 (0x0800)
Internet Protocol Version 4, Src: 0.0.0.0, Dst: 255.255.255.255
User Datagram Protocol, Src Port: 68, Dst Port: 67
Bootstrap Protocol (Discover)

No.     Time           Source                Destination Protocol Length
Info
      852 68.176833686   0.0.0.0               255.255.255.255 DHCP
342    DHCP Discover - Transaction ID 0x421bade3

Frame 852: 342 bytes on wire (2736 bits), 342 bytes captured (2736 bits)
on interface 0
Ethernet II, Src: Commodor_11:11:11 (00:80:10:11:11:11), Dst: Broadcast
(ff:ff:ff:ff:ff:ff)
      Destination: Broadcast (ff:ff:ff:ff:ff:ff)
      Source: Commodor_11:11:11 (00:80:10:11:11:11)
      Type: IPv4 (0x0800)
Internet Protocol Version 4, Src: 0.0.0.0, Dst: 255.255.255.255
User Datagram Protocol, Src Port: 68, Dst Port: 67
Bootstrap Protocol (Discover)

No.     Time           Source                Destination Protocol Length
Info
      990 82.961224895   0.0.0.0               255.255.255.255 DHCP
342    DHCP Discover - Transaction ID 0x421bade3

Frame 990: 342 bytes on wire (2736 bits), 342 bytes captured (2736 bits)
on interface 0
Ethernet II, Src: Commodor_11:11:11 (00:80:10:11:11:11), Dst: Broadcast
(ff:ff:ff:ff:ff:ff)
      Destination: Broadcast (ff:ff:ff:ff:ff:ff)
      Source: Commodor_11:11:11 (00:80:10:11:11:11)
      Type: IPv4 (0x0800)
Internet Protocol Version 4, Src: 0.0.0.0, Dst: 255.255.255.255
User Datagram Protocol, Src Port: 68, Dst Port: 67
Bootstrap Protocol (Discover)

No.     Time           Source                Destination Protocol Length
Info
     3827 95.345964418   0.0.0.0               255.255.255.255 DHCP
342    DHCP Discover - Transaction ID 0x5df47c84

Frame 3827: 342 bytes on wire (2736 bits), 342 bytes captured (2736
bits) on interface 0
Ethernet II, Src: Commodor_11:11:11 (00:80:10:11:11:11), Dst: Broadcast
(ff:ff:ff:ff:ff:ff)
      Destination: Broadcast (ff:ff:ff:ff:ff:ff)
      Source: Commodor_11:11:11 (00:80:10:11:11:11)
      Type: IPv4 (0x0800)
Internet Protocol Version 4, Src: 0.0.0.0, Dst: 255.255.255.255
User Datagram Protocol, Src Port: 68, Dst Port: 67
Bootstrap Protocol (Discover)

No.     Time           Source                Destination Protocol Length
Info
     3887 99.271668572   0.0.0.0               255.255.255.255 DHCP
342    DHCP Discover - Transaction ID 0x5df47c84

Frame 3887: 342 bytes on wire (2736 bits), 342 bytes captured (2736
bits) on interface 0
Ethernet II, Src: Commodor_11:11:11 (00:80:10:11:11:11), Dst: Broadcast
(ff:ff:ff:ff:ff:ff)
      Destination: Broadcast (ff:ff:ff:ff:ff:ff)
      Source: Commodor_11:11:11 (00:80:10:11:11:11)
      Type: IPv4 (0x0800)
Internet Protocol Version 4, Src: 0.0.0.0, Dst: 255.255.255.255
User Datagram Protocol, Src Port: 68, Dst Port: 67
Bootstrap Protocol (Discover)

No.     Time           Source                Destination Protocol Length
Info
     3943 103.383072429  0.0.0.0               255.255.255.255 DHCP
342    DHCP Discover - Transaction ID 0x5df47c84

Frame 3943: 342 bytes on wire (2736 bits), 342 bytes captured (2736
bits) on interface 0
Ethernet II, Src: Commodor_11:11:11 (00:80:10:11:11:11), Dst: Broadcast
(ff:ff:ff:ff:ff:ff)
      Destination: Broadcast (ff:ff:ff:ff:ff:ff)
      Source: Commodor_11:11:11 (00:80:10:11:11:11)
      Type: IPv4 (0x0800)
Internet Protocol Version 4, Src: 0.0.0.0, Dst: 255.255.255.255
User Datagram Protocol, Src Port: 68, Dst Port: 67
Bootstrap Protocol (Discover)
---

The odd thing here is that while the DHCP requests were broadcast to the
outside network (confirming that at least the transmit to the PHY is
working),
I could see no responses from my network's DHCP server to answer these
requests.

It is not a physical networking or routing issue, as I always get a
successful
DHCP response to the X5000/20 when I enable the Realtek 8169 interface
(also
installed [PCIe card] in the X5000/20).

Since initial outgoing traffic *appears* to be working from the DPAA
Ethernet
on the X5000/20, is it possible we are missing an interrupt mapping from
the
Frame Manager to catch the received data?

Any help would be appreciated. Thanks.

--
Best Regards,

Jamie Krueger
BITbyBIT Software Group LLC

-----Original Message-----
From: Jamie Krueger [mailto:jamie@bitbybitsoftwaregroup.com]
Sent: Friday, January 12, 2018 6:36 PM
To: Madalin-cristian Bucur <madalin.bucur@nxp.com>; linuxppc-
dev@lists.ozlabs.org
Cc: netdev@vger.kernel.org
Subject: Re: DPAA Ethernet problems with mainstream Linux kernels

On 01/12/2018 08:22 AM, Madalin-cristian Bucur wrote:

quoted

quoted

-----Original Message-----
From: Linuxppc-dev [mailto:linuxppc-dev-
bounces+madalin.bucur=nxp.com@lists.ozlabs.org] On Behalf Of Jamie

Krueger

quoted

quoted

Sent: Wednesday, January 10, 2018 5:57 PM
To: linuxppc-dev@lists.ozlabs.org
Subject: DPAA Ethernet problems with mainstream Linux kernels

Hello all @ linuxppc-dev,

I have been working with a team of people maintaining PowerPC
Linux for the new AmigaONE X5000/20 (a Freescale p5020 SoC based
machine).

We are trying to determine why the submitted Data Path Acceleration
Architecture (DPAA) Ethernet Driver is not fully functional with
the mainstream Linux kernels.

Hi Jamie,

Hi Madalin,

quoted

We are testing the DPAA driver on several DS and RDB platforms and it
is working properly. The issues you encounter with it on the X5000/20
are likely caused by some issues specific to that particular platform.

It is good to hear that the DPAA driver is functioning correctly
on the reference platforms. I am positive you are correct that
the issue is the difference in implementation on the X5000/20
(Cyrus) motherboard, as compared to the reference boards.

Can you verify which Linux Kernel sources your tests are being
performed on? We have been testing using the mainstream
Linux sources up to linux-4.15-rc6 thus far.

Latest run is on 4.15.0-rc7-00200-gc92a9a4.

quoted

The device tree that you mention, cyrus_p5020.eth.dts is not found in
the Linux kernel sources. The cyrus_p5020.dts file from the fsl ppc
device tree folder does not include the PHY information for the DPAA
interfaces. The problems that you experience may be caused by some
issues with the PHY configuration (i.e. internal delay).

The cyrus_p5020.eth.dts is a modified version of the cyrus_p5020.dts,
which of course was based off the original p5020ds.dts file. As you
noted, the current cyrus_p5020.dts file is incomplete, and does not
map the Ethernet connections properly.

The cyrus_p5020.eth.dts file, along with it's cyrus-pre.dtsi dependent
file, are an attempt to correctly define the Ethernet hardware, as it is
implemented on the X5000/20.

** I have attached both the cyrus_p5020.eth.dts and cyrus-pre.dtsi
      files with this email for comparison. Please let me know if you see
      any corrections that should be made to either file.

At a first glance they look fine to me.

I am not sure what PHY hardware/configuration you are using on the
DS and RDB platforms, but I can confirm that AmigaONE X5000/20
(Cyrus Motherboard with p5020 SoC), has dTSEC 4 and dTSEC 5
wired to two Micrel KSZ9021RN Gigabit Ethernet PHYs, using the
RGMII protocol.

Since it's RGMII, I think you should look into RGMII internal delay
requirements for this board.

quoted

  I suggest
that you connect the DPAA interface to a traffic analyzer or directly
to another device on which you can capture the incoming traffic and
check that the received frames are correct.

I have started testing along that line, using Wireshark to view the
traffic on the X5000/20 itself, and from another machine connected
on the same subnet. So far (as indicated by some details of in my
initial email), I can see outgoing broadcast requests (for DHCP)
being sent out from the X5000/20, and these requests are correctly
constructed and visible outside the X5000/20.

However, no responses to the DHCP broadcasts appear to reach
to X5000/20's DPAA Ethernet. I will need to setup some further
tests to determine if the DHCP server saw the requests and responded
to them. (I assume the DHCP server is getting them, and responding,
as I can always get a successful DHCP response to the X5000/20
when using an add-on Ethernet PICe card on the same subnet).

I will setup some more direct machine-to-machine testing to
see what else I can glean from the network traffic.

That will provide more clarity to the actual issue.

Please have a look at the attached dts files, maybe there is something
obvious there we are not seeing.

Also, given that the X5000/20 uses Micrel KSZ9021RN PHYs in RGMII
mode, what changes to the DPAA hardware configuration should we
expect to see so that the DPAA is configured to talk to them?

The phy-connection-type = "rgmii"; should take care of that.
Try to check the link information reported by the PHY.

quoted

Madalin

--

Best Regards,

Jamie Krueger
BITbyBIT Software Group LLC

quoted

quoted

Here is the results from my latest tests. They were performed using
the linux-4.10.17 ppc64, since that represents when the DPAA Ethernet
code was introduced.

Similar tests, with similar results, were also performed
using the latest Linux kernels:

linux-4.15-rc5
linux-4.15-rc6
linux-4.15-rc7

(Hence the reason for falling back to test the kernel right
   after the introduction of the DPAA Ethernet driver sources)

---

All Kernel builds had the DPAA Ethernet enabled in the kernel,
and are using the correct cyrus_p5020.eth.dtb device tree file
(for use on the X5000/20).

The results are quite similar for all kernels in regards to the DPAA
Ethernet.

All tested kernels setup the two Ethernet interfaces correctly
as eth0 and eth1, and pull the correct MAC addresses from U-Boot
environment variables ethaddr and eth1addr respectively.

So at this point Linux has what it believes is fully configured
hardware, waiting to have an IP Address/Netmask/Gateway
to be set and to bring the interface online.

However, all attempts to communicate with the outside world
do not make it out the physical (PHY) hardware - or do they?

** The following results were captured under linux-4.10.17 **

When I bring the interface up using a static address, in this case
192.168.1.21, I see the following (NOTE TX bytes says 154.0 KB,
while RX bytes says 0.0 B):

jamie@X5000-Linux:$ ifconfig
eth0      Link encap:Ethernet  HWaddr 00:80:10:11:11:11
            inet addr:192.168.1.21  Bcast:192.168.1.255

Mask:255.255.255.0

quoted

quoted

            inet6 addr: fe80::280:10ff:fe11:1111/64 Scope:Link
            UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
            RX packets:0 errors:0 dropped:0 overruns:0 frame:0
            TX packets:1428 errors:0 dropped:0 overruns:0 carrier:0
            collisions:0 txqueuelen:1000
            RX bytes:0 (0.0 B)  TX bytes:154066 (154.0 KB)
            Memory:fe4e6000-fe4e6fff

eth1      Link encap:Ethernet  HWaddr 00:80:10:22:22:22
            UP BROADCAST MULTICAST  MTU:1500  Metric:1
            RX packets:0 errors:0 dropped:0 overruns:0 frame:0
            TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
            collisions:0 txqueuelen:1000
            RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)
            Memory:fe4e8000-fe4e8fff

lo        Link encap:Local Loopback
            inet addr:127.0.0.1  Mask:255.0.0.0
            inet6 addr: ::1/128 Scope:Host
            UP LOOPBACK RUNNING  MTU:65536  Metric:1
            RX packets:1869 errors:0 dropped:0 overruns:0 frame:0
            TX packets:1869 errors:0 dropped:0 overruns:0 carrier:0
            collisions:0 txqueuelen:1000
            RX bytes:156932 (156.9 KB)  TX bytes:156932 (156.9 KB)

Checking the routing table, everything looks fine there:

jamie@X5000-Linux:$ netstat -r
Kernel IP routing table
Destination     Gateway         Genmask         Flags   MSS

Window  irtt

quoted

quoted

Iface
default         192.168.1.1     0.0.0.0         UG        0

0          0

quoted

quoted

eth0
link-local      *               255.255.0.0     U         0

0          0

quoted

quoted

eth0
192.168.1.0     *               255.255.255.0   U         0

0          0

quoted

quoted

eth0

Attempting to PING the interface itself works:

jamie@X5000-Linux:$ ping 192.168.1.21
PING 192.168.1.21 (192.168.1.21) 56(84) bytes of data.
64 bytes from 192.168.1.21: icmp_seq=1 ttl=64 time=0.037 ms
64 bytes from 192.168.1.21: icmp_seq=2 ttl=64 time=0.045 ms
64 bytes from 192.168.1.21: icmp_seq=3 ttl=64 time=0.033 ms
^C

--- 192.168.1.21 ping statistics ---

3 packets transmitted, 3 received, 0% packet loss, time 2043ms

However, attempts to PING the gateway (192.168.1.1) fail as

unreachable:

quoted

quoted

jamie@X5000-Linux:$ ping 192.168.1.1
PING 192.168.1.1 (192.168.1.1) 56(84) bytes of data.
  From 192.168.1.21 icmp_seq=1 Destination Host Unreachable
  From 192.168.1.21 icmp_seq=2 Destination Host Unreachable
  From 192.168.1.21 icmp_seq=3 Destination Host Unreachable
^C

--- 192.168.1.1 ping statistics ---

7 packets transmitted, 0 received, +3 errors, 100% packet loss, time
6077ms

In order to take a closer look at what is going on I installed

Wireshark

quoted

quoted

both on my test X5000/20 Linux install (Ubuntu 16.04.3 LTS), and on
another Linux box connected to the same network switch (in this case
at IP address 192.168.1.210)

In this test I start the capture on eth0 (X5000/20) before it is put
online,
and attempt to bring it up using DHCP to obtain it's address.

What I found was that network traffic *was* being attempted over eth0.
Here is a plain text export of the transmit side (the X5000/20) that
was captured using Wireshark.

(There were more network packets being sent from the X5000/20,
   however, I am only showing DHCP traffic to save space in this post):

---
No.     Time           Source                Destination Protocol

Length

quoted

quoted

Info
        2 0.042259843    0.0.0.0               255.255.255.255 DHCP
342    DHCP Discover - Transaction ID 0x655d91e8

Frame 2: 342 bytes on wire (2736 bits), 342 bytes captured (2736 bits)
on interface 0
Ethernet II, Src: Commodor_11:11:11 (00:80:10:11:11:11), Dst: Broadcast
(ff:ff:ff:ff:ff:ff)
Internet Protocol Version 4, Src: 0.0.0.0, Dst: 255.255.255.255
User Datagram Protocol, Src Port: 68, Dst Port: 67
Bootstrap Protocol (Discover)

No.     Time           Source                Destination Protocol

Length

quoted

quoted

Info
       16 3.830001152    0.0.0.0               255.255.255.255 DHCP
342    DHCP Discover - Transaction ID 0x655d91e8

Frame 16: 342 bytes on wire (2736 bits), 342 bytes captured (2736 bits)
on interface 0
Ethernet II, Src: Commodor_11:11:11 (00:80:10:11:11:11), Dst: Broadcast
(ff:ff:ff:ff:ff:ff)
Internet Protocol Version 4, Src: 0.0.0.0, Dst: 255.255.255.255
User Datagram Protocol, Src Port: 68, Dst Port: 67
Bootstrap Protocol (Discover)

No.     Time           Source                Destination Protocol

Length

quoted

quoted

Info
       21 9.308914533    0.0.0.0               255.255.255.255 DHCP
342    DHCP Discover - Transaction ID 0x655d91e8

Frame 21: 342 bytes on wire (2736 bits), 342 bytes captured (2736 bits)
on interface 0
Ethernet II, Src: Commodor_11:11:11 (00:80:10:11:11:11), Dst: Broadcast
(ff:ff:ff:ff:ff:ff)
Internet Protocol Version 4, Src: 0.0.0.0, Dst: 255.255.255.255
User Datagram Protocol, Src Port: 68, Dst Port: 67
Bootstrap Protocol (Discover)

No.     Time           Source                Destination Protocol

Length

quoted

quoted

Info
       23 18.906405343   0.0.0.0               255.255.255.255 DHCP
342    DHCP Discover - Transaction ID 0x655d91e8

Frame 23: 342 bytes on wire (2736 bits), 342 bytes captured (2736 bits)
on interface 0
Ethernet II, Src: Commodor_11:11:11 (00:80:10:11:11:11), Dst: Broadcast
(ff:ff:ff:ff:ff:ff)
Internet Protocol Version 4, Src: 0.0.0.0, Dst: 255.255.255.255
User Datagram Protocol, Src Port: 68, Dst Port: 67
Bootstrap Protocol (Discover)

No.     Time           Source                Destination Protocol

Length

quoted

quoted

Info
       25 36.390926450   0.0.0.0               255.255.255.255 DHCP
342    DHCP Discover - Transaction ID 0x655d91e8

Frame 25: 342 bytes on wire (2736 bits), 342 bytes captured (2736 bits)
on interface 0
Ethernet II, Src: Commodor_11:11:11 (00:80:10:11:11:11), Dst: Broadcast
(ff:ff:ff:ff:ff:ff)
Internet Protocol Version 4, Src: 0.0.0.0, Dst: 255.255.255.255
User Datagram Protocol, Src Port: 68, Dst Port: 67
Bootstrap Protocol (Discover)

No.     Time           Source                Destination Protocol

Length

quoted

quoted

Info
       26 44.048328412   0.0.0.0               255.255.255.255 DHCP
342    DHCP Discover - Transaction ID 0x655d91e8

Frame 26: 342 bytes on wire (2736 bits), 342 bytes captured (2736 bits)
on interface 0
Ethernet II, Src: Commodor_11:11:11 (00:80:10:11:11:11), Dst: Broadcast
(ff:ff:ff:ff:ff:ff)
Internet Protocol Version 4, Src: 0.0.0.0, Dst: 255.255.255.255
User Datagram Protocol, Src Port: 68, Dst Port: 67
Bootstrap Protocol (Discover)

No.     Time           Source                Destination Protocol

Length

quoted

quoted

Info
       30 44.889049203   0.0.0.0               255.255.255.255 DHCP
342    DHCP Discover - Transaction ID 0x421bade3

Frame 30: 342 bytes on wire (2736 bits), 342 bytes captured (2736 bits)
on interface 0
Ethernet II, Src: Commodor_11:11:11 (00:80:10:11:11:11), Dst: Broadcast
(ff:ff:ff:ff:ff:ff)
Internet Protocol Version 4, Src: 0.0.0.0, Dst: 255.255.255.255
User Datagram Protocol, Src Port: 68, Dst Port: 67
Bootstrap Protocol (Discover)

No.     Time           Source                Destination Protocol

Length

quoted

quoted

Info
       44 48.254495304   0.0.0.0               255.255.255.255 DHCP
342    DHCP Discover - Transaction ID 0x421bade3

Frame 44: 342 bytes on wire (2736 bits), 342 bytes captured (2736 bits)
on interface 0
Ethernet II, Src: Commodor_11:11:11 (00:80:10:11:11:11), Dst: Broadcast
(ff:ff:ff:ff:ff:ff)
Internet Protocol Version 4, Src: 0.0.0.0, Dst: 255.255.255.255
User Datagram Protocol, Src Port: 68, Dst Port: 67
Bootstrap Protocol (Discover)

No.     Time           Source                Destination Protocol

Length

quoted

quoted

Info
       49 54.299052732   0.0.0.0               255.255.255.255 DHCP
342    DHCP Discover - Transaction ID 0x421bade3

Frame 49: 342 bytes on wire (2736 bits), 342 bytes captured (2736 bits)
on interface 0
Ethernet II, Src: Commodor_11:11:11 (00:80:10:11:11:11), Dst: Broadcast
(ff:ff:ff:ff:ff:ff)
Internet Protocol Version 4, Src: 0.0.0.0, Dst: 255.255.255.255
User Datagram Protocol, Src Port: 68, Dst Port: 67
Bootstrap Protocol (Discover)

No.     Time           Source                Destination Protocol

Length

quoted

quoted

Info
       51 62.672007482   0.0.0.0               255.255.255.255 DHCP
342    DHCP Discover - Transaction ID 0x421bade3

Frame 51: 342 bytes on wire (2736 bits), 342 bytes captured (2736 bits)
on interface 0
Ethernet II, Src: Commodor_11:11:11 (00:80:10:11:11:11), Dst: Broadcast
(ff:ff:ff:ff:ff:ff)
Internet Protocol Version 4, Src: 0.0.0.0, Dst: 255.255.255.255
User Datagram Protocol, Src Port: 68, Dst Port: 67
Bootstrap Protocol (Discover)

No.     Time           Source                Destination Protocol

Length

quoted

quoted

Info
       52 77.485896202   0.0.0.0               255.255.255.255 DHCP
342    DHCP Discover - Transaction ID 0x421bade3

Frame 52: 342 bytes on wire (2736 bits), 342 bytes captured (2736 bits)
on interface 0
Ethernet II, Src: Commodor_11:11:11 (00:80:10:11:11:11), Dst: Broadcast
(ff:ff:ff:ff:ff:ff)
Internet Protocol Version 4, Src: 0.0.0.0, Dst: 255.255.255.255
User Datagram Protocol, Src Port: 68, Dst Port: 67
Bootstrap Protocol (Discover)

No.     Time           Source                Destination Protocol

Length

quoted

quoted

Info
       56 89.895304152   0.0.0.0               255.255.255.255 DHCP
342    DHCP Discover - Transaction ID 0x5df47c84

Frame 56: 342 bytes on wire (2736 bits), 342 bytes captured (2736 bits)
on interface 0
Ethernet II, Src: Commodor_11:11:11 (00:80:10:11:11:11), Dst: Broadcast
(ff:ff:ff:ff:ff:ff)
Internet Protocol Version 4, Src: 0.0.0.0, Dst: 255.255.255.255
User Datagram Protocol, Src Port: 68, Dst Port: 67
Bootstrap Protocol (Discover)

No.     Time           Source                Destination Protocol

Length

quoted

quoted

Info
       71 93.828837008   0.0.0.0               255.255.255.255 DHCP
342    DHCP Discover - Transaction ID 0x5df47c84

Frame 71: 342 bytes on wire (2736 bits), 342 bytes captured (2736 bits)
on interface 0
Ethernet II, Src: Commodor_11:11:11 (00:80:10:11:11:11), Dst: Broadcast
(ff:ff:ff:ff:ff:ff)
Internet Protocol Version 4, Src: 0.0.0.0, Dst: 255.255.255.255
User Datagram Protocol, Src Port: 68, Dst Port: 67
Bootstrap Protocol (Discover)

No.     Time           Source                Destination Protocol

Length

quoted

quoted

Info
       74 97.948453158   0.0.0.0               255.255.255.255 DHCP
342    DHCP Discover - Transaction ID 0x5df47c84

Frame 74: 342 bytes on wire (2736 bits), 342 bytes captured (2736 bits)
on interface 0
Ethernet II, Src: Commodor_11:11:11 (00:80:10:11:11:11), Dst: Broadcast
(ff:ff:ff:ff:ff:ff)
Internet Protocol Version 4, Src: 0.0.0.0, Dst: 255.255.255.255
User Datagram Protocol, Src Port: 68, Dst Port: 67
Bootstrap Protocol (Discover)
---

Now, over on the external Linux machine (192.168.1.210), I setup a
Wireshark capture
which filtered for any traffic to/from the MAC address of the

X5000/20's

quoted

quoted

eth0, in this case
shown as Commodor_11:11:11 (00:80:10:11:11:11) below:

This export shows only the DHCP traffic seen from outside the X5000/20,
and as you can see, a matching set of DHCP requests *do in fact* make
it to the outside network.

---
No.     Time           Source                Destination Protocol

Length

quoted

quoted

Info
       39 5.671762509    0.0.0.0               255.255.255.255 DHCP
342    DHCP Discover - Transaction ID 0x655d91e8

Frame 39: 342 bytes on wire (2736 bits), 342 bytes captured (2736 bits)
on interface 0
Ethernet II, Src: Commodor_11:11:11 (00:80:10:11:11:11), Dst: Broadcast
(ff:ff:ff:ff:ff:ff)
      Destination: Broadcast (ff:ff:ff:ff:ff:ff)
      Source: Commodor_11:11:11 (00:80:10:11:11:11)
      Type: IPv4 (0x0800)
Internet Protocol Version 4, Src: 0.0.0.0, Dst: 255.255.255.255
User Datagram Protocol, Src Port: 68, Dst Port: 67
Bootstrap Protocol (Discover)

No.     Time           Source                Destination Protocol

Length

quoted

quoted

Info
       73 9.451895404    0.0.0.0               255.255.255.255 DHCP
342    DHCP Discover - Transaction ID 0x655d91e8

Frame 73: 342 bytes on wire (2736 bits), 342 bytes captured (2736 bits)
on interface 0
Ethernet II, Src: Commodor_11:11:11 (00:80:10:11:11:11), Dst: Broadcast
(ff:ff:ff:ff:ff:ff)
      Destination: Broadcast (ff:ff:ff:ff:ff:ff)
      Source: Commodor_11:11:11 (00:80:10:11:11:11)
      Type: IPv4 (0x0800)
Internet Protocol Version 4, Src: 0.0.0.0, Dst: 255.255.255.255
User Datagram Protocol, Src Port: 68, Dst Port: 67
Bootstrap Protocol (Discover)

No.     Time           Source                Destination Protocol

Length

quoted

quoted

Info
      154 14.919944480   0.0.0.0               255.255.255.255 DHCP
342    DHCP Discover - Transaction ID 0x655d91e8

Frame 154: 342 bytes on wire (2736 bits), 342 bytes captured (2736

bits)

quoted

quoted

on interface 0
Ethernet II, Src: Commodor_11:11:11 (00:80:10:11:11:11), Dst: Broadcast
(ff:ff:ff:ff:ff:ff)
      Destination: Broadcast (ff:ff:ff:ff:ff:ff)
      Source: Commodor_11:11:11 (00:80:10:11:11:11)
      Type: IPv4 (0x0800)
Internet Protocol Version 4, Src: 0.0.0.0, Dst: 255.255.255.255
User Datagram Protocol, Src Port: 68, Dst Port: 67
Bootstrap Protocol (Discover)

No.     Time           Source                Destination Protocol

Length

quoted

quoted

Info
      269 24.498335996   0.0.0.0               255.255.255.255 DHCP
342    DHCP Discover - Transaction ID 0x655d91e8

Frame 269: 342 bytes on wire (2736 bits), 342 bytes captured (2736

bits)

quoted

quoted

on interface 0
Ethernet II, Src: Commodor_11:11:11 (00:80:10:11:11:11), Dst: Broadcast
(ff:ff:ff:ff:ff:ff)
      Destination: Broadcast (ff:ff:ff:ff:ff:ff)
      Source: Commodor_11:11:11 (00:80:10:11:11:11)
      Type: IPv4 (0x0800)
Internet Protocol Version 4, Src: 0.0.0.0, Dst: 255.255.255.255
User Datagram Protocol, Src Port: 68, Dst Port: 67
Bootstrap Protocol (Discover)

No.     Time           Source                Destination Protocol

Length

quoted

quoted

Info
      630 41.948018648   0.0.0.0               255.255.255.255 DHCP
342    DHCP Discover - Transaction ID 0x655d91e8

Frame 630: 342 bytes on wire (2736 bits), 342 bytes captured (2736

bits)

quoted

quoted

on interface 0
Ethernet II, Src: Commodor_11:11:11 (00:80:10:11:11:11), Dst: Broadcast
(ff:ff:ff:ff:ff:ff)
      Destination: Broadcast (ff:ff:ff:ff:ff:ff)
      Source: Commodor_11:11:11 (00:80:10:11:11:11)
      Type: IPv4 (0x0800)
Internet Protocol Version 4, Src: 0.0.0.0, Dst: 255.255.255.255
User Datagram Protocol, Src Port: 68, Dst Port: 67
Bootstrap Protocol (Discover)

No.     Time           Source                Destination Protocol

Length

quoted

quoted

Info
      701 49.590211264   0.0.0.0               255.255.255.255 DHCP
342    DHCP Discover - Transaction ID 0x655d91e8

Frame 701: 342 bytes on wire (2736 bits), 342 bytes captured (2736

bits)

quoted

quoted

on interface 0
Ethernet II, Src: Commodor_11:11:11 (00:80:10:11:11:11), Dst: Broadcast
(ff:ff:ff:ff:ff:ff)
      Destination: Broadcast (ff:ff:ff:ff:ff:ff)
      Source: Commodor_11:11:11 (00:80:10:11:11:11)
      Type: IPv4 (0x0800)
Internet Protocol Version 4, Src: 0.0.0.0, Dst: 255.255.255.255
User Datagram Protocol, Src Port: 68, Dst Port: 67
Bootstrap Protocol (Discover)

No.     Time           Source                Destination Protocol

Length

quoted

quoted

Info
      706 50.429265938   0.0.0.0               255.255.255.255 DHCP
342    DHCP Discover - Transaction ID 0x421bade3

Frame 706: 342 bytes on wire (2736 bits), 342 bytes captured (2736

bits)

quoted

quoted

on interface 0
Ethernet II, Src: Commodor_11:11:11 (00:80:10:11:11:11), Dst: Broadcast
(ff:ff:ff:ff:ff:ff)
      Destination: Broadcast (ff:ff:ff:ff:ff:ff)
      Source: Commodor_11:11:11 (00:80:10:11:11:11)
      Type: IPv4 (0x0800)
Internet Protocol Version 4, Src: 0.0.0.0, Dst: 255.255.255.255
User Datagram Protocol, Src Port: 68, Dst Port: 67
Bootstrap Protocol (Discover)

No.     Time           Source                Destination Protocol

Length

quoted

quoted

Info
      744 53.788035317   0.0.0.0               255.255.255.255 DHCP
342    DHCP Discover - Transaction ID 0x421bade3

Frame 744: 342 bytes on wire (2736 bits), 342 bytes captured (2736

bits)

quoted

quoted

on interface 0
Ethernet II, Src: Commodor_11:11:11 (00:80:10:11:11:11), Dst: Broadcast
(ff:ff:ff:ff:ff:ff)
      Destination: Broadcast (ff:ff:ff:ff:ff:ff)
      Source: Commodor_11:11:11 (00:80:10:11:11:11)
      Type: IPv4 (0x0800)
Internet Protocol Version 4, Src: 0.0.0.0, Dst: 255.255.255.255
User Datagram Protocol, Src Port: 68, Dst Port: 67
Bootstrap Protocol (Discover)

No.     Time           Source                Destination Protocol

Length

quoted

quoted

Info
      797 59.820568614   0.0.0.0               255.255.255.255 DHCP
342    DHCP Discover - Transaction ID 0x421bade3

Frame 797: 342 bytes on wire (2736 bits), 342 bytes captured (2736

bits)

quoted

quoted

on interface 0
Ethernet II, Src: Commodor_11:11:11 (00:80:10:11:11:11), Dst: Broadcast
(ff:ff:ff:ff:ff:ff)
      Destination: Broadcast (ff:ff:ff:ff:ff:ff)
      Source: Commodor_11:11:11 (00:80:10:11:11:11)
      Type: IPv4 (0x0800)
Internet Protocol Version 4, Src: 0.0.0.0, Dst: 255.255.255.255
User Datagram Protocol, Src Port: 68, Dst Port: 67
Bootstrap Protocol (Discover)

No.     Time           Source                Destination Protocol

Length

quoted

quoted

Info
      852 68.176833686   0.0.0.0               255.255.255.255 DHCP
342    DHCP Discover - Transaction ID 0x421bade3

Frame 852: 342 bytes on wire (2736 bits), 342 bytes captured (2736

bits)

quoted

quoted

on interface 0
Ethernet II, Src: Commodor_11:11:11 (00:80:10:11:11:11), Dst: Broadcast
(ff:ff:ff:ff:ff:ff)
      Destination: Broadcast (ff:ff:ff:ff:ff:ff)
      Source: Commodor_11:11:11 (00:80:10:11:11:11)
      Type: IPv4 (0x0800)
Internet Protocol Version 4, Src: 0.0.0.0, Dst: 255.255.255.255
User Datagram Protocol, Src Port: 68, Dst Port: 67
Bootstrap Protocol (Discover)

No.     Time           Source                Destination Protocol

Length

quoted

quoted

Info
      990 82.961224895   0.0.0.0               255.255.255.255 DHCP
342    DHCP Discover - Transaction ID 0x421bade3

Frame 990: 342 bytes on wire (2736 bits), 342 bytes captured (2736

bits)

quoted

quoted

on interface 0
Ethernet II, Src: Commodor_11:11:11 (00:80:10:11:11:11), Dst: Broadcast
(ff:ff:ff:ff:ff:ff)
      Destination: Broadcast (ff:ff:ff:ff:ff:ff)
      Source: Commodor_11:11:11 (00:80:10:11:11:11)
      Type: IPv4 (0x0800)
Internet Protocol Version 4, Src: 0.0.0.0, Dst: 255.255.255.255
User Datagram Protocol, Src Port: 68, Dst Port: 67
Bootstrap Protocol (Discover)

No.     Time           Source                Destination Protocol

Length

quoted

quoted

Info
     3827 95.345964418   0.0.0.0               255.255.255.255 DHCP
342    DHCP Discover - Transaction ID 0x5df47c84

Frame 3827: 342 bytes on wire (2736 bits), 342 bytes captured (2736
bits) on interface 0
Ethernet II, Src: Commodor_11:11:11 (00:80:10:11:11:11), Dst: Broadcast
(ff:ff:ff:ff:ff:ff)
      Destination: Broadcast (ff:ff:ff:ff:ff:ff)
      Source: Commodor_11:11:11 (00:80:10:11:11:11)
      Type: IPv4 (0x0800)
Internet Protocol Version 4, Src: 0.0.0.0, Dst: 255.255.255.255
User Datagram Protocol, Src Port: 68, Dst Port: 67
Bootstrap Protocol (Discover)

No.     Time           Source                Destination Protocol

Length

quoted

quoted

Info
     3887 99.271668572   0.0.0.0               255.255.255.255 DHCP
342    DHCP Discover - Transaction ID 0x5df47c84

Frame 3887: 342 bytes on wire (2736 bits), 342 bytes captured (2736
bits) on interface 0
Ethernet II, Src: Commodor_11:11:11 (00:80:10:11:11:11), Dst: Broadcast
(ff:ff:ff:ff:ff:ff)
      Destination: Broadcast (ff:ff:ff:ff:ff:ff)
      Source: Commodor_11:11:11 (00:80:10:11:11:11)
      Type: IPv4 (0x0800)
Internet Protocol Version 4, Src: 0.0.0.0, Dst: 255.255.255.255
User Datagram Protocol, Src Port: 68, Dst Port: 67
Bootstrap Protocol (Discover)

No.     Time           Source                Destination Protocol

Length

quoted

quoted

Info
     3943 103.383072429  0.0.0.0               255.255.255.255 DHCP
342    DHCP Discover - Transaction ID 0x5df47c84

Frame 3943: 342 bytes on wire (2736 bits), 342 bytes captured (2736
bits) on interface 0
Ethernet II, Src: Commodor_11:11:11 (00:80:10:11:11:11), Dst: Broadcast
(ff:ff:ff:ff:ff:ff)
      Destination: Broadcast (ff:ff:ff:ff:ff:ff)
      Source: Commodor_11:11:11 (00:80:10:11:11:11)
      Type: IPv4 (0x0800)
Internet Protocol Version 4, Src: 0.0.0.0, Dst: 255.255.255.255
User Datagram Protocol, Src Port: 68, Dst Port: 67
Bootstrap Protocol (Discover)
---

The odd thing here is that while the DHCP requests were broadcast to

the

quoted

quoted

outside network (confirming that at least the transmit to the PHY is
working),
I could see no responses from my network's DHCP server to answer these
requests.

It is not a physical networking or routing issue, as I always get a
successful
DHCP response to the X5000/20 when I enable the Realtek 8169 interface
(also
installed [PCIe card] in the X5000/20).

Since initial outgoing traffic *appears* to be working from the DPAA
Ethernet
on the X5000/20, is it possible we are missing an interrupt mapping

from

quoted

quoted

the
Frame Manager to catch the received data?

Any help would be appreciated. Thanks.

--
Best Regards,

Jamie Krueger
BITbyBIT Software Group LLC

Hello Madalin-cristian

On 15/01/2018, Madalin-cristian Bucur wrote:

quoted

quoted

The device tree that you mention, cyrus_p5020.eth.dts is not found in
the Linux kernel sources. The cyrus_p5020.dts file from the fsl ppc
device tree folder does not include the PHY information for the DPAA
interfaces. The problems that you experience may be caused by some
issues with the PHY configuration (i.e. internal delay).

The cyrus_p5020.eth.dts is a modified version of the cyrus_p5020.dts,
which of course was based off the original p5020ds.dts file. As you
noted, the current cyrus_p5020.dts file is incomplete, and does not
map the Ethernet connections properly.

This is because the current linux kernel version of cyrus_p5020.dts includes 'p5020si-pre.dtsi' and 'p5020si-post.dtsi' include files, which orginally gave us working ethernet (when we used the SDK kernel) However at some point you moved the ethernet aliases from the board dts file to the p5020si-pre.dtsi file breaking the linkages for our board.

cyrus-pre.dtsi is simply p5020si-pre.dtsi with only the correct aliases in.

quoted

** I have attached both the cyrus_p5020.eth.dts and cyrus-pre.dtsi
      files with this email for comparison. Please let me know if you see
      any corrections that should be made to either file.

At a first glance they look fine to me.

That's good to know.

quoted

I have started testing along that line, using Wireshark to view the
traffic on the X5000/20 itself, and from another machine connected
on the same subnet. So far (as indicated by some details of in my
initial email), I can see outgoing broadcast requests (for DHCP)
being sent out from the X5000/20, and these requests are correctly
constructed and visible outside the X5000/20.

However, no responses to the DHCP broadcasts appear to reach
to X5000/20's DPAA Ethernet. I will need to setup some further
tests to determine if the DHCP server saw the requests and responded
to them. (I assume the DHCP server is getting them, and responding,
as I can always get a successful DHCP response to the X5000/20
when using an add-on Ethernet PICe card on the same subnet).

This matches what I see, the interface I have connected to the network shows an increasing number of transmitted packets, but no received ones.

Jamie also noticed the following error in dmesg (right after the ethernet port becomes active)

[    4.112165] fsl_dpa dpaa-ethernet.0 eth0: Probed interface eth0
[    4.116616] fsl_dpa dpaa-ethernet.1 eth1: Probed interface eth1
[ ... ]
[  106.501055] IPv6: ADDRCONF(NETDEV_UP): eth1: link is not ready
[  106.570944] IPv6: ADDRCONF(NETDEV_UP): eth1: link is not ready
[  106.605044] IPv6: ADDRCONF(NETDEV_UP): eth0: link is not ready
[  106.674918] IPv6: ADDRCONF(NETDEV_UP): eth0: link is not ready
[  108.702771] IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
[  109.032798] fsl-pamu: pamu_av_isr: access violation interrupt
[  109.032806] fsl-pamu: pamu_av_isr: POES1=00000000
[  109.032808] fsl-pamu: pamu_av_isr: POES2=00000000
[  109.032809] fsl-pamu: pamu_av_isr: AVS1=002d0081
[  109.032811] fsl-pamu: pamu_av_isr: AVS2=00000081
[  109.032813] fsl-pamu: pamu_av_isr: AVA=00000001f1328000
[  109.032815] fsl-pamu: pamu_av_isr: UDAD=002d0001
[  109.032817] fsl-pamu: pamu_av_isr: POEA=0000000000000000

I haven't seen this anywhere else, and wondered if it is relevant.

Regards
Darren

-----Original Message-----
From: Darren Stevens [mailto:darren@stevens-zone.net]
Sent: Tuesday, January 16, 2018 12:40 AM
To: Madalin-cristian Bucur <madalin.bucur@nxp.com>
Cc: Jamie Krueger <redacted>; linuxppc-
dev@lists.ozlabs.org; netdev@vger.kernel.org
Subject: Re: DPAA Ethernet problems with mainstream Linux kernels

Hello Madalin-cristian

On 15/01/2018, Madalin-cristian Bucur wrote:

quoted

quoted

quoted

The device tree that you mention, cyrus_p5020.eth.dts is not found in
the Linux kernel sources. The cyrus_p5020.dts file from the fsl ppc
device tree folder does not include the PHY information for the DPAA
interfaces. The problems that you experience may be caused by some
issues with the PHY configuration (i.e. internal delay).

The cyrus_p5020.eth.dts is a modified version of the cyrus_p5020.dts,
which of course was based off the original p5020ds.dts file. As you
noted, the current cyrus_p5020.dts file is incomplete, and does not
map the Ethernet connections properly.

This is because the current linux kernel version of cyrus_p5020.dts
includes 'p5020si-pre.dtsi' and 'p5020si-post.dtsi' include files, which
orginally gave us working ethernet (when we used the SDK kernel) However
at some point you moved the ethernet aliases from the board dts file to
the p5020si-pre.dtsi file breaking the linkages for our board.

cyrus-pre.dtsi is simply p5020si-pre.dtsi with only the correct aliases
in.

quoted

quoted

** I have attached both the cyrus_p5020.eth.dts and cyrus-pre.dtsi
      files with this email for comparison. Please let me know if you

see

quoted

quoted

      any corrections that should be made to either file.

At a first glance they look fine to me.

That's good to know.

quoted

quoted

I have started testing along that line, using Wireshark to view the
traffic on the X5000/20 itself, and from another machine connected
on the same subnet. So far (as indicated by some details of in my
initial email), I can see outgoing broadcast requests (for DHCP)
being sent out from the X5000/20, and these requests are correctly
constructed and visible outside the X5000/20.

However, no responses to the DHCP broadcasts appear to reach
to X5000/20's DPAA Ethernet. I will need to setup some further
tests to determine if the DHCP server saw the requests and responded
to them. (I assume the DHCP server is getting them, and responding,
as I can always get a successful DHCP response to the X5000/20
when using an add-on Ethernet PICe card on the same subnet).

This matches what I see, the interface I have connected to the network
shows an increasing number of transmitted packets, but no received ones.

Jamie also noticed the following error in dmesg (right after the ethernet
port becomes active)

[    4.112165] fsl_dpa dpaa-ethernet.0 eth0: Probed interface eth0
[    4.116616] fsl_dpa dpaa-ethernet.1 eth1: Probed interface eth1
[ ... ]
[  106.501055] IPv6: ADDRCONF(NETDEV_UP): eth1: link is not ready
[  106.570944] IPv6: ADDRCONF(NETDEV_UP): eth1: link is not ready
[  106.605044] IPv6: ADDRCONF(NETDEV_UP): eth0: link is not ready
[  106.674918] IPv6: ADDRCONF(NETDEV_UP): eth0: link is not ready
[  108.702771] IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
[  109.032798] fsl-pamu: pamu_av_isr: access violation interrupt
[  109.032806] fsl-pamu: pamu_av_isr: POES1=00000000
[  109.032808] fsl-pamu: pamu_av_isr: POES2=00000000
[  109.032809] fsl-pamu: pamu_av_isr: AVS1=002d0081
[  109.032811] fsl-pamu: pamu_av_isr: AVS2=00000081
[  109.032813] fsl-pamu: pamu_av_isr: AVA=00000001f1328000
[  109.032815] fsl-pamu: pamu_av_isr: UDAD=002d0001
[  109.032817] fsl-pamu: pamu_av_isr: POEA=0000000000000000

I haven't seen this anywhere else, and wondered if it is relevant.

Regards
Darren

The PAMU related errors may be relevant to the issue, if you have incorrect
settings you may have no traffic passing through. The PAMU configuration
should be made by the bootloader. Can you try to disable CONFIG_FSL_PAMU?

Madalin

Hi All,

I compiled the RC8 of kernel 4.15 for the X5000 without PAMU support today.

Download: http://www.xenosoft.de/uImage_without_pamu.tar.gz

Please test it on your AmigaOne X5000.

Thanks,
Christian


On 16 January 2018 at 6:33PM, Madalin-cristian Bucur wrote:

quoted

The PAMU related errors may be relevant to the issue, if you have incorrect
settings you may have no traffic passing through. The PAMU configuration
should be made by the bootloader. Can you try to disable CONFIG_FSL_PAMU?

Madalin

Hi Madalin,

On 01/16/2018 11:33 AM, Madalin-cristian Bucur wrote:

quoted

-----Original Message-----
From: Darren Stevens [mailto:darren@stevens-zone.net]
Sent: Tuesday, January 16, 2018 12:40 AM
To: Madalin-cristian Bucur <madalin.bucur@nxp.com>
Cc: Jamie Krueger <redacted>; linuxppc-
dev@lists.ozlabs.org; netdev@vger.kernel.org
Subject: Re: DPAA Ethernet problems with mainstream Linux kernels

Hello Madalin-cristian

On 15/01/2018, Madalin-cristian Bucur wrote:

quoted

quoted

quoted

The device tree that you mention, cyrus_p5020.eth.dts is not found in
the Linux kernel sources. The cyrus_p5020.dts file from the fsl ppc
device tree folder does not include the PHY information for the DPAA
interfaces. The problems that you experience may be caused by some
issues with the PHY configuration (i.e. internal delay).

The cyrus_p5020.eth.dts is a modified version of the cyrus_p5020.dts,
which of course was based off the original p5020ds.dts file. As you
noted, the current cyrus_p5020.dts file is incomplete, and does not
map the Ethernet connections properly.

This is because the current linux kernel version of cyrus_p5020.dts
includes 'p5020si-pre.dtsi' and 'p5020si-post.dtsi' include files, which
orginally gave us working ethernet (when we used the SDK kernel) However
at some point you moved the ethernet aliases from the board dts file to
the p5020si-pre.dtsi file breaking the linkages for our board.

cyrus-pre.dtsi is simply p5020si-pre.dtsi with only the correct aliases
in.

quoted

quoted

** I have attached both the cyrus_p5020.eth.dts and cyrus-pre.dtsi
       files with this email for comparison. Please let me know if you

see

quoted

quoted

       any corrections that should be made to either file.

At a first glance they look fine to me.

That's good to know.

quoted

quoted

I have started testing along that line, using Wireshark to view the
traffic on the X5000/20 itself, and from another machine connected
on the same subnet. So far (as indicated by some details of in my
initial email), I can see outgoing broadcast requests (for DHCP)
being sent out from the X5000/20, and these requests are correctly
constructed and visible outside the X5000/20.

However, no responses to the DHCP broadcasts appear to reach
to X5000/20's DPAA Ethernet. I will need to setup some further
tests to determine if the DHCP server saw the requests and responded
to them. (I assume the DHCP server is getting them, and responding,
as I can always get a successful DHCP response to the X5000/20
when using an add-on Ethernet PICe card on the same subnet).

This matches what I see, the interface I have connected to the network
shows an increasing number of transmitted packets, but no received ones.

Jamie also noticed the following error in dmesg (right after the ethernet
port becomes active)

[    4.112165] fsl_dpa dpaa-ethernet.0 eth0: Probed interface eth0
[    4.116616] fsl_dpa dpaa-ethernet.1 eth1: Probed interface eth1
[ ... ]
[  106.501055] IPv6: ADDRCONF(NETDEV_UP): eth1: link is not ready
[  106.570944] IPv6: ADDRCONF(NETDEV_UP): eth1: link is not ready
[  106.605044] IPv6: ADDRCONF(NETDEV_UP): eth0: link is not ready
[  106.674918] IPv6: ADDRCONF(NETDEV_UP): eth0: link is not ready
[  108.702771] IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
[  109.032798] fsl-pamu: pamu_av_isr: access violation interrupt
[  109.032806] fsl-pamu: pamu_av_isr: POES1=00000000
[  109.032808] fsl-pamu: pamu_av_isr: POES2=00000000
[  109.032809] fsl-pamu: pamu_av_isr: AVS1=002d0081
[  109.032811] fsl-pamu: pamu_av_isr: AVS2=00000081
[  109.032813] fsl-pamu: pamu_av_isr: AVA=00000001f1328000
[  109.032815] fsl-pamu: pamu_av_isr: UDAD=002d0001
[  109.032817] fsl-pamu: pamu_av_isr: POEA=0000000000000000

I haven't seen this anywhere else, and wondered if it is relevant.

Regards
Darren

The PAMU related errors may be relevant to the issue, if you have incorrect
settings you may have no traffic passing through. The PAMU configuration
should be made by the bootloader. Can you try to disable CONFIG_FSL_PAMU?

Madalin

Here are some results from testing on the X5000/20 with 4.15.0-rc8
(w/CONFIG_FSL_PAMU disabled):

In my tests I can now get the interface to obtain an IP Address via DHCP,
at least *part* of the time that is.

Here is a link to a screenshot of a Wireshark capture which shows
a successful DHCP request/response for the X5000/20, followed by some
failed ping attempts.

(This traffic was captured on an external machine on the same subnet, 
and not from the X5000/20 itself)
http://bitbybitsoftwaregroup.com/share/X5000-DHCP-Answered-Wireshark.png

Additionally, here is a link to the export of the packet data shown in 
the above image:
http://bitbybitsoftwaregroup.com/share/Wireshark-X5000-DHCP-Success.html

As you can see by this example, even when the interface does obtain an 
IP address via
DHCP (about half the time), subsequent traffic still fails to receive 
responses.

In this example, I attempt to ping the gateway (192.168.1.1), and one or 
two other
local addresses, and nothing pings back (even when trying Skateman's 
suggestion
of expanding the buffers for the interface -

"A workaround to keep the nic alive a bit longer.... is the following 
command
ip link set eth0 qlen 10000"

For reference:

I have attached the Kernel config that the version I tested was built with.
Also, I have attached the dmesg output I am currently seeing.

-- 
Best Regards,

Jamie Krueger
BITbyBIT Software Group LLC

Hi Jamie,

Many thanks for your effort. If you need a new kernel for testing please let me know.

Cheers,
Christian

Sent from my iPhone

On 18. Jan 2018, at 02:59, Jamie Krueger [off-list ref] wrote:

Hi Madalin,

On 01/16/2018 11:33 AM, Madalin-cristian Bucur wrote:

Here are some results from testing on the X5000/20 with 4.15.0-rc8
(w/CONFIG_FSL_PAMU disabled):

In my tests I can now get the interface to obtain an IP Address via DHCP,
at least *part* of the time that is.

Here is a link to a screenshot of a Wireshark capture which shows
a successful DHCP request/response for the X5000/20, followed by some
failed ping attempts.

(This traffic was captured on an external machine on the same subnet, and not from the X5000/20 itself)
http://bitbybitsoftwaregroup.com/share/X5000-DHCP-Answered-Wireshark.png

Additionally, here is a link to the export of the packet data shown in the above image:
http://bitbybitsoftwaregroup.com/share/Wireshark-X5000-DHCP-Success.html

As you can see by this example, even when the interface does obtain an IP address via
DHCP (about half the time), subsequent traffic still fails to receive responses.

In this example, I attempt to ping the gateway (192.168.1.1), and one or two other
local addresses, and nothing pings back (even when trying Skateman's suggestion
of expanding the buffers for the interface -

"A workaround to keep the nic alive a bit longer.... is the following command
ip link set eth0 qlen 10000"

For reference:

I have attached the Kernel config that the version I tested was built with.
Also, I have attached the dmesg output I am currently seeing.

-- 
Best Regards,

Jamie Krueger
BITbyBIT Software Group LLC

<proc_config-4.15.0-rc8-NO_PAMU.txt>
<fsl_dmesg_output.txt>