On Wed, Jan 13, 2021 at 3:30 AM Jon Hunter [off-list ref] wrote:

On 18/12/2020 03:16, Saravana Kannan wrote:

quoted

As discussed in LPC 2020, cyclic dependencies in firmware that couldn't
be broken using logic was one of the last remaining reasons
fw_devlink=on couldn't be set by default.

This series changes fw_devlink so that when a cyclic dependency is found
in firmware, the links between those devices fallback to permissive mode
behavior. This way, the rest of the system still benefits from
fw_devlink, but the ambiguous cases fallback to permissive mode.

Setting fw_devlink=on by default brings a bunch of benefits (currently,
only for systems with device tree firmware):
* Significantly cuts down deferred probes.
* Device probe is effectively attempted in graph order.
* Makes it much easier to load drivers as modules without having to
  worry about functional dependencies between modules (depmod is still
  needed for symbol dependencies).

One issue we have come across with this is the of_mdio.c driver. On
Tegra194 Jetson Xavier I am seeing the following ...

boot: logs: [       4.194791] WARNING KERN WARNING: CPU: 0 PID: 1 at /dvs/git/dirty/git-master_l4t-upstream/kernel/drivers/base/core.c:1189 device_links_driver_bound+0x240/0x260
boot: logs: [       4.207683] WARNING KERN Modules linked in:
boot: logs: [       4.210691] WARNING KERN CPU: 0 PID: 1 Comm: swapper/0 Not tainted 5.11.0-rc3-next-20210112-gdf869cab4b35 #1
boot: logs: [       4.219221] WARNING KERN Hardware name: NVIDIA Jetson AGX Xavier Developer Kit (DT)
boot: logs: [       4.225628] WARNING KERN pstate: 80400009 (Nzcv daif +PAN -UAO -TCO BTYPE=--)
boot: logs: [       4.231542] WARNING KERN pc : device_links_driver_bound+0x240/0x260
boot: logs: [       4.236587] WARNING KERN lr : device_links_driver_bound+0xf8/0x260
boot: logs: [       4.241560] WARNING KERN sp : ffff800011f4b980
boot: logs: [       4.244819] WARNING KERN x29: ffff800011f4b980 x28: ffff00008208a0a0
boot: logs: [       4.250051] WARNING KERN x27: ffff00008208a080 x26: 00000000ffffffff
boot: logs: [       4.255271] WARNING KERN x25: 0000000000000003 x24: ffff800011b99000
boot: logs: [       4.260489] WARNING KERN x23: 0000000000000001 x22: ffff800011df14f0
boot: logs: [       4.265706] WARNING KERN x21: ffff800011f4b9f8 x20: ffff800011df1000
boot: logs: [       4.270934] WARNING KERN x19: ffff00008208a000 x18: 0000000000000005
boot: logs: [       4.276166] WARNING KERN x17: 0000000000000007 x16: 0000000000000001
boot: logs: [       4.281382] WARNING KERN x15: ffff000080030c90 x14: ffff0000805c9df8
boot: logs: [       4.286618] WARNING KERN x13: 0000000000000000 x12: ffff000080030c90
boot: logs: [       4.291847] WARNING KERN x11: ffff0000805c9da8 x10: 0000000000000040
boot: logs: [       4.297061] WARNING KERN x9 : ffff000080030c98 x8 : 0000000000000000
boot: logs: [       4.302291] WARNING KERN x7 : 0000000000000009 x6 : 0000000000000000
boot: logs: [       4.307509] WARNING KERN x5 : ffff000080100000 x4 : 0000000000000000
boot: logs: [       4.312739] WARNING KERN x3 : ffff800011df1e38 x2 : ffff000080908c10
boot: logs: [       4.317956] WARNING KERN x1 : 0000000000000001 x0 : ffff0000809ca400
boot: logs: [       4.323183] WARNING KERN Call trace:
boot: logs: [       4.325593] WARNING KERN  device_links_driver_bound+0x240/0x260
boot: logs: [       4.330301] WARNING KERN  driver_bound+0x70/0xd0
boot: logs: [       4.333740] WARNING KERN  device_bind_driver+0x50/0x60
boot: logs: [       4.337671] WARNING KERN  phy_attach_direct+0x258/0x2e0
boot: logs: [       4.341718] WARNING KERN  phylink_of_phy_connect+0x7c/0x140
boot: logs: [       4.346081] WARNING KERN  stmmac_open+0xb04/0xc70
boot: logs: [       4.349612] WARNING KERN  __dev_open+0xe0/0x190
boot: logs: [       4.352972] WARNING KERN  __dev_change_flags+0x16c/0x1b8
boot: logs: [       4.357081] WARNING KERN  dev_change_flags+0x20/0x60
boot: logs: [       4.360856] WARNING KERN  ip_auto_config+0x2a0/0xfe8
boot: logs: [       4.364633] WARNING KERN  do_one_initcall+0x58/0x1b8
boot: logs: [       4.368405] WARNING KERN  kernel_init_freeable+0x1ec/0x240
boot: logs: [       4.372698] WARNING KERN  kernel_init+0x10/0x110
boot: logs: [       4.376130] WARNING KERN  ret_from_fork+0x10/0x18


So looking at this change does this mean that the of_mdio needs to be
converted to a proper driver?

Sorry, there's not enough context in this log for me to tell how this
is even related to of_mdio.c. My guess is this is related to network
stack directly calling device_bind_driver() and not updating device
link state correctly. See what device_links_check_suppliers() does in
the normal path. I think I know which warning this is, but can you
check your tree and tell me the code you see in
drivers/base/core.c:1189 ?

Also, can you give me a few more lines above and below this log and
also explain why you think this is related to of_mdio.c? Where is the
DT file for this board in case I need to look at it? And where is this
phy node defined in DT?

If there's an easy way to convert it to a proper driver, that's always
better than calling into driver core in a piecemeal fashion.

I would have thought that this will be
seen on several platforms.

I'm surprised you are seeing this issue only now. I'd have expected it
to have happened even without this series.

-Saravana