lan78xx: WARNING: irq 79 handler enabled interrupts

Hi,
while booting my Raspberry Pi 3 B+ with Linux 4.20 (arm64/defconfig) i'm getting the following warning:

[   11.005738] irq 79 handler irq_default_primary_handler+0x0/0x8 enabled interrupts
[   11.024703] WARNING: CPU: 0 PID: 0 at kernel/irq/handle.c:153 __handle_irq_event_percpu+0x154/0x168
[   11.045293] Modules linked in: brcmfmac vc4 brcmutil cfg80211 cec drm_kms_helper hci_uart drm btbcm bluetooth microchip lan78xx ecdh_generic rfkill crct10dif_ce drm_panel_orientation_quirks bcm2835_rng bcm2835_thermal pwm_bcm2835 raspberrypi_hwmon i2c_bcm2835 vchiq(C) rng_core ip_tables x_tables ipv6
[   11.096874] CPU: 0 PID: 0 Comm: swapper/0 Tainted: G         C        4.20.0 #11
[   11.116736] Hardware name: Raspberry Pi 3 Model B Plus Rev 1.3 (DT)
[   11.135405] pstate: 20000005 (nzCv daif -PAN -UAO)
[   11.152481] pc : __handle_irq_event_percpu+0x154/0x168
[   11.169840] lr : __handle_irq_event_percpu+0x154/0x168
[   11.187018] sp : ffff000008003ce0
[   11.202234] x29: ffff000008003ce0 x28: 0000000000000000 
[   11.219467] x27: 0000000000000001 x26: ffff000008ebed00 
[   11.236619] x25: ffff0000092fe40a x24: ffff800035d7e800 
[   11.253679] x23: ffff0000091b9000 x22: 000000000000004f 
[   11.270636] x21: ffff000008003d84 x20: 0000000000000002 
[   11.287449] x19: ffff800035572980 x18: ffffffffffffffff 
[   11.304177] x17: 0000000000000000 x16: 0000000000000000 
[   11.320827] x15: ffff0000091b96c8 x14: 0720072007200720 
[   11.337364] x13: 0720072007200720 x12: ffff000008bfe738 
[   11.353996] x11: ffff80002d593c70 x10: 0000000000000001 
[   11.370541] x9 : 00000000ffffffff x8 : 0000000000000001 
[   11.387035] x7 : 0000000000000008 x6 : ffff8000393800b8 
[   11.403471] x5 : ffff8000393800b8 x4 : 0000000000000001 
[   11.419849] x3 : 0000000000000000 x2 : ffff0000091c2600 
[   11.436199] x1 : 16ca74a025baa300 x0 : 0000000000000000 
[   11.452495] Call trace:
[   11.465764]  __handle_irq_event_percpu+0x154/0x168
[   11.481439]  handle_irq_event_percpu+0x34/0x88
[   11.496695]  handle_irq_event+0x48/0x78
[   11.511242]  handle_simple_irq+0x80/0xb8
[   11.525787]  generic_handle_irq+0x24/0x38
[   11.540339]  intr_complete+0xbc/0x100 [lan78xx]
[   11.555366]  __usb_hcd_giveback_urb+0x6c/0xd8
[   11.570137]  usb_giveback_urb_bh+0xa8/0x108
[   11.584699]  tasklet_action_common.isra.2+0x80/0x130
[   11.600056]  tasklet_hi_action+0x24/0x30
[   11.614255]  __do_softirq+0x108/0x234
[   11.628097]  irq_exit+0xcc/0xd8
[   11.641323]  __handle_domain_irq+0x60/0xb8
[   11.655516]  bcm2836_arm_irqchip_handle_irq+0x64/0xb8
[   11.670667]  el1_irq+0xb0/0x128
[   11.683770]  arch_cpu_idle+0x10/0x18
[   11.697275]  do_idle+0x1c4/0x280
[   11.710341]  cpu_startup_entry+0x20/0x28
[   11.724058]  rest_init+0xd4/0xe0
[   11.736962]  arch_call_rest_init+0xc/0x14
[   11.750625]  start_kernel+0x3ec/0x418

This seems to come from the lan78xx driver, because i don't get this warning on the RPI 3B (smsc95xx).

What's the approach to fix this issue? 
Is it just simply replace spin_lock_irq() with spin_lock_irqsave() at the right places?

Regards
Stefan

On Sat, 29 Dec 2018 00:02:57 +0000,
Stefan Wahren [off-list ref] wrote:

Hi Stephan,

Hi,
while booting my Raspberry Pi 3 B+ with Linux 4.20 (arm64/defconfig)
i'm getting the following warning:

[   11.005738] irq 79 handler irq_default_primary_handler+0x0/0x8 enabled interrupts

[...]

This seems to come from the lan78xx driver, because i don't get this
warning on the RPI 3B (smsc95xx).

What's the approach to fix this issue?  Is it just simply replace
spin_lock_irq() with spin_lock_irqsave() at the right places?

No, this is more pathological than that. The USB framework seems to be
using threaded interrupts, which implies that interrupts are
re-enabled as soon as it has run.

In turn, this driver is using handle_simple_irq() as its flow, which
definitely expect interrupts to be disabled.

I have no insight in how this driver works, but I'm happy to help if
someone explains the interrupt flow of this thing.

	M.

-- 
Jazz is not dead, it just smell funny.

HI Marc & Stephen,

Most of engineers are out until New Year's Day.
 
LAN78xx driver uses irq_domain for phy interrupt, but smsc95xx uses polling.
Need to check flow again, you can try that comment out "lan78xx_setup_irq_domain" to
make dev->domain_data.phyirq = 0 which forces PHY polling.
 
Can you share if this is filed on RPI forum?
Also, it would be appreciated if you can create a ticket on Microchip site at
https://www.microchip.com/technical-support/hot-topics

Best Regards,
Woojung

-----Original Message-----
From: Marc Zyngier <redacted>
Sent: Saturday, December 29, 2018 4:42 AM
To: Stefan Wahren <redacted>
Cc: Woojung Huh - C21699 <Woojung.Huh@microchip.com>; Eric Anholt
[off-list ref]; UNGLinuxDriver [off-list ref];
netdev@vger.kernel.org; linux-arm-kernel@lists.infradead.org
Subject: Re: lan78xx: WARNING: irq 79 handler enabled interrupts

On Sat, 29 Dec 2018 00:02:57 +0000,
Stefan Wahren [off-list ref] wrote:

Hi Stephan,

quoted

Hi,
while booting my Raspberry Pi 3 B+ with Linux 4.20 (arm64/defconfig)
i'm getting the following warning:

[   11.005738] irq 79 handler irq_default_primary_handler+0x0/0x8 enabled

interrupts

[...]

quoted

This seems to come from the lan78xx driver, because i don't get this
warning on the RPI 3B (smsc95xx).

What's the approach to fix this issue?  Is it just simply replace
spin_lock_irq() with spin_lock_irqsave() at the right places?

No, this is more pathological than that. The USB framework seems to be
using threaded interrupts, which implies that interrupts are
re-enabled as soon as it has run.

In turn, this driver is using handle_simple_irq() as its flow, which
definitely expect interrupts to be disabled.

I have no insight in how this driver works, but I'm happy to help if
someone explains the interrupt flow of this thing.

	M.

--
Jazz is not dead, it just smell funny.

Hi Woojung,

Woojung.Huh@microchip.com hat am 30. Dezember 2018 um 04:25 geschrieben:


HI Marc & Stephen,

Most of engineers are out until New Year's Day.

thanks. I didn't expect a reply that fast.

 
LAN78xx driver uses irq_domain for phy interrupt, but smsc95xx uses polling.
Need to check flow again, you can try that comment out "lan78xx_setup_irq_domain" to
make dev->domain_data.phyirq = 0 which forces PHY polling.

I tested your suggestion with multi_v7_defconfig (32 bit) and arm64/defconfig.
The warning disappeared and Ethernet is still working.

Only the old issue that we can't receive until a first packet has been send out reappear. But this should be manageable.

 
Can you share if this is filed on RPI forum?

I didn't submit this to the raspberrypi.org forum / github. They mostly care about the next LTS release 4.19 and not really about arm64.

But an Arch Linux user (aarch64 using Mainline kernel) already discovered this issue:
https://archlinuxarm.org/forum/viewtopic.php?f=65&t=13302

Also, it would be appreciated if you can create a ticket on Microchip site at
https://www.microchip.com/technical-support/hot-topics

Done

Best regards
Stefan

Best Regards,
Woojung

Hi Stefan,

Thanks for information and creating a ticket.

Did quick check on a PC machine with 4.20 rc. However, not seeing same warning.
Look need time to repro and get details on same build & target you are using.

Regards,
Woojung

-----Original Message-----
From: Stefan Wahren <redacted>
Sent: Tuesday, January 1, 2019 4:32 PM
To: Woojung Huh - C21699 <Woojung.Huh@microchip.com>
Cc: eric@anholt.net; UNGLinuxDriver <UNGLinuxDriver@microchip.com>; marc.zyngier@arm.com;
netdev@vger.kernel.org; linux-arm-kernel@lists.infradead.org
Subject: RE: lan78xx: WARNING: irq 79 handler enabled interrupts

Hi Woojung,

quoted

Woojung.Huh@microchip.com hat am 30. Dezember 2018 um 04:25 geschrieben:


HI Marc & Stephen,

Most of engineers are out until New Year's Day.

thanks. I didn't expect a reply that fast.

quoted

LAN78xx driver uses irq_domain for phy interrupt, but smsc95xx uses polling.
Need to check flow again, you can try that comment out "lan78xx_setup_irq_domain" to
make dev->domain_data.phyirq = 0 which forces PHY polling.

I tested your suggestion with multi_v7_defconfig (32 bit) and arm64/defconfig.
The warning disappeared and Ethernet is still working.

Only the old issue that we can't receive until a first packet has been send out reappear. But this should
be manageable.

quoted

Can you share if this is filed on RPI forum?

I didn't submit this to the raspberrypi.org forum / github. They mostly care about the next LTS release
4.19 and not really about arm64.

But an Arch Linux user (aarch64 using Mainline kernel) already discovered this issue:
https://archlinuxarm.org/forum/viewtopic.php?f=65&t=13302

quoted

Also, it would be appreciated if you can create a ticket on Microchip site at
https://www.microchip.com/technical-support/hot-topics

Done

Best regards
Stefan

quoted

Best Regards,
Woojung

Hi,

Stefan Wahren [off-list ref] hat am 1. Januar 2019 um 22:31 geschrieben:


Hi Woojung,

quoted

Woojung.Huh@microchip.com hat am 30. Dezember 2018 um 04:25 geschrieben:


HI Marc & Stephen,

Most of engineers are out until New Year's Day.

thanks. I didn't expect a reply that fast.

quoted

 
LAN78xx driver uses irq_domain for phy interrupt, but smsc95xx uses polling.
Need to check flow again, you can try that comment out "lan78xx_setup_irq_domain" to
make dev->domain_data.phyirq = 0 which forces PHY polling.

I tested your suggestion with multi_v7_defconfig (32 bit) and arm64/defconfig.
The warning disappeared and Ethernet is still working.

Only the old issue that we can't receive until a first packet has been send out reappear. But this should be manageable.

i got informed that the engineers are busy with other issues and come back later to this :-(

Since i'm getting requests to provide my PHY polling patch, here it is:

diff --git a/drivers/net/usb/lan78xx.c b/drivers/net/usb/lan78xx.c
index e96bc0c..a5bb292 100644
--- a/drivers/net/usb/lan78xx.c
+++ b/drivers/net/usb/lan78xx.c

@@ -2982,13 +2982,6 @@ static int lan78xx_bind(struct lan78xx_net *dev, struct usb_interface *intf)
 
 	dev->net->hw_features = dev->net->features;
 
-	ret = lan78xx_setup_irq_domain(dev);
-	if (ret < 0) {
-		netdev_warn(dev->net,
-			    "lan78xx_setup_irq_domain() failed : %d", ret);
-		goto out1;
-	}
-
 	dev->net->hard_header_len += TX_OVERHEAD;
 	dev->hard_mtu = dev->net->mtu + dev->net->hard_header_len;
 

@@ -2996,13 +2989,13 @@ static int lan78xx_bind(struct lan78xx_net *dev, struct usb_interface *intf)
 	ret = lan78xx_reset(dev);
 	if (ret) {
 		netdev_warn(dev->net, "Registers INIT FAILED....");
-		goto out2;
+		goto out1;
 	}
 
 	ret = lan78xx_mdio_init(dev);
 	if (ret) {
 		netdev_warn(dev->net, "MDIO INIT FAILED.....");
-		goto out2;
+		goto out1;
 	}
 
 	dev->net->flags |= IFF_MULTICAST;

@@ -3011,9 +3004,6 @@ static int lan78xx_bind(struct lan78xx_net *dev, struct usb_interface *intf)
 
 	return ret;
 
-out2:
-	lan78xx_remove_irq_domain(dev);
-
 out1:
 	netdev_warn(dev->net, "Bind routine FAILED");
 	cancel_work_sync(&pdata->set_multicast);