Oops in fs_enet driver with preempt-rt

Hi
I am trying to get preempt-rt (v2.6.25.4-rt1) to work on a Freescale
8280 CPU (arch/powerpc)

I get the follwing oops when trying to enable ethernet:
(looks like a null pointer dereference in
drivers/net/fs_enet/fs_enet-main.c:106)

Unable to handle kernel paging request for data at address 0x00000000
Faulting instruction address: 0xc0197efc
Oops: Kernel access of bad area, sig: 11 [#1]
PREEMPT Innovative Systems ApMax
Modules linked in:
NIP: c0197efc LR: c0197c48 CTR: c01986c4
REGS: df84de90 TRAP: 0300   Not tainted  (2.6.25.4-rt1)
MSR: 00009032 <EE,ME,IR,DR>  CR: 42004022  XER: 00000000
DAR: 00000000, DSISR: 20000000
TASK =3D df82ea70[7] 'sirq-net-rx/0' THREAD: df84c000
GPR00: e105a350 df84df40 df82ea70 df37f000 00000011 c034ed90 c034ed70
00000000
GPR08: 00000000 e105a340 00000000 00000009 0000df80 ffffffff 0fffd000
00000028
GPR16: 00400558 00800000 ffffffff df3e71e0 00000011 00000000 00000000
00000000
GPR24: df37f000 00000000 00000000 00000000 fffb9e82 0000012c 00000011
df37f5a0
NIP [c0197efc] fs_enet_rx_napi+0x2f0/0x3ac
LR [c0197c48] fs_enet_rx_napi+0x3c/0x3ac
Call Trace:
[df84df40] [c024a1b4] __spin_unlock+0x18/0x3c (unreliable)
[df84df80] [c01d9d28] net_rx_action+0x84/0x1ac
[df84dfa0] [c0022f18] ksoftirqd+0x158/0x25c
[df84dfd0] [c0033fa0] kthread+0x4c/0x88
[df84dff0] [c000fb34] kernel_thread+0x44/0x60
Instruction dump:
73202000 3b7b0008 41820008 837f00bc 813f00a0 7f03c378 81290034 7d2903a6
4e800421 7f96a000 409c00a8 7c0004ac <a01b0000> 0c000000 4c00012c
5419043e
---[ end trace c548f7dabfe5b3de ]---

28 more lines

Rune Torgersen wrote:

Hi
I am trying to get preempt-rt (v2.6.25.4-rt1) to work on a Freescale
8280 CPU (arch/powerpc)=20
=20
I get the follwing oops when trying to enable ethernet:
(looks like a null pointer dereference in
drivers/net/fs_enet/fs_enet-main.c:106)

And if I disable NAPI I get this one instead:

Bringing up the eth1 interface...Unable to handle kernel paging request
for data at address 0x00000000
Faulting instruction address: 0xc019706c
Oops: Kernel access of bad area, sig: 11 [#1]
PREEMPT Innovative Systems ApMax
Modules linked in:
NIP: c019706c LR: c0196d84 CTR: c0198790
REGS: ef2c3cb0 TRAP: 0300   Not tainted  (2.6.25.4-rt1)
MSR: 00009032 <EE,ME,IR,DR>  CR: 42004222  XER: 00000000
DAR: 00000000, DSISR: 20000000
TASK =3D ef2ac590[114] 'ifconfig' THREAD: ef2c2000
GPR00: 00000000 ef2c3d60 ef2ac590 ef37f000 00000008 00000000 ef2c3d5c
0000008c
GPR08: ef2c3d38 00000008 ef2c3d40 00000001 22000222 10021454 00000000
100a37b0
GPR16: 10090000 ef3e71e0 ef3e71e0 00000000 10070000 00000008 100192c0
00008914
GPR24: ef37f000 ef37f5a0 00000000 c0196d34 00000000 ef37f5a0 fffffff4
ef2973a0
NIP [c019706c] fs_enet_interrupt+0x338/0x5f4
LR [c0196d84] fs_enet_interrupt+0x50/0x5f4
Call Trace:
[ef2c3d60] [c024a270] __spin_unlock_irqrestore+0x20/0x4c (unreliable)
[ef2c3db0] [c0049a04] request_irq+0xb8/0xfc
[ef2c3de0] [c0198044] fs_enet_open+0x8c/0x210
[ef2c3e00] [c01d9b4c] dev_open+0x80/0xe8
[ef2c3e10] [c01d89ac] dev_change_flags+0xc8/0x19c
[ef2c3e30] [c021bfb0] devinet_ioctl+0x284/0x6f0
[ef2c3e90] [c021c8fc] inet_ioctl+0xc0/0xf4
[ef2c3ea0] [c01cbec4] sock_ioctl+0x200/0x23c
[ef2c3ec0] [c00810c0] vfs_ioctl+0x44/0xa8
[ef2c3ee0] [c0081520] do_vfs_ioctl+0x3fc/0x440
[ef2c3f10] [c00815a4] sys_ioctl+0x40/0x70
[ef2c3f40] [c000fcf0] ret_from_syscall+0x0/0x38

--- Exception: c01 at 0xff6a37c
    LR =3D 0xffecd58

Instruction dump:
7c0004ac b01c0000 72e02000 3b9c0008 41820008 839d00bc 813d00a0 7f03c378
81290034 7d2903a6 4e800421 7c0004ac <a01c0000> 0c000000 4c00012c
5417043e
---[ end trace 61ff5e5c97ddf707 ]---

Seems to be caused by line 471 in fs_enet_main:

0xc0196d84 is in fs_enet_interrupt
(drivers/net/fs_enet/fs_enet-main.c:473).
468                     if (fpi->use_napi)
469                             int_clr_events &=3D ~fep->ev_napi_rx;
470
471                     (*fep->ops->clear_int_events)(dev,
int_clr_events);
472
473                     if (int_events & fep->ev_err)
474                             (*fep->ops->ev_error)(dev, int_events);
475
476                     if (int_events & fep->ev_rx) {
477                             if (!fpi->use_napi)
(gdb)    =20

Rune Torgersen wrote:

0xc0196d84 is in fs_enet_interrupt
(drivers/net/fs_enet/fs_enet-main.c:473).
468                     if (fpi->use_napi)
469                             int_clr_events &= ~fep->ev_napi_rx;
470
471                     (*fep->ops->clear_int_events)(dev,
int_clr_events);
472
473                     if (int_events & fep->ev_err)
474                             (*fep->ops->ev_error)(dev, int_events);
475
476                     if (int_events & fep->ev_rx) {
477                             if (!fpi->use_napi)
(gdb)    

Do you have shared interrupt debugging turned on?  That breaks this driver, and a patch to remove the shared flag was nacked in favor of actually fixing the driver, which I haven't gotten around to.

-Scott

Scott Wood wrote:

Rune Torgersen wrote:

quoted

0xc0196d84 is in fs_enet_interrupt
(drivers/net/fs_enet/fs_enet-main.c:473).
468                     if (fpi->use_napi)
469                             int_clr_events &=3D ~fep->ev_napi_rx;
470 471                     (*fep->ops->clear_int_events)(dev,
int_clr_events); 472
473                     if (int_events & fep->ev_err)
474                             (*fep->ops->ev_error)(dev,
int_events); 475 476                     if (int_events &
fep->ev_rx) { 477                             if (!fpi->use_napi)
(gdb)

=20
Do you have shared interrupt debugging turned on?  That breaks this
driver, and a patch to remove the shared flag was nacked in favor of
actually fixing the driver, which I haven't gotten around to.
=20
-Scott

Thanks!!
That worked. Now I just have to get highmem support...

Scott Wood wrote:

Rune Torgersen wrote:

quoted

0xc0196d84 is in fs_enet_interrupt
(drivers/net/fs_enet/fs_enet-main.c:473).
468                     if (fpi->use_napi)
469                             int_clr_events &=3D ~fep->ev_napi_rx;
470 471                     (*fep->ops->clear_int_events)(dev,
int_clr_events); 472
473                     if (int_events & fep->ev_err)
474                             (*fep->ops->ev_error)(dev,
int_events); 475 476                     if (int_events &
fep->ev_rx) { 477                             if (!fpi->use_napi)
(gdb)

=20
Do you have shared interrupt debugging turned on?  That breaks this
driver, and a patch to remove the shared flag was nacked in favor of
actually fixing the driver, which I haven't gotten around to.
=20
-Scott

Hmm.... ttyCPM1 output seems to be kinda garbled... Any ideas?
Incomplete locking between printk and userland use of ttyCPM1 as
console?

kjour[  OK  ]   nal
 stdrtiag. nCom it mnteivalr5 s cones
dXT3EFS  n soa7,dint rnae jolrnau
ElT3-Xs: founmed tilefystsm weth irdeoed ratadmod .
ek/*    [  Removing /var/run/* and /var/locp... [  OK  ]
Creating new /var/run/utmlogin /fastboot and /forcefsck...      OK  ]
Removing possible /etc/noBringing up the loopback interface...  [  OK  ]
OK  ]
OK  ] Entering runlevel: 3 hostname to 9919_unit1_c
Starting the i2c_8266 driver: i2
_82c6: 6odume llcenie 'snspucifeed'itai ts nernkl.
                                                  e[ OK ]

Rune Torgersen wrote:

Hmm.... ttyCPM1 output seems to be kinda garbled... Any ideas?
Incomplete locking between printk and userland use of ttyCPM1 as
console?

s/incomplete/nonexistent/ :-P

Try acquiring port->lock in cpm_uart_console_write.

-Scott

Scott Wood wrote:

Rune Torgersen wrote:

quoted

Hmm.... ttyCPM1 output seems to be kinda garbled... Any ideas?
Incomplete locking between printk and userland use of ttyCPM1 as
console?

=20
s/incomplete/nonexistent/ :-P
=20
Try acquiring port->lock in cpm_uart_console_write.
=20
-Scott

That worked. Console output is sane again.

On May 20, 2008, at 11:31 AM, Rune Torgersen wrote:

Scott Wood wrote:

quoted

Rune Torgersen wrote:

quoted

Hmm.... ttyCPM1 output seems to be kinda garbled... Any ideas?
Incomplete locking between printk and userland use of ttyCPM1 as
console?

s/incomplete/nonexistent/ :-P

Try acquiring port->lock in cpm_uart_console_write.

-Scott

That worked. Console output is sane again.

I'm hoping to see some patches related to these fixes :)

- k

Kumar Gala wrote:

quoted

Rune Torgersen wrote:
That worked. Console output is sane again.

=20
I'm hoping to see some patches related to these fixes :)

Working on it.