2.6.23-rc8 dies somewhere during boot!?

Hi,

I'm working on a 2.6.23 kernel for the AmigaOne. I implemented the device
tree and the platform setup code, which all compiles fine. I built a
cuImage target, loaded it on my target machine with TFTP and booted it.
The kernel passes the platform setup code and then dies somewhere in the
driver init code (AFAICT), but before the keyboard driver is initialized
(thus magic sysrq key doesn't work). Can somebody help me to track down
this problem?

I tried to recover the kernel log buffer and attached it to this e-mail.

regards,

Gerhard
-- 
Psssst! Schon vom neuen GMX MultiMessenger gehört?
Der kanns mit allen: http://www.gmx.net/de/go/multimessenger

On Thu, Sep 27, 2007 at 09:12:33PM +0200, Gerhard Pircher wrote:

Hi,

I'm working on a 2.6.23 kernel for the AmigaOne.

[...]

<6>PCI: Probing PCI hardware.
<7>PCI: Scanning bus 0...
...00:00:07.0.
<7>PCI: Calling quirk...
...CI: Found 0000:00:07.2 [1106/303...

Any chance that this thing has an e100 ethernet card in it? 
If so, edit drivers/pci/quirks.c and ifdef out the readb()
in the e100_quirk routine.

We're debating the proper fix on the pci mailing list now.

--linas

-------- Original-Nachricht --------

Datum: Thu, 27 Sep 2007 14:25:51 -0500
Von: linas@austin.ibm.com
An: Gerhard Pircher [off-list ref]
CC: linuxppc-dev@ozlabs.org
Betreff: Re: 2.6.23-rc8 dies somewhere during boot!?

On Thu, Sep 27, 2007 at 09:12:33PM +0200, Gerhard Pircher wrote:

quoted

Hi,

I'm working on a 2.6.23 kernel for the AmigaOne.

[...]

quoted

<6>PCI: Probing PCI hardware.
<7>PCI: Scanning bus 0...
...00:00:07.0.
<7>PCI: Calling quirk...
...CI: Found 0000:00:07.2 [1106/303...

Any chance that this thing has an e100 ethernet card in it? 
If so, edit drivers/pci/quirks.c and ifdef out the readb()
in the e100_quirk routine.

No, it has a 3Com 3c905C/3c920 10/100Mbit controller.

Gerhard

-- 
Ist Ihr Browser Vista-kompatibel? Jetzt die neuesten 
Browser-Versionen downloaden: http://www.gmx.net/de/go/browser

On Thu, Sep 27, 2007 at 09:31:31PM +0200, Gerhard Pircher wrote:

quoted

Betreff: Re: 2.6.23-rc8 dies somewhere during boot!?

quoted

I'm working on a 2.6.23 kernel for the AmigaOne.

Have you tried 2.6.22, or does that fail also?

--linas

-------- Original-Nachricht --------

Datum: Thu, 27 Sep 2007 15:50:12 -0500
Von: linas@austin.ibm.com
An: Gerhard Pircher [off-list ref]
CC: linuxppc-dev@ozlabs.org
Betreff: Re: 2.6.23-rc8 dies somewhere during boot!?

On Thu, Sep 27, 2007 at 09:31:31PM +0200, Gerhard Pircher wrote:

quoted

quoted

Betreff: Re: 2.6.23-rc8 dies somewhere during boot!?

quoted

I'm working on a 2.6.23 kernel for the AmigaOne.

Have you tried 2.6.22, or does that fail also?

--linas

No, not yet. The last kernel that is working on the AmigaOne is a 2.6.16.x
kernel (still arch/ppc). All further kernels (2.6.17 and 2.6.18) that I
compiled with the old platform code for arch/ppc showed a similar
behavior. I hoped that this would only be a side effect of the ppc ->
powerpc transition. I also tried to search through the changelog files
to find an indication for the problem.

Do you have an idea how to debug it?
I guess it would be possible with a JTAG debugger, but the AmigaOne is
a desktop system and I don't know, if the JTAG pins of the CPU are
accessible on the board.

Gerhard
-- 
GMX FreeMail: 1 GB Postfach, 5 E-Mail-Adressen, 10 Free SMS.
Alle Infos und kostenlose Anmeldung: http://www.gmx.net/de/go/freemail

On Thu, Sep 27, 2007 at 11:17:00PM +0200, Gerhard Pircher wrote:

quoted

Betreff: Re: 2.6.23-rc8 dies somewhere during boot!?

Do you have an idea how to debug it?

Not particularly. What caught my eye was the failure right near the
PCI quirk stuff, as I was having problems there as well (but apearntly,
for very different reasons).  Based on your boot messages, it looks 
like you are failing somewhere in pci probe.  My olde-fashioned, slow,
but-usually-works method is to sprinkle enough printk's into the code
to catch it in the act.

--linas

-------- Original-Nachricht --------

Datum: Thu, 27 Sep 2007 16:35:03 -0500
Von: linas@austin.ibm.com
An: Gerhard Pircher [off-list ref]
CC: linuxppc-dev@ozlabs.org
Betreff: Re: 2.6.23-rc8 dies somewhere during boot!?

On Thu, Sep 27, 2007 at 11:17:00PM +0200, Gerhard Pircher wrote:

quoted

quoted

Betreff: Re: 2.6.23-rc8 dies somewhere during boot!?

Do you have an idea how to debug it?

Not particularly. What caught my eye was the failure right near the
PCI quirk stuff, as I was having problems there as well (but apearntly,
for very different reasons).

Yes, I also suspect the PCI code. The AmigaOne has a VIA southbridge.
AFAIK there were some changes in the PCI quirks for VIA chipsets, but so
far I couldn't see any significant changes in the code.

Based on your boot messages, it looks like you are failing somewhere in
pci probe.  My olde-fashioned, slow, but-usually-works method is to
sprinkle enough printk's into the code to catch it in the act.

I guess the code in arch/powerpc/pci*.c is the right place to sprinkle
some printk's into the code?

Gerhard
-- 
GMX FreeMail: 1 GB Postfach, 5 E-Mail-Adressen, 10 Free SMS.
Alle Infos und kostenlose Anmeldung: http://www.gmx.net/de/go/freemail

On Thu, Sep 27, 2007 at 11:57:35PM +0200, Gerhard Pircher wrote:

quoted

Based on your boot messages, it looks like you are failing somewhere in
pci probe.  My olde-fashioned, slow, but-usually-works method is to
sprinkle enough printk's into the code to catch it in the act.

I guess the code in arch/powerpc/pci*.c is the right place to sprinkle
some printk's into the code?

The last identifiable message I was

<7>PCI: Calling quirk...

which is from drivers/pci/quirks.c

...CI: Found 0000:00:07.2 [1106/303...

and this is from pci_setup_device() in drivers/pci/probe.c  So I'd look
to see if pci_setup_device() ever returned, and then I'd look to see
what happened next.

--linas

-------- Original-Nachricht --------

Datum: Thu, 27 Sep 2007 17:20:07 -0500
Von: linas@austin.ibm.com
An: Gerhard Pircher [off-list ref]
CC: linuxppc-dev@ozlabs.org
Betreff: Re: 2.6.23-rc8 dies somewhere during boot!?

On Thu, Sep 27, 2007 at 11:57:35PM +0200, Gerhard Pircher wrote:

The last identifiable message I was

<7>PCI: Calling quirk...

which is from drivers/pci/quirks.c

...CI: Found 0000:00:07.2 [1106/303...

and this is from pci_setup_device() in drivers/pci/probe.c  So I'd look
to see if pci_setup_device() ever returned, and then I'd look to see
what happened next.

Ah, I thought probing is done in the architecture's PCI code. I'll take
a look at pci_setup_device() function.

Thanks!

Gerhard
-- 
Der GMX SmartSurfer hilft bis zu 70% Ihrer Onlinekosten zu sparen! 
Ideal für Modem und ISDN: http://www.gmx.net/de/go/smartsurfer

On Fri, 2007-09-28 at 00:27 +0200, Gerhard Pircher wrote:

-------- Original-Nachricht --------

quoted

Datum: Thu, 27 Sep 2007 17:20:07 -0500
Von: linas@austin.ibm.com
An: Gerhard Pircher [off-list ref]
CC: linuxppc-dev@ozlabs.org
Betreff: Re: 2.6.23-rc8 dies somewhere during boot!?

quoted

On Thu, Sep 27, 2007 at 11:57:35PM +0200, Gerhard Pircher wrote:

The last identifiable message I was

<7>PCI: Calling quirk...

which is from drivers/pci/quirks.c

...CI: Found 0000:00:07.2 [1106/303...

and this is from pci_setup_device() in drivers/pci/probe.c  So I'd look
to see if pci_setup_device() ever returned, and then I'd look to see
what happened next.

Ah, I thought probing is done in the architecture's PCI code. I'll take
a look at pci_setup_device() function.

It's one of the quirks I suppose. Those are often full of x86 only
bogoziness that hurts everybody else if you happen to hit them.

The "late" quirks are called at pci_enable_device() time.

Ben

Gerhard Pircher writes:

I'm working on a 2.6.23 kernel for the AmigaOne. I implemented the device
tree and the platform setup code, which all compiles fine. I built a
cuImage target, loaded it on my target machine with TFTP and booted it.
The kernel passes the platform setup code and then dies somewhere in the
driver init code (AFAICT), but before the keyboard driver is initialized
(thus magic sysrq key doesn't work). Can somebody help me to track down
this problem?

You appear to have a working 16550-style serial port which the udbg
infrastructure sees.  Thus you should be able to use xmon, talking to
it via the serial port.  Put "xmon" on the kernel command line and the
kernel will drop into xmon early in the boot process (in setup_arch).
Then, either the kernel will oops at some point and drop into xmon,
and you can then inspect memory and registers and get a stack trace,
or it will hang at some point.  If it hangs, you can work out where it
hangs by putting in breakpoints at various points and seeing which
breakpoints you get to (this might take several boots).

Paul.

-------- Original-Nachricht --------

Datum: Fri, 28 Sep 2007 09:31:31 +1000
Von: Paul Mackerras [off-list ref]
An: "Gerhard Pircher" [off-list ref]
CC: linuxppc-dev@ozlabs.org
Betreff: Re: 2.6.23-rc8 dies somewhere during boot!?

You appear to have a working 16550-style serial port which the udbg
infrastructure sees.  Thus you should be able to use xmon, talking to

I couldn't get udbg running yet. Maybe i didn't specify a proper kernel
boot argument.

it via the serial port.  Put "xmon" on the kernel command line and the
kernel will drop into xmon early in the boot process (in setup_arch).
Then, either the kernel will oops at some point and drop into xmon,
and you can then inspect memory and registers and get a stack trace,
or it will hang at some point.  If it hangs, you can work out where it
hangs by putting in breakpoints at various points and seeing which
breakpoints you get to (this might take several boots).

Okay, I'll investigate it. Is there a documentation for xmon (Google
wasn't really helpful in this case).

Thanks!

regards,

Gerhard
-- 
Der GMX SmartSurfer hilft bis zu 70% Ihrer Onlinekosten zu sparen! 
Ideal für Modem und ISDN: http://www.gmx.net/de/go/smartsurfer