From: Michael Ellerman <mpe@ellerman.id.au> Date: 2016-10-31 05:29:06
Andreas Schwab [off-list ref] writes:
Any news?
We discovered it also breaks VGA on qemu, which presumably is not the
type of news you were hoping for.
To reproduce you just need to build a ppc64le kernel:
$ apt-get install gcc-powerpc64le-linux-gnu
$ make ARCH=powerpc CROSS_COMPILE=powerpc64le-linux-gnu- pseries_le_defconfig
$ make ARCH=powerpc CROSS_COMPILE=powerpc64le-linux-gnu-
And then run qemu:
$ qemu-system-ppc64 -M pseries -m 1G -kernel vmlinux
Which will display the Tux logo but then nothing else and then reboot
after 10 seconds or so.
If you revert 05fd007e4629 on top of rc3 it boots fine.
Paul I tried your "console: use first console if stdout-path device
doesn't appear" patch, but it didn't help. I'll try and work out why,
but it's a bit hard with no output :)
If we can't come up with a fix soon I'm inclined to ask for a revert.
cheers
From: Paul Burton <hidden> Date: 2016-10-31 12:15:26
If a device tree specified a preferred device for kernel console output
via the stdout-path or linux,stdout-path chosen node properties there's
no guarantee that it will have specified a device for which we have a
driver. It may also be the case that we do have a driver but it doesn't
call of_console_check() to register as a preferred console (eg. offb
driver as used on powermac systems). In these cases try to ensure that
we provide some console output by enabling the first usable registered
console, which we keep track of with the of_fallback_console variable.
Tested in QEMU with a PowerPC pseries_defconfig kernel.
Signed-off-by: Paul Burton <redacted>
Fixes: 05fd007e4629 ("console: don't prefer first registered if DT specifies stdout-path")
Reported-by: Andreas Schwab <redacted>
Reported-by: Larry Finger <redacted>
Reported-by: Michael Ellerman <mpe@ellerman.id.au>
Cc: Andreas Schwab <redacted>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Borislav Petkov <redacted>
Cc: Larry Finger <redacted>
Cc: Michael Ellerman <mpe@ellerman.id.au>
Cc: Petr Mladek <pmladek@suse.com>
Cc: Sergey Senozhatsky <redacted>
Cc: Tejun Heo <tj@kernel.org>
Cc: linux-kernel@vger.kernel.org
Cc: linuxppc-dev@lists.ozlabs.org
---
Changes in v2:
- Split enable_console() out of register_console() & call in the fallback case.
- Track the console we would have enabled as of_fallback_console.
kernel/printk/printk.c | 60 +++++++++++++++++++++++++++++++++++++++++++++-----
1 file changed, 55 insertions(+), 5 deletions(-)
From: Paul Burton <hidden> Date: 2016-10-31 15:51:07
On Monday, 31 October 2016 12:14:55 GMT Paul Burton wrote:
If a device tree specified a preferred device for kernel console output
via the stdout-path or linux,stdout-path chosen node properties there's
no guarantee that it will have specified a device for which we have a
driver. It may also be the case that we do have a driver but it doesn't
call of_console_check() to register as a preferred console (eg. offb
driver as used on powermac systems). In these cases try to ensure that
we provide some console output by enabling the first usable registered
console, which we keep track of with the of_fallback_console variable.
Tested in QEMU with a PowerPC pseries_defconfig kernel.
Actually whilst this fixes the output in QEMU it has other problems. I'm still
digging...
Thanks,
Paul
quoted hunk
Signed-off-by: Paul Burton <redacted>
Fixes: 05fd007e4629 ("console: don't prefer first registered if DT specifies
stdout-path") Reported-by: Andreas Schwab [off-list ref]
Reported-by: Larry Finger <redacted>
Reported-by: Michael Ellerman <mpe@ellerman.id.au>
Cc: Andreas Schwab <redacted>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Borislav Petkov <redacted>
Cc: Larry Finger <redacted>
Cc: Michael Ellerman <mpe@ellerman.id.au>
Cc: Petr Mladek <pmladek@suse.com>
Cc: Sergey Senozhatsky <redacted>
Cc: Tejun Heo <tj@kernel.org>
Cc: linux-kernel@vger.kernel.org
Cc: linuxppc-dev@lists.ozlabs.org
---
Changes in v2:
- Split enable_console() out of register_console() & call in the fallback
case. - Track the console we would have enabled as of_fallback_console.
kernel/printk/printk.c | 60
+++++++++++++++++++++++++++++++++++++++++++++----- 1 file changed, 55
insertions(+), 5 deletions(-)
* rid of the boot console as soon as the proper console shows up, so there
* won't be side-effects from postponing the removal.
+ *
+ * Additionally we may be using a device tree which specifies valid
+ * stdout-path referencing a device for which we don't have a driver, or
for + * which we have a driver that doesn't register itself as preferred
console + * using of_console_check(). In these cases we attempt here to
enable the + * first usable registered console, which we assigned to
of_fallback_console. */
static int __init printk_late_init(void)
{
struct console *con;
+ bool any_enabled = false;
+
+ for_each_console(con) {
+ if (!(con->flags & CON_ENABLED))
+ continue;
+
+ any_enabled = true;
+ break;
+ }
+
+ if (!any_enabled && of_fallback_console) {
+ if (of_fallback_console->index < 0)
+ of_fallback_console->index = 0;
+
+ if (!of_fallback_console->setup ||
+ !of_fallback_console->setup(of_fallback_console, NULL)) {
+ of_fallback_console->flags |= CON_ENABLED;
+ if (of_fallback_console->device) {
+ of_fallback_console->flags |= CON_CONSDEV;
+ preferred_console = 0;
+ }
+ }
+
+ enable_console(of_fallback_console);
+ }
for_each_console(con) {
if (!keep_bootcon && con->flags & CON_BOOT) {
From: Larry Finger <hidden> Date: 2016-10-31 19:21:35
On 10/31/2016 10:50 AM, Paul Burton wrote:
On Monday, 31 October 2016 12:14:55 GMT Paul Burton wrote:
quoted
If a device tree specified a preferred device for kernel console output
via the stdout-path or linux,stdout-path chosen node properties there's
no guarantee that it will have specified a device for which we have a
driver. It may also be the case that we do have a driver but it doesn't
call of_console_check() to register as a preferred console (eg. offb
driver as used on powermac systems). In these cases try to ensure that
we provide some console output by enabling the first usable registered
console, which we keep track of with the of_fallback_console variable.
Tested in QEMU with a PowerPC pseries_defconfig kernel.
Actually whilst this fixes the output in QEMU it has other problems. I'm still
digging...
As expected, it does n ot work with 4.9-rc3 on the real PowerPC.
Larry
From: Paul Burton <hidden> Date: 2016-11-03 13:00:00
If a device tree specified a preferred device for kernel console output
via the stdout-path or linux,stdout-path chosen node properties there's
no guarantee that it will have specified a device for which we have a
driver. It may also be the case that we do have a driver but it doesn't
call of_console_check() to register as a preferred console (eg. offb
driver as used on powermac systems).
In these cases try to ensure that we provide some console output by
enabling the first usable registered console, which we keep track of
with the of_fallback_console variable. Affected systems will enable
their console later than they did prior to commit 05fd007e4629
("console: don't prefer first registered if DT specifies stdout-path")
but should otherwise produce the same output.
Tested in QEMU with a PowerPC pseries_defconfig kernel.
Signed-off-by: Paul Burton <redacted>
Fixes: 05fd007e4629 ("console: don't prefer first registered if DT specifies stdout-path")
Reported-by: Andreas Schwab <redacted>
Reported-by: Larry Finger <redacted>
Reported-by: Michael Ellerman <mpe@ellerman.id.au>
Cc: Andreas Schwab <redacted>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Borislav Petkov <redacted>
Cc: Larry Finger <redacted>
Cc: Michael Ellerman <mpe@ellerman.id.au>
Cc: Petr Mladek <pmladek@suse.com>
Cc: Sergey Senozhatsky <redacted>
Cc: Tejun Heo <tj@kernel.org>
Cc: linux-kernel@vger.kernel.org
Cc: linuxppc-dev@lists.ozlabs.org
---
Changes in v3:
- Be less invasive by simply calling register_console() again rather than splitting it.
- Check for CON_CONSDEV on the first console driver rather than for CON_ENABLED on any.
- Clean up the tracking of the fallback console.
Changes in v2:
- Split enable_console() out of register_console() & call in the fallback case.
- Track the console we would have enabled as of_fallback_console.
kernel/printk/printk.c | 44 ++++++++++++++++++++++++++++++++++++++++++--
1 file changed, 42 insertions(+), 2 deletions(-)
If a device tree specified a preferred device for kernel console output
via the stdout-path or linux,stdout-path chosen node properties there's
no guarantee that it will have specified a device for which we have a
driver. It may also be the case that we do have a driver but it doesn't
call of_console_check() to register as a preferred console (eg. offb
driver as used on powermac systems).
so why that driver doesn't call of_console_check() then? if there is a
misconfiguration then why do we want to fix it/fallback in printk code?
[..]
@@ -2657,10 +2665,26 @@ void register_console(struct console *newcon) * didn't select a console we take the first one * that registers here. */- if (preferred_console < 0 && !of_specified_console) {+ if (preferred_console < 0) { if (newcon->index < 0) newcon->index = 0;- if (newcon->setup == NULL ||+ if (of_specified_console) {+ /*+ * The device tree stdout-path chosen node property was+ * specified so we don't want to enable the first+ * registered console just now in order to give the+ * device indicated by stdout-path a chance to be+ * registered first. Do however keep track of the+ * first console we see so that we can fall back to+ * using it if we don't see the desired device, either+ * because stdout-path isn't valid, or because we have+ * no driver for the device or our driver doesn't call+ * of_console_check(). See printk_late_init() for this+ * fallback.
if the path is not valid then correct the path. no?
quoted hunk
+ */
+ if (!of_fallback_console)
+ of_fallback_console = newcon;
+ } else if (newcon->setup == NULL ||
newcon->setup(newcon, NULL) == 0) {
newcon->flags |= CON_ENABLED;
if (newcon->device) {
@@ -2844,6 +2868,22 @@ static int __init printk_late_init(void) { struct console *con;+ if (of_specified_console && of_fallback_console &&+ (!console_drivers || !(console_drivers->flags & CON_CONSDEV))) {+ /*+ * The system has a device tree which specified stdout-path,+ * but we haven't seen a console associated with the device+ * specified by the stdout-path chosen node property.+ *+ * We do however know which console would have been used+ * if stdout-path weren't specified at all, so in an attempt+ * to provide some output we'll re-register that console+ * pretending that we never saw stdout-path.+ */
DT screwed up, so why would printk() care? does any other
sub-system/driver fixes up a DT misconfiguration?
-ss
From: Paul Burton <hidden> Date: 2016-11-03 21:18:08
Hi Sergey,
On Friday, 4 November 2016 02:40:40 GMT Sergey Senozhatsky wrote:
On (11/03/16 12:57), Paul Burton wrote:
quoted
If a device tree specified a preferred device for kernel console output
via the stdout-path or linux,stdout-path chosen node properties there's
no guarantee that it will have specified a device for which we have a
driver. It may also be the case that we do have a driver but it doesn't
call of_console_check() to register as a preferred console (eg. offb
driver as used on powermac systems).
so why that driver doesn't call of_console_check() then? if there is a
misconfiguration then why do we want to fix it/fallback in printk code?
Ideally I think the driver (or perhaps fbdev/fbcon) ought to call
of_console_check() which would allow the console to be enabled earlier again.
However there isn't any single place I can see that currently has all the
information required to do so - the device tree node, the name & index of the
console.
Even if we change that in the future & do call of_console_check(), I can't
guarantee that offb is the only driver that would encounter this case, and it
still wouldn't cover the case of us not having any driver for the specified
stdout-path device. The fallback thus seems like a sensible thing to do.
@@ -2657,10 +2665,26 @@ void register_console(struct console *newcon) * didn't select a console we take the first one * that registers here. */- if (preferred_console < 0 && !of_specified_console) {+ if (preferred_console < 0) { if (newcon->index < 0) newcon->index = 0;- if (newcon->setup == NULL ||+ if (of_specified_console) {+ /*+ * The device tree stdout-path chosen node property was+ * specified so we don't want to enable the first+ * registered console just now in order to give the+ * device indicated by stdout-path a chance to be+ * registered first. Do however keep track of the+ * first console we see so that we can fall back to+ * using it if we don't see the desired device, either+ * because stdout-path isn't valid, or because we have+ * no driver for the device or our driver doesn't call+ * of_console_check(). See printk_late_init() for this+ * fallback.
if the path is not valid then correct the path. no?
...but what if the path is valid and we simply don't have a driver for the
device it references? As I said in that comment we may not have a driver at
all.
quoted
+ */
+ if (!of_fallback_console)
+ of_fallback_console = newcon;
+ } else if (newcon->setup == NULL ||
newcon->setup(newcon, NULL) == 0) {
newcon->flags |= CON_ENABLED;
if (newcon->device) {
@@ -2844,6 +2868,22 @@ static int __init printk_late_init(void) { struct console *con;+ if (of_specified_console && of_fallback_console &&+ (!console_drivers || !(console_drivers->flags & CON_CONSDEV))) {+ /*+ * The system has a device tree which specified stdout-path,+ * but we haven't seen a console associated with the device+ * specified by the stdout-path chosen node property.+ *+ * We do however know which console would have been used+ * if stdout-path weren't specified at all, so in an attempt+ * to provide some output we'll re-register that console+ * pretending that we never saw stdout-path.+ */
DT screwed up, so why would printk() care? does any other
sub-system/driver fixes up a DT misconfiguration?
-ss
...and again, it may not be a misconfiguration - that's one possibility of a
few that I mentioned. Even if the DT is misconfigured & stdout-path is
completely bonkers it's still arguable that falling back to the first console
registered (ie. our previous behaviour) is the sensible thing to do. Perhaps
we ought to warn in such cases, but even then we can't distinguish between the
3 cases I mentioned (invalid stdout-path, driver which doesn't call
of_console_check() or no driver at all) so we'd certainly end up warning on
systems like these affected PowerPC systems, which makes me think that may be
a better warning to introduce once we expect systems to not hit it.
Thanks,
Paul
+ * The device tree stdout-path chosen node property was
+ * specified so we don't want to enable the first
+ * registered console just now in order to give the
+ * device indicated by stdout-path a chance to be
+ * registered first. Do however keep track of the
+ * first console we see so that we can fall back to
+ * using it if we don't see the desired device, either
+ * because stdout-path isn't valid, or because we have
+ * no driver for the device or our driver doesn't call
+ * of_console_check(). See printk_late_init() for this
+ * fallback.
if the path is not valid then correct the path. no?
...but what if the path is valid and we simply don't have a driver for the
device it references? As I said in that comment we may not have a driver at
all.
well, I suppose, in this case normally one would go and enable the
missing .config option. no?
-ss
From: Michael Ellerman <mpe@ellerman.id.au> Date: 2016-11-07 08:27:33
Paul Burton [off-list ref] writes:
If a device tree specified a preferred device for kernel console output
via the stdout-path or linux,stdout-path chosen node properties there's
no guarantee that it will have specified a device for which we have a
driver. It may also be the case that we do have a driver but it doesn't
call of_console_check() to register as a preferred console (eg. offb
driver as used on powermac systems).
In these cases try to ensure that we provide some console output by
enabling the first usable registered console, which we keep track of
with the of_fallback_console variable. Affected systems will enable
their console later than they did prior to commit 05fd007e4629
("console: don't prefer first registered if DT specifies stdout-path")
but should otherwise produce the same output.
Tested in QEMU with a PowerPC pseries_defconfig kernel.
Hi Paul,
This does "work", as in it boots and I get a console. But the delay in
getting output on the VGA is not workable. I get pretty much no output
until the machine is booted entirely to userspace, meaning any crash
prior to that will be undebuggable.
I also note Andreas reports it doesn't work at all on PowerMac.
Please send a revert and we can try again next cycle.
cheers
From: Paul Burton <hidden> Date: 2016-11-07 09:18:16
On Monday, 7 November 2016 19:27:32 GMT Michael Ellerman wrote:
Paul Burton [off-list ref] writes:
quoted
If a device tree specified a preferred device for kernel console output
via the stdout-path or linux,stdout-path chosen node properties there's
no guarantee that it will have specified a device for which we have a
driver. It may also be the case that we do have a driver but it doesn't
call of_console_check() to register as a preferred console (eg. offb
driver as used on powermac systems).
In these cases try to ensure that we provide some console output by
enabling the first usable registered console, which we keep track of
with the of_fallback_console variable. Affected systems will enable
their console later than they did prior to commit 05fd007e4629
("console: don't prefer first registered if DT specifies stdout-path")
but should otherwise produce the same output.
Tested in QEMU with a PowerPC pseries_defconfig kernel.
Hi Paul,
This does "work", as in it boots and I get a console. But the delay in
getting output on the VGA is not workable. I get pretty much no output
until the machine is booted entirely to userspace, meaning any crash
prior to that will be undebuggable.
I also note Andreas reports it doesn't work at all on PowerMac.
Please send a revert and we can try again next cycle.
cheers
From: Larry Finger <hidden> Date: 2016-11-07 15:26:42
On 11/07/2016 03:18 AM, Paul Burton wrote:
On Monday, 7 November 2016 19:27:32 GMT Michael Ellerman wrote:
quoted
Paul Burton [off-list ref] writes:
quoted
If a device tree specified a preferred device for kernel console output
via the stdout-path or linux,stdout-path chosen node properties there's
no guarantee that it will have specified a device for which we have a
driver. It may also be the case that we do have a driver but it doesn't
call of_console_check() to register as a preferred console (eg. offb
driver as used on powermac systems).
In these cases try to ensure that we provide some console output by
enabling the first usable registered console, which we keep track of
with the of_fallback_console variable. Affected systems will enable
their console later than they did prior to commit 05fd007e4629
("console: don't prefer first registered if DT specifies stdout-path")
but should otherwise produce the same output.
Tested in QEMU with a PowerPC pseries_defconfig kernel.
Hi Paul,
This does "work", as in it boots and I get a console. But the delay in
getting output on the VGA is not workable. I get pretty much no output
until the machine is booted entirely to userspace, meaning any crash
prior to that will be undebuggable.
I also note Andreas reports it doesn't work at all on PowerMac.
Please send a revert and we can try again next cycle.
cheers
I am a little surprised that I was not CCd on that thread. To reiterate, my
PowerBook G4 with a PPC32 processor CRASHES on boot. That is a lot more serious
than the console output disappearing.
As it seems unlikely that this regression will be fixed in the current cycle, I
recommend that the reversion of commit 05fd007e4629 until a proper fix is available.
Larry
I am a little surprised that I was not CCd on that thread.
Hans started that thread without copying you just as you started your thread
without copying Andreas, who reported issues first.
To reiterate, my
PowerBook G4 with a PPC32 processor CRASHES on boot. That is a lot more
serious than the console output disappearing.
Crashes as in init dies due to lack of a console, right?
As it seems unlikely that this regression will be fixed in the current
cycle, I recommend that the reversion of commit 05fd007e4629 until a proper
fix is available.
I agree that reverting is probably the best option for now, and have replied
with that in the other thread too.
Thanks,
Paul
I am a little surprised that I was not CCd on that thread.
Hans started that thread without copying you just as you started your thread
without copying Andreas, who reported issues first.
My searching had failed to find his report.
quoted
To reiterate, my
PowerBook G4 with a PPC32 processor CRASHES on boot. That is a lot more
serious than the console output disappearing.
Crashes as in init dies due to lack of a console, right?
It gets a kernel panic because of an attempt to kill process 1 (init). It then
waits 120 seconds and tries to reboot.
quoted
As it seems unlikely that this regression will be fixed in the current
cycle, I recommend that the reversion of commit 05fd007e4629 until a proper
fix is available.
I agree that reverting is probably the best option for now, and have replied
with that in the other thread too.
From: Paul Burton <hidden> Date: 2016-11-03 13:04:25
On Monday, 31 October 2016 18:31:43 GMT Larry Finger wrote:
On 10/31/2016 06:09 PM, Sergey Senozhatsky wrote:
quoted
Hi,
On (10/31/16 15:50), Paul Burton wrote:
[..]
quoted
Actually whilst this fixes the output in QEMU it has other problems. I'm
still digging...
I propose a revert of '05fd007e46296', so you guys can find the
problem and fix it, not being under 'rc3' pressure.
If we were at -rc4 or -rc5, then I would agree; however, I think there is
still time to fix the problem.
Larry
Hi Larry et al,
If you could give the v3 I just submitted a try that would be great:
https://patchwork.kernel.org/patch/9410849/
If it still doesn't work for you then we're back to a place where I can't test
an affected system, since QEMU pseries now works fine with my patch. Would you
perhaps be able to get some console output by specifying console=<whatever> on
the kernel command line? Doing that ought to be unaffected by 05fd007e4629
("console: don't prefer first registered if DT specifies stdout-path") and
could be a way to get us some debug output.
Thanks,
Paul
From: Larry Finger <hidden> Date: 2016-10-31 15:58:21
On 10/31/2016 07:14 AM, Paul Burton wrote:
If a device tree specified a preferred device for kernel console output
via the stdout-path or linux,stdout-path chosen node properties there's
no guarantee that it will have specified a device for which we have a
driver. It may also be the case that we do have a driver but it doesn't
call of_console_check() to register as a preferred console (eg. offb
driver as used on powermac systems). In these cases try to ensure that
we provide some console output by enabling the first usable registered
console, which we keep track of with the of_fallback_console variable.
Tested in QEMU with a PowerPC pseries_defconfig kernel.
The patch fails to fix my real PowerPC. I still get a kernel panic for an
attempt to kill init. This first test was done with 4.9-rc2. I am in the process
of updating to -rc3 to see if that changes anything. Later today when that build
finishes, I will report those results.
Larry
From: Paul Burton <hidden> Date: 2016-10-31 12:23:36
Hi Michael et al,
On Monday, 31 October 2016 16:28:59 GMT Michael Ellerman wrote:
Andreas Schwab [off-list ref] writes:
quoted
Any news?
We discovered it also breaks VGA on qemu, which presumably is not the
type of news you were hoping for.
On the contrary, that's wonderful news - I can test that! Huzzah for
brokenness!
Thanks for your steps to reproduce the issue. Could you try my v2 patch? It
works for me in QEMU, at least as far as seeing the kernel console output for
a panic due to not having a rootfs. Hopefully it works for you (& Andreas &
Larry) too:
https://patchwork.kernel.org/patch/9405415/
Thanks,
Paul