3.8-rc2: EFI framebuffer lock inversion...

Started by Daniel J Blueman on Jan 3, 2013 · 6 messages from 4 authors · Last reply Jan 3, 2013

open the first message on its own page

Message details and related content
On 3.8-rc2 with lockdep enabled and dual-GPU setup (Macbook Pro
Retina), I see two releated lock inversion issues with the EFI
framebuffer, leading to possible deadlock: when X takes over from the
EFI framebuffer [1] and when nouveau releases the framebuffer when
being vgaswitcherood [2].

Let me know if you'd like any testing or analysis when I can get the time.

Many thanks,
  Daniel
--- [1]
init: lightdm main process (950) terminated with status 1

===========================
[ INFO: possible circular locking dependency detected ]
3.8.0-rc2-expert #1 Not tainted
-------------------------------------------------------
Xorg/1193 is trying to acquire lock:
 ((fb_notifier_list).rwsem){++++.+}, at: [<ffffffff810697c1>]
__blocking_notifier_call_chain+0x51/0xc0

but task is already holding lock:
 (console_lock){+.+.+.}, at: [<ffffffff81263f95>] do_fb_ioctl+0x2e5/0x5f0

which lock already depends on the new lock.


the existing dependency chain (in reverse order) is:

-> #1 (console_lock){+.+.+.}:
    [<ffffffff81090a61>] __lock_acquire+0x3a1/0xb60
    [<ffffffff810916ea>] lock_acquire+0x5a/0x70
    [<ffffffff810407a7>] console_lock+0x77/0x80
    [<ffffffff812c6d84>] register_con_driver+0x34/0x140
    [<ffffffff812c84e9>] take_over_console+0x29/0x60
    [<ffffffff8126e76b>] fbcon_takeover+0x5b/0xb0
    [<ffffffff81272bb5>] fbcon_event_notify+0x715/0x820
    [<ffffffff810693a5>] notifier_call_chain+0x55/0x110
    [<ffffffff810697d7>] __blocking_notifier_call_chain+0x67/0xc0
    [<ffffffff81069841>] blocking_notifier_call_chain+0x11/0x20
    [<ffffffff81262a16>] fb_notifier_call_chain+0x16/0x20
    [<ffffffff81264c1d>] register_framebuffer+0x1bd/0x2f0
    [<ffffffff81ac2bd4>] efifb_probe+0x40f/0x496
    [<ffffffff81308dfe>] platform_drv_probe+0x3e/0x70
    [<ffffffff81306dc6>] driver_probe_device+0x76/0x240
    [<ffffffff81307033>] __driver_attach+0xa3/0xb0
    [<ffffffff8130503d>] bus_for_each_dev+0x4d/0x90
    [<ffffffff81306929>] driver_attach+0x19/0x20
    [<ffffffff813064e0>] bus_add_driver+0x1a0/0x270
    [<ffffffff813076c2>] driver_register+0x72/0x170
    [<ffffffff81308671>] platform_driver_register+0x41/0x50

64 more lines

--- [2]
hda-intel 0000:01:00.1: Disabling via VGA-switcheroo
hda-intel 0000:01:00.1: Cannot lock devices!
VGA switcheroo: switched nouveau off
nouveau [   DRM] suspending fbcon...

===========================
[ INFO: possible circular locking dependency detected ]
3.8.0-rc2-expert #1 Not tainted
-------------------------------------------------------
sh/1017 is trying to acquire lock:
 ((fb_notifier_list).rwsem){++++.+}, at: [<ffffffff810697c1>]
__blocking_notifier_call_chain+0x51/0xc0

but task is already holding lock:
 (console_lock){+.+.+.}, at: [<ffffffffa0204d35>]
nouveau_fbcon_set_suspend+0x25/0xc0 [nouveau]

which lock already depends on the new lock.


the existing dependency chain (in reverse order) is:

-> #1 (console_lock){+.+.+.}:
    [<ffffffff81090a61>] __lock_acquire+0x3a1/0xb60
    [<ffffffff810916ea>] lock_acquire+0x5a/0x70
    [<ffffffff810407a7>] console_lock+0x77/0x80
    [<ffffffff812c6d84>] register_con_driver+0x34/0x140
    [<ffffffff812c84e9>] take_over_console+0x29/0x60
    [<ffffffff8126e76b>] fbcon_takeover+0x5b/0xb0
    [<ffffffff81272bb5>] fbcon_event_notify+0x715/0x820
    [<ffffffff810693a5>] notifier_call_chain+0x55/0x110
    [<ffffffff810697d7>] __blocking_notifier_call_chain+0x67/0xc0
    [<ffffffff81069841>] blocking_notifier_call_chain+0x11/0x20
    [<ffffffff81262a16>] fb_notifier_call_chain+0x16/0x20
    [<ffffffff81264c1d>] register_framebuffer+0x1bd/0x2f0
    [<ffffffff81ac2bd4>] efifb_probe+0x40f/0x496
    [<ffffffff81308dfe>] platform_drv_probe+0x3e/0x70
    [<ffffffff81306dc6>] driver_probe_device+0x76/0x240
    [<ffffffff81307033>] __driver_attach+0xa3/0xb0
    [<ffffffff8130503d>] bus_for_each_dev+0x4d/0x90

85 more lines

Message details and related content
On Thu, 3 Jan 2013 20:56:30 +0800
Daniel J Blueman [off-list ref] wrote:
On 3.8-rc2 with lockdep enabled and dual-GPU setup (Macbook Pro
Retina), I see two releated lock inversion issues with the EFI
framebuffer, leading to possible deadlock: when X takes over from the
EFI framebuffer [1] and when nouveau releases the framebuffer when
being vgaswitcherood [2].

Let me know if you'd like any testing or analysis when I can get the time.
The fb layer locking was broken. I posted patches early December which
should have fixed the ones we know about. ('fb: Rework locking to fix
lock ordering on takeover').

Alan
Message details and related content
On 3 January 2013 21:11, Alan Cox [off-list ref] wrote:
On Thu, 3 Jan 2013 20:56:30 +0800
Daniel J Blueman [off-list ref] wrote:
quoted
On 3.8-rc2 with lockdep enabled and dual-GPU setup (Macbook Pro
Retina), I see two releated lock inversion issues with the EFI
framebuffer, leading to possible deadlock: when X takes over from the
EFI framebuffer [1] and when nouveau releases the framebuffer when
being vgaswitcherood [2].

Let me know if you'd like any testing or analysis when I can get the time.
The fb layer locking was broken. I posted patches early December which
should have fixed the ones we know about. ('fb: Rework locking to fix
lock ordering on takeover').
Superb work, Alan!

The only patch I could find [1] (mid Nov) looks like it needs another
sites updating, since we now see an i915 vs efifb lock ordering issue
[2].

I can get some time next week to take a look if it helps.

Thanks,
  Daniel
--- [1] https://patchwork.kernel.org/patch/1757061/

--- [2]
[drm] Memory usable by graphics device = 2048M
checking generic (b0000000 1440000) vs hw (b0000000 10000000)
fb: conflicting fb hw usage inteldrmfb vs EFI VGA - removing generic driver

===========================
[ INFO: possible circular locking dependency detected ]
3.8.0-rc2-expert+ #2 Not tainted
-------------------------------------------------------
modprobe/603 is trying to acquire lock:
 (console_lock){+.+.+.}, at: [<ffffffff812c869f>] unbind_con_driver+0x3f/0x200

but task is already holding lock:
 ((fb_notifier_list).rwsem){++++.+}, at: [<ffffffff810697c1>]
__blocking_notifier_call_chain+0x51/0xc0

which lock already depends on the new lock.


the existing dependency chain (in reverse order) is:

-> #1 ((fb_notifier_list).rwsem){++++.+}:
    [<ffffffff81090a61>] __lock_acquire+0x3a1/0xb60
    [<ffffffff810916ea>] lock_acquire+0x5a/0x70
    [<ffffffff81557c97>] down_read+0x47/0x5c
    [<ffffffff810697c1>] __blocking_notifier_call_chain+0x51/0xc0
    [<ffffffff81069841>] blocking_notifier_call_chain+0x11/0x20
    [<ffffffff81262a16>] fb_notifier_call_chain+0x16/0x20
    [<ffffffff81264c20>] register_framebuffer+0x1c0/0x300
    [<ffffffff81ac2bd4>] efifb_probe+0x40f/0x496
    [<ffffffff81308fbe>] platform_drv_probe+0x3e/0x70
    [<ffffffff81306f86>] driver_probe_device+0x76/0x240
    [<ffffffff813071f3>] __driver_attach+0xa3/0xb0
    [<ffffffff813051fd>] bus_for_each_dev+0x4d/0x90
    [<ffffffff81306ae9>] driver_attach+0x19/0x20
    [<ffffffff813066a0>] bus_add_driver+0x1a0/0x270
    [<ffffffff81307882>] driver_register+0x72/0x170
    [<ffffffff81308831>] platform_driver_register+0x41/0x50
    [<ffffffff81308856>] platform_driver_probe+0x16/0xa0
    [<ffffffff81ac2ece>] efifb_init+0x273/0x292
    [<ffffffff810002da>] do_one_initcall+0x11a/0x170

111 more lines

Message details and related content
The only patch I could find [1] (mid Nov) looks like it needs another
sites updating, since we now see an i915 vs efifb lock ordering issue
[2].

I can get some time next week to take a look if it helps.
That would be great. I've not got any EFI afflicted hardware and I'm
doing my best to avoid it.

Alan
Message details and related content
On Thu, Jan 3, 2013 at 5:11 AM, Alan Cox [off-list ref] wrote:
The fb layer locking was broken. I posted patches early December which
should have fixed the ones we know about. ('fb: Rework locking to fix
lock ordering on takeover').
That patch causes compile errors with "allmodconfig":

  ERROR: "do_take_over_console" [drivers/video/console/fbcon.ko] undefined!
  make[1]: *** [__modpost] Error 1
  make: *** [modules] Error 2
  make: *** Waiting for unfinished jobs....

Hmm?

               Linus
Message details and related content
On Thu, 3 Jan 2013 11:40:47 -0800
Linus Torvalds [off-list ref] wrote:
On Thu, Jan 3, 2013 at 5:11 AM, Alan Cox [off-list ref] wrote:
quoted
The fb layer locking was broken. I posted patches early December which
should have fixed the ones we know about. ('fb: Rework locking to fix
lock ordering on takeover').
That patch causes compile errors with "allmodconfig":

  ERROR: "do_take_over_console" [drivers/video/console/fbcon.ko] undefined!
  make[1]: *** [__modpost] Error 1
  make: *** [modules] Error 2
  make: *** Waiting for unfinished jobs....

Hmm?
I have a couple of fixes against
fb-rework-locking-to-fix-lock-ordering-on-takeover.patch:

http://ozlabs.org/~akpm/mmots/broken-out/fb-rework-locking-to-fix-lock-ordering-on-takeover-fix.patch
http://ozlabs.org/~akpm/mmots/broken-out/fb-rework-locking-to-fix-lock-ordering-on-takeover-fix-2.patch

Florian has been busy for a month or two - I've been waiting for him to
reappear to consider this patch.
Keyboard shortcuts
hback out one level
jnext message in thread / next series-diff position
kprevious message in thread / previous series-diff position
ldrill in
Escclose help / fold thread tree
?toggle this help