Lockdep warning in iwl_pcie_rx_handle()
From: Jiri Kosina <jikos@kernel.org>
Date: 2021-03-02 07:09:06
Also in:
lkml
Hi, I am getting the splat below with Linus' tree as of today (5.11-rc1, fe07bfda2fb). I haven't started to look into the code yet, but apparently this has been already reported by Heiner here: https://www.spinics.net/lists/linux-wireless/msg208353.html so before I start digging deep into it (the previous kernel this particular machine had is 5.9, so I'd rather avoid lenghty bisect for now in case someone has already looked into it and has ideas where the problem is), I thought I'd ask whether this has been root-caused elsewhere already. Thanks. [ 9.970414] ======================================================== [ 9.973157] WARNING: possible irq lock inversion dependency detected [ 9.976186] 5.12.0-rc1-00002-gbada49429032 #5 Not tainted [ 9.979239] -------------------------------------------------------- [ 9.982223] irq/136-iwlwifi/565 just changed the state of lock: [ 9.984904] ffff89f28433b0b0 (&rxq->lock){+.-.}-{2:2}, at: iwl_pcie_rx_handle+0x7f/0x960 [iwlwifi] [ 9.987932] but this lock took another, SOFTIRQ-unsafe lock in the past: [ 9.990980] (napi_hash_lock){+.+.}-{2:2} [ 9.990986] and interrupts could create inverse lock ordering between them. [ 10.003955] other info that might help us debug this: [ 10.008989] Possible interrupt unsafe locking scenario: [ 10.013966] CPU0 CPU1 [ 10.016404] ---- ---- [ 10.018976] lock(napi_hash_lock); [ 10.021529] local_irq_disable(); [ 10.023985] lock(&rxq->lock); [ 10.026340] lock(napi_hash_lock); [ 10.028644] <Interrupt> [ 10.030934] lock(&rxq->lock); [ 10.033119] *** DEADLOCK *** [ 10.039456] 1 lock held by irq/136-iwlwifi/565: [ 10.041549] #0: ffff89f2b1440170 (sync_cmd_lockdep_map){+.+.}-{0:0}, at: iwl_pcie_irq_handler+0x5/0xb30 [iwlwifi] [ 10.043728] the shortest dependencies between 2nd lock and 1st lock: [ 10.047836] -> (napi_hash_lock){+.+.}-{2:2} { [ 10.049845] HARDIRQ-ON-W at: [ 10.051860] lock_acquire+0x277/0x3d0 [ 10.053881] _raw_spin_lock+0x2c/0x40 [ 10.056170] netif_napi_add+0x14b/0x270 [ 10.058349] e1000_probe+0x2fe/0xee0 [e1000e] [ 10.060539] local_pci_probe+0x42/0x90 [ 10.062609] pci_device_probe+0x10b/0x1c0 [ 10.064579] really_probe+0xef/0x4b0 [ 10.066578] driver_probe_device+0xde/0x150 [ 10.068765] device_driver_attach+0x4f/0x60 [ 10.070890] __driver_attach+0x9c/0x140 [ 10.073115] bus_for_each_dev+0x79/0xc0 [ 10.075304] bus_add_driver+0x18d/0x220 [ 10.077464] driver_register+0x5b/0xf0 [ 10.079615] do_one_initcall+0x5b/0x300 [ 10.081686] do_init_module+0x5b/0x21c [ 10.083730] load_module+0x1dae/0x22c0 [ 10.085751] __do_sys_finit_module+0xad/0x110 [ 10.087799] do_syscall_64+0x33/0x80 [ 10.089758] entry_SYSCALL_64_after_hwframe+0x44/0xae [ 10.091747] SOFTIRQ-ON-W at: [ 10.093634] lock_acquire+0x277/0x3d0 [ 10.095514] _raw_spin_lock+0x2c/0x40 [ 10.097353] netif_napi_add+0x14b/0x270 [ 10.099208] e1000_probe+0x2fe/0xee0 [e1000e] [ 10.101032] local_pci_probe+0x42/0x90 [ 10.102835] pci_device_probe+0x10b/0x1c0 [ 10.104648] really_probe+0xef/0x4b0 [ 10.106439] driver_probe_device+0xde/0x150 [ 10.106443] device_driver_attach+0x4f/0x60 [ 10.106446] __driver_attach+0x9c/0x140 [ 10.111451] bus_for_each_dev+0x79/0xc0 [ 10.111454] bus_add_driver+0x18d/0x220 [ 10.111456] driver_register+0x5b/0xf0 [ 10.111459] do_one_initcall+0x5b/0x300 [ 10.111462] do_init_module+0x5b/0x21c [ 10.111467] load_module+0x1dae/0x22c0 [ 10.111469] __do_sys_finit_module+0xad/0x110 [ 10.122761] do_syscall_64+0x33/0x80 [ 10.122768] entry_SYSCALL_64_after_hwframe+0x44/0xae [ 10.122772] INITIAL USE at: [ 10.122775] lock_acquire+0x277/0x3d0 [ 10.128672] _raw_spin_lock+0x2c/0x40 [ 10.130160] netif_napi_add+0x14b/0x270 [ 10.131725] e1000_probe+0x2fe/0xee0 [e1000e] [ 10.133268] local_pci_probe+0x42/0x90 [ 10.134783] pci_device_probe+0x10b/0x1c0 [ 10.136325] really_probe+0xef/0x4b0 [ 10.137864] driver_probe_device+0xde/0x150 [ 10.139394] device_driver_attach+0x4f/0x60 [ 10.140908] __driver_attach+0x9c/0x140 [ 10.142429] bus_for_each_dev+0x79/0xc0 [ 10.143947] bus_add_driver+0x18d/0x220 [ 10.145490] driver_register+0x5b/0xf0 [ 10.147024] do_one_initcall+0x5b/0x300 [ 10.148566] do_init_module+0x5b/0x21c [ 10.150013] load_module+0x1dae/0x22c0 [ 10.151293] __do_sys_finit_module+0xad/0x110 [ 10.152571] do_syscall_64+0x33/0x80 [ 10.153740] entry_SYSCALL_64_after_hwframe+0x44/0xae [ 10.154889] } [ 10.156023] ... key at: [<ffffffffae84ef38>] napi_hash_lock+0x18/0x40 [ 10.157191] ... acquired at: [ 10.158318] _raw_spin_lock+0x2c/0x40 [ 10.159418] netif_napi_add+0x14b/0x270 [ 10.160500] _iwl_pcie_rx_init+0x1f4/0x710 [iwlwifi] [ 10.161609] iwl_pcie_rx_init+0x1b/0x3b0 [iwlwifi] [ 10.162704] iwl_trans_pcie_start_fw+0x2ac/0x6a0 [iwlwifi] [ 10.163813] iwl_mvm_load_ucode_wait_alive+0x116/0x460 [iwlmvm] [ 10.164955] iwl_run_init_mvm_ucode+0xa4/0x3a0 [iwlmvm] [ 10.166079] iwl_op_mode_mvm_start+0x9ed/0xbf0 [iwlmvm] [ 10.167206] _iwl_op_mode_start.isra.4+0x42/0x80 [iwlwifi] [ 10.168326] iwl_opmode_register+0x71/0xe0 [iwlwifi] [ 10.169451] iwl_mvm_init+0x34/0x1000 [iwlmvm] [ 10.170585] do_one_initcall+0x5b/0x300 [ 10.171699] do_init_module+0x5b/0x21c [ 10.172818] load_module+0x1dae/0x22c0 [ 10.173933] __do_sys_finit_module+0xad/0x110 [ 10.175048] do_syscall_64+0x33/0x80 [ 10.176164] entry_SYSCALL_64_after_hwframe+0x44/0xae [ 10.178421] -> (&rxq->lock){+.-.}-{2:2} { [ 10.179564] HARDIRQ-ON-W at: [ 10.180709] lock_acquire+0x277/0x3d0 [ 10.181874] _raw_spin_lock_bh+0x34/0x40 [ 10.183049] _iwl_pcie_rx_init+0x130/0x710 [iwlwifi] [ 10.184229] iwl_pcie_rx_init+0x1b/0x3b0 [iwlwifi] [ 10.185397] iwl_trans_pcie_start_fw+0x2ac/0x6a0 [iwlwifi] [ 10.186572] iwl_mvm_load_ucode_wait_alive+0x116/0x460 [iwlmvm] [ 10.187759] iwl_run_init_mvm_ucode+0xa4/0x3a0 [iwlmvm] [ 10.188950] iwl_op_mode_mvm_start+0x9ed/0xbf0 [iwlmvm] [ 10.190132] _iwl_op_mode_start.isra.4+0x42/0x80 [iwlwifi] [ 10.191312] iwl_opmode_register+0x71/0xe0 [iwlwifi] [ 10.192497] iwl_mvm_init+0x34/0x1000 [iwlmvm] [ 10.193685] do_one_initcall+0x5b/0x300 [ 10.194868] do_init_module+0x5b/0x21c [ 10.196048] load_module+0x1dae/0x22c0 [ 10.197225] __do_sys_finit_module+0xad/0x110 [ 10.198409] do_syscall_64+0x33/0x80 [ 10.199601] entry_SYSCALL_64_after_hwframe+0x44/0xae [ 10.200801] IN-SOFTIRQ-W at: [ 10.202000] lock_acquire+0x277/0x3d0 [ 10.203202] _raw_spin_lock+0x2c/0x40 [ 10.204412] iwl_pcie_rx_handle+0x7f/0x960 [iwlwifi] [ 10.205653] iwl_pcie_napi_poll+0x2c/0x100 [iwlwifi] [ 10.206879] __napi_poll+0x2d/0x220 [ 10.208075] net_rx_action+0x106/0x280 [ 10.209276] __do_softirq+0xd4/0x461 [ 10.210486] do_softirq+0x63/0xa0 [ 10.211677] __local_bh_enable_ip+0xd1/0xf0 [ 10.212879] iwl_pcie_irq_handler+0x67d/0xb30 [iwlwifi] [ 10.214092] irq_thread_fn+0x21/0x60 [ 10.215298] irq_thread+0x107/0x1a0 [ 10.216491] kthread+0x13e/0x160 [ 10.217670] ret_from_fork+0x22/0x30 [ 10.218855] INITIAL USE at: [ 10.220030] lock_acquire+0x277/0x3d0 [ 10.221239] _raw_spin_lock_bh+0x34/0x40 [ 10.222438] _iwl_pcie_rx_init+0x130/0x710 [iwlwifi] [ 10.223667] iwl_pcie_rx_init+0x1b/0x3b0 [iwlwifi] [ 10.224894] iwl_trans_pcie_start_fw+0x2ac/0x6a0 [iwlwifi] [ 10.226167] iwl_mvm_load_ucode_wait_alive+0x116/0x460 [iwlmvm] [ 10.227448] iwl_run_init_mvm_ucode+0xa4/0x3a0 [iwlmvm] [ 10.228718] iwl_op_mode_mvm_start+0x9ed/0xbf0 [iwlmvm] [ 10.229977] _iwl_op_mode_start.isra.4+0x42/0x80 [iwlwifi] [ 10.231236] iwl_opmode_register+0x71/0xe0 [iwlwifi] [ 10.232492] iwl_mvm_init+0x34/0x1000 [iwlmvm] [ 10.233758] do_one_initcall+0x5b/0x300 [ 10.235002] do_init_module+0x5b/0x21c [ 10.236224] load_module+0x1dae/0x22c0 [ 10.237454] __do_sys_finit_module+0xad/0x110 [ 10.238679] do_syscall_64+0x33/0x80 [ 10.239906] entry_SYSCALL_64_after_hwframe+0x44/0xae [ 10.241137] } [ 10.242327] ... key at: [<ffffffffc111f0c0>] __key.69825+0x0/0xfffffffffffe2f40 [iwlwifi] [ 10.243587] ... acquired at: [ 10.244814] __lock_acquire+0x27d/0x1a50 [ 10.246055] lock_acquire+0x277/0x3d0 [ 10.247296] _raw_spin_lock+0x2c/0x40 [ 10.248518] iwl_pcie_rx_handle+0x7f/0x960 [iwlwifi] [ 10.249789] iwl_pcie_napi_poll+0x2c/0x100 [iwlwifi] [ 10.251028] __napi_poll+0x2d/0x220 [ 10.252254] net_rx_action+0x106/0x280 [ 10.253484] __do_softirq+0xd4/0x461 [ 10.254731] do_softirq+0x63/0xa0 [ 10.255946] __local_bh_enable_ip+0xd1/0xf0 [ 10.257157] iwl_pcie_irq_handler+0x67d/0xb30 [iwlwifi] [ 10.258382] irq_thread_fn+0x21/0x60 [ 10.259608] irq_thread+0x107/0x1a0 [ 10.260819] kthread+0x13e/0x160 [ 10.262022] ret_from_fork+0x22/0x30 [ 10.264433] stack backtrace: [ 10.266755] CPU: 2 PID: 565 Comm: irq/136-iwlwifi Not tainted 5.12.0-rc1-00002-gbada49429032 #5 [ 10.267964] Hardware name: LENOVO 20K5S22R00/20K5S22R00, BIOS R0IET38W (1.16 ) 05/31/2017 [ 10.269175] Call Trace: [ 10.270376] <IRQ> [ 10.271566] dump_stack+0x7f/0xa1 [ 10.272750] mark_lock.part.49+0x8cb/0xc70 [ 10.273950] ? __lock_acquire+0x2e1/0x1a50 [ 10.275164] ? __lock_acquire+0x27d/0x1a50 [ 10.276372] __lock_acquire+0x27d/0x1a50 [ 10.277562] ? __lock_acquire+0x2e1/0x1a50 [ 10.278754] ? __lock_acquire+0x2e1/0x1a50 [ 10.279937] lock_acquire+0x277/0x3d0 [ 10.281110] ? iwl_pcie_rx_handle+0x7f/0x960 [iwlwifi] [ 10.282317] ? lock_is_held_type+0xb4/0x120 [ 10.283483] _raw_spin_lock+0x2c/0x40 [ 10.284637] ? iwl_pcie_rx_handle+0x7f/0x960 [iwlwifi] [ 10.285813] iwl_pcie_rx_handle+0x7f/0x960 [iwlwifi] [ 10.286985] ? __lock_acquire+0x2e1/0x1a50 [ 10.288148] ? try_to_wake_up+0x41c/0x600 [ 10.289323] ? print_dl_rq+0xcc/0x160 [ 10.290490] iwl_pcie_napi_poll+0x2c/0x100 [iwlwifi] [ 10.291684] __napi_poll+0x2d/0x220 [ 10.292862] net_rx_action+0x106/0x280 [ 10.294034] __do_softirq+0xd4/0x461 [ 10.295185] do_softirq+0x63/0xa0 [ 10.296349] </IRQ> [ 10.297514] __local_bh_enable_ip+0xd1/0xf0 [ 10.298683] ? iwl_pcie_irq_handler+0x66c/0xb30 [iwlwifi] [ 10.299882] iwl_pcie_irq_handler+0x67d/0xb30 [iwlwifi] [ 10.301076] ? irq_forced_thread_fn+0x80/0x80 [ 10.302265] irq_thread_fn+0x21/0x60 [ 10.303434] ? irq_thread+0xa1/0x1a0 [ 10.304596] irq_thread+0x107/0x1a0 [ 10.305766] ? wake_threads_waitq+0x30/0x30 [ 10.306922] ? irq_thread_dtor+0xa0/0xa0 [ 10.308078] kthread+0x13e/0x160 [ 10.309246] ? kthread_create_worker_on_cpu+0x70/0x70 [ 10.310410] ret_from_fork+0x22/0x30