Re: System freeze with BTRFS corruption on 4 systems with kernel 5.12 (MANJARO)
From: Qu Wenruo <hidden>
Date: 2021-05-19 10:02:43
On 2021/5/19 下午5:17, Swâmi Petaramesh wrote:
On 5/19/21 9:25 AM, Qu Wenruo wrote:quoted
quoted
Kernel affected : Linux 5.12.1-2-MANJARO (Not sure if I tried Manjaro Linux 5.12.1-1-MANJARO) Kernel not affected : All previous versions up to 5.11.18-1-MANJARO Symptoms : Under heavy disk usage (such as performing a system backup onto external USB HD) the machine soon completely freezes and only a hard power cycle can get it out of it.Any dying message?No, just a sudden and complete system and disks freeze. Thus no messages, nothing logged.
Have you tried something like net-console to catch something? If it's some hang, after 120s it would have some dmesg popping out. But in that hang case, you should still be able to do a lot of things. If it's something like BUG_ON(), it would immediately show up. (And if the trace is not btrfs related, I bet it's something in the dm layer) Without the dying message, it's really hard to further debug. Considering you have so many devices, it should be pretty simple to setup a device running nc to receive all the net-console output: https://www.kernel.org/doc/html/latest/networking/netconsole.html
quoted
quoted
After reboot, systems on which BTRFS is built over bcache may show heavy filesystem corruption.Which kind of corruption? Just data csum mismatch?AFAIR it was some kind of “generation mismatch”, expected something, found another, in very large quantities.
That means flush command doesn't work as expected. Considering there are extra layers involved, it's pretty hard to tell which is the cause, btrfs or dm-* modules.
The machine with BTRFS RAID-1 could heal itself out of this by running a simple btrfs scrub,
This further proves it may be lower layer doing something wrong. As if it's btrfs itself causing the bug, the transid mismatch shouldn't be recoverable at all. For btrfs caused error, it would be broken COW, thus all copies should be corrupted. It's really a good practice to have LUKS under all your fs, but it also introduces an extra layer of flush problems. Did you have any raw btrfs directly over HDD/SDD experiencing such problem? Thanks, Qu
I gave up on the non-RAID one my previous experience with similar errors making me think the FS was beyond repair, I reformatted and restored from backups.quoted
Does `btrfs check` reports other problems?I didn't try. Thanks for the quick help :) ॐ