Thread (5 messages) 5 messages, 2 authors, 2021-05-19

Re: System freeze with BTRFS corruption on 4 systems with kernel 5.12 (MANJARO)

From: Qu Wenruo <hidden>
Date: 2021-05-19 10:02:43


On 2021/5/19 下午5:17, Swâmi Petaramesh wrote:
On 5/19/21 9:25 AM, Qu Wenruo wrote:
quoted
quoted
Kernel affected : Linux 5.12.1-2-MANJARO
(Not sure if I tried Manjaro Linux 5.12.1-1-MANJARO)

Kernel not affected : All previous versions up to 5.11.18-1-MANJARO

Symptoms : Under heavy disk usage (such as performing a system backup
onto external USB HD) the machine soon completely freezes and only a
hard power cycle can get it out of it.
Any dying message?
No, just a sudden and complete system and disks freeze. Thus no
messages, nothing logged.
Have you tried something like net-console to catch something?

If it's some hang, after 120s it would have some dmesg popping out.
But in that hang case, you should still be able to do a lot of things.

If it's something like BUG_ON(), it would immediately show up.
(And if the trace is not btrfs related, I bet it's something in the dm
layer)

Without the dying message, it's really hard to further debug.

Considering you have so many devices, it should be pretty simple to
setup a device running nc to receive all the net-console output:
https://www.kernel.org/doc/html/latest/networking/netconsole.html
quoted
quoted
After reboot, systems on which BTRFS is built over bcache may show heavy
filesystem corruption.
Which kind of corruption? Just data csum mismatch?
AFAIR it was some kind of “generation mismatch”, expected something,
found another, in very large quantities.
That means flush command doesn't work as expected.

Considering there are extra layers involved, it's pretty hard to tell
which is the cause, btrfs or dm-* modules.
The machine with BTRFS RAID-1 could heal itself out of this by running a
simple btrfs scrub,
This further proves it may be lower layer doing something wrong.

As if it's btrfs itself causing the bug, the transid mismatch shouldn't
be recoverable at all.

For btrfs caused error, it would be broken COW, thus all copies should
be corrupted.

It's really a good practice to have LUKS under all your fs, but it also
introduces an extra layer of flush problems.
Did you have any raw btrfs directly over HDD/SDD experiencing such problem?

Thanks,
Qu
I gave up on the non-RAID one my previous experience
with similar errors making me think the FS was beyond repair, I
reformatted and restored from backups.
quoted
Does `btrfs check` reports other problems?
I didn't try.

Thanks for the quick help :)

ॐ
Keyboard shortcuts
hback out one level
jnext message in thread
kprevious message in thread
ldrill in
Escclose help / fold thread tree
?toggle this help