Re: Kernel GPF on RAID6 replace
From: Thiago Ramon <hidden>
Date: 2021-06-18 19:50:44
Setup is a bit unusual, BTRFS is running over BCACHE, whole
unpartitioned disks, here's the raw data, removed the unrelated disks:
lsblk -o NAME,FSTYPE,SIZE,FSUSE%,MOUNTPOINT,UUID,MIN-IO,SCHED,DISC-GRAN,MODEL:
sda bcache 7.3T
bc7c0748-4cb4-43cc-ad5d-d736e5e0db82 4096 mq-deadline 0B
ST8000DM004-2CX1
└─bcache1 btrfs 7.3T
224936e2-df99-4952-952e-9fc8cf87fc34 512 512B
sdb bcache 7.3T
0a642b3a-78c8-435f-89bd-4fce86c270a7 4096 mq-deadline 0B
ST8000DM004-2CX1
└─bcache8 btrfs 7.3T
224936e2-df99-4952-952e-9fc8cf87fc34 512 512B
sdc bcache 7.3T
5f2a88df-1465-464a-96f7-53432f5d4acc 4096 mq-deadline 0B
ST8000DM004-2CX1
└─bcache9 btrfs 7.3T
224936e2-df99-4952-952e-9fc8cf87fc34 512 512B
sdd bcache 7.3T
f48df2c8-b665-485b-ae82-ac0cb256c1de 4096 mq-deadline 0B
ST8000DM004-2CX1
└─bcache11 btrfs 7.3T 66% /data
224936e2-df99-4952-952e-9fc8cf87fc34 512 512B
sdg bcache 7.3T
ba8101e5-6d9e-44a1-8339-3bbbfa9db030 4096 mq-deadline 0B
ST8000DM004-2CX1
└─bcache6 btrfs 7.3T
224936e2-df99-4952-952e-9fc8cf87fc34 512 512B
sdh bcache 7.3T
301fb860-2eb3-488c-94b7-7013f2a9ad0c 4096 mq-deadline 0B
ST8000DM004-2CX1
└─bcache3 btrfs 7.3T
224936e2-df99-4952-952e-9fc8cf87fc34 512 512B
sdi bcache 7.3T
7ba65110-6a17-4c67-be63-060d2f721a87 4096 mq-deadline 0B
ST8000DM004-2CX1
└─bcache5 btrfs 7.3T
224936e2-df99-4952-952e-9fc8cf87fc34 512 512B
sdj bcache 7.3T
36d72ae0-b03d-458c-b75d-c62804ddb240 4096 mq-deadline 0B
ST8000DM004-2CX1
└─bcache4 btrfs 7.3T
224936e2-df99-4952-952e-9fc8cf87fc34 512 512B
sdk bcache 7.3T
2b494639-5863-4bb9-b462-686f62e75091 4096 mq-deadline 0B
ST8000DM004-2CX1
└─bcache2 btrfs 7.3T
224936e2-df99-4952-952e-9fc8cf87fc34 512 512B
sdl bcache 7.3T
03b62951-3532-40f5-9a67-1ef287caa890 4096 mq-deadline 0B
ST8000DM004-2CX1
└─bcache0 btrfs 7.3T
224936e2-df99-4952-952e-9fc8cf87fc34 512 512B
sdm bcache 7.3T
8b978d80-c6c3-426b-a260-86221212610c 4096 mq-deadline 0B
ST8000DM004-2CX1
└─bcache7 btrfs 7.3T
224936e2-df99-4952-952e-9fc8cf87fc34 512 512B
sdn bcache 7.3T
5e36e48a-c550-402e-b249-11fae77cc740 4096 mq-deadline 0B
ST8000DM004-2CX1
└─bcache10 btrfs 7.3T
224936e2-df99-4952-952e-9fc8cf87fc34 512 512B
sdp bcache 223.6G
517b4831-9c41-4c86-8c8f-f5e5a2d74d27 512 mq-deadline 512B
KINGSTON_SA400S3
├─bcache0 btrfs 7.3T
224936e2-df99-4952-952e-9fc8cf87fc34 512 512B
├─bcache1 btrfs 7.3T
224936e2-df99-4952-952e-9fc8cf87fc34 512 512B
├─bcache2 btrfs 7.3T
224936e2-df99-4952-952e-9fc8cf87fc34 512 512B
├─bcache3 btrfs 7.3T
224936e2-df99-4952-952e-9fc8cf87fc34 512 512B
├─bcache4 btrfs 7.3T
224936e2-df99-4952-952e-9fc8cf87fc34 512 512B
├─bcache5 btrfs 7.3T
224936e2-df99-4952-952e-9fc8cf87fc34 512 512B
├─bcache6 btrfs 7.3T
224936e2-df99-4952-952e-9fc8cf87fc34 512 512B
├─bcache7 btrfs 7.3T
224936e2-df99-4952-952e-9fc8cf87fc34 512 512B
├─bcache8 btrfs 7.3T
224936e2-df99-4952-952e-9fc8cf87fc34 512 512B
├─bcache9 btrfs 7.3T
224936e2-df99-4952-952e-9fc8cf87fc34 512 512B
├─bcache10 btrfs 7.3T
224936e2-df99-4952-952e-9fc8cf87fc34 512 512B
├─bcache11 btrfs 7.3T 66% /data
224936e2-df99-4952-952e-9fc8cf87fc34 512 512B
└─bcache12 ext4 3.7T 42% /scratch
3590506e-c0f0-45c1-8fd7-3fafe3a73024 512 512B
btrfs filesystem usage -T /data
WARNING: RAID56 detected, not implemented
Overall:
Device size: 94.61TiB
Device allocated: 488.12GiB
Device unallocated: 94.13TiB
Device missing: 7.28TiB
Used: 447.63GiB
Free (estimated): 0.00B (min: 8.00EiB)
Data ratio: 0.00
Metadata ratio: 4.00
Global reserve: 512.00MiB (used: 0.00B)
Data Metadata System
Id Path RAID6 RAID1C4 RAID1C4 Unallocated
-- ------------- -------- --------- -------- -----------
0 /dev/bcache11 - - - 7.28TiB
1 /dev/bcache2 5.71TiB 41.00GiB - 1.53TiB
4 /dev/bcache8 5.71TiB 40.00GiB 32.00MiB 1.53TiB
5 /dev/bcache0 5.71TiB 41.00GiB - 1.53TiB
6 missing 5.71TiB 41.00GiB - -5.75TiB
7 /dev/bcache1 5.71TiB 40.00GiB 32.00MiB 1.53TiB
8 /dev/bcache9 5.71TiB 41.00GiB - 1.53TiB
9 /dev/bcache7 5.71TiB 41.00GiB - 1.53TiB
10 /dev/bcache10 5.71TiB 40.00GiB 32.00MiB 1.53TiB
11 /dev/bcache6 5.71TiB 41.00GiB - 1.53TiB
12 /dev/bcache3 5.71TiB 40.00GiB 32.00MiB 1.53TiB
13 /dev/bcache5 5.71TiB 41.00GiB - 1.53TiB
14 /dev/bcache4 5.71TiB 41.00GiB - 1.53TiB
-- ------------- -------- --------- -------- -----------
Total 57.11TiB 122.00GiB 32.00MiB 18.32TiB
Used 56.95TiB 111.91GiB 2.69MiB
Currently mounted degraded,readonly. Didn't explore much yet, but
overall seems "fine" (as much as fine as a RAID6 missing a disk can
be).
On Fri, Jun 18, 2021 at 4:24 PM Chris Murphy [off-list ref] wrote:It'd be useful to see the storage stack: lsblk -o NAME,FSTYPE,SIZE,FSUSE%,MOUNTPOINT,UUID,MIN-IO,SCHED,DISC-GRAN,MODEL It's fine to omit the UUID. Mainly I just want to see all the physical devices, their partitions, and how everything is being assembled to make up this Btrfs file system. Also useful info sudo btrfs filesystem usage -T /mntpoint Chris Murphy