From: Zhilong Liu <hidden> Date: 2017-08-29 02:37:28
Hi, Neil;
Thanks for your pointing and sorry for the incorrect dmesg for last
mail.
Here update the pure steps and paste the dmesg.
ENV:
OS: 4.13-rc7 upstream
linux-apta:~/mdadm-test # df -T /mnt/
Filesystem Type 1K-blocks Used Available Use% Mounted on
/dev/sda2 ext4 44248848 24416952 18778472 57% /
Reproduce: 100%
Steps:
linux-apta:~/mdadm-test # ./mdadm -CR /dev/md0 -l1 -b /mnt/3 -n2 -x1
/dev/loop[0-2] --force
mdadm: Note: this array has metadata at the start and
may not be suitable as a boot device. If you plan to
store '/boot' on this device please ensure that
your boot-loader understands md/v1.x metadata, or use
--metadata=0.90
mdadm: Defaulting to version 1.2 metadata
mdadm: array /dev/md0 started.
linux-apta:~/mdadm-test # cat /proc/mdstat
Personalities : [raid1]
md0 : active raid1 loop2[2](S) loop1[1] loop0[0]
18944 blocks super 1.2 [2/2] [UU]
bitmap: 3/3 pages [12KB], 4KB chunk, file: /mnt/3
unused devices: <none>
linux-apta:~/mdadm-test # dmesg -c
[ 181.378209] md/raid1:md0: not clean -- starting background reconstruction
[ 181.378211] md/raid1:md0: active with 2 out of 2 mirrors
[ 181.379354] md0: detected capacity change from 0 to 19398656
[ 181.379773] md: resync of RAID array md0
[ 190.396162] md: md0: resync done.
linux-apta:~/mdadm-test # ./mdadm --grow /dev/md0 --size 128
Segmentation fault
linux-apta:~/mdadm-test # cat /sys/block/md0/md/component_size
18944 "here is incorrect also."
linux-apta:~/mdadm-test # dmesg -c
[ 208.027505] ------------[ cut here ]------------
[ 208.027508] kernel BUG at drivers/md/bitmap.c:298!
[ 208.027511] invalid opcode: 0000 [#1] SMP
[ 208.027516] Modules linked in: raid1(E) md_mod(E) loop(E) uinput(E)
af_packet(E) iscsi_ibft(E) iscsi_boot_sysfs(E) snd_hda_codec_generic(E)
snd_hda_intel(E) snd_hda_codec(E) snd_hda_core(E) snd_hwdep(E)
snd_pcm(E) snd_timer(E) snd(E) virtio_balloon(E) virtio_net(E)
soundcore(E) i2c_piix4(E) crct10dif_pclmul(E) crc32_pclmul(E)
crc32c_intel(E) ghash_clmulni_intel(E) pcbc(E) ppdev(E) parport_pc(E)
parport(E) joydev(E) pvpanic(E) aesni_intel(E) aes_x86_64(E)
crypto_simd(E) glue_helper(E) cryptd(E) button(E) pcspkr(E) ext4(E)
crc16(E) mbcache(E) jbd2(E) hid_generic(E) usbhid(E) ata_generic(E)
sd_mod(E) virtio_scsi(E) virtio_console(E) floppy(E) ata_piix(E) qxl(E)
drm_kms_helper(E) syscopyarea(E) sysfillrect(E) sysimgblt(E)
fb_sys_fops(E) ttm(E) serio_raw(E) ahci(E) libahci(E) drm(E) ehci_pci(E)
libata(E)
[ 208.027565] uhci_hcd(E) ehci_hcd(E) usbcore(E) virtio_pci(E)
virtio_ring(E) virtio(E) sg(E) dm_multipath(E) dm_mod(E) scsi_dh_rdac(E)
scsi_dh_emc(E) scsi_dh_alua(E) scsi_mod(E) autofs4(E)
[ 208.027579] CPU: 2 PID: 9592 Comm: mdadm Tainted: G E
4.13.0-rc7-up-latest #1
[ 208.027581] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996),
BIOS Bochs 01/01/2011
[ 208.027584] task: ffff8800714286c0 task.stack: ffffc90000a68000
[ 208.027595] RIP: 0010:write_page+0x304/0x310 [md_mod]
[ 208.027597] RSP: 0018:ffffc90000a6bbd8 EFLAGS: 00010246
[ 208.027600] RAX: 000fffffc0000000 RBX: ffff88006c65a300 RCX:
0000000000000001
[ 208.027602] RDX: 0000000000000000 RSI: ffffea0001cbe100 RDI:
ffff88006c65a300
[ 208.027603] RBP: ffffc90000a6bc30 R08: ffff880072f84000 R09:
ffff88006b8033a0
[ 208.027605] R10: 0000000000000003 R11: 0000000000000020 R12:
ffff88006c65a300
[ 208.027607] R13: 0000000000000000 R14: ffffea0001cbe100 R15:
0000000000000001
[ 208.027610] FS: 00007fe15ce1a700(0000) GS:ffff88007fd00000(0000)
knlGS:0000000000000000
[ 208.027612] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 208.027614] CR2: 00000000018fc888 CR3: 0000000072fd4000 CR4:
00000000000406e0
[ 208.027625] Call Trace:
[ 208.027634] ? __put_page+0x46/0x80
[ 208.027642] bitmap_unplug+0xac/0x100 [md_mod]
[ 208.027648] bitmap_resize+0x6ca/0x7f0 [md_mod]
[ 208.027653] raid1_resize+0x4e/0xb0 [raid1]
[ 208.027659] update_size+0x9e/0x120 [md_mod]
[ 208.027665] md_ioctl+0xdcc/0x1830 [md_mod]
[ 208.027671] ? layout_show+0x40/0x60 [md_mod]
[ 208.027677] blkdev_ioctl+0x475/0x8b0
[ 208.027681] ? mntput+0x24/0x40
[ 208.027685] block_ioctl+0x41/0x50
[ 208.027689] do_vfs_ioctl+0x96/0x5b0
[ 208.027693] ? ____fput+0xe/0x10
[ 208.027697] ? task_work_run+0x88/0xb0
[ 208.027700] SyS_ioctl+0x79/0x90
[ 208.027704] entry_SYSCALL_64_fastpath+0x1a/0xa5
[ 208.027706] RIP: 0033:0x7fe15c7534b7
[ 208.027708] RSP: 002b:00007ffe88a37788 EFLAGS: 00000246 ORIG_RAX:
0000000000000010
[ 208.027711] RAX: ffffffffffffffda RBX: 0000000000000004 RCX:
00007fe15c7534b7
[ 208.027713] RDX: 00007ffe88a37850 RSI: 0000000040480923 RDI:
0000000000000003
[ 208.027715] RBP: 00007ffe88a37730 R08: 00000000004711e0 R09:
0000000000000004
[ 208.027717] R10: 00000000004631fe R11: 0000000000000246 R12:
00000000ffffffff
[ 208.027718] R13: 0000000000000000 R14: 00000000018f45b0 R15:
0000000000000001
[ 208.027721] Code: 85 c0 0f 8f 3a ff ff ff 48 8b 53 68 49 8b 85 f0 00
00 00 48 03 43 58 48 01 d6 48 39 f0 0f 87 1f ff ff ff 48 01 fa e9 8c fe
ff ff <0f> 0b 66 2e 0f 1f 84 00 00 00 00 00 66 66 66 66 90 48 8b 4f 20
[ 208.027762] RIP: write_page+0x304/0x310 [md_mod] RSP: ffffc90000a6bbd8
[ 208.027765] ---[ end trace dfbe72f2783c5398 ]---
Thanks for your confirmation;
-Zhilong
On 08/29/2017 08:41 AM, NeilBrown wrote:
On Mon, Aug 28 2017, Zhilong Liu wrote:
quoted
Hi, Neil;
Here report a new bug with latest 4.13-rc7 and latest mdadm when I'm
testing for grow size; please refer to the following detail call-trace
stack.
I found that there is no testing covered "--grow --size", thus I
simulate the testing steps as:
Thanks for the report - but something doesn't make sense here.
This presumably marks when the array was created and finished resync.
quoted
[ 3220.651913] md0: detected capacity change from 19398656 to 65536
5 seconds later this shows the array being resized to 64K. Did you do
that and not tell me?
quoted
[ 3236.930481] md0: bitmap file is out of date (0 < 20) -- forcing full
recovery
[ 3236.930488] md0: bitmap file is out of date, doing full recovery
16 seconds later something happens to the bitmap. It looks like the
bitmap was removed and re-added. Is that possible?
quoted
[ 3241.272597] ------------[ cut here ]------------
Another 5 seconds later it crashes.
I think you must have performed some other steps that you didn't
mention.
Please give a complete description of the steps you used which produced
the problem.
Thanks,
NeilBrown
Hi, Neil;
Thanks for your pointing and sorry for the incorrect dmesg for last
mail.
Here update the pure steps and paste the dmesg.
ENV:
OS: 4.13-rc7 upstream
linux-apta:~/mdadm-test # df -T /mnt/
Filesystem Type 1K-blocks Used Available Use% Mounted on
/dev/sda2 ext4 44248848 24416952 18778472 57% /
Reproduce: 100%
Steps:
linux-apta:~/mdadm-test # ./mdadm -CR /dev/md0 -l1 -b /mnt/3 -n2 -x1
/dev/loop[0-2] --force
mdadm: Note: this array has metadata at the start and
may not be suitable as a boot device. If you plan to
store '/boot' on this device please ensure that
your boot-loader understands md/v1.x metadata, or use
--metadata=0.90
mdadm: Defaulting to version 1.2 metadata
mdadm: array /dev/md0 started.
linux-apta:~/mdadm-test # cat /proc/mdstat
Personalities : [raid1]
md0 : active raid1 loop2[2](S) loop1[1] loop0[0]
18944 blocks super 1.2 [2/2] [UU]
bitmap: 3/3 pages [12KB], 4KB chunk, file: /mnt/3
unused devices: <none>
linux-apta:~/mdadm-test # dmesg -c
[ 181.378209] md/raid1:md0: not clean -- starting background reconstruction
[ 181.378211] md/raid1:md0: active with 2 out of 2 mirrors
[ 181.379354] md0: detected capacity change from 0 to 19398656
[ 181.379773] md: resync of RAID array md0
[ 190.396162] md: md0: resync done.
linux-apta:~/mdadm-test # ./mdadm --grow /dev/md0 --size 128
Segmentation fault
linux-apta:~/mdadm-test # cat /sys/block/md0/md/component_size
18944 "here is incorrect also."
linux-apta:~/mdadm-test # dmesg -c
[ 208.027505] ------------[ cut here ]------------
[ 208.027508] kernel BUG at drivers/md/bitmap.c:298!
Thanks. Less confusing now.
The problem is that when the bitmap is resized, new pages are allocated
to store the on-disk copy, but these are not read from the file, the
contents are set from the in-memory bitmap.
So read_page() isn't called and particularly
bh = alloc_page_buffers(page, 1<<inode->i_blkbits, 0);
...
attach_page_buffers(page, bh);
doesn't happen.
Maybe something like this will work.
Can you test it?
Thanks,
NeilBrown
From: Zhilong Liu <hidden> Date: 2017-08-29 04:39:35
On 08/29/2017 11:12 AM, NeilBrown wrote:
On Tue, Aug 29 2017, Zhilong Liu wrote:
quoted
Hi, Neil;
Thanks for your pointing and sorry for the incorrect dmesg for last
mail.
Here update the pure steps and paste the dmesg.
ENV:
OS: 4.13-rc7 upstream
linux-apta:~/mdadm-test # df -T /mnt/
Filesystem Type 1K-blocks Used Available Use% Mounted on
/dev/sda2 ext4 44248848 24416952 18778472 57% /
Reproduce: 100%
Steps:
linux-apta:~/mdadm-test # ./mdadm -CR /dev/md0 -l1 -b /mnt/3 -n2 -x1
/dev/loop[0-2] --force
mdadm: Note: this array has metadata at the start and
may not be suitable as a boot device. If you plan to
store '/boot' on this device please ensure that
your boot-loader understands md/v1.x metadata, or use
--metadata=0.90
mdadm: Defaulting to version 1.2 metadata
mdadm: array /dev/md0 started.
linux-apta:~/mdadm-test # cat /proc/mdstat
Personalities : [raid1]
md0 : active raid1 loop2[2](S) loop1[1] loop0[0]
18944 blocks super 1.2 [2/2] [UU]
bitmap: 3/3 pages [12KB], 4KB chunk, file: /mnt/3
unused devices: <none>
linux-apta:~/mdadm-test # dmesg -c
[ 181.378209] md/raid1:md0: not clean -- starting background reconstruction
[ 181.378211] md/raid1:md0: active with 2 out of 2 mirrors
[ 181.379354] md0: detected capacity change from 0 to 19398656
[ 181.379773] md: resync of RAID array md0
[ 190.396162] md: md0: resync done.
linux-apta:~/mdadm-test # ./mdadm --grow /dev/md0 --size 128
Segmentation fault
linux-apta:~/mdadm-test # cat /sys/block/md0/md/component_size
18944 "here is incorrect also."
linux-apta:~/mdadm-test # dmesg -c
[ 208.027505] ------------[ cut here ]------------
[ 208.027508] kernel BUG at drivers/md/bitmap.c:298!
Thanks. Less confusing now.
The problem is that when the bitmap is resized, new pages are allocated
to store the on-disk copy, but these are not read from the file, the
contents are set from the in-memory bitmap.
So read_page() isn't called and particularly
bh = alloc_page_buffers(page, 1<<inode->i_blkbits, 0);
...
attach_page_buffers(page, bh);
doesn't happen.
Maybe something like this will work.
Can you test it?
Hi, Neil;
Thanks for your pointing and sorry for the incorrect dmesg for last
mail.
Here update the pure steps and paste the dmesg.
ENV:
OS: 4.13-rc7 upstream
linux-apta:~/mdadm-test # df -T /mnt/
Filesystem Type 1K-blocks Used Available Use% Mounted on
/dev/sda2 ext4 44248848 24416952 18778472 57% /
Reproduce: 100%
Steps:
linux-apta:~/mdadm-test # ./mdadm -CR /dev/md0 -l1 -b /mnt/3 -n2 -x1
/dev/loop[0-2] --force
mdadm: Note: this array has metadata at the start and
may not be suitable as a boot device. If you plan to
store '/boot' on this device please ensure that
your boot-loader understands md/v1.x metadata, or use
--metadata=0.90
mdadm: Defaulting to version 1.2 metadata
mdadm: array /dev/md0 started.
linux-apta:~/mdadm-test # cat /proc/mdstat
Personalities : [raid1]
md0 : active raid1 loop2[2](S) loop1[1] loop0[0]
18944 blocks super 1.2 [2/2] [UU]
bitmap: 3/3 pages [12KB], 4KB chunk, file: /mnt/3
unused devices: <none>
linux-apta:~/mdadm-test # dmesg -c
[ 181.378209] md/raid1:md0: not clean -- starting background reconstruction
[ 181.378211] md/raid1:md0: active with 2 out of 2 mirrors
[ 181.379354] md0: detected capacity change from 0 to 19398656
[ 181.379773] md: resync of RAID array md0
[ 190.396162] md: md0: resync done.
linux-apta:~/mdadm-test # ./mdadm --grow /dev/md0 --size 128
Segmentation fault
linux-apta:~/mdadm-test # cat /sys/block/md0/md/component_size
18944 "here is incorrect also."
linux-apta:~/mdadm-test # dmesg -c
[ 208.027505] ------------[ cut here ]------------
[ 208.027508] kernel BUG at drivers/md/bitmap.c:298!
Thanks. Less confusing now.
The problem is that when the bitmap is resized, new pages are allocated
to store the on-disk copy, but these are not read from the file, the
contents are set from the in-memory bitmap.
So read_page() isn't called and particularly
bh = alloc_page_buffers(page, 1<<inode->i_blkbits, 0);
...
attach_page_buffers(page, bh);
doesn't happen.
Maybe something like this will work.
Can you test it?
Another panic happens when I built with the below patch.
Steps:
1. patching the codes to bitmap.c
2. rebuilt the kernel source code.
3. reboot and test.
linux-apta:~/mdadm-test # ./mdadm -CR /dev/md0 -l1 -b /mnt/3 -n2 -x1
/dev/loop[0-2] --force
mdadm: Note: this array has metadata at the start and
may not be suitable as a boot device. If you plan to
store '/boot' on this device please ensure that
your boot-loader understands md/v1.x metadata, or use
--metadata=0.90
mdadm: Defaulting to version 1.2 metadata
Segmentation fault
linux-apta:~/mdadm-test # dmesg -c
[ 46.416567] md/raid1:md0: not clean -- starting background reconstruction
[ 46.416570] md/raid1:md0: active with 2 out of 2 mirrors
[ 46.417003] ------------[ cut here ]------------
[ 46.417004] kernel BUG at drivers/md/bitmap.c:371!
Thanks. I see what I missed. Please try this patch instead.
NeilBrown
From: Zhilong Liu <hidden> Date: 2017-08-30 02:52:45
On 08/29/2017 06:47 PM, NeilBrown wrote:
On Tue, Aug 29 2017, Zhilong Liu wrote:
quoted
On 08/29/2017 11:12 AM, NeilBrown wrote:
quoted
On Tue, Aug 29 2017, Zhilong Liu wrote:
quoted
Hi, Neil;
Thanks for your pointing and sorry for the incorrect dmesg for last
mail.
Here update the pure steps and paste the dmesg.
ENV:
OS: 4.13-rc7 upstream
linux-apta:~/mdadm-test # df -T /mnt/
Filesystem Type 1K-blocks Used Available Use% Mounted on
/dev/sda2 ext4 44248848 24416952 18778472 57% /
Reproduce: 100%
Steps:
linux-apta:~/mdadm-test # ./mdadm -CR /dev/md0 -l1 -b /mnt/3 -n2 -x1
/dev/loop[0-2] --force
mdadm: Note: this array has metadata at the start and
may not be suitable as a boot device. If you plan to
store '/boot' on this device please ensure that
your boot-loader understands md/v1.x metadata, or use
--metadata=0.90
mdadm: Defaulting to version 1.2 metadata
mdadm: array /dev/md0 started.
linux-apta:~/mdadm-test # cat /proc/mdstat
Personalities : [raid1]
md0 : active raid1 loop2[2](S) loop1[1] loop0[0]
18944 blocks super 1.2 [2/2] [UU]
bitmap: 3/3 pages [12KB], 4KB chunk, file: /mnt/3
unused devices: <none>
linux-apta:~/mdadm-test # dmesg -c
[ 181.378209] md/raid1:md0: not clean -- starting background reconstruction
[ 181.378211] md/raid1:md0: active with 2 out of 2 mirrors
[ 181.379354] md0: detected capacity change from 0 to 19398656
[ 181.379773] md: resync of RAID array md0
[ 190.396162] md: md0: resync done.
linux-apta:~/mdadm-test # ./mdadm --grow /dev/md0 --size 128
Segmentation fault
linux-apta:~/mdadm-test # cat /sys/block/md0/md/component_size
18944 "here is incorrect also."
linux-apta:~/mdadm-test # dmesg -c
[ 208.027505] ------------[ cut here ]------------
[ 208.027508] kernel BUG at drivers/md/bitmap.c:298!
Thanks. Less confusing now.
The problem is that when the bitmap is resized, new pages are allocated
to store the on-disk copy, but these are not read from the file, the
contents are set from the in-memory bitmap.
So read_page() isn't called and particularly
bh = alloc_page_buffers(page, 1<<inode->i_blkbits, 0);
...
attach_page_buffers(page, bh);
doesn't happen.
Maybe something like this will work.
Can you test it?
Another panic happens when I built with the below patch.
Steps:
1. patching the codes to bitmap.c
2. rebuilt the kernel source code.
3. reboot and test.
linux-apta:~/mdadm-test # ./mdadm -CR /dev/md0 -l1 -b /mnt/3 -n2 -x1
/dev/loop[0-2] --force
mdadm: Note: this array has metadata at the start and
may not be suitable as a boot device. If you plan to
store '/boot' on this device please ensure that
your boot-loader understands md/v1.x metadata, or use
--metadata=0.90
mdadm: Defaulting to version 1.2 metadata
Segmentation fault
linux-apta:~/mdadm-test # dmesg -c
[ 46.416567] md/raid1:md0: not clean -- starting background reconstruction
[ 46.416570] md/raid1:md0: active with 2 out of 2 mirrors
[ 46.417003] ------------[ cut here ]------------
[ 46.417004] kernel BUG at drivers/md/bitmap.c:371!
Thanks. I see what I missed. Please try this patch instead.
Hi, Neil;
I have tested the following patch, I still got the call-trace after
I built with it.
If you need other infos, I would append.
linux-apta:~/mdadm-test # ./mdadm -CR /dev/md0 -l1 -b /mnt/3 -n2 -x1
/dev/loop[0-2] --force
mdadm: Note: this array has metadata at the start and
may not be suitable as a boot device. If you plan to
store '/boot' on this device please ensure that
your boot-loader understands md/v1.x metadata, or use
--metadata=0.90
mdadm: Defaulting to version 1.2 metadata
Segmentation fault
linux-apta:~/mdadm-test # dmesg -c
[ 88.787135] md/raid1:md0: not clean -- starting background reconstruction
[ 88.787137] md/raid1:md0: active with 2 out of 2 mirrors
[ 88.787590] ------------[ cut here ]------------
[ 88.787592] kernel BUG at drivers/md/bitmap.c:371!
[ 88.787594] invalid opcode: 0000 [#1] SMP
[ 88.787597] Modules linked in: raid1(E) md_mod(E) loop(E) uinput(E)
af_packet(E) iscsi_ibft(E) iscsi_boot_sysfs(E) snd_hda_codec_generic(E)
snd_hda_intel(E) snd_hda_codec(E) snd_hda_core(E) snd_hwdep(E)
crct10dif_pclmul(E) crc32_pclmul(E) crc32c_intel(E)
ghash_clmulni_intel(E) pcbc(E) snd_pcm(E) snd_timer(E) aesni_intel(E)
aes_x86_64(E) ppdev(E) parport_pc(E) snd(E) crypto_simd(E) joydev(E)
glue_helper(E) parport(E) soundcore(E) virtio_net(E) pvpanic(E)
cryptd(E) button(E) i2c_piix4(E) pcspkr(E) virtio_balloon(E) ext4(E)
crc16(E) mbcache(E) jbd2(E) hid_generic(E) usbhid(E) ata_generic(E)
sd_mod(E) ata_piix(E) virtio_console(E) virtio_scsi(E) ahci(E)
libahci(E) serio_raw(E) ehci_pci(E) libata(E) qxl(E) drm_kms_helper(E)
syscopyarea(E) uhci_hcd(E) ehci_hcd(E) sysfillrect(E) sysimgblt(E)
fb_sys_fops(E)
[ 88.787627] usbcore(E) floppy(E) ttm(E) virtio_pci(E) drm(E)
virtio_ring(E) virtio(E) sg(E) dm_multipath(E) dm_mod(E) scsi_dh_rdac(E)
scsi_dh_emc(E) scsi_dh_alua(E) scsi_mod(E) autofs4(E)
[ 88.787637] CPU: 2 PID: 9435 Comm: mdadm Tainted: G E
4.13.0-rc7-up-latest #1
[ 88.787639] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996),
BIOS Bochs 01/01/2011
[ 88.787640] task: ffff880070ad1440 task.stack: ffffc90000c7c000
[ 88.787648] RIP: 0010:read_page+0x1dd/0x1e0 [md_mod]
[ 88.787650] RSP: 0018:ffffc90000c7fbc0 EFLAGS: 00010246
[ 88.787652] RAX: 000fffffc0000000 RBX: 0000000000000000 RCX:
0000000000000350
[ 88.787653] RDX: ffff88006c4d1900 RSI: 0000000000000000 RDI:
ffff88006bc95000
[ 88.787654] RBP: ffffc90000c7fc20 R08: ffffea0001bef140 R09:
000000000000563a
[ 88.787656] R10: 0000000000000010 R11: 000000006fbc5000 R12:
ffff88006c4d1900
[ 88.787657] R13: ffff88006c4d1900 R14: 0000000000000350 R15:
ffff88006a920790
[ 88.787659] FS: 00007f267dce4700(0000) GS:ffff88007fd00000(0000)
knlGS:0000000000000000
[ 88.787660] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 88.787661] CR2: 00000000006a6708 CR3: 000000006fb3f000 CR4:
00000000000406e0
[ 88.787669] Call Trace:
[ 88.787676] bitmap_create+0x264/0x990 [md_mod]
[ 88.787680] ? raid1_run+0x1d0/0x2e0 [raid1]
[ 88.787684] md_run+0x5d2/0xb00 [md_mod]
[ 88.787688] ? locks_dispose_list+0x36/0x50
[ 88.787690] ? flock_lock_inode+0x124/0x280
[ 88.787695] do_md_run+0x14/0xb0 [md_mod]
[ 88.787699] md_ioctl+0x13ed/0x1830 [md_mod]
[ 88.787703] ? kzfree+0x2d/0x30
[ 88.787707] blkdev_ioctl+0x475/0x8b0
[ 88.787710] ? mntput+0x24/0x40
[ 88.787713] block_ioctl+0x41/0x50
[ 88.787715] do_vfs_ioctl+0x96/0x5b0
[ 88.787718] ? ____fput+0xe/0x10
[ 88.787721] ? task_work_run+0x88/0xb0
[ 88.787723] SyS_ioctl+0x79/0x90
[ 88.787726] entry_SYSCALL_64_fastpath+0x1a/0xa5
[ 88.787728] RIP: 0033:0x7f267d61d4b7
[ 88.787729] RSP: 002b:00007ffdbe091138 EFLAGS: 00000246 ORIG_RAX:
0000000000000010
[ 88.787731] RAX: ffffffffffffffda RBX: 00000000009d8700 RCX:
00007f267d61d4b7
[ 88.787732] RDX: 00007ffdbe091450 RSI: 00000000400c0930 RDI:
0000000000000004
[ 88.787734] RBP: 0000000000000000 R08: 0000000000001000 R09:
00007f267d8d8678
[ 88.787735] R10: 0000000000000000 R11: 0000000000000246 R12:
0000000000000000
[ 88.787736] R13: 0000000000000004 R14: 00007ffdbe091630 R15:
00000000009d8640
[ 88.787738] Code: ff ff 48 8b 55 a0 89 d9 be 00 10 00 00 48 c7 c7 d0
8d 4a a0 31 c0 48 c1 e2 0c e8 62 e4 c3 e0 e9 59 ff ff ff bb fb ff ff ff
eb d7 <0f> 0b 90 66 66 66 66 90 55 48 89 e5 41 57 4c 8d 7f 18 41 56 45
[ 88.787764] RIP: read_page+0x1dd/0x1e0 [md_mod] RSP: ffffc90000c7fbc0
[ 88.787767] ---[ end trace 48d7caff74360162 ]---
Thanks,
-Zhilong
Thanks. I see what I missed. Please try this patch instead.
Hi, Neil;
I have tested the following patch, I still got the call-trace after
I built with it.
If you need other infos, I would append.
Thanks for testing.
I looked more completely and I think it is easiest just to disable the
functionality rather than try to fix it.
Resizing the file in the kernel is extra complexity that I don't
want to get in to.
We could adjust the bitmap chunk size so that the file doesn't
need to grow, but it started getting more complicated than I really
wanted to deal with.
If there is anyone actually using file-backed bitmaps who wants to
be able to resize the array without removing the bitmap first, then
we can look at the problem again. For now I've sent a patch which
just returns an error instead of crashing when someone tries to resize
an array with a file-backed bitmap.
Thanks,
NeilBrown
From: Zhilong Liu <hidden> Date: 2017-08-31 02:57:11
On 08/31/2017 08:27 AM, NeilBrown wrote:
On Wed, Aug 30 2017, Zhilong Liu wrote:
quoted
On 08/29/2017 06:47 PM, NeilBrown wrote:
quoted
Thanks. I see what I missed. Please try this patch instead.
Hi, Neil;
I have tested the following patch, I still got the call-trace after
I built with it.
If you need other infos, I would append.
Thanks for testing.
I looked more completely and I think it is easiest just to disable the
functionality rather than try to fix it.
Resizing the file in the kernel is extra complexity that I don't
want to get in to.
We could adjust the bitmap chunk size so that the file doesn't
need to grow, but it started getting more complicated than I really
wanted to deal with.
If there is anyone actually using file-backed bitmaps who wants to
be able to resize the array without removing the bitmap first, then
we can look at the problem again. For now I've sent a patch which
just returns an error instead of crashing when someone tries to resize
an array with a file-backed bitmap.
Hi, Neil;
Shall update the "SIZE CHANGES" under "GROW MODE" of man-page for
resize like following after kernel patch merged?
Thanks,
-Zhilong
@@ -2758,6 +2758,11 @@ Also the size of an array cannot be changed while
it has an active
bitmap. If an array has a bitmap, it must be removed before the size
can be changed. Once the change is complete a new bitmap can be created.
+.PP
+Note:
+.B "--grow --size"
+is not yet supported for external file bitmap.
+
.SS RAID\-DEVICES CHANGES
A RAID1 array can work with any number of devices from 1 upwards
Thanks. I see what I missed. Please try this patch instead.
Hi, Neil;
I have tested the following patch, I still got the call-trace after
I built with it.
If you need other infos, I would append.
Thanks for testing.
I looked more completely and I think it is easiest just to disable the
functionality rather than try to fix it.
Resizing the file in the kernel is extra complexity that I don't
want to get in to.
We could adjust the bitmap chunk size so that the file doesn't
need to grow, but it started getting more complicated than I really
wanted to deal with.
If there is anyone actually using file-backed bitmaps who wants to
be able to resize the array without removing the bitmap first, then
we can look at the problem again. For now I've sent a patch which
just returns an error instead of crashing when someone tries to resize
an array with a file-backed bitmap.
Hi, Neil;
Shall update the "SIZE CHANGES" under "GROW MODE" of man-page for
resize like following after kernel patch merged?
Good idea, but it isn't just "--grow --size". It is anything that
changes the size of the array, which includes changing the number of
devices in a RAID5 etc. So a more general statement would be better.
NeilBrown
@@ -2758,6 +2758,11 @@ Also the size of an array cannot be changed while
it has an active
bitmap. If an array has a bitmap, it must be removed before the size
can be changed. Once the change is complete a new bitmap can be created.
+.PP
+Note:
+.B "--grow --size"
+is not yet supported for external file bitmap.
+
.SS RAID\-DEVICES CHANGES
A RAID1 array can work with any number of devices from 1 upwards
From: Zhilong Liu <hidden> Date: 2017-09-01 09:49:07
On 08/31/2017 11:30 AM, NeilBrown wrote:
On Thu, Aug 31 2017, Zhilong Liu wrote:
quoted
On 08/31/2017 08:27 AM, NeilBrown wrote:
quoted
On Wed, Aug 30 2017, Zhilong Liu wrote:
quoted
On 08/29/2017 06:47 PM, NeilBrown wrote:
quoted
Thanks. I see what I missed. Please try this patch instead.
Hi, Neil;
I have tested the following patch, I still got the call-trace after
I built with it.
If you need other infos, I would append.
Thanks for testing.
I looked more completely and I think it is easiest just to disable the
functionality rather than try to fix it.
Resizing the file in the kernel is extra complexity that I don't
want to get in to.
We could adjust the bitmap chunk size so that the file doesn't
need to grow, but it started getting more complicated than I really
wanted to deal with.
If there is anyone actually using file-backed bitmaps who wants to
be able to resize the array without removing the bitmap first, then
we can look at the problem again. For now I've sent a patch which
just returns an error instead of crashing when someone tries to resize
an array with a file-backed bitmap.
Hi, Neil;
Shall update the "SIZE CHANGES" under "GROW MODE" of man-page for
resize like following after kernel patch merged?
Good idea, but it isn't just "--grow --size". It is anything that
changes the size of the array, which includes changing the number of
devices in a RAID5 etc. So a more general statement would be better.
I have tested the changing number of devices within your patch. Here are
my steps,
and all works well.
1. create one raid1 with same size disks, loop[0-2], 2 active and 1 spare.
2. grow array to raid5.
3. "grow and add" new same size devices into raid5.
4. continue the step 3.
5. "grow and add" new lager size device into raid5.
6. continue step 5.
7. "manage and set failure" smaller disk in raid5.
8. "manage and remove" the failure disk from raid5.
Sorry for the long logs.
Thanks,
-Zhilong
linux-apta:~/mdadm-test # lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
loop1 7:1 0 19.5M 0 loop
loop13 7:13 0 64M 0 loop
loop11 7:11 0 64M 0 loop
loop8 7:8 0 64M 0 loop
loop6 7:6 0 19.5M 0 loop
loop4 7:4 0 19.5M 0 loop
loop2 7:2 0 19.5M 0 loop
loop0 7:0 0 19.5M 0 loop
loop12 7:12 0 64M 0 loop
loop9 7:9 0 64M 0 loop
loop10 7:10 0 64M 0 loop
sda 8:0 0 45G 0 disk
├─sda2 8:2 0 43G 0 part /
└─sda1 8:1 0 2G 0 part [SWAP]
loop7 7:7 0 19.5M 0 loop
loop5 7:5 0 19.5M 0 loop
loop3 7:3 0 19.5M 0 loop
linux-apta:~/mdadm-test # ./mdadm -CR /dev/md0 -l1 -b /mnt/3 -n2 -x1
/dev/loop[0-2] --force
mdadm: /dev/loop0 appears to contain an ext2fs file system
size=38912K mtime=Wed Dec 31 19:00:00 1969
mdadm: Note: this array has metadata at the start and
may not be suitable as a boot device. If you plan to
store '/boot' on this device please ensure that
your boot-loader understands md/v1.x metadata, or use
--metadata=0.90
mdadm: /dev/loop1 appears to contain an ext2fs file system
size=19968K mtime=Wed Dec 31 19:00:00 1969
mdadm: Defaulting to version 1.2 metadata
mdadm: array /dev/md0 started.
linux-apta:~/mdadm-test # cat /proc/mdstat
Personalities : [raid1]
md0 : active raid1 loop2[2](S) loop1[1] loop0[0]
18944 blocks super 1.2 [2/2] [UU]
bitmap: 2/3 pages [8KB], 4KB chunk, file: /mnt/3
unused devices: <none>
linux-apta:~/mdadm-test # ./mdadm --grow /dev/md0 --size max
mdadm: Cannot set device size for /dev/md0: Invalid argument
linux-apta:~/mdadm-test # dmesg
[ 2218.652119] md: md0: resync done.
[ 2235.258392] md: cannot resize file-based bitmap
[ 2235.325163] md: couldn't update array info. -22
linux-apta:~/mdadm-test # ./mdadm --grow /dev/md0 -l5
mdadm: level of /dev/md0 changed to raid5
linux-apta:~/mdadm-test # cat /proc/mdstat
Personalities : [raid1] [raid6] [raid5] [raid4]
md0 : active raid5 loop0[0] loop2[2](S) loop1[1]
18944 blocks super 1.2 level 5, 64k chunk, algorithm 2 [2/2] [UU]
bitmap: 0/3 pages [0KB], 4KB chunk, file: /mnt/3
unused devices: <none>
linux-apta:~/mdadm-test # ./mdadm --grow /dev/md0 -n3 -a /dev/loop3
mdadm: added /dev/loop3
linux-apta:~/mdadm-test # cat /proc/mdstat
Personalities : [raid1] [raid6] [raid5] [raid4]
md0 : active raid5 loop3[3] loop0[0] loop2[2](S) loop1[1]
18944 blocks super 1.2 level 5, 64k chunk, algorithm 2 [3/3] [UUU]
[======>..............] reshape = 31.5% (6528/18944)
finish=0.0min speed=3264K/sec
bitmap: 0/3 pages [0KB], 4KB chunk, file: /mnt/3
unused devices: <none>
linux-apta:~/mdadm-test # cat /proc/mdstat
Personalities : [raid1] [raid6] [raid5] [raid4]
md0 : active raid5 loop3[3] loop0[0] loop2[2](S) loop1[1]
37888 blocks super 1.2 level 5, 64k chunk, algorithm 2 [3/3] [UUU]
bitmap: 0/3 pages [0KB], 4KB chunk, file: /mnt/3
unused devices: <none>
linux-apta:~/mdadm-test # ./mdadm --grow /dev/md0 -n4 -a /dev/loop4
mdadm: added /dev/loop4
linux-apta:~/mdadm-test # cat /proc/mdstat
Personalities : [raid1] [raid6] [raid5] [raid4]
md0 : active raid5 loop4[4] loop3[3] loop0[0] loop2[2](S) loop1[1]
37888 blocks super 1.2 level 5, 64k chunk, algorithm 2 [4/4] [UUUU]
[======>..............] reshape = 31.5% (6272/18944)
finish=0.1min speed=2090K/sec
bitmap: 0/3 pages [0KB], 4KB chunk, file: /mnt/3
unused devices: <none>
linux-apta:~/mdadm-test # cat /proc/mdstat
Personalities : [raid1] [raid6] [raid5] [raid4]
md0 : active raid5 loop4[4] loop3[3] loop0[0] loop2[2](S) loop1[1]
56832 blocks super 1.2 level 5, 64k chunk, algorithm 2 [4/4] [UUUU]
bitmap: 0/3 pages [0KB], 4KB chunk, file: /mnt/3
unused devices: <none>
linux-apta:~/mdadm-test # ./mdadm --grow /dev/md0 -n5 -a /dev/loop10
mdadm: added /dev/loop10
linux-apta:~/mdadm-test # cat /proc/mdstat
Personalities : [raid1] [raid6] [raid5] [raid4]
md0 : active raid5 loop10[5] loop0[0] loop2[2](S) loop4[4] loop3[3] loop1[1]
56832 blocks super 1.2 level 5, 64k chunk, algorithm 2 [5/5] [UUUUU]
[==========>..........] reshape = 52.6% (10240/18944)
finish=0.0min speed=2560K/sec
bitmap: 0/3 pages [0KB], 4KB chunk, file: /mnt/3
unused devices: <none>
linux-apta:~/mdadm-test # cat /proc/mdstat
Personalities : [raid1] [raid6] [raid5] [raid4]
md0 : active raid5 loop10[5] loop0[0] loop2[2](S) loop4[4] loop3[3] loop1[1]
75776 blocks super 1.2 level 5, 64k chunk, algorithm 2 [5/5] [UUUUU]
bitmap: 0/3 pages [0KB], 4KB chunk, file: /mnt/3
unused devices: <none>
linux-apta:~/mdadm-test # ./mdadm --grow /dev/md0 -n6 -a /dev/loop11
mdadm: added /dev/loop11
linux-apta:~/mdadm-test # cat /proc/mdstat
Personalities : [raid1] [raid6] [raid5] [raid4]
md0 : active raid5 loop11[6] loop10[5] loop0[0] loop2[2](S) loop4[4]
loop3[3] loop1[1]
75776 blocks super 1.2 level 5, 64k chunk, algorithm 2 [6/6] [UUUUUU]
[=====>...............] reshape = 26.3% (5248/18944)
finish=0.0min speed=2624K/sec
bitmap: 0/3 pages [0KB], 4KB chunk, file: /mnt/3
unused devices: <none>
linux-apta:~/mdadm-test # ./mdadm --manage /dev/md0 -f /dev/loop11
mdadm: set /dev/loop11 faulty in /dev/md0
linux-apta:~/mdadm-test # cat /proc/mdstat
Personalities : [raid1] [raid6] [raid5] [raid4]
md0 : active raid5 loop11[6](F) loop10[5] loop0[0] loop2[2] loop4[4]
loop3[3] loop1[1]
94720 blocks super 1.2 level 5, 64k chunk, algorithm 2 [6/5] [UUUUU_]
[====>................] recovery = 23.6% (4752/18944)
finish=0.0min speed=2376K/sec
bitmap: 0/3 pages [0KB], 4KB chunk, file: /mnt/3
unused devices: <none>
linux-apta:~/mdadm-test # cat /proc/mdstat
Personalities : [raid1] [raid6] [raid5] [raid4]
md0 : active raid5 loop11[6](F) loop10[5] loop0[0] loop2[2] loop4[4]
loop3[3] loop1[1]
94720 blocks super 1.2 level 5, 64k chunk, algorithm 2 [6/6] [UUUUUU]
bitmap: 0/3 pages [0KB], 4KB chunk, file: /mnt/3
unused devices: <none>
linux-apta:~/mdadm-test # ./mdadm --manage /dev/md0 -r /dev/loop11
mdadm: hot removed /dev/loop11 from /dev/md0
linux-apta:~/mdadm-test # cat /proc/mdstat
Personalities : [raid1] [raid6] [raid5] [raid4]
md0 : active raid5 loop10[5] loop0[0] loop2[2] loop4[4] loop3[3] loop1[1]
94720 blocks super 1.2 level 5, 64k chunk, algorithm 2 [6/6] [UUUUUU]
bitmap: 0/3 pages [0KB], 4KB chunk, file: /mnt/3
unused devices: <none>
@@ -2758,6 +2758,11 @@ Also the size of an array cannot be changed while
it has an active
bitmap. If an array has a bitmap, it must be removed before the size
can be changed. Once the change is complete a new bitmap can be created.
+.PP
+Note:
+.B "--grow --size"
+is not yet supported for external file bitmap.
+
.SS RAID\-DEVICES CHANGES
A RAID1 array can work with any number of devices from 1 upwards