From: Thomas Backlund <hidden> Date: 2008-07-27 14:26:33
Hi,
(please cc me as I'm not subscribed)
I have hit a bug with mdadm 2.6.7
It rebuilds my raid5 array on every boot
(raid0 and raid1 arrays are not affected)
This didn't happend with 2.6.4
kernels tested are 2.6.24.7 and 2.6.25.12
Arch is x86_64
Distro Mandriva 2008.1, but I've tested wich kernel.org kernels and
upstream mdadm 2.6.7 and have the same problem
Now I could try to bisect it, but every raid5 rebuild takes 6-7 hours,
so I thought about asking for pointers before...
Any ideas where to start looking ?
here is the info on the array that gets rebuilt...
[root@tmb ~]# mdadm --detail /dev/md8
/dev/md8:
Version : 00.90
Creation Time : Fri Feb 1 17:44:23 2008
Raid Level : raid5
Array Size : 1465143808 (1397.27 GiB 1500.31 GB)
Used Dev Size : 732571904 (698.64 GiB 750.15 GB)
Raid Devices : 3
Total Devices : 3
Preferred Minor : 8
Persistence : Superblock is persistent
Update Time : Sun Jul 27 13:32:30 2008
State : clean, degraded, recovering
Active Devices : 2
Working Devices : 3
Failed Devices : 0
Spare Devices : 1
Layout : left-symmetric
Chunk Size : 128K
Rebuild Status : 1% complete
UUID : dc482f3f:ad67b9ef:bb6636b8:e9392071
Events : 0.16162
Number Major Minor RaidDevice State
0 8 33 0 active sync /dev/sdc1
1 8 49 1 active sync /dev/sdd1
3 8 65 2 spare rebuilding /dev/sde1
On Sun, 2008-07-27 at 17:26 +0300, Thomas Backlund wrote:
Hi,
(please cc me as I'm not subscribed)
I have hit a bug with mdadm 2.6.7
It rebuilds my raid5 array on every boot
(raid0 and raid1 arrays are not affected)
This didn't happend with 2.6.4
kernels tested are 2.6.24.7 and 2.6.25.12
Arch is x86_64
Distro Mandriva 2008.1, but I've tested wich kernel.org kernels and
upstream mdadm 2.6.7 and have the same problem
Now I could try to bisect it, but every raid5 rebuild takes 6-7 hours,
so I thought about asking for pointers before...
Any ideas where to start looking ?
Are you using mkinitrd (or something similar) to start the arrays, or
are you using udev rules that call mdadm --incremental --run? If it's
the later, then this is what you get when A) the array is started as
soon as there are enough devices to run in degraded mode and B)
something writes to the array before the last device gets added and C)
you don't have a bitmap to allow the array to keep track of what blocks
need resynced and therefore it resynces the entire drive.
here is the info on the array that gets rebuilt...
[root@tmb ~]# mdadm --detail /dev/md8
/dev/md8:
Version : 00.90
Creation Time : Fri Feb 1 17:44:23 2008
Raid Level : raid5
Array Size : 1465143808 (1397.27 GiB 1500.31 GB)
Used Dev Size : 732571904 (698.64 GiB 750.15 GB)
Raid Devices : 3
Total Devices : 3
Preferred Minor : 8
Persistence : Superblock is persistent
Update Time : Sun Jul 27 13:32:30 2008
State : clean, degraded, recovering
Active Devices : 2
Working Devices : 3
Failed Devices : 0
Spare Devices : 1
Layout : left-symmetric
Chunk Size : 128K
Rebuild Status : 1% complete
UUID : dc482f3f:ad67b9ef:bb6636b8:e9392071
Events : 0.16162
Number Major Minor RaidDevice State
0 8 33 0 active sync /dev/sdc1
1 8 49 1 active sync /dev/sdd1
3 8 65 2 spare rebuilding /dev/sde1
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
From: Thomas Backlund <hidden> Date: 2008-07-27 15:24:24
Doug Ledford skrev:
On Sun, 2008-07-27 at 17:26 +0300, Thomas Backlund wrote:
quoted
Hi,
(please cc me as I'm not subscribed)
I have hit a bug with mdadm 2.6.7
It rebuilds my raid5 array on every boot
(raid0 and raid1 arrays are not affected)
This didn't happend with 2.6.4
kernels tested are 2.6.24.7 and 2.6.25.12
Arch is x86_64
Distro Mandriva 2008.1, but I've tested wich kernel.org kernels and
upstream mdadm 2.6.7 and have the same problem
Now I could try to bisect it, but every raid5 rebuild takes 6-7 hours,
so I thought about asking for pointers before...
Any ideas where to start looking ?
Are you using mkinitrd (or something similar) to start the arrays, or
are you using udev rules that call mdadm --incremental --run? If it's
the later, then this is what you get when A) the array is started as
soon as there are enough devices to run in degraded mode and B)
something writes to the array before the last device gets added and C)
you don't have a bitmap to allow the array to keep track of what blocks
need resynced and therefore it resynces the entire drive.
I'm using udev.
but looking at the difference between 2.6.4 and 2.6.7:
diff -Nurp mdadm-2.6.4/etc/udev/rules.d/70-mdadm.rules
mdadm-2.6.7/etc/udev/rules.d/70-mdadm.rules
@@ -3,4 +3,4 @@ # See udev(8) for syntax SUBSYSTEM=="block", ACTION=="add|change",
ENV{ID_FS_TYPE}=="linux_raid*", \
- RUN+="/sbin/mdadm --incremental $root/%k"
+ RUN+="/sbin/mdadm --incremental --run --scan $root/%k"
I see that --incremental was already there in 2.6.4, so I guess the
--run is the one messing with me...
as for --bitmap, can it be added to an existing array ?
What is the better choice, bitmap=internal or bitmap=<some_file> ?
I'd hate to have to recreate the array, as I have about 1.2GB of data on
it...
--
Thomas
On Sun, 2008-07-27 at 18:24 +0300, Thomas Backlund wrote:
quoted hunk
Doug Ledford skrev:
quoted
On Sun, 2008-07-27 at 17:26 +0300, Thomas Backlund wrote:
quoted
Hi,
(please cc me as I'm not subscribed)
I have hit a bug with mdadm 2.6.7
It rebuilds my raid5 array on every boot
(raid0 and raid1 arrays are not affected)
This didn't happend with 2.6.4
kernels tested are 2.6.24.7 and 2.6.25.12
Arch is x86_64
Distro Mandriva 2008.1, but I've tested wich kernel.org kernels and
upstream mdadm 2.6.7 and have the same problem
Now I could try to bisect it, but every raid5 rebuild takes 6-7 hours,
so I thought about asking for pointers before...
Any ideas where to start looking ?
Are you using mkinitrd (or something similar) to start the arrays, or
are you using udev rules that call mdadm --incremental --run? If it's
the later, then this is what you get when A) the array is started as
soon as there are enough devices to run in degraded mode and B)
something writes to the array before the last device gets added and C)
you don't have a bitmap to allow the array to keep track of what blocks
need resynced and therefore it resynces the entire drive.
I'm using udev.
but looking at the difference between 2.6.4 and 2.6.7:
diff -Nurp mdadm-2.6.4/etc/udev/rules.d/70-mdadm.rules
mdadm-2.6.7/etc/udev/rules.d/70-mdadm.rules
@@ -3,4 +3,4 @@ # See udev(8) for syntax SUBSYSTEM=="block", ACTION=="add|change",
ENV{ID_FS_TYPE}=="linux_raid*", \
- RUN+="/sbin/mdadm --incremental $root/%k"
+ RUN+="/sbin/mdadm --incremental --run --scan $root/%k"
I see that --incremental was already there in 2.6.4, so I guess the
--run is the one messing with me...
Yep. That'd be it.
as for --bitmap, can it be added to an existing array ?
What is the better choice, bitmap=internal or bitmap=<some_file> ?
I'd hate to have to recreate the array, as I have about 1.2GB of data on
it...
I use bitmap=internal on my arrays and never have a problem.