On 04/02/2013 19:39, Phil Turmel wrote:

On 02/04/2013 07:51 AM, Dominique wrote:

quoted

Back from weekend, and still with my problem. Output of lsdrv is as follow:

Yuck, mangled utf-8.  Does it look that way on your console?
[output w/ fixed tree characters pasted below]

First item, greatest importance:  You are using Western Digital Green
drives.  These are *unsafe* to use in raid arrays of any kind "out of
the box".  They do *not* support SCTERC.  You *must* use a boot-up
script to set the linux driver timeouts to two minutes or more or you
*will* crash this array.

I recommend a three minute timeout.  Something like this:

quoted

for x in /sys/block/sd[abcdef]/driver/timeout ; do
	echo 180 > $x
done

in "rc.local" or wherever your distribution likes such things.  And
don't wait for your next reboot--execute it now.

If you don't have that in place, it suggests that your array is very new
or you are not running any regular "scrub".  It is important that you
not be vulnerable during the reshape, as the array will be heavily
exercised.  If you aren't regularly scrubbing, execute:

quoted

echo "check" >/sys/block/md2/md/sync_action

then monitor /proc/mdstat until it completes.  (Several hours)  Then
look at the mismatch count:

quoted

cat /sys/block/md2/md/mismatch_count

It should be zero.

Otherwise, your array is a simple ext4 filesystem without any apparent
system dependencies.  So you should be able to do everything needed in
your normal boot environment, just shutting down the services that are
using the data in "/srv".  Once the filesystem itself is resized
(possibly quite quickly if your array has a great deal of free space),
the remainder of the work can occur online (with /srv mounted again and
your services restarted).

Here's your recipe:

1) Stop all services using /srv, then unmount with:

quoted

umount /srv

2) Make sure the filesystem is consistent:

quoted

fsck /dev/md2

3) Determine resizing options:

quoted

resize2fs -P /dev/md2

This will report the number of blocks needed for the current contents
(usually in 4k blocks).  I don't recommend resizing all the way to the
minimum, as it may take much longer.  Just make sure you can shrink to
~10 terabytes.

4) Resize:

quoted

resize2fs /dev/md2 10240G

5) Verify Ok:

quoted

fsck -n /dev/md2

6) Instruct md to temporarily use less than the future size of md2 (but
more than the filesystem):

quoted

mdadm --grow /dev/md2 --array-size=10241G

7) Verify Again:

quoted

fsck -n /dev/md2

8) Instruct md to reshape to raid6 and view its progress:

quoted

mdadm --grow /dev/md2 --level=raid6 --raid-devices=6
cat /proc/mdstat

  (The reshape will continue in the background.)

9) If you need the services as soon as possible, the filesystem can be
remounted at this point, and the services restarted.  If /srv is in your
fstab, just use:

quoted

mount /srv

10) Depending on the speed of your components, and how heavily you use
the array, the reshape can take several hours to days.  I would expect
yours to take at least 7 hours, best case.  Once complete, you can
resize once more to the maximum available (this can be done while
mounted and the services running):

quoted

mdadm --grow /dev/md2 --array-size=max
resize2fs /dev/md2

If you run into any problems (or questions), let us know.

Regards,

Phil

--

PCI [ahci] 00:1f.2 SATA controller: Intel Corporation 6 Series/C200
Series Chipset Family SATA AHCI Controller (rev 05)
├scsi 0:0:0:0 ATA      WDC WD30EZRX-00M {WD-WCAWZ1618149}
│└sda 2.73t [8:0] Partitioned (gpt)
│ ├sda1 95.37m [8:1] vfat {89EF-00F4}
│ │└Mounted as /dev/sda1 @ /boot/efi
│ ├sda2 29.80g [8:2] MD raid1 (0/6) (w/ sdd2,sde2,sdc2,sdb2,sdf2)
in_sync 'solipym:0' {d2e6885b-2d25-6c5a-5f3d-e9a9a5daa736}
│ │└md0 29.80g [9:0] MD v1.2 raid1 (6) clean
{d2e6885b:2d256c5a:5f3de9a9:a5daa736}
│ │                  swap {8e565dfc-c5fa-4c76-abe9-f3a5a6e8fcff}
│ ├sda3 186.26g [8:3] MD raid1 (0/6) (w/ sdf3,sdd3,sde3,sdc3,sdb3)
in_sync 'solipym:1' {89d69c6f-2ea2-23f5-aec8-b67c403488ad}
│ │└md1 186.26g [9:1] MD v1.2 raid1 (6) clean
{89d69c6f:2ea223f5:aec8b67c:403488ad}
│ │ │                 ext4 {ca4bff22-40d8-4b31-859e-bba063f01df1}
│ │ ├Mounted as /dev/disk/by-uuid/ca4bff22-40d8-4b31-859e-bba063f01df1 @ /
│ │ └Mounted as /dev/disk/by-uuid/ca4bff22-40d8-4b31-859e-bba063f01df1 @
/var/spool/hylafax/etc
│ └sda4 2.52t [8:4] MD raid5 (0/6) (w/ sdd4,sdb4,sde4,sdc4,sdf4) in_sync
'solipym:2' {84672b4c-8fae-7f38-bb4c-c911aa9d7444}
│  └md2 12.59t [9:2] MD v1.2 raid5 (6) clean, 512k Chunk
{84672b4c:8fae7f38:bb4cc911:aa9d7444}
│   │                ext4 {a9cfe95c-1e62-4cf6-b2aa-65abbd0c77e4}
│   └Mounted as /dev/md2 @ /srv
├scsi 1:0:0:0 ATA      WDC WD30EZRX-00M {WD-WCAWZ1382990}
│└sdb 2.73t [8:16] Partitioned (gpt)
│ ├sdb1 95.37m [8:17] vfat {6EFD-1659}
│ ├sdb2 29.80g [8:18] MD raid1 (1/6) (w/ sda2,sdd2,sde2,sdc2,sdf2)
in_sync 'solipym:0' {d2e6885b-2d25-6c5a-5f3d-e9a9a5daa736}
│ │└md0 29.80g [9:0] MD v1.2 raid1 (6) clean
{d2e6885b:2d256c5a:5f3de9a9:a5daa736}
│ │                  swap {8e565dfc-c5fa-4c76-abe9-f3a5a6e8fcff}
│ ├sdb3 186.26g [8:19] MD raid1 (1/6) (w/ sdf3,sdd3,sde3,sda3,sdc3)
in_sync 'solipym:1' {89d69c6f-2ea2-23f5-aec8-b67c403488ad}
│ │└md1 186.26g [9:1] MD v1.2 raid1 (6) clean
{89d69c6f:2ea223f5:aec8b67c:403488ad}
│ │                   ext4 {ca4bff22-40d8-4b31-859e-bba063f01df1}
│ └sdb4 2.52t [8:20] MD raid5 (1/6) (w/ sdd4,sda4,sde4,sdc4,sdf4)
in_sync 'solipym:2' {84672b4c-8fae-7f38-bb4c-c911aa9d7444}
│  └md2 12.59t [9:2] MD v1.2 raid5 (6) clean, 512k Chunk
{84672b4c:8fae7f38:bb4cc911:aa9d7444}
│                    ext4 {a9cfe95c-1e62-4cf6-b2aa-65abbd0c77e4}
├scsi 2:0:0:0 ATA      WDC WD30EZRX-00M {WD-WCAWZ0995502}
│└sdc 2.73t [8:32] Partitioned (gpt)
│ ├sdc1 95.37m [8:33] vfat {16BF-AABE}
│ ├sdc2 29.80g [8:34] MD raid1 (2/6) (w/ sda2,sdd2,sde2,sdb2,sdf2)
in_sync 'solipym:0' {d2e6885b-2d25-6c5a-5f3d-e9a9a5daa736}
│ │└md0 29.80g [9:0] MD v1.2 raid1 (6) clean
{d2e6885b:2d256c5a:5f3de9a9:a5daa736}
│ │                  swap {8e565dfc-c5fa-4c76-abe9-f3a5a6e8fcff}
│ ├sdc3 186.26g [8:35] MD raid1 (2/6) (w/ sdf3,sdd3,sde3,sda3,sdb3)
in_sync 'solipym:1' {89d69c6f-2ea2-23f5-aec8-b67c403488ad}
│ │└md1 186.26g [9:1] MD v1.2 raid1 (6) clean
{89d69c6f:2ea223f5:aec8b67c:403488ad}
│ │                   ext4 {ca4bff22-40d8-4b31-859e-bba063f01df1}
│ └sdc4 2.52t [8:36] MD raid5 (2/6) (w/ sdd4,sda4,sdb4,sde4,sdf4)
in_sync 'solipym:2' {84672b4c-8fae-7f38-bb4c-c911aa9d7444}
│  └md2 12.59t [9:2] MD v1.2 raid5 (6) clean, 512k Chunk
{84672b4c:8fae7f38:bb4cc911:aa9d7444}
│                    ext4 {a9cfe95c-1e62-4cf6-b2aa-65abbd0c77e4}
├scsi 3:0:0:0 ATA      WDC WD30EZRX-00M {WD-WCAWZ1118226}
│└sdd 2.73t [8:48] Partitioned (gpt)
│ ├sdd1 95.37m [8:49] vfat {978F-21B2}
│ ├sdd2 29.80g [8:50] MD raid1 (3/6) (w/ sda2,sde2,sdc2,sdb2,sdf2)
in_sync 'solipym:0' {d2e6885b-2d25-6c5a-5f3d-e9a9a5daa736}
│ │└md0 29.80g [9:0] MD v1.2 raid1 (6) clean
{d2e6885b:2d256c5a:5f3de9a9:a5daa736}
│ │                  swap {8e565dfc-c5fa-4c76-abe9-f3a5a6e8fcff}
│ ├sdd3 186.26g [8:51] MD raid1 (3/6) (w/ sdf3,sde3,sda3,sdc3,sdb3)
in_sync 'solipym:1' {89d69c6f-2ea2-23f5-aec8-b67c403488ad}
│ │└md1 186.26g [9:1] MD v1.2 raid1 (6) clean
{89d69c6f:2ea223f5:aec8b67c:403488ad}
│ │                   ext4 {ca4bff22-40d8-4b31-859e-bba063f01df1}
│ └sdd4 2.52t [8:52] MD raid5 (3/6) (w/ sda4,sdb4,sde4,sdc4,sdf4)
in_sync 'solipym:2' {84672b4c-8fae-7f38-bb4c-c911aa9d7444}
│  └md2 12.59t [9:2] MD v1.2 raid5 (6) clean, 512k Chunk
{84672b4c:8fae7f38:bb4cc911:aa9d7444}
│                    ext4 {a9cfe95c-1e62-4cf6-b2aa-65abbd0c77e4}
├scsi 4:0:0:0 ATA      WDC WD30EZRX-00M {WD-WCAWZ1649385}
│└sde 2.73t [8:64] Partitioned (gpt)
│ ├sde1 95.37m [8:65] vfat {8875-2D50}
│ ├sde2 29.80g [8:66] MD raid1 (4/6) (w/ sda2,sdd2,sdc2,sdb2,sdf2)
in_sync 'solipym:0' {d2e6885b-2d25-6c5a-5f3d-e9a9a5daa736}
│ │└md0 29.80g [9:0] MD v1.2 raid1 (6) clean
{d2e6885b:2d256c5a:5f3de9a9:a5daa736}
│ │                  swap {8e565dfc-c5fa-4c76-abe9-f3a5a6e8fcff}
│ ├sde3 186.26g [8:67] MD raid1 (4/6) (w/ sdf3,sdd3,sda3,sdc3,sdb3)
in_sync 'solipym:1' {89d69c6f-2ea2-23f5-aec8-b67c403488ad}
│ │└md1 186.26g [9:1] MD v1.2 raid1 (6) clean
{89d69c6f:2ea223f5:aec8b67c:403488ad}
│ │                   ext4 {ca4bff22-40d8-4b31-859e-bba063f01df1}
│ └sde4 2.52t [8:68] MD raid5 (4/6) (w/ sdd4,sda4,sdb4,sdc4,sdf4)
in_sync 'solipym:2' {84672b4c-8fae-7f38-bb4c-c911aa9d7444}
│  └md2 12.59t [9:2] MD v1.2 raid5 (6) clean, 512k Chunk
{84672b4c:8fae7f38:bb4cc911:aa9d7444}
│                    ext4 {a9cfe95c-1e62-4cf6-b2aa-65abbd0c77e4}
└scsi 5:0:0:0 ATA      WDC WD30EZRX-00M {WD-WCAWZ1383151}
  └sdf 2.73t [8:80] Partitioned (gpt)
   ├sdf1 95.37m [8:81] vfat {A89B-DC05}
   ├sdf2 29.80g [8:82] MD raid1 (5/6) (w/ sda2,sdd2,sde2,sdc2,sdb2)
in_sync 'solipym:0' {d2e6885b-2d25-6c5a-5f3d-e9a9a5daa736}
   │└md0 29.80g [9:0] MD v1.2 raid1 (6) clean
{d2e6885b:2d256c5a:5f3de9a9:a5daa736}
   │                  swap {8e565dfc-c5fa-4c76-abe9-f3a5a6e8fcff}
   ├sdf3 186.26g [8:83] MD raid1 (5/6) (w/ sdd3,sde3,sda3,sdc3,sdb3)
in_sync 'solipym:1' {89d69c6f-2ea2-23f5-aec8-b67c403488ad}
   │└md1 186.26g [9:1] MD v1.2 raid1 (6) clean
{89d69c6f:2ea223f5:aec8b67c:403488ad}
   │                   ext4 {ca4bff22-40d8-4b31-859e-bba063f01df1}
   └sdf4 2.52t [8:84] MD raid5 (5/6) (w/ sdd4,sda4,sdb4,sde4,sdc4)
in_sync 'solipym:2' {84672b4c-8fae-7f38-bb4c-c911aa9d7444}
    └md2 12.59t [9:2] MD v1.2 raid5 (6) clean, 512k Chunk
{84672b4c:8fae7f38:bb4cc911:aa9d7444}
                      ext4 {a9cfe95c-1e62-4cf6-b2aa-65abbd0c77e4}
USB [usb-storage] Bus 002 Device 004: ID 059f:1010 LaCie, Ltd Desktop
Hard Drive {ST3500830A            9QG6RC54}
└scsi 6:0:0:0 ST350083 0AS
  └sdg 465.76g [8:96] Partitioned (dos)
   └sdg1 465.76g [8:97] ext4 {4c2f6b92-829d-4e53-b553-c07e0f571e02}
    └Mounted as /dev/sdg1 @ /mnt
Other Block Devices
├loop0 0.00k [7:0] Empty/Unknown
├loop1 0.00k [7:1] Empty/Unknown
├loop2 0.00k [7:2] Empty/Unknown
├loop3 0.00k [7:3] Empty/Unknown
├loop4 0.00k [7:4] Empty/Unknown
├loop5 0.00k [7:5] Empty/Unknown
├loop6 0.00k [7:6] Empty/Unknown
├loop7 0.00k [7:7] Empty/Unknown
├ram0 64.00m [1:0] Empty/Unknown
├ram1 64.00m [1:1] Empty/Unknown
├ram2 64.00m [1:2] Empty/Unknown
├ram3 64.00m [1:3] Empty/Unknown
├ram4 64.00m [1:4] Empty/Unknown
├ram5 64.00m [1:5] Empty/Unknown
├ram6 64.00m [1:6] Empty/Unknown
├ram7 64.00m [1:7] Empty/Unknown
├ram8 64.00m [1:8] Empty/Unknown
├ram9 64.00m [1:9] Empty/Unknown
├ram10 64.00m [1:10] Empty/Unknown
├ram11 64.00m [1:11] Empty/Unknown
├ram12 64.00m [1:12] Empty/Unknown
├ram13 64.00m [1:13] Empty/Unknown
├ram14 64.00m [1:14] Empty/Unknown
└ram15 64.00m [1:15] Empty/Unknown
--

Thanks for the detailled answer. I read through all of it to identify 
what I did not understood before starting. I have a few clarificacions 
for you and a few questions.

Yes, that's the way my console looks like. I usually don't have problem 
with UTF8 output (i.e. not garbled), I figured it was the result of the 
lsdrv.... Nasty to read, but it's all there.

I am running Ubuntu 12.04.1 server on this relatively new raid5 setup.
I think I understood most of the reshaping instructions, but I need to 
clarify your point related to the timeout. I tried to execute it in a 
simple bash file and got stopped at the beginning just for the lack of a 
timeoutfile?

After looking for it, I realised that sd[abcdef] were all symlinks 
pointing to another part of the system, where there is no driver 
directory. After browsing, i figured out that /sys/block/sda/device did 
led to a timeout file. Can you please confirm, it is the one we want ?

A last point I need to clarify: is the reshaping (although long) data 
destructive (backup will be done in all cases)?

Thanks,

Dominique

--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html