Re: [PATCH] RAID-6 check standalone
- From
- Piergiorgio Sartor <hidden>
- Date
- 2011-03-21 10:40:07
- Reply to
- NeilBrown, 2011-03-21
Hi Neil, On Mon, Mar 21, 2011 at 02:02:44PM +1100, NeilBrown wrote:
On Mon, 21 Feb 2011 21:45:51 +0100 Piergiorgio Sartor [off-list ref] wrote:quoted
Hi Neil, please find attached a patch, to mdadm-3.2 base, including a standalone versione of the raid-6 check. This is basically a re-working (and hopefully improvement) of the already implemented check in "restripe.c". I splitted the check function into "collect" and "stats", so that the second one could be easily replaced. The API is also simplified. The command line option are reduced, since we only level is raid-6, but the ":offset" option is included. The output reports the block/stripe rotation, P/Q errors and the possible HDD (or unknown). BTW, the patch applies also to the already patched "restripe.c", including the last ":offset" patch (which is not yet in git).Sorry for not replying for a month!! I manage to lose track of this then couldn't find it when I went looking, then got distracted, then....
don't worry, I know how easy is to loose track of such items. That's why I kept (slowly) insisting.
Thanks for your patience and persistence. I have applied this patch to me git now - with some minor changes.
Thanks.
One question, I did not find the previous patch to "restripe.c",
which was adding the ":offset" capability. This was in order to
be able to cope with metadata 1.1 or 1.2 (offset to be determined).
quoted
Other item is that due to "sysfs.c" linking (see below) the "Makefile" needed some changes, I hope this is not a problem.I have reverted most of these changes and the linking with sysfs.c isn't needed yet. When it is we can sort out how best to achieve it.quoted
Next steps (TODO list you like) would be: 1) Add the "sysfs.c" code in order to retrieve the HDDs info from the MD device. It is already linked, together with the whole (mdadm) universe, since it seems it cannot leave alone. I'll need some advice or hint on how to do use it. I checked "sysfs.c", but before I dig deep into it maybe better to have some advice (maybe just one function call will do it).What exactly do you want to achieve? I suspect you want to a open the md device, then info = sysfs_read(fd, -1, GET_LEVEL|GET_LAYOUT_GET_DISKS|GET_CHUNK|GET_DEVS|GET_OFFSET); (possibly with other flags) and then extract the info you want from the data structure returned - but I'm only guessing at what you might want.
I think this more or less correct.
I would like to pass, to "raid6check", the md device as only
parameter, maybe with some start/stop position, and derive all
the needed information from that.
That is, it should be: raid6check /dev/mdX start stop
Or, possibly: raid6check /dev/mdX start length
Or just: raid6check /dev/mdX
This means, the information that should be derived is:
1) level, in order to confirm it is 6
2) layout
3) disk components
4) offset for each component
5) chunk size
That should be all, I guess.
Your "sysfs_read" seems to do exactly this, please confirm.
quoted
2) Add the suspend lo/hi control. Fellow John Robinson was suggesting to look into "Grow.c", which I did, but I guess the same story as 1) is valid: better to have some hint on where to look before wasting time.This would be: sysfs_set_num(info, NULL, "suspend_lo", offset*data_disks); sysfs_set_num(info, NULL, "suspend_hi", (offset+chunksize)*data_disks); to freeze one stripe. Then work in there. The addresses are addresses in the array, hence the multiplication by data_disks (which is raid_disks - 2 for RAID6). Don't hold the array suspended for too long or something might get upset. And allocate any memory you need first, and call mlockall(MCL_CURRENT | MCL_FUTURE); first to be even more safe.
Thanks for the explanation. How is, then, the "unfreeze"?
Just writing (0) to both hi and lo?
quoted
3) Add a repair option (future). This should have different levels, like "all", "disk", "stripe". That is, fix everything (more or less like "repair"), fix only if a disk is clearly having problems, fix each stripe which has clearly a problem (but maybe different stripes may belong to different HDDs). So, for the point 1) and 2) would be nice to have some more detail on where to look what. Point 3) we will discuss later.Agreed. Hopefully the info I have provided is sufficient to help you progress.
I'll ask, if I need more information.
Thanks, NeilBrown
Thank you, bye, pg
quoted
Thanks, please consider for inclusion, bye, pgquoted
diff -urN a/Makefile b/Makefile--- a/Makefile 2011-02-01 05:31:07.000000000 +0100 +++ b/Makefile 2011-02-20 00:03:44.929998038 +0100@@ -98,7 +98,7 @@ MAN5DIR = $(MANDIR)/man5 MAN8DIR = $(MANDIR)/man8 -OBJS = mdadm.o config.o policy.o mdstat.o ReadMe.o util.o Manage.o Assemble.o Build.o \ +OBJSX = config.o policy.o mdstat.o ReadMe.o util.o Manage.o Assemble.o Build.o \ Create.o Detail.o Examine.o Grow.o Monitor.o dlink.o Kill.o Query.o \ Incremental.o \ mdopen.o super0.o super1.o super-ddf.o super-intel.o bitmap.o \@@ -106,6 +106,8 @@ restripe.o sysfs.o sha1.o mapfile.o crc32.o sg_io.o msg.o \ platform-intel.o probe_roms.o +OBJS = mdadm.o $(OBJSX) + SRCS = mdadm.c config.c policy.c mdstat.c ReadMe.c util.c Manage.c Assemble.c Build.c \ Create.c Detail.c Examine.c Grow.c Monitor.c dlink.c Kill.c Query.c \ Incremental.c \@@ -182,6 +184,9 @@ test_stripe : restripe.c mdadm.h $(CC) $(CXFLAGS) $(LDFLAGS) -o test_stripe -DMAIN restripe.c +raid6check : raid6check.o mdadm.h $(OBJSX) + $(CC) $(CXFLAGS) $(LDFLAGS) -o raid6check raid6check.o $(OBJSX) + mdassemble : $(ASSEMBLE_SRCS) $(INCL) rm -f $(OBJS) $(DIET_GCC) $(ASSEMBLE_FLAGS) -o mdassemble $(ASSEMBLE_SRCS) $(STATICSRC)@@ -265,7 +270,7 @@ mdadm.Os mdadm.O2 mdmon.O2 \ mdassemble mdassemble.static mdassemble.auto mdassemble.uclibc \ mdassemble.klibc swap_super \ - init.cpio.gz mdadm.uclibc.static test_stripe mdmon \ + init.cpio.gz mdadm.uclibc.static test_stripe raid6check raid6check.o mdmon \ mdadm.8 dist : cleandiff -urN a/raid6check.c b/raid6check.c--- a/raid6check.c 1970-01-01 01:00:00.000000000 +0100 +++ b/raid6check.c 2011-02-21 21:40:27.813656214 +0100@@ -0,0 +1,263 @@ +/* + * raid6check - extended consistency check for RAID-6 + * + * Copyright (C) 2011 Piergiorgio Sartor + * + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + * + * Author: Piergiorgio Sartor + * Based on "restripe.c" from "mdadm" codebase + */ + +#include "mdadm.h" +#include <stdint.h> + +int geo_map(int block, unsigned long long stripe, int raid_disks, + int level, int layout); +void qsyndrome(uint8_t *p, uint8_t *q, uint8_t **sources, int disks, int size); +void make_tables(void); + +/* Collect per stripe consistency information */ +void raid6_collect(int chunk_size, uint8_t *p, uint8_t *q, + char *chunkP, char *chunkQ, int *results) +{ + int i; + int data_id; + uint8_t Px, Qx; + extern uint8_t raid6_gflog[]; + + for(i = 0; i < chunk_size; i++) { + Px = (uint8_t)chunkP[i] ^ (uint8_t)p[i]; + Qx = (uint8_t)chunkQ[i] ^ (uint8_t)q[i]; + + if((Px != 0) && (Qx == 0)) + results[i] = -1; + + if((Px == 0) && (Qx != 0)) + results[i] = -2; + + if((Px != 0) && (Qx != 0)) { + data_id = (raid6_gflog[Qx] - raid6_gflog[Px]); + if(data_id < 0) data_id += 255; + results[i] = data_id; + } + + if((Px == 0) && (Qx == 0)) + results[i] = -255; + } +} + +/* Try to find out if a specific disk has problems */ +int raid6_stats(int *results, int raid_disks, int chunk_size) +{ + int i; + int curr_broken_disk = -255; + int prev_broken_disk = -255; + int broken_status = 0; + + for(i = 0; i < chunk_size; i++) { + + if(results[i] != -255) + curr_broken_disk = results[i]; + + if(curr_broken_disk >= raid_disks) + broken_status = 2; + + switch(broken_status) { + case 0: + if(curr_broken_disk != -255) { + prev_broken_disk = curr_broken_disk; + broken_status = 1; + } + break; + + case 1: + if(curr_broken_disk != prev_broken_disk) + broken_status = 2; + break; + + case 2: + default: + curr_broken_disk = prev_broken_disk = -65535; + break; + } + } + + return curr_broken_disk; +} + +int check_stripes(int *source, unsigned long long *offsets, + int raid_disks, int chunk_size, int level, int layout, + unsigned long long start, unsigned long long length, char *name[]) +{ + /* read the data and p and q blocks, and check we got them right */ + char *stripe_buf = malloc(raid_disks * chunk_size); + char **stripes = malloc(raid_disks * sizeof(char*)); + char **blocks = malloc(raid_disks * sizeof(char*)); + uint8_t *p = malloc(chunk_size); + uint8_t *q = malloc(chunk_size); + int *results = malloc(chunk_size * sizeof(int)); + + int i; + int diskP, diskQ; + int data_disks = raid_disks - 2; + + extern int tables_ready; + + if (!tables_ready) + make_tables(); + + for ( i = 0 ; i < raid_disks ; i++) + stripes[i] = stripe_buf + i * chunk_size; + + while (length > 0) { + int disk; + + for (i = 0 ; i < raid_disks ; i++) { + lseek64(source[i], offsets[i]+start, 0); + read(source[i], stripes[i], chunk_size); + } + for (i = 0 ; i < data_disks ; i++) { + int disk = geo_map(i, start/chunk_size, raid_disks, + level, layout); + blocks[i] = stripes[disk]; + printf("%d->%d\n", i, disk); + } + + qsyndrome(p, q, (uint8_t**)blocks, data_disks, chunk_size); + diskP = geo_map(-1, start/chunk_size, raid_disks, + level, layout); + if (memcmp(p, stripes[diskP], chunk_size) != 0) { + printf("P(%d) wrong at %llu\n", diskP, + start / chunk_size); + } + diskQ = geo_map(-2, start/chunk_size, raid_disks, + level, layout); + if (memcmp(q, stripes[diskQ], chunk_size) != 0) { + printf("Q(%d) wrong at %llu\n", diskQ, + start / chunk_size); + } + raid6_collect(chunk_size, p, q, + stripes[diskP], stripes[diskQ], results); + disk = raid6_stats(results, raid_disks, chunk_size); + + if(disk >= -2) { + disk = geo_map(disk, start/chunk_size, raid_disks, + level, layout); + } + if(disk >= 0) { + printf("Possible failed disk: %d --> %s\n", disk, name[disk]); + } + if(disk == -65535) { + printf("Failure detected, but disk unknown\n"); + } + + length -= chunk_size; + start += chunk_size; + } + + free(stripe_buf); + free(stripes); + free(blocks); + free(p); + free(q); + free(results); + + return 0; +} + +unsigned long long getnum(char *str, char **err) +{ + char *e; + unsigned long long rv = strtoull(str, &e, 10); + if (e==str || *e) { + *err = str; + return 0; + } + return rv; +} + +int main(int argc, char *argv[]) +{ + /* raid_disks chunk_size layout start length devices... + */ + int *fds; + char *buf; + unsigned long long *offsets; + int raid_disks, chunk_size, layout; + int level = 6; + unsigned long long start, length; + int i; + + char *err = NULL; + if (argc < 8) { + fprintf(stderr, "Usage: raid6check raid_disks" + " chunk_size layout start length devices...\n"); + exit(1); + } + + raid_disks = getnum(argv[1], &err); + chunk_size = getnum(argv[2], &err); + layout = getnum(argv[3], &err); + start = getnum(argv[4], &err); + length = getnum(argv[5], &err); + if (err) { + fprintf(stderr, "test_stripe: Bad number: %s\n", err); + exit(2); + } + if (argc != raid_disks + 6) { + fprintf(stderr, "test_stripe: wrong number of devices: want %d found %d\n", + raid_disks, argc-6); + exit(2); + } + fds = malloc(raid_disks * sizeof(*fds)); + offsets = malloc(raid_disks * sizeof(*offsets)); + memset(offsets, 0, raid_disks * sizeof(*offsets)); + + for (i=0; i<raid_disks; i++) { + char *p; + p = strchr(argv[6+i], ':'); + + if(p != NULL) { + *p++ = '\0'; + offsets[i] = atoll(p) * 512; + } + fds[i] = open(argv[6+i], O_RDWR); + if (fds[i] < 0) { + perror(argv[6+i]); + fprintf(stderr,"test_stripe: cannot open %s.\n", argv[6+i]); + exit(3); + } + } + + buf = malloc(raid_disks * chunk_size); + + int rv = check_stripes(fds, offsets, + raid_disks, chunk_size, level, layout, + start, length, &argv[6]); + if (rv != 0) { + fprintf(stderr, + "test_stripe: test_stripes returned %d\n", rv); + exit(1); + } + + free(fds); + free(offsets); + free(buf); + + exit(0); +} +diff -urN a/restripe.c b/restripe.c--- a/restripe.c 2011-02-19 22:36:29.707651642 +0100 +++ b/restripe.c 2011-02-19 18:13:14.468626986 +0100@@ -33,7 +33,7 @@ * */ -static int geo_map(int block, unsigned long long stripe, int raid_disks, +int geo_map(int block, unsigned long long stripe, int raid_disks, int level, int layout) { /* On the given stripe, find which disk in the array will have@@ -223,7 +223,7 @@ } } -static void qsyndrome(uint8_t *p, uint8_t *q, uint8_t **sources, int disks, int size) +void qsyndrome(uint8_t *p, uint8_t *q, uint8_t **sources, int disks, int size) { int d, z; uint8_t wq0, wp0, wd0, w10, w20;-- To unsubscribe from this list: send the line "unsubscribe linux-raid" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
-- piergiorgio