Fwd: Re: Device utilization with RAID-1

7 messages, 6 authors, 2011-08-20 · open the first message on its own page

Fwd: Re: Device utilization with RAID-1

From: Harald Nikolisin <hidden>
Date: 2011-08-18 00:26:17 

hi,

I didn't want to complain in general about SW RAID-1 performance. I
simply think something is wrong with my setup and I have currently no
idea how to improve.

The basic questions (where I did not find an answer, neither in FAQ's
nor in forum discussions) are.
a) Is it normal that the hard drives show an permanent utilization
(around 20%) without any noticeable actions on the computer?
b) Should (as long as no resync happens) the state of mdadm active or clean?

cheers,
  harald

well, I have only 2 hard drives and no space for more..

Am 16.08.2011 03:29, schrieb Roberto Spadim:
try raid10 far layout

2011/8/15 Harald Nikolisin <hochglanz@gmail.com
<mailto:hochglanz@gmail.com>>

    Since a long time I'm unhappy with the performance of my RAID-1 system.
    Investigation with atop and iostat unveils that the disk utilization is
    always on a certain level although nothing happens on the system. In the
    case of reading or writing files the utilization boosts always to 100%
    for a long time. Very ugly examples are "Firefox starting" or "zypper
    updates".
    That is snapshot of the output of iostat:


    Device:         rrqm/s   wrqm/s     r/s     w/s   rsec/s   wsec/s
    avgrq-sz avgqu-sz   await  svctm  %util
    sda               0,00     0,00    0,00    7,33     0,00    43,33
    5,91     0,33   43,18  33,32  24,43
    sdb               0,00     0,00    0,00    7,33     0,00    43,33
    5,91     0,35   45,59  39,73  29,13
    md0               0,00     0,00    0,00    0,67     0,00     5,33
    8,00     0,00    0,00   0,00   0,00
    md1               0,00     0,00    0,00    0,33     0,00     5,33
    16,00     0,00    0,00   0,00   0,00
    md2               0,00     0,00    0,00    0,33     0,00     1,00
    3,00     0,00    0,00   0,00   0,00
    md3               0,00     0,00    0,00    0,00     0,00     0,00
    0,00     0,00    0,00   0,00   0,00
    md4               0,00     0,00    0,00    0,00     0,00     0,00
    0,00     0,00    0,00   0,00   0,00
    md5               0,00     0,00    0,00    0,33     0,00     0,67
    2,00     0,00    0,00   0,00   0,00

    I checked with mdadm if a resync happens or so, but this is not the
    case. The state says "active" on all RAID devices - btw. what is the
    difference to "clean" ?

    thanks for any hints,
     harald
    --
    To unsubscribe from this list: send the line "unsubscribe linux-raid" in
    the body of a message to majordomo@vger.kernel.org
    <mailto:majordomo@vger.kernel.org>
    More majordomo info at  http://vger.kernel.org/majordomo-info.html




-- 
Roberto Spadim
Spadim Technology / SPAEmpresarial





  

    
Re: Device utilization with RAID-1

    

      From: NeilBrown <hidden>

      Date: 2011-08-18 01:42:05 
    

  

  
On Thu, 18 Aug 2011 02:26:17 +0200 Harald Nikolisin [off-list ref]
wrote:

hi,

I didn't want to complain in general about SW RAID-1 performance. I
simply think something is wrong with my setup and I have currently no
idea how to improve.

The basic questions (where I did not find an answer, neither in FAQ's
nor in forum discussions) are.
a) Is it normal that the hard drives show an permanent utilization
(around 20%) without any noticeable actions on the computer?
No.  If the array is resyncing or recovering then you would expect
utilization for as many hours as it takes - but that would show
in /proc/mdstat.

b) Should (as long as no resync happens) the state of mdadm active or clean?
If anything has been written to the device in the last 200msec (including
e.g. access time updates) then expect it to be 'active'.
If nothing has been written for 200msecc or more, then expect it to be clean.

If you crash while it is active, a resync is needed.
If you crash while it is clean, no resync is needed.
If you don't crash at all .... that is best :-)

NeilBrown



cheers,
  harald

well, I have only 2 hard drives and no space for more..

Am 16.08.2011 03:29, schrieb Roberto Spadim:
quoted
try raid10 far layout

2011/8/15 Harald Nikolisin <hochglanz@gmail.com
<mailto:hochglanz@gmail.com>>

    Since a long time I'm unhappy with the performance of my RAID-1 system.
    Investigation with atop and iostat unveils that the disk utilization is
    always on a certain level although nothing happens on the system. In the
    case of reading or writing files the utilization boosts always to 100%
    for a long time. Very ugly examples are "Firefox starting" or "zypper
    updates".
    That is snapshot of the output of iostat:


    Device:         rrqm/s   wrqm/s     r/s     w/s   rsec/s   wsec/s
    avgrq-sz avgqu-sz   await  svctm  %util
    sda               0,00     0,00    0,00    7,33     0,00    43,33
    5,91     0,33   43,18  33,32  24,43
    sdb               0,00     0,00    0,00    7,33     0,00    43,33
    5,91     0,35   45,59  39,73  29,13
    md0               0,00     0,00    0,00    0,67     0,00     5,33
    8,00     0,00    0,00   0,00   0,00
    md1               0,00     0,00    0,00    0,33     0,00     5,33
    16,00     0,00    0,00   0,00   0,00
    md2               0,00     0,00    0,00    0,33     0,00     1,00
    3,00     0,00    0,00   0,00   0,00
    md3               0,00     0,00    0,00    0,00     0,00     0,00
    0,00     0,00    0,00   0,00   0,00
    md4               0,00     0,00    0,00    0,00     0,00     0,00
    0,00     0,00    0,00   0,00   0,00
    md5               0,00     0,00    0,00    0,33     0,00     0,67
    2,00     0,00    0,00   0,00   0,00

    I checked with mdadm if a resync happens or so, but this is not the
    case. The state says "active" on all RAID devices - btw. what is the
    difference to "clean" ?

    thanks for any hints,
     harald
    --
    To unsubscribe from this list: send the line "unsubscribe linux-raid" in
    the body of a message to majordomo@vger.kernel.org
    <mailto:majordomo@vger.kernel.org>
    More majordomo info at  http://vger.kernel.org/majordomo-info.html




-- 
Roberto Spadim
Spadim Technology / SPAEmpresarial

--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html





  

    
Re: Device utilization with RAID-1

    

      From: Stefan /*St0fF*/ Hübner <hidden>

      Date: 2011-08-18 06:44:29 
    

  

  
Am 18.08.2011 03:42, schrieb NeilBrown:
On Thu, 18 Aug 2011 02:26:17 +0200 Harald Nikolisin [off-list ref]
wrote:

quoted
hi,

I didn't want to complain in general about SW RAID-1 performance. I
simply think something is wrong with my setup and I have currently no
idea how to improve.

The basic questions (where I did not find an answer, neither in FAQ's
nor in forum discussions) are.
a) Is it normal that the hard drives show an permanent utilization
(around 20%) without any noticeable actions on the computer?
No.  If the array is resyncing or recovering then you would expect
utilization for as many hours as it takes - but that would show
in /proc/mdstat.

quoted
b) Should (as long as no resync happens) the state of mdadm active or clean?
If anything has been written to the device in the last 200msec (including
e.g. access time updates) then expect it to be 'active'.
If nothing has been written for 200msecc or more, then expect it to be clean.

If you crash while it is active, a resync is needed.
If you crash while it is clean, no resync is needed.
If you don't crash at all .... that is best :-)

NeilBrown
I second that ;)  Have you checked the SMART-Attributes of your disks,
are they still OK?  But if they weren't, you wouldn't see that they're a
bit more busy, you'd only feel it from bad performance.

Indeed I think you need to find out which processes create your I/O
load, as it seems to be kind of a badly configured service/daemon which
slows down your whole computer that way... It's probably a good idea to
start with dstat and a wide screen :)

Stefan

quoted
cheers,
  harald

well, I have only 2 hard drives and no space for more..

Am 16.08.2011 03:29, schrieb Roberto Spadim:
quoted
try raid10 far layout

2011/8/15 Harald Nikolisin <hochglanz@gmail.com
<mailto:hochglanz@gmail.com>>

    Since a long time I'm unhappy with the performance of my RAID-1 system.
    Investigation with atop and iostat unveils that the disk utilization is
    always on a certain level although nothing happens on the system. In the
    case of reading or writing files the utilization boosts always to 100%
    for a long time. Very ugly examples are "Firefox starting" or "zypper
    updates".
    That is snapshot of the output of iostat:


    Device:         rrqm/s   wrqm/s     r/s     w/s   rsec/s   wsec/s
    avgrq-sz avgqu-sz   await  svctm  %util
    sda               0,00     0,00    0,00    7,33     0,00    43,33
    5,91     0,33   43,18  33,32  24,43
    sdb               0,00     0,00    0,00    7,33     0,00    43,33
    5,91     0,35   45,59  39,73  29,13
    md0               0,00     0,00    0,00    0,67     0,00     5,33
    8,00     0,00    0,00   0,00   0,00
    md1               0,00     0,00    0,00    0,33     0,00     5,33
    16,00     0,00    0,00   0,00   0,00
    md2               0,00     0,00    0,00    0,33     0,00     1,00
    3,00     0,00    0,00   0,00   0,00
    md3               0,00     0,00    0,00    0,00     0,00     0,00
    0,00     0,00    0,00   0,00   0,00
    md4               0,00     0,00    0,00    0,00     0,00     0,00
    0,00     0,00    0,00   0,00   0,00
    md5               0,00     0,00    0,00    0,33     0,00     0,67
    2,00     0,00    0,00   0,00   0,00

    I checked with mdadm if a resync happens or so, but this is not the
    case. The state says "active" on all RAID devices - btw. what is the
    difference to "clean" ?

    thanks for any hints,
     harald
    --
    To unsubscribe from this list: send the line "unsubscribe linux-raid" in
    the body of a message to majordomo@vger.kernel.org
    <mailto:majordomo@vger.kernel.org>
    More majordomo info at  http://vger.kernel.org/majordomo-info.html




-- 
Roberto Spadim
Spadim Technology / SPAEmpresarial

--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html





  

    
Re: Device utilization with RAID-1

    

      From: CoolCold <hidden>

      Date: 2011-08-18 13:44:19 
    

  

  
On Thu, Aug 18, 2011 at 5:42 AM, NeilBrown [off-list ref] wrote:
On Thu, 18 Aug 2011 02:26:17 +0200 Harald Nikolisin [off-list ref]
wrote:

quoted
hi,

I didn't want to complain in general about SW RAID-1 performance. I
simply think something is wrong with my setup and I have currently no
idea how to improve.

The basic questions (where I did not find an answer, neither in FAQ's
nor in forum discussions) are.
a) Is it normal that the hard drives show an permanent utilization
(around 20%) without any noticeable actions on the computer?
No.  If the array is resyncing or recovering then you would expect
utilization for as many hours as it takes - but that would show
in /proc/mdstat.

quoted
b) Should (as long as no resync happens) the state of mdadm active or clean?
If anything has been written to the device in the last 200msec (including
e.g. access time updates) then expect it to be 'active'.
If nothing has been written for 200msecc or more, then expect it to be clean.

If you crash while it is active, a resync is needed.
If you crash while it is clean, no resync is needed.
If you don't crash at all .... that is best :-)
I think this info should be wikified if not yet.

btw, I've experimented a bit on my /boot array (it doesn't being
updated, checked with iostat ), and:
root@m2:~# for i in {1..5};do mdname="md0"; echo "iteration $i";
(mdadm --detail /dev/$mdname|grep 'State ';cat
/sys/block/$mdname/md/array_state;grep "$mdname :" /proc/mdstat);sleep
1;done
iteration 1
          State : clean
clean
md0 : active raid1 sda1[0] sdb1[1]
iteration 2
          State : clean
clean
md0 : active raid1 sda1[0] sdb1[1]
iteration 3
          State : clean
clean
md0 : active raid1 sda1[0] sdb1[1]
iteration 4
          State : clean
clean
md0 : active raid1 sda1[0] sdb1[1]
iteration 5
          State : clean
clean
md0 : active raid1 sda1[0] sdb1[1]

so, mdadm --detail & array_state shows array is "clean", while
/proc/mdstat shows array is "active" (no reads/writes happen).

Some value is lieing or being misunderdstanded by me...



quoted
cheers,
  harald

well, I have only 2 hard drives and no space for more..

Am 16.08.2011 03:29, schrieb Roberto Spadim:
quoted
try raid10 far layout

2011/8/15 Harald Nikolisin <hochglanz@gmail.com
<mailto:hochglanz@gmail.com>>

    Since a long time I'm unhappy with the performance of my RAID-1 system.
    Investigation with atop and iostat unveils that the disk utilization is
    always on a certain level although nothing happens on the system. In the
    case of reading or writing files the utilization boosts always to 100%
    for a long time. Very ugly examples are "Firefox starting" or "zypper
    updates".
    That is snapshot of the output of iostat:


    Device:         rrqm/s   wrqm/s     r/s     w/s   rsec/s   wsec/s
    avgrq-sz avgqu-sz   await  svctm  %util
    sda               0,00     0,00    0,00    7,33     0,00    43,33
    5,91     0,33   43,18  33,32  24,43
    sdb               0,00     0,00    0,00    7,33     0,00    43,33
    5,91     0,35   45,59  39,73  29,13
    md0               0,00     0,00    0,00    0,67     0,00     5,33
    8,00     0,00    0,00   0,00   0,00
    md1               0,00     0,00    0,00    0,33     0,00     5,33
    16,00     0,00    0,00   0,00   0,00
    md2               0,00     0,00    0,00    0,33     0,00     1,00
    3,00     0,00    0,00   0,00   0,00
    md3               0,00     0,00    0,00    0,00     0,00     0,00
    0,00     0,00    0,00   0,00   0,00
    md4               0,00     0,00    0,00    0,00     0,00     0,00
    0,00     0,00    0,00   0,00   0,00
    md5               0,00     0,00    0,00    0,33     0,00     0,67
    2,00     0,00    0,00   0,00   0,00

    I checked with mdadm if a resync happens or so, but this is not the
    case. The state says "active" on all RAID devices - btw. what is the
    difference to "clean" ?

    thanks for any hints,
     harald
    --
    To unsubscribe from this list: send the line "unsubscribe linux-raid" in
    the body of a message to majordomo@vger.kernel.org
    <mailto:majordomo@vger.kernel.org>
    More majordomo info at  http://vger.kernel.org/majordomo-info.html




--
Roberto Spadim
Spadim Technology / SPAEmpresarial

--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


-- 
Best regards,
[COOLCOLD-RIPN]
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html





  

    
Re: Device utilization with RAID-1

    

      From: NeilBrown <hidden>

      Date: 2011-08-19 00:31:35 
    

  

  
On Thu, 18 Aug 2011 17:44:19 +0400 CoolCold [off-list ref] wrote:

On Thu, Aug 18, 2011 at 5:42 AM, NeilBrown [off-list ref] wrote:
quoted
On Thu, 18 Aug 2011 02:26:17 +0200 Harald Nikolisin [off-list ref]
wrote:

quoted
hi,

I didn't want to complain in general about SW RAID-1 performance. I
simply think something is wrong with my setup and I have currently no
idea how to improve.

The basic questions (where I did not find an answer, neither in FAQ's
nor in forum discussions) are.
a) Is it normal that the hard drives show an permanent utilization
(around 20%) without any noticeable actions on the computer?
No.  If the array is resyncing or recovering then you would expect
utilization for as many hours as it takes - but that would show
in /proc/mdstat.

quoted
b) Should (as long as no resync happens) the state of mdadm active or clean?
If anything has been written to the device in the last 200msec (including
e.g. access time updates) then expect it to be 'active'.
If nothing has been written for 200msecc or more, then expect it to be clean.

If you crash while it is active, a resync is needed.
If you crash while it is clean, no resync is needed.
If you don't crash at all .... that is best :-)
I think this info should be wikified if not yet.

btw, I've experimented a bit on my /boot array (it doesn't being
updated, checked with iostat ), and:
root@m2:~# for i in {1..5};do mdname="md0"; echo "iteration $i";
(mdadm --detail /dev/$mdname|grep 'State ';cat
/sys/block/$mdname/md/array_state;grep "$mdname :" /proc/mdstat);sleep
1;done
iteration 1
          State : clean
clean
md0 : active raid1 sda1[0] sdb1[1]
iteration 2
          State : clean
clean
md0 : active raid1 sda1[0] sdb1[1]
iteration 3
          State : clean
clean
md0 : active raid1 sda1[0] sdb1[1]
iteration 4
          State : clean
clean
md0 : active raid1 sda1[0] sdb1[1]
iteration 5
          State : clean
clean
md0 : active raid1 sda1[0] sdb1[1]

so, mdadm --detail & array_state shows array is "clean", while
/proc/mdstat shows array is "active" (no reads/writes happen).

Some value is lieing or being misunderdstanded by me...
In mdstat you have 'active' or 'inactive'.  You cannot access an array at all
until it is active.  If you are assembling an array bit by bit with "mdadm
-I", it will be inactive until all the devices appear.  Then it will be
active.

In mdadm "State :" you have 'active' or 'clean'. as described above.  It used
to be 'dirty' or 'clean' but people were confused by having 'dirty' arrays in
normal operation.  So I changed it to 'active' and now it confuses a
different set of people.  You just can't win can you :-)

NeilBrown


quoted


quoted
cheers,
  harald

well, I have only 2 hard drives and no space for more..

Am 16.08.2011 03:29, schrieb Roberto Spadim:
quoted
try raid10 far layout

2011/8/15 Harald Nikolisin <hochglanz@gmail.com
<mailto:hochglanz@gmail.com>>

    Since a long time I'm unhappy with the performance of my RAID-1 system.
    Investigation with atop and iostat unveils that the disk utilization is
    always on a certain level although nothing happens on the system. In the
    case of reading or writing files the utilization boosts always to 100%
    for a long time. Very ugly examples are "Firefox starting" or "zypper
    updates".
    That is snapshot of the output of iostat:


    Device:         rrqm/s   wrqm/s     r/s     w/s   rsec/s   wsec/s
    avgrq-sz avgqu-sz   await  svctm  %util
    sda               0,00     0,00    0,00    7,33     0,00    43,33
    5,91     0,33   43,18  33,32  24,43
    sdb               0,00     0,00    0,00    7,33     0,00    43,33
    5,91     0,35   45,59  39,73  29,13
    md0               0,00     0,00    0,00    0,67     0,00     5,33
    8,00     0,00    0,00   0,00   0,00
    md1               0,00     0,00    0,00    0,33     0,00     5,33
    16,00     0,00    0,00   0,00   0,00
    md2               0,00     0,00    0,00    0,33     0,00     1,00
    3,00     0,00    0,00   0,00   0,00
    md3               0,00     0,00    0,00    0,00     0,00     0,00
    0,00     0,00    0,00   0,00   0,00
    md4               0,00     0,00    0,00    0,00     0,00     0,00
    0,00     0,00    0,00   0,00   0,00
    md5               0,00     0,00    0,00    0,33     0,00     0,67
    2,00     0,00    0,00   0,00   0,00

    I checked with mdadm if a resync happens or so, but this is not the
    case. The state says "active" on all RAID devices - btw. what is the
    difference to "clean" ?

    thanks for any hints,
     harald
    --
    To unsubscribe from this list: send the line "unsubscribe linux-raid" in
    the body of a message to majordomo@vger.kernel.org
    <mailto:majordomo@vger.kernel.org>
    More majordomo info at  http://vger.kernel.org/majordomo-info.html




--
Roberto Spadim
Spadim Technology / SPAEmpresarial

--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html





  

    
Re: Device utilization with RAID-1

    

      From: maurice <hidden>

      Date: 2011-08-19 18:32:37 
    

  

  
On 8/18/2011 6:31 PM, NeilBrown wrote:
..
In mdstat you have 'active' or 'inactive'.  You cannot access an array at all
until it is active.  If you are assembling an array bit by bit with "mdadm
-I", it will be inactive until all the devices appear.  Then it will be
active.

In mdadm "State :" you have 'active' or 'clean'. as described above.  It used
to be 'dirty' or 'clean' but people were confused by having 'dirty' arrays in
normal operation.  So I changed it to 'active' and now it confuses a
different set of people.  You just can't win can you :-)

NeilBrown
mdstat:
"Enabled" or "Disabled" perhaps?

That matches what most commercial hardware RAID interfaces use.

-- 
Cheers,
Maurice Hilarius
eMail: /mhilarius@gmail.com/





  

    
Re: Device utilization with RAID-1

    

      From: John Robinson <hidden>

      Date: 2011-08-20 03:13:03 
    

  

  
On 19/08/2011 19:32, maurice wrote:
On 8/18/2011 6:31 PM, NeilBrown wrote:
quoted
..
In mdstat you have 'active' or 'inactive'. You cannot access an array
at all
until it is active. If you are assembling an array bit by bit with "mdadm
-I", it will be inactive until all the devices appear. Then it will be
active.

In mdadm "State :" you have 'active' or 'clean'. as described above.
It used
to be 'dirty' or 'clean' but people were confused by having 'dirty'
arrays in
normal operation. So I changed it to 'active' and now it confuses a
different set of people. You just can't win can you :-)

NeilBrown
mdstat:
"Enabled" or "Disabled" perhaps?

That matches what most commercial hardware RAID interfaces use.
Does it? It sounds more like an administrative action than a current 
status. I would have thought "online" or "offline" - unless that means 
something else somewhere else.

And for mdadm state: how about "busy" and "idle"? Hmm maybe just "busy" 
instead of "active" or "dirty"; we don't want to start an array with 
--assume-idle...

Cheers,

John.





    
  

  
    

      

        Keyboard shortcuts
      

      

        
          
hback out one level

          
jnext message in thread

          
kprevious message in thread

          
ldrill in

          
Escclose help / fold thread tree

          
?toggle this help