raidd5:Only move IO_THRESHOLD stripes from delay_list to hold_list once.

Started by majianpeng on Jul 13, 2012 · 5 messages from 3 authors · Last reply Jul 16, 2012

Subsystems (2)

Maintainers (3)

STALE5241d

open the first message on its own page

Message details and related content
To improve write perfomance by decreasing the preread stripe,only move
IO_THRESHOLD stripes from delay_list to hold_list once.

Using the follow command:
dd if=/dev/zero of=/dev/md0 bs=2M count=52100.

At default condition: speed is 95MB/s.
At the condition of preread_bypass_threshold was equal zero:speed is 105MB/s.
Using this patch:speed is 123MB/s.

If preread_bypass_threshold was zero,the performance will be better,but
not better than this patch.
I think maybe two reason:
1:If bio is REQ_SYNC
2:In function __get_priority_stripe():
quoted
} else if (!list_empty(&conf->hold_list) &&
	   ((conf->bypass_threshold &&
	     conf->bypass_count > conf->bypass_threshold) ||
	    atomic_read(&conf->pending_full_writes) == 0)) {
Preread_bypass_threshold is one condition of getting stripe from
hold_list.So only control the number of hold_list can get better
performance.

Signed-off-by: Jianpeng Ma <redacted>
---
 drivers/md/raid5.c |    3 +++
 1 files changed, 3 insertions(+), 0 deletions(-)
diff --git a/drivers/md/raid5.c b/drivers/md/raid5.c
index 04348d7..a6749bb 100644
--- a/drivers/md/raid5.c
+++ b/drivers/md/raid5.c
@@ -3662,6 +3662,7 @@ finish:
 
 static void raid5_activate_delayed(struct r5conf *conf)
 {
+	int count = 0;
 	if (atomic_read(&conf->preread_active_stripes) < IO_THRESHOLD) {
 		while (!list_empty(&conf->delayed_list)) {
 			struct list_head *l = conf->delayed_list.next;
@@ -3672,6 +3673,8 @@ static void raid5_activate_delayed(struct r5conf *conf)
 			if (!test_and_set_bit(STRIPE_PREREAD_ACTIVE, &sh->state))
 				atomic_inc(&conf->preread_active_stripes);
 			list_add_tail(&sh->lru, &conf->hold_list);
+			if (++count >= IO_THRESHOLD)
+				break;
 		}
 	}
 }
-- 
1.7.5.4
Message details and related content
[ adding Shaohua ]

On Fri, Jul 13, 2012 at 3:31 AM, majianpeng [off-list ref] wrote:
To improve write perfomance by decreasing the preread stripe,only move
IO_THRESHOLD stripes from delay_list to hold_list once.

Using the follow command:
dd if=/dev/zero of=/dev/md0 bs=2M count=52100.

At default condition: speed is 95MB/s.
At the condition of preread_bypass_threshold was equal zero:speed is 105MB/s.
Using this patch:speed is 123MB/s.

If preread_bypass_threshold was zero,the performance will be better,but
not better than this patch.
I think maybe two reason:
1:If bio is REQ_SYNC
2:In function __get_priority_stripe():
quoted
quoted
} else if (!list_empty(&conf->hold_list) &&
                ((conf->bypass_threshold &&
                  conf->bypass_count > conf->bypass_threshold) ||
                 atomic_read(&conf->pending_full_writes) == 0)) {
Preread_bypass_threshold is one condition of getting stripe from
hold_list.So only control the number of hold_list can get better
performance.
So this is a pretty obvious tradeoff of increased latency for improved
throughput.  Any idea how much this change affects latency?
Especially in the fast device case?
Message details and related content
On 2012-07-14 07:56 Dan Williams [off-list ref] Wrote:
[ adding Shaohua ]

On Fri, Jul 13, 2012 at 3:31 AM, majianpeng [off-list ref] wrote:
quoted
To improve write perfomance by decreasing the preread stripe,only move
IO_THRESHOLD stripes from delay_list to hold_list once.

Using the follow command:
dd if=/dev/zero of=/dev/md0 bs=2M count=52100.

At default condition: speed is 95MB/s.
At the condition of preread_bypass_threshold was equal zero:speed is 105MB/s.
Using this patch:speed is 123MB/s.

If preread_bypass_threshold was zero,the performance will be better,but
not better than this patch.
I think maybe two reason:
1:If bio is REQ_SYNC
2:In function __get_priority_stripe():
quoted
quoted
} else if (!list_empty(&conf->hold_list) &&
                ((conf->bypass_threshold &&
                  conf->bypass_count > conf->bypass_threshold) ||
                 atomic_read(&conf->pending_full_writes) == 0)) {
Preread_bypass_threshold is one condition of getting stripe from
hold_list.So only control the number of hold_list can get better
performance.
So this is a pretty obvious tradeoff of increased latency for improved
throughput.  Any idea how much this change affects latency?
Especially in the fast device case?
I did not think the latency.If it only fetch preread_bypass_threshold stripes from delay_list to 
host_list,the latency can be control by userspace.
The code like :

static void raid5_activate_delayed(struct r5conf *conf)
 {
+ int count = 0;
  if (atomic_read(&conf->preread_active_stripes) < IO_THRESHOLD) {
  while (!list_empty(&conf->delayed_list)) {
  struct list_head *l = conf->delayed_list.next;
@@ -3672,6 +3673,8 @@ static void raid5_activate_delayed(struct r5conf *conf)
  if (!test_and_set_bit(STRIPE_PREREAD_ACTIVE, &sh->state))
  atomic_inc(&conf->preread_active_stripes);
  list_add_tail(&sh->lru, &conf->hold_list);
+ if (++count >= conf->preread_active_stripes)
+ break;
  }
 

Message details and related content
On Fri, 13 Jul 2012 18:31:11 +0800 majianpeng [off-list ref] wrote:
quoted hunk
To improve write perfomance by decreasing the preread stripe,only move
IO_THRESHOLD stripes from delay_list to hold_list once.

Using the follow command:
dd if=/dev/zero of=/dev/md0 bs=2M count=52100.

At default condition: speed is 95MB/s.
At the condition of preread_bypass_threshold was equal zero:speed is 105MB/s.
Using this patch:speed is 123MB/s.

If preread_bypass_threshold was zero,the performance will be better,but
not better than this patch.
I think maybe two reason:
1:If bio is REQ_SYNC
2:In function __get_priority_stripe():
quoted
quoted
} else if (!list_empty(&conf->hold_list) &&
	   ((conf->bypass_threshold &&
	     conf->bypass_count > conf->bypass_threshold) ||
	    atomic_read(&conf->pending_full_writes) == 0)) {
Preread_bypass_threshold is one condition of getting stripe from
hold_list.So only control the number of hold_list can get better
performance.

Signed-off-by: Jianpeng Ma <redacted>
---
 drivers/md/raid5.c |    3 +++
 1 files changed, 3 insertions(+), 0 deletions(-)
diff --git a/drivers/md/raid5.c b/drivers/md/raid5.c
index 04348d7..a6749bb 100644
--- a/drivers/md/raid5.c
+++ b/drivers/md/raid5.c
@@ -3662,6 +3662,7 @@ finish:
 
 static void raid5_activate_delayed(struct r5conf *conf)
 {
+	int count = 0;
 	if (atomic_read(&conf->preread_active_stripes) < IO_THRESHOLD) {
 		while (!list_empty(&conf->delayed_list)) {
 			struct list_head *l = conf->delayed_list.next;

9 more quoted lines


I tried this patch - against my current for-next tree - on my own modest
hardware and could not measure any difference in write throughput.

Maybe some other patch has fixed something.

However it is still reading a lot during a write-only test and that is not
ideal.  It would be nice if we could arrange that it didn't read at all.

NeilBRown
Message details and related content
On 2012-07-16 15:46 NeilBrown [off-list ref] Wrote:
On Fri, 13 Jul 2012 18:31:11 +0800 majianpeng [off-list ref] wrote:
quoted
To improve write perfomance by decreasing the preread stripe,only move
IO_THRESHOLD stripes from delay_list to hold_list once.

Using the follow command:
dd if=/dev/zero of=/dev/md0 bs=2M count=52100.

At default condition: speed is 95MB/s.
At the condition of preread_bypass_threshold was equal zero:speed is 105MB/s.
Using this patch:speed is 123MB/s.

If preread_bypass_threshold was zero,the performance will be better,but
not better than this patch.
I think maybe two reason:
1:If bio is REQ_SYNC
2:In function __get_priority_stripe():
quoted
quoted
} else if (!list_empty(&conf->hold_list) &&
	   ((conf->bypass_threshold &&
	     conf->bypass_count > conf->bypass_threshold) ||
	    atomic_read(&conf->pending_full_writes) == 0)) {
Preread_bypass_threshold is one condition of getting stripe from
hold_list.So only control the number of hold_list can get better
performance.

Signed-off-by: Jianpeng Ma <redacted>
---
 drivers/md/raid5.c |    3 +++
 1 files changed, 3 insertions(+), 0 deletions(-)
diff --git a/drivers/md/raid5.c b/drivers/md/raid5.c
index 04348d7..a6749bb 100644
--- a/drivers/md/raid5.c
+++ b/drivers/md/raid5.c
@@ -3662,6 +3662,7 @@ finish:
 
 static void raid5_activate_delayed(struct r5conf *conf)
 {
+	int count = 0;
 	if (atomic_read(&conf->preread_active_stripes) < IO_THRESHOLD) {

21 more quoted lines

By compare to kernel 2.6.18/2.6.32, there are not any reading.
So i think  it should more work to do .
NeilBRown
Keyboard shortcuts
hback out one level
jnext message in thread / next series-diff position
kprevious message in thread / previous series-diff position
ldrill in
Escclose help / fold thread tree
?toggle this help