From: Yu Kuai <redacted>
Changes in v2:
- rebase with latest md-next
- remove some follow up cleanup patches, these patches will be sent
later after this patchset.
After previous four patchset of preparatory work, this patchset impelement
a new version of mddev_suspend(), the new apis:
- reconfig_mutex is not required;
- the weird logical that suspend array hold 'reconfig_mutex' for
mddev_check_recovery() to update superblock is not needed;
- the special handling, 'pers->prepare_suspend', for raid456 is not
needed;
- It's safe to be called at any time once mddev is allocated, and it's
designed to be used from slow path where array configuration is changed;
And use the new api to replace:
mddev_lock
mddev_suspend or not
// array reconfiguration
mddev_resume or not
mddev_unlock
With:
mddev_suspend
mddev_lock
// array reconfiguration
mddev_unlock
mddev_resume
However, the above change is not possible for raid5 and raid-cluster in
some corner cases, and mddev_suspend/resume() is replaced with quiesce()
callback, which will suspend the array as well.
This patchset is tested in my VM with mdadm testsuite with loop device
except for 10ddf tests(they always fail before this patchset).
A lot of cleanups will be started after this patchset.
Yu Kuai (28):
md: use READ_ONCE/WRITE_ONCE for 'suspend_lo' and 'suspend_hi'
md: use 'mddev->suspended' for is_md_suspended()
md: add new helpers to suspend/resume array
md: add new helpers to suspend/resume and lock/unlock array
md: use new apis to suspend array for suspend_lo/hi_store()
md: use new apis to suspend array for level_store()
md: use new apis to suspend array for serialize_policy_store()
md/dm-raid: use new apis to suspend array
md/md-bitmap: use new apis to suspend array for location_store()
md/raid5-cache: use READ_ONCE/WRITE_ONCE for 'conf->log'
md/raid5-cache: use new apis to suspend array for
r5c_disable_writeback_async()
md/raid5-cache: use new apis to suspend array for
r5c_journal_mode_store()
md/raid5: use new apis to suspend array for raid5_store_stripe_size()
md/raid5: use new apis to suspend array for raid5_store_skip_copy()
md/raid5: use new apis to suspend array for
raid5_store_group_thread_cnt()
md/raid5: use new apis to suspend array for
raid5_change_consistency_policy()
md/raid5: replace suspend with quiesce() callback
md: quiesce before md_kick_rdev_from_array() for md-cluster
md: use new apis to suspend array for ioctls involed array
reconfiguration
md: use new apis to suspend array for adding/removing rdev from
state_store()
md: use new apis to suspend array for bind_rdev_to_array()
md: use new apis to suspend array related to serial pool in
state_store()
md: use new apis to suspend array in backlog_store()
md: suspend array in md_start_sync() if array need reconfiguration
md: cleanup mddev_create/destroy_serial_pool()
md/md-linear: cleanup linear_add()
md: remove old apis to suspend the array
md: rename __mddev_suspend/resume() back to mddev_suspend/resume()
drivers/md/dm-raid.c | 8 +-
drivers/md/md-autodetect.c | 4 +-
drivers/md/md-bitmap.c | 18 ++-
drivers/md/md-linear.c | 2 -
drivers/md/md.c | 250 ++++++++++++++++++++++---------------
drivers/md/md.h | 52 ++++++--
drivers/md/raid5-cache.c | 61 +++++----
drivers/md/raid5.c | 56 ++++-----
8 files changed, 253 insertions(+), 198 deletions(-)
--
2.39.2
From: Yu Kuai <redacted>
'pers->prepare_suspend' is introduced to prevent a deadlock for raid456,
this change prepares to clean this up in later patches while refactoring
mddev_suspend(). Specifically allow reshape to make progress while
waiting for 'active_io' to be 0.
Signed-off-by: Yu Kuai <redacted>
---
drivers/md/md.c | 2 +-
drivers/md/md.h | 2 +-
2 files changed, 2 insertions(+), 2 deletions(-)
From: Yu Kuai <redacted>
Convert to use new apis, the old apis will be removed eventually.
This is not hot path, so performance is not concerned.
Signed-off-by: Yu Kuai <redacted>
---
drivers/md/md.c | 6 ++----
1 file changed, 2 insertions(+), 4 deletions(-)
From: Yu Kuai <redacted>
Convert to use new apis, the old apis will be removed eventually.
This is not hot path, so performance is not concerned.
Signed-off-by: Yu Kuai <redacted>
---
drivers/md/md.c | 6 ++----
1 file changed, 2 insertions(+), 4 deletions(-)
@@ -4090,7 +4090,6 @@ level_store(struct mddev *mddev, const char *buf, size_t len)}/* Looks like we have a winner */-mddev_suspend(mddev);mddev_detach(mddev);spin_lock(&mddev->lock);
From: Yu Kuai <redacted>
Advantages for new apis:
- reconfig_mutex is not required;
- the weird logical that suspend array hold 'reconfig_mutex' for
mddev_check_recovery() to update superblock is not needed;
- the specail handling, 'pers->prepare_suspend', for raid456 is not
needed;
- It's safe to be called at any time once mddev is allocated, and it's
designed to be used from slow path where array configuration is changed;
Signed-off-by: Yu Kuai <redacted>
---
drivers/md/md.c | 85 +++++++++++++++++++++++++++++++++++++++++++++++--
drivers/md/md.h | 3 ++
2 files changed, 86 insertions(+), 2 deletions(-)
@@ -443,12 +443,22 @@ void mddev_suspend(struct mddev *mddev)lockdep_is_held(&mddev->reconfig_mutex));WARN_ON_ONCE(thread&¤t==thread->tsk);-if(mddev->suspended++)++/* can't concurrent with __mddev_suspend() and __mddev_resume() */+mutex_lock(&mddev->suspend_mutex);+if(mddev->suspended++){+mutex_unlock(&mddev->suspend_mutex);return;+}+wake_up(&mddev->sb_wait);set_bit(MD_ALLOW_SB_UPDATE,&mddev->flags);percpu_ref_kill(&mddev->active_io);+/*+*TODO:cleanup'pers->prepare_suspendafterallcallersarereplaced+*by__mddev_suspend().+*/if(mddev->pers&&mddev->pers->prepare_suspend)mddev->pers->prepare_suspend(mddev);
@@ -459,14 +469,21 @@ void mddev_suspend(struct mddev *mddev)del_timer_sync(&mddev->safemode_timer);/* restrict memory reclaim I/O during raid array is suspend */mddev->noio_flag=memalloc_noio_save();++mutex_unlock(&mddev->suspend_mutex);}EXPORT_SYMBOL_GPL(mddev_suspend);voidmddev_resume(structmddev*mddev){lockdep_assert_held(&mddev->reconfig_mutex);-if(--mddev->suspended)++/* can't concurrent with __mddev_suspend() and __mddev_resume() */+mutex_lock(&mddev->suspend_mutex);+if(--mddev->suspended){+mutex_unlock(&mddev->suspend_mutex);return;+}/* entred the memalloc scope from mddev_suspend() */memalloc_noio_restore(mddev->noio_flag);
@@ -477,9 +494,72 @@ void mddev_resume(struct mddev *mddev)set_bit(MD_RECOVERY_NEEDED,&mddev->recovery);md_wakeup_thread(mddev->thread);md_wakeup_thread(mddev->sync_thread);/* possibly kick off a reshape */++mutex_unlock(&mddev->suspend_mutex);}EXPORT_SYMBOL_GPL(mddev_resume);+void__mddev_suspend(structmddev*mddev)+{++/*+*holdreconfig_mutextowaitfornormaliowilldeadlock,because+*othercontextcan'tupdatesuper_block,andnormaliocanrelyon+*updatingsuper_block.+*/+lockdep_assert_not_held(&mddev->reconfig_mutex);++mutex_lock(&mddev->suspend_mutex);++if(mddev->suspended){+WRITE_ONCE(mddev->suspended,mddev->suspended+1);+mutex_unlock(&mddev->suspend_mutex);+return;+}++percpu_ref_kill(&mddev->active_io);+wait_event(mddev->sb_wait,percpu_ref_is_zero(&mddev->active_io));++/*+*Forraid456,iomightbewaitingforreshapetomakeprogress,+*allownewreshapetostartwhilewaitingforiotobedoneto+*preventdeadlock.+*/+WRITE_ONCE(mddev->suspended,mddev->suspended+1);++del_timer_sync(&mddev->safemode_timer);+/* restrict memory reclaim I/O during raid array is suspend */+mddev->noio_flag=memalloc_noio_save();++mutex_unlock(&mddev->suspend_mutex);+}+EXPORT_SYMBOL_GPL(__mddev_suspend);++void__mddev_resume(structmddev*mddev)+{+lockdep_assert_not_held(&mddev->reconfig_mutex);++mutex_lock(&mddev->suspend_mutex);+WRITE_ONCE(mddev->suspended,mddev->suspended-1);+if(mddev->suspended){+mutex_unlock(&mddev->suspend_mutex);+return;+}++/* entred the memalloc scope from __mddev_suspend() */+memalloc_noio_restore(mddev->noio_flag);++percpu_ref_resurrect(&mddev->active_io);+wake_up(&mddev->sb_wait);++set_bit(MD_RECOVERY_NEEDED,&mddev->recovery);+md_wakeup_thread(mddev->thread);+md_wakeup_thread(mddev->sync_thread);/* possibly kick off a reshape */++mutex_unlock(&mddev->suspend_mutex);+}+EXPORT_SYMBOL_GPL(__mddev_resume);+/**Genericflushhandlingformd*/
@@ -667,6 +747,7 @@ int mddev_init(struct mddev *mddev)mutex_init(&mddev->open_mutex);mutex_init(&mddev->reconfig_mutex);mutex_init(&mddev->sync_mutex);+mutex_init(&mddev->suspend_mutex);mutex_init(&mddev->bitmap_info.mutex);INIT_LIST_HEAD(&mddev->disks);INIT_LIST_HEAD(&mddev->all_mddevs);
From: Yu Kuai <redacted>
Convert to use new apis, the old apis will be removed eventually.
Signed-off-by: Yu Kuai <redacted>
---
drivers/md/md.c | 18 ++++--------------
1 file changed, 4 insertions(+), 14 deletions(-)
From: Yu Kuai <redacted>
The new helpers suspend the array first and then lock the array,
Prepare to refactor from:
mddev_lock/trylock/lock_nointr
mddev_suspend
...
mddev_resuem
mddev_lock
With:
mddev_suspend_and_lock/trylock/lock_nointr
...
mddev_unlock_and_resume
After all the use cases is refactored, mddev_suspend/resume() will be
removed.
And mddev_suspend_and_lock() will also replace mddev_lock() for the case
that the array will be reconfigured, in order to synchronize with io to
prevent problems in many corner cases.
Signed-off-by: Yu Kuai <redacted>
---
drivers/md/md.h | 36 ++++++++++++++++++++++++++++++++++++
1 file changed, 36 insertions(+)
From: Yu Kuai <redacted>
Convert to use new apis, the old apis will be removed eventually.
This is not hot path, so performance is not concerned.
Signed-off-by: Yu Kuai <redacted>
---
drivers/md/md-bitmap.c | 6 ++----
1 file changed, 2 insertions(+), 4 deletions(-)
From: Yu Kuai <redacted>
Convert to use new apis, the old apis will be removed eventually.
These are not hot path, so performance is not concerned.
Signed-off-by: Yu Kuai <redacted>
---
drivers/md/dm-raid.c | 12 ++++--------
1 file changed, 4 insertions(+), 8 deletions(-)
@@ -3244,7 +3244,7 @@ static int raid_ctr(struct dm_target *ti, unsigned int argc, char **argv)set_bit(MD_RECOVERY_FROZEN,&rs->md.recovery);/* Has to be held on running the array */-mddev_lock_nointr(&rs->md);+mddev_suspend_and_lock_nointr(&rs->md);r=md_run(&rs->md);rs->md.in_sync=0;/* Assume already marked dirty */if(r){
@@ -3270,7 +3270,6 @@ static int raid_ctr(struct dm_target *ti, unsigned int argc, char **argv)}}-mddev_suspend(&rs->md);set_bit(RT_FLAG_RS_SUSPENDED,&rs->runtime_flags);/* Try to adjust the raid4/5/6 stripe cache size to the stripe size */
@@ -4014,7 +4011,7 @@ static int raid_preresume(struct dm_target *ti)}/* Check for any resize/reshape on @rs and adjust/initiate */-/* Be prepared for mddev_resume() in raid_resume() */+/* Be prepared for __mddev_resume() in raid_resume() */set_bit(MD_RECOVERY_FROZEN,&mddev->recovery);if(mddev->recovery_cp&&mddev->recovery_cp<MaxSector){set_bit(MD_RECOVERY_REQUESTED,&mddev->recovery);
From: Yu Kuai <redacted>
Convert to use new apis, the old apis will be removed eventually.
Signed-off-by: Yu Kuai <redacted>
---
drivers/md/raid5-cache.c | 12 +++++-------
1 file changed, 5 insertions(+), 7 deletions(-)
From: Yu Kuai <redacted>
Convert to use new apis, the old apis will be removed eventually.
This is not hot path, so performance is not concerned.
Signed-off-by: Yu Kuai <redacted>
---
drivers/md/raid5.c | 6 ++----
1 file changed, 2 insertions(+), 4 deletions(-)
From: Yu Kuai <redacted>
'conf->log' is set with 'reconfig_mutex' grabbed, however, readers are
not procted, hence use READ_ONCE/WRITE_ONCE to prevent reading abnormal
value.
Signed-off-by: Yu Kuai <redacted>
---
drivers/md/raid5-cache.c | 47 +++++++++++++++++++++-------------------
1 file changed, 25 insertions(+), 22 deletions(-)
@@ -1591,7 +1594,7 @@ void r5l_quiesce(struct r5l_log *log, int quiesce)boolr5l_log_disk_error(structr5conf*conf){-structr5l_log*log=conf->log;+structr5l_log*log=READ_ONCE(conf->log);/* don't allow write if journal disk is missing */if(!log)
@@ -2635,7 +2638,7 @@ int r5c_try_caching_write(struct r5conf *conf,structstripe_head_state*s,intdisks){-structr5l_log*log=conf->log;+structr5l_log*log=READ_ONCE(conf->log);inti;structr5dev*dev;intto_cache=0;
@@ -2941,7 +2944,7 @@ int r5c_cache_data(struct r5l_log *log, struct stripe_head *sh)/* check whether this big stripe is in write back cache. */boolr5c_big_stripe_cached(structr5conf*conf,sector_tsect){-structr5l_log*log=conf->log;+structr5l_log*log=READ_ONCE(conf->log);sector_ttree_index;void*slot;
@@ -3049,14 +3052,14 @@ int r5l_start(struct r5l_log *log)voidr5c_update_on_rdev_error(structmddev*mddev,structmd_rdev*rdev){structr5conf*conf=mddev->private;-structr5l_log*log=conf->log;+structr5l_log*log=READ_ONCE(conf->log);if(!log)return;if((raid5_calc_degraded(conf)>0||test_bit(Journal,&rdev->flags))&&-conf->log->r5c_journal_mode==R5C_JOURNAL_MODE_WRITE_BACK)+log->r5c_journal_mode==R5C_JOURNAL_MODE_WRITE_BACK)schedule_work(&log->disable_writeback_work);}
From: Yu Kuai <redacted>
r5c_journal_mode_set() will suspend array and it has only 2 caller, the
other caller raid_ctl() already suspend the array with new apis.
This is not hot path, so performance is not concerned.
Signed-off-by: Yu Kuai <redacted>
---
drivers/md/raid5-cache.c | 6 ++----
1 file changed, 2 insertions(+), 4 deletions(-)
From: Yu Kuai <redacted>
Convert to use new apis, the old apis will be removed eventually.
This is not hot path, so performance is not concerned.
Signed-off-by: Yu Kuai <redacted>
---
drivers/md/raid5.c | 7 ++-----
1 file changed, 2 insertions(+), 5 deletions(-)
From: Yu Kuai <redacted>
Convert to use new apis, the old apis will be removed eventually.
This is not hot path, so performance is not concerned.
Signed-off-by: Yu Kuai <redacted>
---
drivers/md/raid5.c | 6 ++----
1 file changed, 2 insertions(+), 4 deletions(-)
From: Yu Kuai <redacted>
raid5 is the only personality to suspend array in check_reshape() and
start_reshape() callback, suspend and quiesce() callback can both wait
for all normal io to be done, and prevent new io to be dispatched, the
difference is that suspend is implemented in common layer, and quiesce()
callback is implemented in raid5.
In order to cleanup all the usage of mddev_suspend(), the new apis
__mddev_suspend() need to be called before 'reconfig_mutex' is held,
and it's not good to affect all the personalities in common layer just
for raid5. Hence replace suspend with quiesce() callaback, prepare to
reomove all the users of mddev_suspend().
Signed-off-by: Yu Kuai <redacted>
---
drivers/md/raid5.c | 18 +++++++++---------
1 file changed, 9 insertions(+), 9 deletions(-)
@@ -70,6 +70,8 @@ MODULE_PARM_DESC(devices_handle_discard_safely,"Set to Y if all devices in each array reliably return zeroes on reads from discarded regions");staticstructworkqueue_struct*raid5_wq;+staticvoidraid5_quiesce(structmddev*mddev,intquiesce);+staticinlinestructhlist_head*stripe_hash(structr5conf*conf,sector_tsect){inthash=(sect>>RAID5_STRIPE_SHIFT(conf))&HASH_MASK;
@@ -2492,15 +2494,12 @@ static int resize_chunks(struct r5conf *conf, int new_disks, int new_sectors)unsignedlongcpu;interr=0;-/*-*Nevershrink.Andmddev_suspend()coulddeadlockifthisiscalled-*fromraid5d.Inthatcase,scribble_disksandscribble_sectors-*shouldequaltonew_disksandnew_sectors-*/+/* Never shrink. */if(conf->scribble_disks>=new_disks&&conf->scribble_sectors>=new_sectors)return0;-mddev_suspend(conf->mddev);++raid5_quiesce(conf->mddev,true);cpus_read_lock();for_each_present_cpu(cpu){
@@ -2514,7 +2513,8 @@ static int resize_chunks(struct r5conf *conf, int new_disks, int new_sectors)}cpus_read_unlock();-mddev_resume(conf->mddev);+raid5_quiesce(conf->mddev,false);+if(!err){conf->scribble_disks=new_disks;conf->scribble_sectors=new_sectors;
@@ -8551,8 +8551,8 @@ static int raid5_start_reshape(struct mddev *mddev)*thereshapewasn'trunning-likeDiscardorRead-have*completed.*/-mddev_suspend(mddev);-mddev_resume(mddev);+raid5_quiesce(mddev,true);+raid5_quiesce(mddev,false);/* Add some new drives, as many as will fit.*Weknowthereareenoughtomakethenewlysizedarraywork.
From: Yu Kuai <redacted>
User can write 'remove' and 're-add' to trigger array reconfiguration
through sysfs, suspend array in this case so that io won't concurrent
with array reconfiguration.
Signed-off-by: Yu Kuai <redacted>
---
drivers/md/md.c | 18 ++++++++++++------
1 file changed, 12 insertions(+), 6 deletions(-)
From: Yu Kuai <redacted>
Convert to use new apis, the old apis will be removed eventually.
This is not hot path, so performance is not concerned.
Signed-off-by: Yu Kuai <redacted>
---
drivers/md/raid5.c | 19 ++++++-------------
1 file changed, 6 insertions(+), 13 deletions(-)
@@ -9007,11 +9002,9 @@ static int raid5_change_consistency_policy(struct mddev *mddev, const char *buf)break;}-if(!journal_dev_exists){-mddev_suspend(mddev);+if(!journal_dev_exists)clear_bit(MD_HAS_JOURNAL,&mddev->flags);-mddev_resume(mddev);-}else/* need remove journal device first */+else/* need remove journal device first */err=-EBUSY;}elseerr=-EINVAL;
From: Yu Kuai <redacted>
'reconfig_mutex' will be grabbed before these ioctls, suspend array
before holding the lock, so that io won't concurrent with array
reconfiguration through ioctls.
This is not hot path, so performance is not concerned.
Signed-off-by: Yu Kuai <redacted>
---
drivers/md/md.c | 29 ++++++++++++++++++++---------
1 file changed, 20 insertions(+), 9 deletions(-)
From: Yu Kuai <redacted>
mddev_create/destroy_serial_pool() will be called from state_store() if
user write 'writemostly'/'-writemostly', and mddev_suspend() will be
called later.
Prepare to remove the mddev_suspend() from
mddev_create/destroy_serial_pool().
Signed-off-by: Yu Kuai <redacted>
---
drivers/md/md.c | 8 +++++---
1 file changed, 5 insertions(+), 3 deletions(-)
From: Yu Kuai <redacted>
md_kick_rdev_from_array() can be called from md_check_recovery() and
md_reload_sb() for md-cluster, it's very complicated to use new apis to
suspend the array before holding 'reconfig_mutex' in this case.
Fortunately, md-cluster is only supported for raid1 and raid10, and they
both impelement quiesce() callback that is safe to be called from daemon
thread. Hence use quiesce() callback to prevent io concurrent with
removing rdev from the array.
Signed-off-by: Yu Kuai <redacted>
---
drivers/md/md.c | 38 ++++++++++++++++++++++++++++++++++++++
1 file changed, 38 insertions(+)
@@ -9918,6 +9937,22 @@ static void check_sb_changes(struct mddev *mddev, struct md_rdev *rdev)md_bitmap_update_sb(mddev->bitmap);}+/*+*md-clusterisusedforraid1/raid10,andtheyboth+*implementquiesce()callback.+*/+rdev_for_each(rdev2,mddev){+if(test_bit(Faulty,&rdev2->flags))+continue;+role=le16_to_cpu(sb->dev_roles[rdev2->desc_nr]);+if(test_bit(Candidate,&rdev2->flags)&&+role==MD_DISK_ROLE_FAULTY){+mddev->pers->quiesce(mddev,true);+suspended=true;+break;+}+}+/* Check for change of roles in the active devices */rdev_for_each_safe(rdev2,tmp,mddev){if(test_bit(Faulty,&rdev2->flags))
From: Yu Kuai <redacted>
mddev_create_serial_pool() will be called from bind_rdev_to_array(), and
mddev_suspend() will be called if serial pool is used.
Prepare to remove the mddev_suspend() from mddev_create_serial_pool().
Signed-off-by: Yu Kuai <redacted>
---
drivers/md/md-autodetect.c | 4 ++--
drivers/md/md.c | 14 +++++++-------
2 files changed, 9 insertions(+), 9 deletions(-)
@@ -6619,13 +6619,13 @@ static void autorun_devices(int part)if(IS_ERR(mddev))break;-if(mddev_lock(mddev))+if(mddev_suspend_and_lock(mddev))pr_warn("md: %s locked, cannot run\n",mdname(mddev));elseif(mddev->raid_disks||mddev->major_version||!list_empty(&mddev->disks)){pr_warn("md: %s already running, cannot run %pg\n",mdname(mddev),rdev0->bdev);-mddev_unlock(mddev);+mddev_unlock_and_resume(mddev);}else{pr_debug("md: created %s\n",mdname(mddev));mddev->persistent=1;
@@ -6635,7 +6635,7 @@ static void autorun_devices(int part)export_rdev(rdev,mddev);}autorun_array(mddev);-mddev_unlock(mddev);+mddev_unlock_and_resume(mddev);}/* on success, candidates will be empty, on error*itwon't...
From: Yu Kuai <redacted>
So that io won't concurrent with array reconfiguration, and it's safe to
suspend the array directly because normal io won't rely on
md_start_sync().
Signed-off-by: Yu Kuai <redacted>
---
drivers/md/md.c | 11 +++++++++++
1 file changed, 11 insertions(+)
From: Yu Kuai <redacted>
Now that caller already suspend the array, there is no need to suspend
array in liner_add().
Note that mddev_suspend/resume() is not used anymore.
Signed-off-by: Yu Kuai <redacted>
---
drivers/md/md-linear.c | 2 --
1 file changed, 2 deletions(-)
From: Yu Kuai <redacted>
mddev_create/destroy_serial_pool() will be called from backlog_store(),
and mddev_suspend() will be called later.
Prepare to remove the mddev_suspend() from
mddev_create/destroy_serial_pool().
Signed-off-by: Yu Kuai <redacted>
---
drivers/md/md-bitmap.c | 8 ++++----
1 file changed, 4 insertions(+), 4 deletions(-)
@@ -2562,16 +2562,16 @@ backlog_store(struct mddev *mddev, const char *buf, size_t len)if(!backlog&&mddev->serial_info_pool){/* serial_info_pool is not needed if backlog is zero */if(!mddev->serialize_policy)-mddev_destroy_serial_pool(mddev,NULL,false);+mddev_destroy_serial_pool(mddev,NULL,true);}elseif(backlog&&!mddev->serial_info_pool){/* serial_info_pool is needed since backlog is not zero */rdev_for_each(rdev,mddev)-mddev_create_serial_pool(mddev,rdev,false);+mddev_create_serial_pool(mddev,rdev,true);}if(old_mwb!=backlog)md_bitmap_update_sb(mddev->bitmap);-mddev_unlock(mddev);+mddev_unlock_and_resume(mddev);returnlen;}
From: Yu Kuai <redacted>
Now that mddev_suspend() and mddev_resume() is not used anywhere, remove
them, and remove 'MD_ALLOW_SB_UPDATE' and 'MD_UPDATING_SB' as well.
Signed-off-by: Yu Kuai <redacted>
---
drivers/md/md.c | 82 ++-----------------------------------------------
drivers/md/md.h | 8 -----
2 files changed, 3 insertions(+), 87 deletions(-)
@@ -418,74 +418,10 @@ static void md_submit_bio(struct bio *bio)md_handle_request(mddev,bio);}-/* mddev_suspend makes sure no new requests are submitted-*tothedevice,andthatanyrequeststhathavebeensubmitted-*arecompletelyhandled.-*Oncemddev_detach()iscalledandcompletes,themodulewillbe-*completelyunused.+/*+*Makesurenonewrequestsaresubmittedtothedevice,andanyrequeststhat+*havebeensubmittedarecompletelyhandled.*/-voidmddev_suspend(structmddev*mddev)-{-structmd_thread*thread=rcu_dereference_protected(mddev->thread,-lockdep_is_held(&mddev->reconfig_mutex));--WARN_ON_ONCE(thread&¤t==thread->tsk);--/* can't concurrent with __mddev_suspend() and __mddev_resume() */-mutex_lock(&mddev->suspend_mutex);-if(mddev->suspended++){-mutex_unlock(&mddev->suspend_mutex);-return;-}--wake_up(&mddev->sb_wait);-set_bit(MD_ALLOW_SB_UPDATE,&mddev->flags);-percpu_ref_kill(&mddev->active_io);--/*-*TODO:cleanup'pers->prepare_suspendafterallcallersarereplaced-*by__mddev_suspend().-*/-if(mddev->pers&&mddev->pers->prepare_suspend)-mddev->pers->prepare_suspend(mddev);--wait_event(mddev->sb_wait,percpu_ref_is_zero(&mddev->active_io));-clear_bit_unlock(MD_ALLOW_SB_UPDATE,&mddev->flags);-wait_event(mddev->sb_wait,!test_bit(MD_UPDATING_SB,&mddev->flags));--del_timer_sync(&mddev->safemode_timer);-/* restrict memory reclaim I/O during raid array is suspend */-mddev->noio_flag=memalloc_noio_save();--mutex_unlock(&mddev->suspend_mutex);-}-EXPORT_SYMBOL_GPL(mddev_suspend);--voidmddev_resume(structmddev*mddev)-{-lockdep_assert_held(&mddev->reconfig_mutex);--/* can't concurrent with __mddev_suspend() and __mddev_resume() */-mutex_lock(&mddev->suspend_mutex);-if(--mddev->suspended){-mutex_unlock(&mddev->suspend_mutex);-return;-}--/* entred the memalloc scope from mddev_suspend() */-memalloc_noio_restore(mddev->noio_flag);--percpu_ref_resurrect(&mddev->active_io);-wake_up(&mddev->sb_wait);--set_bit(MD_RECOVERY_NEEDED,&mddev->recovery);-md_wakeup_thread(mddev->thread);-md_wakeup_thread(mddev->sync_thread);/* possibly kick off a reshape */--mutex_unlock(&mddev->suspend_mutex);-}-EXPORT_SYMBOL_GPL(mddev_resume);-void__mddev_suspend(structmddev*mddev){
@@ -9536,18 +9472,6 @@ static void md_start_sync(struct work_struct *ws)*/voidmd_check_recovery(structmddev*mddev){-if(test_bit(MD_ALLOW_SB_UPDATE,&mddev->flags)&&mddev->sb_flags){-/* Write superblock - thread that called mddev_suspend()-*holdsreconfig_mutexforus.-*/-set_bit(MD_UPDATING_SB,&mddev->flags);-smp_mb__after_atomic();-if(test_bit(MD_ALLOW_SB_UPDATE,&mddev->flags))-md_update_sb(mddev,0);-clear_bit_unlock(MD_UPDATING_SB,&mddev->flags);-wake_up(&mddev->sb_wait);-}-if(is_md_suspended(mddev))return;
From: Yu Kuai <redacted>
Now that the old apis are removed, __mddev_suspend/resume() can be
renamed to their original names.
This is done by:
sed -i "s/__mddev_suspend/mddev_suspend/g" *.[ch]
sed -i "s/__mddev_resume/mddev_resume/g" *.[ch]
Signed-off-by: Yu Kuai <redacted>
---
drivers/md/dm-raid.c | 4 ++--
drivers/md/md.c | 30 +++++++++++++++---------------
drivers/md/md.h | 16 ++++++++--------
drivers/md/raid5-cache.c | 4 ++--
4 files changed, 27 insertions(+), 27 deletions(-)
@@ -4011,7 +4011,7 @@ static int raid_preresume(struct dm_target *ti)}/* Check for any resize/reshape on @rs and adjust/initiate */-/* Be prepared for __mddev_resume() in raid_resume() */+/* Be prepared for mddev_resume() in raid_resume() */set_bit(MD_RECOVERY_FROZEN,&mddev->recovery);if(mddev->recovery_cp&&mddev->recovery_cp<MaxSector){set_bit(MD_RECOVERY_REQUESTED,&mddev->recovery);
From: Yu Kuai <redacted>
Now that except for stopping the array, all the callers already suspend
the array, there is no need to suspend anymore, hence remove the second
parameter.
Signed-off-by: Yu Kuai <redacted>
---
drivers/md/md-bitmap.c | 8 ++++----
drivers/md/md.c | 33 ++++++++++-----------------------
drivers/md/md.h | 7 +++----
3 files changed, 17 insertions(+), 31 deletions(-)
@@ -1977,7 +1977,7 @@ int md_bitmap_load(struct mddev *mddev)gotoout;rdev_for_each(rdev,mddev)-mddev_create_serial_pool(mddev,rdev,true);+mddev_create_serial_pool(mddev,rdev);if(mddev_is_clustered(mddev))md_cluster_ops->load_bitmaps(mddev,mddev->bitmap_info.nodes);
@@ -2562,11 +2562,11 @@ backlog_store(struct mddev *mddev, const char *buf, size_t len)if(!backlog&&mddev->serial_info_pool){/* serial_info_pool is not needed if backlog is zero */if(!mddev->serialize_policy)-mddev_destroy_serial_pool(mddev,NULL,true);+mddev_destroy_serial_pool(mddev,NULL);}elseif(backlog&&!mddev->serial_info_pool){/* serial_info_pool is needed since backlog is not zero */rdev_for_each(rdev,mddev)-mddev_create_serial_pool(mddev,rdev,true);+mddev_create_serial_pool(mddev,rdev);}if(old_mwb!=backlog)md_bitmap_update_sb(mddev->bitmap);
@@ -260,8 +251,6 @@ void mddev_destroy_serial_pool(struct mddev *mddev, struct md_rdev *rdev,structmd_rdev*temp;intnum=0;/* used to track if other rdevs need the pool */-if(!is_suspend)-mddev_suspend(mddev);rdev_for_each(temp,mddev){if(!rdev){if(!mddev->serialize_policy||
On Mon, Aug 28, 2023 at 10:04 AM Yu Kuai [off-list ref] wrote:
From: Yu Kuai <redacted>
Because reading 'suspend_lo' and 'suspend_hi' from md_handle_request()
is not protected, use READ_ONCE/WRITE_ONCE to prevent reading abnormal
value.
Hi Kuai
If we don't use READ_ONCE/WRITE_ONCE, What's the risk here? Could you
explain in detail or give an example?
Regards
Xiao
On Mon, Aug 28, 2023 at 10:04 AM Yu Kuai [off-list ref] wrote:
From: Yu Kuai <redacted>
'pers->prepare_suspend' is introduced to prevent a deadlock for raid456,
this change prepares to clean this up in later patches while refactoring
mddev_suspend(). Specifically allow reshape to make progress while
waiting for 'active_io' to be 0.
Hi Kuai
From my side, I can't understand the comments. The change has
relationship with pers->prepare_suspend? And why this change can
affect reshape? If this change indeed can affect these two things, can
you explain more?
If we use mddev->suspended to judge if the raid is suspended, it
should be enough? Because mddev->suspended must be true when active_io
is dying.
Best Regards
Xiao
quoted hunk
return true;
if (bio_data_dir(bio) != WRITE)
return false;
On Mon, Aug 28, 2023 at 10:04 AM Yu Kuai [off-list ref] wrote:
quoted
From: Yu Kuai <redacted>
Because reading 'suspend_lo' and 'suspend_hi' from md_handle_request()
is not protected, use READ_ONCE/WRITE_ONCE to prevent reading abnormal
value.
Hi Kuai
If we don't use READ_ONCE/WRITE_ONCE, What's the risk here? Could you
explain in detail or give an example?
Sorry for the late reply.
That depends on the architecture, a load/store may not be atomice,
for example:
// assume a is 10
t1 write 01
// write half first
a = 11
t2 read
//read
a = 11 -> read abnormal value.
// write other half
a = 01
READ_ONCE/WRITE_ONCE can guarantee that either old value or new value is
read.
Thanks,
Kuai
On Mon, Aug 28, 2023 at 10:04 AM Yu Kuai [off-list ref] wrote:
quoted
From: Yu Kuai <redacted>
'pers->prepare_suspend' is introduced to prevent a deadlock for raid456,
this change prepares to clean this up in later patches while refactoring
mddev_suspend(). Specifically allow reshape to make progress while
waiting for 'active_io' to be 0.
Hi Kuai
quoted
From my side, I can't understand the comments. The change has
relationship with pers->prepare_suspend? And why this change can
affect reshape? If this change indeed can affect these two things, can
you explain more?
First of all, 'prepare_suspend' is used to fix a deadlock in raid456:
1) suspend is waiting for normal io to be done.
mddev_suspend
mddev->suspended++ -> new sync_thread can't start
percpu_ref_kill(active_io)
wait_event(percpu_ref_is_zero(active_io))
2) normal io is waiting for reshape to make progress.
3) reshape is waiting for suspended array to be resumed.
md_check_recovery
if (is_md_suspended(mddev))
return
Then prepare_suspend will failed the io that is waiting for reshape to
make progress:
mddev_suspend
mddev->suspended++
percpu_ref_kill(active_io)
-> new io will be stuck in md_handle_request
pers->prepare_suspend() -> raid5_prepare_suspend
-> wake_up(wait_for_overlap)
// woke up
raid5_make_request
make_stripe_request
!reshape_inprogress(mddev) && reshape_disabled(mddev)
// return io error for the io that is waiting for
// reshape to make progress
wait_event(percpu_ref_is_zero(active_io))
With this patch and the new api to suspend array:
mddev_suspend
percpu_ref_kill(active_io)
wait_event(percpu_ref_is_zero(active_io))
-> while waiting for normal io to be done, new sync_thread can still
start, and reshape can still make progress.
mddev->suspended++
If we use mddev->suspended to judge if the raid is suspended, it
should be enough? Because mddev->suspended must be true when active_io
is dying.
In the new api, active_io is killed before increasing suspended, and the
difference is that the timing that array is suspended will be delayed
from the start of mddev_suspend() to when all dispatched io is done.
I think this is OK because this doesn't change behaviour when
mddev_suspend() returns.
Thanks,
Kuai
`
Best Regards
Xiao
quoted
return true;
if (bio_data_dir(bio) != WRITE)
return false;
On Mon, Aug 28, 2023 at 10:04 AM Yu Kuai [off-list ref] wrote:
quoted hunk
From: Yu Kuai <redacted>
Advantages for new apis:
- reconfig_mutex is not required;
- the weird logical that suspend array hold 'reconfig_mutex' for
mddev_check_recovery() to update superblock is not needed;
- the specail handling, 'pers->prepare_suspend', for raid456 is not
needed;
- It's safe to be called at any time once mddev is allocated, and it's
designed to be used from slow path where array configuration is changed;
Signed-off-by: Yu Kuai <redacted>
---
drivers/md/md.c | 85 +++++++++++++++++++++++++++++++++++++++++++++++--
drivers/md/md.h | 3 ++
2 files changed, 86 insertions(+), 2 deletions(-)
@@ -443,12 +443,22 @@ void mddev_suspend(struct mddev *mddev)lockdep_is_held(&mddev->reconfig_mutex));WARN_ON_ONCE(thread&¤t==thread->tsk);-if(mddev->suspended++)++/* can't concurrent with __mddev_suspend() and __mddev_resume() */+mutex_lock(&mddev->suspend_mutex);+if(mddev->suspended++){+mutex_unlock(&mddev->suspend_mutex);return;+}+wake_up(&mddev->sb_wait);set_bit(MD_ALLOW_SB_UPDATE,&mddev->flags);percpu_ref_kill(&mddev->active_io);+/*+*TODO:cleanup'pers->prepare_suspendafterallcallersarereplaced+*by__mddev_suspend().+*/if(mddev->pers&&mddev->pers->prepare_suspend)mddev->pers->prepare_suspend(mddev);
@@ -459,14 +469,21 @@ void mddev_suspend(struct mddev *mddev)del_timer_sync(&mddev->safemode_timer);/* restrict memory reclaim I/O during raid array is suspend */mddev->noio_flag=memalloc_noio_save();++mutex_unlock(&mddev->suspend_mutex);}EXPORT_SYMBOL_GPL(mddev_suspend);voidmddev_resume(structmddev*mddev){lockdep_assert_held(&mddev->reconfig_mutex);-if(--mddev->suspended)++/* can't concurrent with __mddev_suspend() and __mddev_resume() */+mutex_lock(&mddev->suspend_mutex);+if(--mddev->suspended){+mutex_unlock(&mddev->suspend_mutex);return;+}/* entred the memalloc scope from mddev_suspend() */memalloc_noio_restore(mddev->noio_flag);
@@ -477,9 +494,72 @@ void mddev_resume(struct mddev *mddev)set_bit(MD_RECOVERY_NEEDED,&mddev->recovery);md_wakeup_thread(mddev->thread);md_wakeup_thread(mddev->sync_thread);/* possibly kick off a reshape */++mutex_unlock(&mddev->suspend_mutex);}EXPORT_SYMBOL_GPL(mddev_resume);+void__mddev_suspend(structmddev*mddev)+{++/*+*holdreconfig_mutextowaitfornormaliowilldeadlock,because+*othercontextcan'tupdatesuper_block,andnormaliocanrelyon+*updatingsuper_block.+*/+lockdep_assert_not_held(&mddev->reconfig_mutex);++mutex_lock(&mddev->suspend_mutex);++if(mddev->suspended){+WRITE_ONCE(mddev->suspended,mddev->suspended+1);+mutex_unlock(&mddev->suspend_mutex);+return;+}++percpu_ref_kill(&mddev->active_io);+wait_event(mddev->sb_wait,percpu_ref_is_zero(&mddev->active_io));++/*+*Forraid456,iomightbewaitingforreshapetomakeprogress,+*allownewreshapetostartwhilewaitingforiotobedoneto+*preventdeadlock.+*/+WRITE_ONCE(mddev->suspended,mddev->suspended+1);
It changes the order of setting suspended and checking active_io.
suspended is used to stop I/O. Now it checks active_io first and then
adds suspended, if the i/o doesn't stop, it looks like active_io can't
be 0. So it will stuck at waiting active_io to be 0?
Best Regards
Xiao
On Mon, Sep 25, 2023 at 3:21 PM Xiao Ni [off-list ref] wrote:
On Mon, Aug 28, 2023 at 10:04 AM Yu Kuai [off-list ref] wrote:
quoted
From: Yu Kuai <redacted>
Advantages for new apis:
- reconfig_mutex is not required;
- the weird logical that suspend array hold 'reconfig_mutex' for
mddev_check_recovery() to update superblock is not needed;
- the specail handling, 'pers->prepare_suspend', for raid456 is not
needed;
- It's safe to be called at any time once mddev is allocated, and it's
designed to be used from slow path where array configuration is changed;
Signed-off-by: Yu Kuai <redacted>
---
drivers/md/md.c | 85 +++++++++++++++++++++++++++++++++++++++++++++++--
drivers/md/md.h | 3 ++
2 files changed, 86 insertions(+), 2 deletions(-)
@@ -443,12 +443,22 @@ void mddev_suspend(struct mddev *mddev)lockdep_is_held(&mddev->reconfig_mutex));WARN_ON_ONCE(thread&¤t==thread->tsk);-if(mddev->suspended++)++/* can't concurrent with __mddev_suspend() and __mddev_resume() */+mutex_lock(&mddev->suspend_mutex);+if(mddev->suspended++){+mutex_unlock(&mddev->suspend_mutex);return;+}+wake_up(&mddev->sb_wait);set_bit(MD_ALLOW_SB_UPDATE,&mddev->flags);percpu_ref_kill(&mddev->active_io);+/*+*TODO:cleanup'pers->prepare_suspendafterallcallersarereplaced+*by__mddev_suspend().+*/if(mddev->pers&&mddev->pers->prepare_suspend)mddev->pers->prepare_suspend(mddev);
@@ -459,14 +469,21 @@ void mddev_suspend(struct mddev *mddev)del_timer_sync(&mddev->safemode_timer);/* restrict memory reclaim I/O during raid array is suspend */mddev->noio_flag=memalloc_noio_save();++mutex_unlock(&mddev->suspend_mutex);}EXPORT_SYMBOL_GPL(mddev_suspend);voidmddev_resume(structmddev*mddev){lockdep_assert_held(&mddev->reconfig_mutex);-if(--mddev->suspended)++/* can't concurrent with __mddev_suspend() and __mddev_resume() */+mutex_lock(&mddev->suspend_mutex);+if(--mddev->suspended){+mutex_unlock(&mddev->suspend_mutex);return;+}/* entred the memalloc scope from mddev_suspend() */memalloc_noio_restore(mddev->noio_flag);
@@ -477,9 +494,72 @@ void mddev_resume(struct mddev *mddev)set_bit(MD_RECOVERY_NEEDED,&mddev->recovery);md_wakeup_thread(mddev->thread);md_wakeup_thread(mddev->sync_thread);/* possibly kick off a reshape */++mutex_unlock(&mddev->suspend_mutex);}EXPORT_SYMBOL_GPL(mddev_resume);+void__mddev_suspend(structmddev*mddev)+{++/*+*holdreconfig_mutextowaitfornormaliowilldeadlock,because+*othercontextcan'tupdatesuper_block,andnormaliocanrelyon+*updatingsuper_block.+*/+lockdep_assert_not_held(&mddev->reconfig_mutex);++mutex_lock(&mddev->suspend_mutex);++if(mddev->suspended){+WRITE_ONCE(mddev->suspended,mddev->suspended+1);+mutex_unlock(&mddev->suspend_mutex);+return;+}++percpu_ref_kill(&mddev->active_io);+wait_event(mddev->sb_wait,percpu_ref_is_zero(&mddev->active_io));++/*+*Forraid456,iomightbewaitingforreshapetomakeprogress,+*allownewreshapetostartwhilewaitingforiotobedoneto+*preventdeadlock.+*/+WRITE_ONCE(mddev->suspended,mddev->suspended+1);
It changes the order of setting suspended and checking active_io.
suspended is used to stop I/O. Now it checks active_io first and then
adds suspended, if the i/o doesn't stop, it looks like active_io can't
be 0. So it will stuck at waiting active_io to be 0?
Ah, I c, you add the state of active_io to judge if a raid is suspended.
Regards
Xiao
From: Yu Kuai <redacted>
Changes in v2:
- rebase with latest md-next
- remove some follow up cleanup patches, these patches will be sent
later after this patchset.
After previous four patchset of preparatory work, this patchset impelement
a new version of mddev_suspend(), the new apis:
- reconfig_mutex is not required;
- the weird logical that suspend array hold 'reconfig_mutex' for
mddev_check_recovery() to update superblock is not needed;
- the special handling, 'pers->prepare_suspend', for raid456 is not
needed;
- It's safe to be called at any time once mddev is allocated, and it's
designed to be used from slow path where array configuration is changed;
And use the new api to replace:
mddev_lock
mddev_suspend or not
// array reconfiguration
mddev_resume or not
mddev_unlock
With:
mddev_suspend
mddev_lock
// array reconfiguration
mddev_unlock
mddev_resume
However, the above change is not possible for raid5 and raid-cluster in
some corner cases, and mddev_suspend/resume() is replaced with quiesce()
callback, which will suspend the array as well.
This patchset is tested in my VM with mdadm testsuite with loop device
except for 10ddf tests(they always fail before this patchset).
A lot of cleanups will be started after this patchset.
Yu Kuai (28):
md: use READ_ONCE/WRITE_ONCE for 'suspend_lo' and 'suspend_hi'
md: use 'mddev->suspended' for is_md_suspended()
md: add new helpers to suspend/resume array
md: add new helpers to suspend/resume and lock/unlock array
md: use new apis to suspend array for suspend_lo/hi_store()
md: use new apis to suspend array for level_store()
md: use new apis to suspend array for serialize_policy_store()
md/dm-raid: use new apis to suspend array
md/md-bitmap: use new apis to suspend array for location_store()
md/raid5-cache: use READ_ONCE/WRITE_ONCE for 'conf->log'
md/raid5-cache: use new apis to suspend array for
r5c_disable_writeback_async()
md/raid5-cache: use new apis to suspend array for
r5c_journal_mode_store()
md/raid5: use new apis to suspend array for raid5_store_stripe_size()
md/raid5: use new apis to suspend array for raid5_store_skip_copy()
md/raid5: use new apis to suspend array for
raid5_store_group_thread_cnt()
md/raid5: use new apis to suspend array for
raid5_change_consistency_policy()
md/raid5: replace suspend with quiesce() callback
md: quiesce before md_kick_rdev_from_array() for md-cluster
md: use new apis to suspend array for ioctls involed array
reconfiguration
md: use new apis to suspend array for adding/removing rdev from
state_store()
md: use new apis to suspend array for bind_rdev_to_array()
md: use new apis to suspend array related to serial pool in
state_store()
md: use new apis to suspend array in backlog_store()
md: suspend array in md_start_sync() if array need reconfiguration
md: cleanup mddev_create/destroy_serial_pool()
md/md-linear: cleanup linear_add()
md: remove old apis to suspend the array
md: rename __mddev_suspend/resume() back to mddev_suspend/resume()
drivers/md/dm-raid.c | 8 +-
drivers/md/md-autodetect.c | 4 +-
drivers/md/md-bitmap.c | 18 ++-
drivers/md/md-linear.c | 2 -
drivers/md/md.c | 250 ++++++++++++++++++++++---------------
drivers/md/md.h | 52 ++++++--
drivers/md/raid5-cache.c | 61 +++++----
drivers/md/raid5.c | 56 ++++-----
8 files changed, 253 insertions(+), 198 deletions(-)
--
2.39.2
From: Yu Kuai <redacted>
Changes in v2:
- rebase with latest md-next
- remove some follow up cleanup patches, these patches will be sent
later after this patchset.
After previous four patchset of preparatory work, this patchset impelement
a new version of mddev_suspend(), the new apis:
- reconfig_mutex is not required;
- the weird logical that suspend array hold 'reconfig_mutex' for
mddev_check_recovery() to update superblock is not needed;
- the special handling, 'pers->prepare_suspend', for raid456 is not
needed;
- It's safe to be called at any time once mddev is allocated, and it's
designed to be used from slow path where array configuration is changed;
And use the new api to replace:
mddev_lock
mddev_suspend or not
// array reconfiguration
mddev_resume or not
mddev_unlock
With:
mddev_suspend
mddev_lock
// array reconfiguration
mddev_unlock
mddev_resume
However, the above change is not possible for raid5 and raid-cluster in
some corner cases, and mddev_suspend/resume() is replaced with quiesce()
callback, which will suspend the array as well.
This patchset is tested in my VM with mdadm testsuite with loop device
except for 10ddf tests(they always fail before this patchset).
A lot of cleanups will be started after this patchset.
Yu Kuai (28):
md: use READ_ONCE/WRITE_ONCE for 'suspend_lo' and 'suspend_hi'
md: use 'mddev->suspended' for is_md_suspended()
md: add new helpers to suspend/resume array
md: add new helpers to suspend/resume and lock/unlock array
md: use new apis to suspend array for suspend_lo/hi_store()
md: use new apis to suspend array for level_store()
md: use new apis to suspend array for serialize_policy_store()
md/dm-raid: use new apis to suspend array
md/md-bitmap: use new apis to suspend array for location_store()
md/raid5-cache: use READ_ONCE/WRITE_ONCE for 'conf->log'
md/raid5-cache: use new apis to suspend array for
r5c_disable_writeback_async()
md/raid5-cache: use new apis to suspend array for
r5c_journal_mode_store()
md/raid5: use new apis to suspend array for raid5_store_stripe_size()
md/raid5: use new apis to suspend array for raid5_store_skip_copy()
md/raid5: use new apis to suspend array for
raid5_store_group_thread_cnt()
md/raid5: use new apis to suspend array for
raid5_change_consistency_policy()
md/raid5: replace suspend with quiesce() callback
md: quiesce before md_kick_rdev_from_array() for md-cluster
md: use new apis to suspend array for ioctls involed array
reconfiguration
md: use new apis to suspend array for adding/removing rdev from
state_store()
md: use new apis to suspend array for bind_rdev_to_array()
md: use new apis to suspend array related to serial pool in
state_store()
md: use new apis to suspend array in backlog_store()
md: suspend array in md_start_sync() if array need reconfiguration
md: cleanup mddev_create/destroy_serial_pool()
md/md-linear: cleanup linear_add()
md: remove old apis to suspend the array
md: rename __mddev_suspend/resume() back to mddev_suspend/resume()
drivers/md/dm-raid.c | 8 +-
drivers/md/md-autodetect.c | 4 +-
drivers/md/md-bitmap.c | 18 ++-
drivers/md/md-linear.c | 2 -
drivers/md/md.c | 250 ++++++++++++++++++++++---------------
drivers/md/md.h | 52 ++++++--
drivers/md/raid5-cache.c | 61 +++++----
drivers/md/raid5.c | 56 ++++-----
8 files changed, 253 insertions(+), 198 deletions(-)
--
2.39.2