Hi, all
These 3 patches add ata port runtime pm support, which is done through scsi layer.
scsi host runtime pm mechanism is already there(scsi_pm.c).
These patches add hooks for ata port.
Lin Ming (3):
scsi: fix potential dead lock for host runtime pm
scsi: add hooks for host runtime power management
ata: implement ata port runtime pm hooks
drivers/ata/libata-core.c | 96 +++++++++++++++++++++++++++++---------------
drivers/scsi/scsi_error.c | 4 +-
drivers/scsi/scsi_pm.c | 45 +++++++++++++++++++-
drivers/scsi/scsi_priv.h | 2 +
include/linux/libata.h | 7 +++
include/scsi/scsi_host.h | 8 ++++
6 files changed, 125 insertions(+), 37 deletions(-)
Thanks for any comment.
Lin Ming
In later patch hooks will be added to do ata port runtime pm through scsi layer.
libata schedules scsi EH to handle suspend, then dead lock happens
because scsi EH in turn waits for the ongoing suspend, as below.
<scsi host runtime suspend>
scsi_autopm_put_host
pm_runtime_put_sync
<scsi_host runtime pm status updated to RPM_SUSPENDING>
......
<call libata hook to do suspend>
<wake up scsi EH to handle suspend>
<wait for scsi EH ...>
<scsi EH wake up>
scsi_error_handler
<resume scsi host>
scsi_autopm_get_host
pm_runtime_get_sync
.....
<sleep to wait for the ongoing scsi host suspend>
This patch fixes the dead lock by checking if there is ongoing runtime PM request.
If there is ongoing runtime PM request, scsi_autopm_get_host_noresume is called to
increase the usage count, but don't resume the host.
Signed-off-by: Lin Ming <redacted>
---
drivers/scsi/scsi_error.c | 4 +++-
drivers/scsi/scsi_pm.c | 11 +++++++++++
drivers/scsi/scsi_priv.h | 2 ++
3 files changed, 16 insertions(+), 1 deletions(-)
Split a new function ata_port_request_pm from ata_host_request_pm.
Implement runtime suspend and resume hooks for scsi layer.
Signed-off-by: Lin Ming <redacted>
---
drivers/ata/libata-core.c | 96 +++++++++++++++++++++++++++++---------------
include/linux/libata.h | 7 +++
2 files changed, 70 insertions(+), 33 deletions(-)
@@ -5234,51 +5235,62 @@ bool ata_link_offline(struct ata_link *link)}#ifdef CONFIG_PM-staticintata_host_request_pm(structata_host*host,pm_message_tmesg,+staticintata_port_request_pm(structata_port*ap,pm_message_tmesg,unsignedintaction,unsignedintehi_flags,intwait){+structata_link*link;unsignedlongflags;-inti,rc;+intrc;-for(i=0;i<host->n_ports;i++){-structata_port*ap=host->ports[i];-structata_link*link;+/* Previous resume operation might still be in+*progress.WaitforPM_PENDINGtoclear.+*/+if(ap->pflags&ATA_PFLAG_PM_PENDING){+ata_port_wait_eh(ap);+WARN_ON(ap->pflags&ATA_PFLAG_PM_PENDING);+}-/* Previous resume operation might still be in-*progress.WaitforPM_PENDINGtoclear.-*/-if(ap->pflags&ATA_PFLAG_PM_PENDING){-ata_port_wait_eh(ap);-WARN_ON(ap->pflags&ATA_PFLAG_PM_PENDING);-}+/* request PM ops to EH */+spin_lock_irqsave(ap->lock,flags);-/* request PM ops to EH */-spin_lock_irqsave(ap->lock,flags);+ap->pm_mesg=mesg;+if(wait){+rc=0;+ap->pm_result=&rc;+}-ap->pm_mesg=mesg;-if(wait){-rc=0;-ap->pm_result=&rc;-}+ap->pflags|=ATA_PFLAG_PM_PENDING;+ata_for_each_link(link,ap,HOST_FIRST){+link->eh_info.action|=action;+link->eh_info.flags|=ehi_flags;+}-ap->pflags|=ATA_PFLAG_PM_PENDING;-ata_for_each_link(link,ap,HOST_FIRST){-link->eh_info.action|=action;-link->eh_info.flags|=ehi_flags;-}+ata_port_schedule_eh(ap);-ata_port_schedule_eh(ap);+spin_unlock_irqrestore(ap->lock,flags);-spin_unlock_irqrestore(ap->lock,flags);+/* wait and check result */+if(wait){+ata_port_wait_eh(ap);+WARN_ON(ap->pflags&ATA_PFLAG_PM_PENDING);+}-/* wait and check result */-if(wait){-ata_port_wait_eh(ap);-WARN_ON(ap->pflags&ATA_PFLAG_PM_PENDING);-if(rc)-returnrc;-}+returnrc;+}++staticintata_host_request_pm(structata_host*host,pm_message_tmesg,+unsignedintaction,unsignedintehi_flags,+intwait)+{+inti,rc;++for(i=0;i<host->n_ports;i++){+structata_port*ap=host->ports[i];++rc=ata_port_request_pm(ap,mesg,action,ehi_flags,wait);+if(rc)+returnrc;}return0;
Adds ->suspend and ->resume callbacks in struct scsi_host_template
to do host runtime power management.
Signed-off-by: Lin Ming <redacted>
---
drivers/scsi/scsi_pm.c | 34 +++++++++++++++++++++++++++++++---
include/scsi/scsi_host.h | 8 ++++++++
2 files changed, 39 insertions(+), 3 deletions(-)
@@ -42,6 +42,28 @@ static int scsi_dev_type_resume(struct device *dev)returnerr;}+staticintscsi_host_suspend(structdevice*dev)+{+structScsi_Host*shost=dev_to_shost(dev);+interr=0;++if(shost->hostt->suspend)+err=shost->hostt->suspend(shost);++returnerr;+}++staticintscsi_host_resume(structdevice*dev)+{+structScsi_Host*shost=dev_to_shost(dev);+interr=0;++if(shost->hostt->resume)+err=shost->hostt->resume(shost);++returnerr;+}+#ifdef CONFIG_PM_SLEEPstaticintscsi_bus_suspend_common(structdevice*dev,pm_message_tmsg)
@@ -106,7 +128,10 @@ static int scsi_runtime_suspend(struct device *dev)round_jiffies_up_relative(HZ/10)));}-/* Insert hooks here for targets, hosts, and transport classes */+/* Insert hooks here for targets and transport classes */++if(scsi_is_host_device(dev))+err=scsi_host_suspend(dev);returnerr;}
@@ -119,7 +144,10 @@ static int scsi_runtime_resume(struct device *dev)if(scsi_is_sdev_device(dev))err=scsi_dev_type_resume(dev);-/* Insert hooks here for targets, hosts, and transport classes */+/* Insert hooks here for targets and transport classes */++if(scsi_is_host_device(dev))+err=scsi_host_resume(dev);returnerr;}
@@ -132,7 +160,7 @@ static int scsi_runtime_idle(struct device *dev)/* Insert hooks here for targets, hosts, and transport classes */-if(scsi_is_sdev_device(dev))+if(scsi_is_sdev_device(dev)||scsi_is_host_device(dev))err=pm_schedule_suspend(dev,100);elseerr=pm_runtime_suspend(dev);
From: Alan Stern <stern@rowland.harvard.edu> Date: 2011-11-02 14:41:52
On Wed, 2 Nov 2011, Lin Ming wrote:
quoted hunk
In later patch hooks will be added to do ata port runtime pm through scsi layer.
libata schedules scsi EH to handle suspend, then dead lock happens
because scsi EH in turn waits for the ongoing suspend, as below.
<scsi host runtime suspend>
scsi_autopm_put_host
pm_runtime_put_sync
<scsi_host runtime pm status updated to RPM_SUSPENDING>
......
<call libata hook to do suspend>
<wake up scsi EH to handle suspend>
<wait for scsi EH ...>
<scsi EH wake up>
scsi_error_handler
<resume scsi host>
scsi_autopm_get_host
pm_runtime_get_sync
.....
<sleep to wait for the ongoing scsi host suspend>
This patch fixes the dead lock by checking if there is ongoing runtime PM request.
If there is ongoing runtime PM request, scsi_autopm_get_host_noresume is called to
increase the usage count, but don't resume the host.
Signed-off-by: Lin Ming <redacted>
---
drivers/scsi/scsi_error.c | 4 +++-
drivers/scsi/scsi_pm.c | 11 +++++++++++
drivers/scsi/scsi_priv.h | 2 ++
3 files changed, 16 insertions(+), 1 deletions(-)
@@ -1804,7 +1804,9 @@ int scsi_error_handler(void *data)*whatweneedtodotogetitupandonlineagain(ifwecan).*Ifwefail,weenduptakingthethingoffline.*/-if(scsi_autopm_get_host(shost)!=0){+if(scsi_autopm_host_busy(shost))+scsi_autopm_get_host_noresume(shost);+elseif(scsi_autopm_get_host(shost)!=0){SCSI_LOG_ERROR_RECOVERY(1,printk(KERN_ERR"Error handler scsi_eh_%d ""unable to autoresume\n",
Here's what I'm worried about: Suppose during normal operation, an
error occurs. When the command is completed, runtime PM might start to
suspend the host. But then the error-handler thread starts up, and
of course it needs the host to be powered-up in order to recover from
the error. The code you're adding could prevent this from working.
What we really need is a way to prevent host from going into runtime
suspend while the error-handler is pending or running. Do you have any
ideas on how to do this?
From: Alan Stern <stern@rowland.harvard.edu> Date: 2011-11-02 14:53:21
On Wed, 2 Nov 2011, Lin Ming wrote:
quoted hunk
Adds ->suspend and ->resume callbacks in struct scsi_host_template
to do host runtime power management.
Signed-off-by: Lin Ming <redacted>
---
drivers/scsi/scsi_pm.c | 34 +++++++++++++++++++++++++++++++---
include/scsi/scsi_host.h | 8 ++++++++
2 files changed, 39 insertions(+), 3 deletions(-)
@@ -42,6 +42,28 @@ static int scsi_dev_type_resume(struct device *dev)returnerr;}+staticintscsi_host_suspend(structdevice*dev)+{+structScsi_Host*shost=dev_to_shost(dev);+interr=0;++if(shost->hostt->suspend)+err=shost->hostt->suspend(shost);++returnerr;+}++staticintscsi_host_resume(structdevice*dev)+{+structScsi_Host*shost=dev_to_shost(dev);+interr=0;++if(shost->hostt->resume)+err=shost->hostt->resume(shost);++returnerr;+}
These don't need to be independent routines. You can put them inline
in scsi_runtime_suspend() and scsi_runtime_resume() below; that's how
the device code is written.
quoted hunk
@@ -132,7 +160,7 @@ static int scsi_runtime_idle(struct device *dev) /* Insert hooks here for targets, hosts, and transport classes */- if (scsi_is_sdev_device(dev))+ if (scsi_is_sdev_device(dev) || scsi_is_host_device(dev)) err = pm_schedule_suspend(dev, 100);
This is wrong. The 100-ms delay is meant for devices only. There's
no reason to delay suspending a host once everything beneath it is
suspended. Just leave the code the way it is now.
The comment should explain what drivers are supposed to do when these
hooks are called. In particular, the suspend callback should _not_
power-down the entire host adapter; it should power-down only the
circuitry that controls the interface to the SCSI bus. The upstream
interface to the CPU should remain at full power.
If there's no way to power-down just that part of the host adapter then
there's no need to have these callbacks at all. The low-level driver
can simply use the runtime PM API on its own struct device.
Alan Stern
Hello,
On Wed, Nov 02, 2011 at 02:21:37PM +0800, Lin Ming wrote:
These 3 patches add ata port runtime pm support, which is done through scsi layer.
scsi host runtime pm mechanism is already there(scsi_pm.c).
These patches add hooks for ata port.
Lin Ming (3):
scsi: fix potential dead lock for host runtime pm
scsi: add hooks for host runtime power management
ata: implement ata port runtime pm hooks
Hmmm.... Some questions.
* How much does this add on top of hardware timed auto spindown and
dynamic link power management?
* Do the added benefits justify yet another runtime powersaving
mechanism?
* Any way we can tie the other stuff with overall runtime PM?
Thanks.
--
tejun
On Wed, 2011-11-02 at 22:53 +0800, Alan Stern wrote:
On Wed, 2 Nov 2011, Lin Ming wrote:
quoted
Adds ->suspend and ->resume callbacks in struct scsi_host_template
to do host runtime power management.
Signed-off-by: Lin Ming <redacted>
---
drivers/scsi/scsi_pm.c | 34 +++++++++++++++++++++++++++++++---
include/scsi/scsi_host.h | 8 ++++++++
2 files changed, 39 insertions(+), 3 deletions(-)
@@ -42,6 +42,28 @@ static int scsi_dev_type_resume(struct device *dev)returnerr;}+staticintscsi_host_suspend(structdevice*dev)+{+structScsi_Host*shost=dev_to_shost(dev);+interr=0;++if(shost->hostt->suspend)+err=shost->hostt->suspend(shost);++returnerr;+}++staticintscsi_host_resume(structdevice*dev)+{+structScsi_Host*shost=dev_to_shost(dev);+interr=0;++if(shost->hostt->resume)+err=shost->hostt->resume(shost);++returnerr;+}
These don't need to be independent routines. You can put them inline
in scsi_runtime_suspend() and scsi_runtime_resume() below; that's how
the device code is written.
OK.
quoted
@@ -132,7 +160,7 @@ static int scsi_runtime_idle(struct device *dev) /* Insert hooks here for targets, hosts, and transport classes */- if (scsi_is_sdev_device(dev))+ if (scsi_is_sdev_device(dev) || scsi_is_host_device(dev)) err = pm_schedule_suspend(dev, 100);
This is wrong. The 100-ms delay is meant for devices only. There's
no reason to delay suspending a host once everything beneath it is
suspended. Just leave the code the way it is now.
The comment should explain what drivers are supposed to do when these
hooks are called. In particular, the suspend callback should _not_
power-down the entire host adapter; it should power-down only the
circuitry that controls the interface to the SCSI bus. The upstream
interface to the CPU should remain at full power.
If there's no way to power-down just that part of the host adapter then
there's no need to have these callbacks at all. The low-level driver
can simply use the runtime PM API on its own struct device.
Good point.
I realize that this is not the natural way to do ata port runtime pm.
Hooking it to scsi host runtime pm is not good. It does not deal with
the races with system suspend/resume of host controller.
How about making ata port as the parent device of scsi host?
Then, for example, the runtime suspend happens as below,
disk suspend --> scsi target suspend --> scsi host suspend --> ata port
suspend.
Current device tree is:
/sys/devices/pci0000:00/0000:00:1f.2/
|-- ata1
|-- host0
After the change, the tree will become as:
/sys/devices/pci0000:00/0000:00:1f.2/ata1/
|-- host0
The tricky part is ata port(parent device) suspend need to schedule scsi
EH which will resume scsi host(child device). Then the child device
resume will in turn make parent device resume first. This is kind of
recursive.
We can fix this by adding a flag somewhere to tell scsi EH don't resume
the host in ata port pm request handling case.
What do you think?
Thanks,
Lin Ming
From: Alan Stern <stern@rowland.harvard.edu> Date: 2011-11-03 14:22:47
On Thu, 3 Nov 2011, Lin Ming wrote:
I realize that this is not the natural way to do ata port runtime pm.
Hooking it to scsi host runtime pm is not good. It does not deal with
the races with system suspend/resume of host controller.
How about making ata port as the parent device of scsi host?
Then, for example, the runtime suspend happens as below,
disk suspend --> scsi target suspend --> scsi host suspend --> ata port
suspend.
Current device tree is:
/sys/devices/pci0000:00/0000:00:1f.2/
|-- ata1
|-- host0
After the change, the tree will become as:
/sys/devices/pci0000:00/0000:00:1f.2/ata1/
|-- host0
I don't know enough about the ATA subsystem to say much, except that
this looks more logical.
The tricky part is ata port(parent device) suspend need to schedule scsi
EH which will resume scsi host(child device). Then the child device
resume will in turn make parent device resume first. This is kind of
recursive.
We can fix this by adding a flag somewhere to tell scsi EH don't resume
the host in ata port pm request handling case.
What do you think?
Yes, it is a problem. But it looks like the underlying issue is that
you're using the SCSI error handler to do something it was not intended
for. Can't you suspend and resume the ATA port without using the error
handler?
Alan Stern
On Wed, 2011-11-02 at 23:17 +0800, Tejun Heo wrote:
Hello,
On Wed, Nov 02, 2011 at 02:21:37PM +0800, Lin Ming wrote:
quoted
These 3 patches add ata port runtime pm support, which is done through scsi layer.
scsi host runtime pm mechanism is already there(scsi_pm.c).
These patches add hooks for ata port.
Lin Ming (3):
scsi: fix potential dead lock for host runtime pm
scsi: add hooks for host runtime power management
ata: implement ata port runtime pm hooks
Hmmm.... Some questions.
* How much does this add on top of hardware timed auto spindown and
dynamic link power management?
Sorry, I don't understand this question.
What do you mean?
* Do the added benefits justify yet another runtime powersaving
mechanism?
There will be ACPI firmware which supports ata port runtime D3Cold.
To enable ata port runtime PM is the first step.
* Any way we can tie the other stuff with overall runtime PM?
What's the other stuff?
Do you mean tie disk/link/port/controller stuff with overall runtime PM?
I'm not sure for now. Need to think about it more.
Hello,
On Thu, Nov 3, 2011 at 7:21 AM, Lin Ming [off-list ref] wrote:
quoted
* How much does this add on top of hardware timed auto spindown and
dynamic link power management?
Sorry, I don't understand this question.
What do you mean?
I was curious about the amount of additional power saving on top of
the existing PM features.
quoted
* Do the added benefits justify yet another runtime powersaving
mechanism?
There will be ACPI firmware which supports ata port runtime D3Cold.
To enable ata port runtime PM is the first step.
libata can already turn off unused ports although it wouldn't go as
far as putting the whole controller into D3.
quoted
* Any way we can tie the other stuff with overall runtime PM?
What's the other stuff?
Do you mean tie disk/link/port/controller stuff with overall runtime PM?
Hardware initiated spin-down (hdparm -S) and link power saving (look
for ata_lpm_*). It would be great if there's an overall design how
they interact before adding yet another PM vector.
Thanks.
--
tejun
On Thu, 2011-11-03 at 22:22 +0800, Alan Stern wrote:
On Thu, 3 Nov 2011, Lin Ming wrote:
quoted
I realize that this is not the natural way to do ata port runtime pm.
Hooking it to scsi host runtime pm is not good. It does not deal with
the races with system suspend/resume of host controller.
How about making ata port as the parent device of scsi host?
Then, for example, the runtime suspend happens as below,
disk suspend --> scsi target suspend --> scsi host suspend --> ata port
suspend.
Current device tree is:
/sys/devices/pci0000:00/0000:00:1f.2/
|-- ata1
|-- host0
After the change, the tree will become as:
/sys/devices/pci0000:00/0000:00:1f.2/ata1/
|-- host0
I don't know enough about the ATA subsystem to say much, except that
this looks more logical.
quoted
The tricky part is ata port(parent device) suspend need to schedule scsi
EH which will resume scsi host(child device). Then the child device
resume will in turn make parent device resume first. This is kind of
recursive.
We can fix this by adding a flag somewhere to tell scsi EH don't resume
the host in ata port pm request handling case.
What do you think?
Yes, it is a problem. But it looks like the underlying issue is that
you're using the SCSI error handler to do something it was not intended
for. Can't you suspend and resume the ATA port without using the error
handler?
The system suspend and resume of the ATA port uses the error handler.
I think the runtime suspend and resume should use the error handler too.
Tejun,
Could you comment more on this?
Thanks.
Hello,
On Thu, Nov 3, 2011 at 7:37 AM, Lin Ming [off-list ref] wrote:
quoted
Yes, it is a problem. But it looks like the underlying issue is that
you're using the SCSI error handler to do something it was not intended
for. Can't you suspend and resume the ATA port without using the error
handler?
The system suspend and resume of the ATA port uses the error handler.
I think the runtime suspend and resume should use the error handler too.
Tejun,
Could you comment more on this?
I don't know. I haven't really thought about it but as it's currently
designed, I don't think it'll be possible to avoid going through EH to
put ATA ports into suspend mode.
Thanks.
--
tejun
--
To unsubscribe from this list: send the line "unsubscribe linux-scsi" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
On Thu, 2011-11-03 at 22:30 +0800, Tejun Heo wrote:
Hello,
On Thu, Nov 3, 2011 at 7:21 AM, Lin Ming [off-list ref] wrote:
quoted
quoted
* How much does this add on top of hardware timed auto spindown and
dynamic link power management?
Sorry, I don't understand this question.
What do you mean?
I was curious about the amount of additional power saving on top of
the existing PM features.
I'll measure the power saving.
quoted
quoted
* Do the added benefits justify yet another runtime powersaving
mechanism?
There will be ACPI firmware which supports ata port runtime D3Cold.
To enable ata port runtime PM is the first step.
libata can already turn off unused ports although it wouldn't go as
far as putting the whole controller into D3.
At runtime?
Could you point me which piece of code implement this?
quoted
quoted
* Any way we can tie the other stuff with overall runtime PM?
What's the other stuff?
Do you mean tie disk/link/port/controller stuff with overall runtime PM?
Hardware initiated spin-down (hdparm -S) and link power saving (look
for ata_lpm_*). It would be great if there's an overall design how
they interact before adding yet another PM vector.
From: Alan Stern <stern@rowland.harvard.edu> Date: 2011-11-03 15:39:36
On Thu, 3 Nov 2011, Tejun Heo wrote:
Hello,
On Thu, Nov 3, 2011 at 7:37 AM, Lin Ming [off-list ref] wrote:
quoted
quoted
Yes, it is a problem. �But it looks like the underlying issue is that
you're using the SCSI error handler to do something it was not intended
for. �Can't you suspend and resume the ATA port without using the error
handler?
The system suspend and resume of the ATA port uses the error handler.
I think the runtime suspend and resume should use the error handler too.
Tejun,
Could you comment more on this?
I don't know. I haven't really thought about it but as it's currently
designed, I don't think it'll be possible to avoid going through EH to
put ATA ports into suspend mode.
Hello,
On Thu, Nov 3, 2011 at 8:39 AM, Alan Stern [off-list ref] wrote:
quoted
I don't know. I haven't really thought about it but as it's currently
designed, I don't think it'll be possible to avoid going through EH to
put ATA ports into suspend mode.
Why not?
It's just how things are designed now. There's no reason why it's
fundamentally impossible but going around that would require some
amount of hackery or preferably re-design. Synchronization against
command processing, interrupts and all are built around EH.
Thanks.
--
tejun
From: Alan Stern <stern@rowland.harvard.edu> Date: 2011-11-03 16:08:57
On Thu, 3 Nov 2011, Tejun Heo wrote:
Hello,
On Thu, Nov 3, 2011 at 8:39 AM, Alan Stern [off-list ref] wrote:
quoted
quoted
I don't know. I haven't really thought about it but as it's currently
designed, I don't think it'll be possible to avoid going through EH to
put ATA ports into suspend mode.
Why not?
It's just how things are designed now. There's no reason why it's
fundamentally impossible but going around that would require some
amount of hackery or preferably re-design. Synchronization against
command processing, interrupts and all are built around EH.
Then it sounds like the best idea is something like what Ming proposed
earlier: Have the error handler make some sort of test to determine
whether it has been invoked for suspend/resume handling, and skip the
runtime-PM calls on the host if it has.
This new test, whatever it is, should apply both to system suspend and
runtime suspend. By the way, what would happen if a system suspend
occurred while the ATA port was already runtime-suspended?
Alan Stern