From: Xiongfeng Wang <hidden> Date: 2020-01-09 02:44:02
Hi Will, Mark,
Sorry to bother you. It's just that we have come across some problems about PMU recently.
We are working on deep power state on CPU cores. In the deepest idle state, the core will be
powered down. In our implementation, the PMU and the core are in the same power domain,
so the PMU will also be powered down. But I didn't find where we saved the PMU context
in kernel before entering the deepest idle state.
Before we enter the system sleep state, we update the kernel PMU counter and stop the PMU
in 'cpu_pm_pmu_notify()'. But we didn't do that before we enter idle state.
I only find some system registers saving in 'psci_cpu_suspend_enter()->cpu_susend()->cpu_do_suspend()'
Do you have some suggestion about how we can handle this problem ?
Should we save and restore the PMU context before and after the deepest idle state . I don't know
if we can take care of the PMU counters and interrupts properly by saving and restoring
the PMU registers.
Or we should update the kernel counter and stop the PMU, like what we did in system sleep situation.
Or we should let the firmware to handle the saving and restoring problem.
Thanks,
Xiongfeng
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
From: Will Deacon <will@kernel.org> Date: 2020-01-09 15:03:35
[+Lorenzo]
On Thu, Jan 09, 2020 at 10:43:40AM +0800, Xiongfeng Wang wrote:
Sorry to bother you. It's just that we have come across some problems
about PMU recently.
No bother, and thanks for including the mailing list.
We are working on deep power state on CPU cores. In the deepest idle state, the core will be
powered down. In our implementation, the PMU and the core are in the same power domain,
so the PMU will also be powered down. But I didn't find where we saved the PMU context
in kernel before entering the deepest idle state.
Before we enter the system sleep state, we update the kernel PMU counter and stop the PMU
in 'cpu_pm_pmu_notify()'. But we didn't do that before we enter idle state.
I only find some system registers saving in 'psci_cpu_suspend_enter()->cpu_susend()->cpu_do_suspend()'
I'm not sure what you mean by "system sleep state", but if you're putting
the CPU into a state where register contents is lost, then I think you need
those PM notifiers to run. My understanding was that CPUidle took care of
this. Why is that not sufficient for you?
Do you have some suggestion about how we can handle this problem ?
Should we save and restore the PMU context before and after the deepest idle state . I don't know
if we can take care of the PMU counters and interrupts properly by saving and restoring
the PMU registers.
Or we should update the kernel counter and stop the PMU, like what we did in system sleep situation.
Or we should let the firmware to handle the saving and restoring problem.
If the idle state is somehow autonomous (i.e. the kernel is not aware of
it), then I suppose firmware has to save/restore the register state that
is not otherwise preserved. However, I'd like to understand if this is
really what's happening and why CPUidle isn't just doing the right thing.
Thanks,
Will
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
From: Lorenzo Pieralisi <hidden> Date: 2020-01-09 16:47:15
On Thu, Jan 09, 2020 at 03:03:19PM +0000, Will Deacon wrote:
[+Lorenzo]
On Thu, Jan 09, 2020 at 10:43:40AM +0800, Xiongfeng Wang wrote:
quoted
Sorry to bother you. It's just that we have come across some problems
about PMU recently.
No bother, and thanks for including the mailing list.
quoted
We are working on deep power state on CPU cores. In the deepest idle
state, the core will be powered down. In our implementation, the PMU
and the core are in the same power domain, so the PMU will also be
powered down. But I didn't find where we saved the PMU context in
kernel before entering the deepest idle state.
Before we enter the system sleep state, we update the kernel PMU
counter and stop the PMU in 'cpu_pm_pmu_notify()'. But we didn't do
that before we enter idle state.
ACPI or DT firmware ? I suspect that's ACPI, with LPI idle state
flags set to 0x0 (3.1.3 - save and restore flags):
http://infocenter.arm.com/help/topic/com.arm.doc.den0048a/DEN0048A_ARM_FFH_Specification.pdf
If that's the case a firmware update is needed (ie currently the kernel
expects the PMU state to be retained).
arch/arm64/kernel/cpuidle.c
ARM64_LPI_IS_RETENTION_STATE()
In DT in the PSCI CPUidle driver we run the notifiers irrespective
of the idle state depth so I don't think this behaviour can happen
in a DT bootstrapped system.
I am just guessing - please let me know if my assumption is correct.
quoted
I only find some system registers saving in 'psci_cpu_suspend_enter()->cpu_susend()->cpu_do_suspend()'
I'm not sure what you mean by "system sleep state"
I think they mean suspend-to-RAM - in suspend-to-RAM the notifiers
are run through syscore operations which are decoupled from CPUidle.
Regardless, CPUidle should call the notifiers if instructed by firmware
correctly.
Thanks,
Lorenzo
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
From: Xiongfeng Wang <hidden> Date: 2020-01-10 03:29:01
On 2020/1/10 0:46, Lorenzo Pieralisi wrote:
On Thu, Jan 09, 2020 at 03:03:19PM +0000, Will Deacon wrote:
quoted
[+Lorenzo]
On Thu, Jan 09, 2020 at 10:43:40AM +0800, Xiongfeng Wang wrote:
quoted
Sorry to bother you. It's just that we have come across some problems
about PMU recently.
No bother, and thanks for including the mailing list.
quoted
We are working on deep power state on CPU cores. In the deepest idle
state, the core will be powered down. In our implementation, the PMU
and the core are in the same power domain, so the PMU will also be
powered down. But I didn't find where we saved the PMU context in
kernel before entering the deepest idle state.
Before we enter the system sleep state, we update the kernel PMU
counter and stop the PMU in 'cpu_pm_pmu_notify()'. But we didn't do
that before we enter idle state.
ACPI or DT firmware ? I suspect that's ACPI, with LPI idle state
flags set to 0x0 (3.1.3 - save and restore flags):
http://infocenter.arm.com/help/topic/com.arm.doc.den0048a/DEN0048A_ARM_FFH_Specification.pdf
If that's the case a firmware update is needed (ie currently the kernel
expects the PMU state to be retained).
arch/arm64/kernel/cpuidle.c
ARM64_LPI_IS_RETENTION_STATE()
That totally solved my problem.
I set the LPI idle state flag in firmware, and the 'cpu_pm_pmu_notify()' can be called
before I enter the context-lost idle state. Thanks a lot !
In DT in the PSCI CPUidle driver we run the notifiers irrespective
of the idle state depth so I don't think this behaviour can happen
in a DT bootstrapped system.
I am just guessing - please let me know if my assumption is correct.
Yes, it's correct. We are using ACPI.
quoted
quoted
I only find some system registers saving in 'psci_cpu_suspend_enter()->cpu_susend()->cpu_do_suspend()'
I'm not sure what you mean by "system sleep state"
I think they mean suspend-to-RAM - in suspend-to-RAM the notifiers
are run through syscore operations which are decoupled from CPUidle.
Yes, I mean suspend-to-RAM.
Thanks,
Xiongfeng
Regardless, CPUidle should call the notifiers if instructed by firmware
correctly.
Thanks,
Lorenzo
.
From: Xiongfeng Wang <hidden> Date: 2020-01-10 03:46:14
On 2020/1/9 23:03, Will Deacon wrote:
[+Lorenzo]
On Thu, Jan 09, 2020 at 10:43:40AM +0800, Xiongfeng Wang wrote:
quoted
Sorry to bother you. It's just that we have come across some problems
about PMU recently.
No bother, and thanks for including the mailing list.
quoted
We are working on deep power state on CPU cores. In the deepest idle state, the core will be
powered down. In our implementation, the PMU and the core are in the same power domain,
so the PMU will also be powered down. But I didn't find where we saved the PMU context
in kernel before entering the deepest idle state.
Before we enter the system sleep state, we update the kernel PMU counter and stop the PMU
in 'cpu_pm_pmu_notify()'. But we didn't do that before we enter idle state.
I only find some system registers saving in 'psci_cpu_suspend_enter()->cpu_susend()->cpu_do_suspend()'
I'm not sure what you mean by "system sleep state", but if you're putting
the CPU into a state where register contents is lost, then I think you need
those PM notifiers to run. My understanding was that CPUidle took care of
this. Why is that not sufficient for you?
Sorry, I didn't notice the PM notifiers are called in CPUidle when 'arch_flags' is set.
I thought it's only called in 'cpu_pm_syscore_ops', I mean Susepnd-to-RAM.
Thanks,
Xiongfeng
quoted
Do you have some suggestion about how we can handle this problem ?
Should we save and restore the PMU context before and after the deepest idle state . I don't know
if we can take care of the PMU counters and interrupts properly by saving and restoring
the PMU registers.
Or we should update the kernel counter and stop the PMU, like what we did in system sleep situation.
Or we should let the firmware to handle the saving and restoring problem.
If the idle state is somehow autonomous (i.e. the kernel is not aware of
it), then I suppose firmware has to save/restore the register state that
is not otherwise preserved. However, I'd like to understand if this is
really what's happening and why CPUidle isn't just doing the right thing.
Thanks,
Will
.