From: Hector Martin <hidden> Date: 2021-10-11 16:57:34
Hi folks, here's a first attempt at cpufreq support for the Apple M1.
I'd appreciate any feedback on this approach.
The hardware in these SoCs is very simple: you just poke a single
register to change the performance state of a CPU cluster. There's
some init required on older firmware versions, but we're going to
say that's the bootloader's responsibility. This keeps the driver
nice and simple and generic and likely to work on future SoCs too.
However, there's a catch: the memory controller config should also be
modified when higher clock states are engaged on the P-cores, to
get optimal performance.
This series implements this using two drivers, on top of the existing
cpufreq-dt infrastructure. The cpu clock driver represents the p-state
switching as if it were a standard clock driver, so it can work with
cpufreq-dt. To do this, it also needs access to the OPP table, so it can
map the incoming clock frequences back to P-State index numbers, so that
is present in the binding. This might be a bit strange, since the same
OPP table is referenced by the CPUs themselves, and by the clocks driver
that provides the actual switching for them...
The memory controller stuff is implemented as a genpd provider that
exposes two performance states that the CPU OPP tables can depend on.
Unfortunately, the cpufreq path doesn't work properly for this, since
the CPUs aren't typical devices participating in runtime-pm. So instead
I opted to put that logic in the clock driver, which means it gets a
power-domains reference to the memory controller. This required a hack
to the OPP core so that it wouldn't complain about the missing parent
domain when evaluating the OPPs in the context of the CPUs...
The actual memory controller config is two magic numbers per performance
state. I'd love to find out what they do, but this seems unlikely
without docs or a deep memory performance analysis expedition... so I
think we're going to have to settle for this way, at least for now. If
things become better understood in the future, we can modify the binding
and keep the driver backwards-compatible with old DTs at least.
I did benchmark the CPU p-state switching, so the latency numbers there
have been experimentally measured. The CPU capacity numbers are also
based on real benchmarks (oddly enough, Dhrystone was a big outlier
here that was not representative of everything else, so we didn't use
it).
Patches:
#1: MAINTAINERS updates, split out so this can go through the SoC
tree so we can spare all the subsystem maintainers the merge
conflicts, since we have a bunch of other changes here going on
in parallel.
#2-3: DT bindings
#4: The aforementioned hack for the OPP core
#5: Add of_genpd_add_provider_simple_noclk()
#6: Memory controller driver
#7: CPU p-state clock driver
#8: Add some deps for ARCH_APPLE
#9: DT updates (for asahi-soc tree)
Hector Martin (9):
MAINTAINERS: apple: Add apple-mcc and clk-apple-cluster paths
dt-bindings: memory-controller: Add apple,mcc binding
dt-bindings: clock: Add apple,cluster-clk binding
opp: core: Don't warn if required OPP device does not exist
PM: domains: Add of_genpd_add_provider_simple_noclk()
memory: apple: Add apple-mcc driver to manage MCC perf in Apple SoCs
clk: apple: Add clk-apple-cluster driver to manage CPU p-states
arm64: apple: Select MEMORY and APPLE_MCC
arm64: apple: Add CPU frequency scaling support for t8103
.../bindings/clock/apple,cluster-clk.yaml | 115 ++++++++
.../memory-controllers/apple,mcc.yaml | 80 ++++++
.../opp/apple,mcc-operating-points.yaml | 62 +++++
MAINTAINERS | 5 +
arch/arm64/Kconfig.platforms | 2 +
arch/arm64/boot/dts/apple/t8103.dtsi | 255 +++++++++++++++++-
drivers/base/power/domain.c | 39 ++-
drivers/clk/Kconfig | 9 +
drivers/clk/Makefile | 1 +
drivers/clk/clk-apple-cluster.c | 184 +++++++++++++
drivers/memory/Kconfig | 9 +
drivers/memory/Makefile | 1 +
drivers/memory/apple-mcc.c | 130 +++++++++
drivers/opp/core.c | 5 +-
include/linux/pm_domain.h | 8 +
15 files changed, 887 insertions(+), 18 deletions(-)
create mode 100644 Documentation/devicetree/bindings/clock/apple,cluster-clk.yaml
create mode 100644 Documentation/devicetree/bindings/memory-controllers/apple,mcc.yaml
create mode 100644 Documentation/devicetree/bindings/opp/apple,mcc-operating-points.yaml
create mode 100644 drivers/clk/clk-apple-cluster.c
create mode 100644 drivers/memory/apple-mcc.c
--
2.33.0
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
From: Hector Martin <hidden> Date: 2021-10-11 16:57:38
Splitting this out from the drivers, since we have enough things going
on in parallel that MAINTAINERS merges are going to cause pain for
subsystems. This will go through the SoC tree instead.
Signed-off-by: Hector Martin <redacted>
---
MAINTAINERS | 5 +++++
1 file changed, 5 insertions(+)
From: Hector Martin <hidden> Date: 2021-10-11 16:57:45
This device represents the memory controller in Apple SoCs, and is
chiefly in charge of adjusting performance characteristics according to
system demand.
Signed-off-by: Hector Martin <redacted>
---
.../memory-controllers/apple,mcc.yaml | 80 +++++++++++++++++++
.../opp/apple,mcc-operating-points.yaml | 62 ++++++++++++++
2 files changed, 142 insertions(+)
create mode 100644 Documentation/devicetree/bindings/memory-controllers/apple,mcc.yaml
create mode 100644 Documentation/devicetree/bindings/opp/apple,mcc-operating-points.yaml
@@ -0,0 +1,80 @@+# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)+%YAML1.2+---+$id:http://devicetree.org/schemas/memory-controllers/apple,mcc.yaml#+$schema:http://devicetree.org/meta-schemas/core.yaml#++title:Apple SoC MCC memory controller performance controls++maintainers:+-Hector Martin <marcan@marcan.st>++description:|+Apple SoCs contain a multichannel memory controller that can have its+configuration changed to adjust to changing performance requirements from+the rest of the SoC. This node represents the controller and provides a+power domain provider that downstream devices can use to adjust the memory+controller performance level.++properties:+compatible:+items:+-enum:+-apple,t8103-mcc+-const:apple,mcc++reg:+maxItems:1++"#power-domain-cells":+const:0++operating-points-v2:+$ref:/schemas/types.yaml#/definitions/phandle-array+description:+A reference to the OPP table describing the memory controller performance+levels. Each OPP node should contain an `apple,memory-perf-config`+property that contains the configuration values for that performance+level.++apple,num-channels:+$ref:/schemas/types.yaml#/definitions/uint32+description:+The number of memory channels in use.++required:+-compatible+-reg+-"#power-domain-cells"+-operating-points-v2+-apple,num-channels++additionalProperties:false++examples:+# See clock/apple,cluster-clock.yaml for an example of downstream usage.+-|+mcc_opp:opp-table-2 {+compatible = "operating-points-v2";++mcc_lowperf:opp0 {+opp-level = <0>;+apple,memory-perf-config = <0x813057f 0x1800180>;+};+mcc_highperf:opp1 {+opp-level = <1>;+apple,memory-perf-config = <0x133 0x55555340>;+};+};+soc {+#address-cells = <2>;+#size-cells = <2>;++mcc:memory-controller@200200000 {+compatible = "apple,t8103-mcc", "apple,mcc";+#power-domain-cells = <0>;+reg = <0x2 0x200000 0x0 0x200000>;+operating-points-v2 = <&mcc_opp>;+apple,num-channels = <8>;+};+};
@@ -0,0 +1,62 @@+# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)+%YAML1.2+---+$id:http://devicetree.org/schemas/opp/apple,mcc-operating-points.yaml#+$schema:http://devicetree.org/meta-schemas/core.yaml#++title:Apple SoC memory controller OPP bindings++maintainers:+-Hector Martin <marcan@marcan.st>++description:|+Apple SoCs can have their memory controller performance adjusted depending on+system requirements. These performance states are represented by specific+memory controller register values. The apple-mcc driver uses these values+to change the MCC performance.++allOf:+-$ref:opp-v2-base.yaml#++properties:+compatible:+const:apple,mcc-operating-points++required:+-compatible++patternProperties:+"opp[0-9]+":+type:object++properties:+opp-level:true+apple,memory-perf-config:+$ref:/schemas/types.yaml#/definitions/uint32-array+description:|+A pair of register values used to configure this performance state.+minItems:2+maxItems:2++required:+-opp-level+-apple,memory-perf-config++unevaluatedProperties:false++additionalProperties:false++examples:+-|+mcc_opp:opp-table-2 {+compatible = "operating-points-v2";++mcc_lowperf:opp0 {+opp-level = <0>;+apple,memory-perf-config = <0x813057f 0x1800180>;+};+mcc_highperf:opp1 {+opp-level = <1>;+apple,memory-perf-config = <0x133 0x55555340>;+};+};
--
2.33.0
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
From: Krzysztof Kozlowski <hidden> Date: 2021-10-12 08:48:20
On 11/10/2021 18:57, Hector Martin wrote:
quoted hunk
This device represents the memory controller in Apple SoCs, and is
chiefly in charge of adjusting performance characteristics according to
system demand.
Signed-off-by: Hector Martin <redacted>
---
.../memory-controllers/apple,mcc.yaml | 80 +++++++++++++++++++
.../opp/apple,mcc-operating-points.yaml | 62 ++++++++++++++
2 files changed, 142 insertions(+)
create mode 100644 Documentation/devicetree/bindings/memory-controllers/apple,mcc.yaml
create mode 100644 Documentation/devicetree/bindings/opp/apple,mcc-operating-points.yaml
@@ -0,0 +1,80 @@+# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)+%YAML1.2+---+$id:http://devicetree.org/schemas/memory-controllers/apple,mcc.yaml#+$schema:http://devicetree.org/meta-schemas/core.yaml#++title:Apple SoC MCC memory controller performance controls++maintainers:+-Hector Martin <marcan@marcan.st>++description:|+Apple SoCs contain a multichannel memory controller that can have its+configuration changed to adjust to changing performance requirements from+the rest of the SoC. This node represents the controller and provides a+power domain provider that downstream devices can use to adjust the memory+controller performance level.++properties:+compatible:+items:+-enum:+-apple,t8103-mcc+-const:apple,mcc++reg:+maxItems:1++"#power-domain-cells":+const:0++operating-points-v2:+$ref:/schemas/types.yaml#/definitions/phandle-array+description:+A reference to the OPP table describing the memory controller performance+levels. Each OPP node should contain an `apple,memory-perf-config`+property that contains the configuration values for that performance+level.++apple,num-channels:+$ref:/schemas/types.yaml#/definitions/uint32+description:+The number of memory channels in use.++required:+-compatible+-reg+-"#power-domain-cells"+-operating-points-v2+-apple,num-channels++additionalProperties:false++examples:+# See clock/apple,cluster-clock.yaml for an example of downstream usage.+-|+mcc_opp:opp-table-2 {+compatible = "operating-points-v2";
@@ -0,0 +1,62 @@+# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)+%YAML1.2+---+$id:http://devicetree.org/schemas/opp/apple,mcc-operating-points.yaml#+$schema:http://devicetree.org/meta-schemas/core.yaml#++title:Apple SoC memory controller OPP bindings++maintainers:+-Hector Martin <marcan@marcan.st>++description:|+Apple SoCs can have their memory controller performance adjusted depending on+system requirements. These performance states are represented by specific+memory controller register values. The apple-mcc driver uses these values+to change the MCC performance.++allOf:+-$ref:opp-v2-base.yaml#++properties:+compatible:+const:apple,mcc-operating-points++required:+-compatible++patternProperties:+"opp[0-9]+":+type:object++properties:+opp-level:true
From: Rob Herring <robh@kernel.org> Date: 2021-10-19 22:44:04
On Tue, Oct 12, 2021 at 10:48:12AM +0200, Krzysztof Kozlowski wrote:
On 11/10/2021 18:57, Hector Martin wrote:
quoted
This device represents the memory controller in Apple SoCs, and is
chiefly in charge of adjusting performance characteristics according to
system demand.
Signed-off-by: Hector Martin <redacted>
---
.../memory-controllers/apple,mcc.yaml | 80 +++++++++++++++++++
.../opp/apple,mcc-operating-points.yaml | 62 ++++++++++++++
2 files changed, 142 insertions(+)
create mode 100644 Documentation/devicetree/bindings/memory-controllers/apple,mcc.yaml
create mode 100644 Documentation/devicetree/bindings/opp/apple,mcc-operating-points.yaml
@@ -0,0 +1,80 @@+# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)+%YAML1.2+---+$id:http://devicetree.org/schemas/memory-controllers/apple,mcc.yaml#+$schema:http://devicetree.org/meta-schemas/core.yaml#++title:Apple SoC MCC memory controller performance controls++maintainers:+-Hector Martin <marcan@marcan.st>++description:|+Apple SoCs contain a multichannel memory controller that can have its+configuration changed to adjust to changing performance requirements from+the rest of the SoC. This node represents the controller and provides a+power domain provider that downstream devices can use to adjust the memory+controller performance level.++properties:+compatible:+items:+-enum:+-apple,t8103-mcc+-const:apple,mcc++reg:+maxItems:1++"#power-domain-cells":+const:0++operating-points-v2:+$ref:/schemas/types.yaml#/definitions/phandle-array+description:+A reference to the OPP table describing the memory controller performance+levels. Each OPP node should contain an `apple,memory-perf-config`+property that contains the configuration values for that performance+level.++apple,num-channels:+$ref:/schemas/types.yaml#/definitions/uint32+description:+The number of memory channels in use.++required:+-compatible+-reg+-"#power-domain-cells"+-operating-points-v2+-apple,num-channels++additionalProperties:false++examples:+# See clock/apple,cluster-clock.yaml for an example of downstream usage.+-|+mcc_opp:opp-table-2 {+compatible = "operating-points-v2";
@@ -0,0 +1,62 @@+# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)+%YAML1.2+---+$id:http://devicetree.org/schemas/opp/apple,mcc-operating-points.yaml#+$schema:http://devicetree.org/meta-schemas/core.yaml#++title:Apple SoC memory controller OPP bindings++maintainers:+-Hector Martin <marcan@marcan.st>++description:|+Apple SoCs can have their memory controller performance adjusted depending on+system requirements. These performance states are represented by specific+memory controller register values. The apple-mcc driver uses these values+to change the MCC performance.++allOf:+-$ref:opp-v2-base.yaml#++properties:+compatible:+const:apple,mcc-operating-points++required:+-compatible++patternProperties:+"opp[0-9]+":+type:object++properties:+opp-level:true
You don't need to mention it.
Actually, you do.
You are thinking unevaluatedProperties takes care of it, but it doesn't
here. The problem is if you have 2 schemas (this one and
opp-v2-base.yaml) with child nodes, the child nodes in each schema are
evaluated separately.
So anywhere we have child nodes, we need the child node schema to be a
separate file or able to be directly referenced (i.e. under $defs). I
only realized this when testing out unevaluatedProperties support.
Rob
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
From: Hector Martin <hidden> Date: 2021-10-11 16:57:54
This device represents the CPU performance state switching mechanism as
a clock controller, to be used with the standard cpufreq-dt
infrastructure.
Signed-off-by: Hector Martin <redacted>
---
.../bindings/clock/apple,cluster-clk.yaml | 115 ++++++++++++++++++
1 file changed, 115 insertions(+)
create mode 100644 Documentation/devicetree/bindings/clock/apple,cluster-clk.yaml
@@ -0,0 +1,115 @@+# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)+%YAML1.2+---+$id:http://devicetree.org/schemas/clock/apple,cluster-clk.yaml#+$schema:http://devicetree.org/meta-schemas/core.yaml#++title:CPU cluster frequency scaling for Apple SoCs++maintainers:+-Hector Martin <marcan@marcan.st>++description:|+Apple SoCs control CPU cluster frequencies by using a performance state+index. This node represents the feature as a clock controller, and uses+a reference to the CPU OPP table to translate clock frequencies into+performance states. This allows the CPUs to use the standard cpufreq-dt+mechanism for frequency scaling.++properties:+compatible:+items:+-enum:+-apple,t8103-cluster-clk+-const:apple,cluster-clk++reg:+maxItems:1++'#clock-cells':+const:0++operating-points-v2:+$ref:/schemas/types.yaml#/definitions/phandle-array+description:+A reference to the OPP table used for the CPU cluster controlled by this+device instance. The table should contain an `opp-level` property for+every OPP, which represents the p-state index used by the hardware to+represent this performance level.++OPPs may also have a `required-opps` property (see power-domains).++power-domains:+maxItems:1+description:+An optional reference to a power domain provider that links its+performance state to the CPU cluster performance state. This is typically+a memory controller. If set, the `required-opps` property in the CPU+frequency OPP nodes will be used to change the performance state of this+provider state in tandem with CPU frequency changes.++required:+-compatible+-reg+-'#clock-cells'+-operating-points-v2++additionalProperties:false+++examples:+-|+pcluster_opp:opp-table-1 {+compatible = "operating-points-v2";+opp-shared;++opp01 {+opp-hz = /bits/ 64 <600000000>;+opp-microvolt = <781000>;+opp-level = <1>;+clock-latency-ns = <8000>;+required-opps = <&mcc_lowperf>;+};+/* intermediate p-states omitted */+opp15 {+opp-hz = /bits/ 64 <3204000000>;+opp-microvolt = <1081000>;+opp-level = <15>;+clock-latency-ns = <56000>;+required-opps = <&mcc_highperf>;+};+};++mcc_opp:opp-table-2 {+compatible = "operating-points-v2";++mcc_lowperf:opp0 {+opp-level = <0>;+apple,memory-perf-config = <0x813057f 0x1800180>;+};+mcc_highperf:opp1 {+opp-level = <1>;+apple,memory-perf-config = <0x133 0x55555340>;+};+};++soc {+#address-cells = <2>;+#size-cells = <2>;++mcc:memory-controller@200200000 {+compatible = "apple,t8103-mcc", "apple,mcc";+#power-domain-cells = <0>;+reg = <0x2 0x200000 0x0 0x200000>;+operating-points-v2 = <&mcc_opp>;+apple,num-channels = <8>;+};++clk_pcluster:clock-controller@211e20000 {+compatible = "apple,t8103-cluster-clk", "apple,cluster-clk";+#clock-cells = <0>;+reg = <0x2 0x11e20000 0x0 0x4000>;+operating-points-v2 = <&pcluster_opp>;+power-domains = <&mcc>;+};+};
--
2.33.0
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
From: Krzysztof Kozlowski <hidden> Date: 2021-10-12 08:51:25
On 11/10/2021 18:57, Hector Martin wrote:
quoted hunk
This device represents the CPU performance state switching mechanism as
a clock controller, to be used with the standard cpufreq-dt
infrastructure.
Signed-off-by: Hector Martin <redacted>
---
.../bindings/clock/apple,cluster-clk.yaml | 115 ++++++++++++++++++
1 file changed, 115 insertions(+)
create mode 100644 Documentation/devicetree/bindings/clock/apple,cluster-clk.yaml
@@ -0,0 +1,115 @@+# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)+%YAML1.2+---+$id:http://devicetree.org/schemas/clock/apple,cluster-clk.yaml#+$schema:http://devicetree.org/meta-schemas/core.yaml#++title:CPU cluster frequency scaling for Apple SoCs++maintainers:+-Hector Martin <marcan@marcan.st>++description:|+Apple SoCs control CPU cluster frequencies by using a performance state+index. This node represents the feature as a clock controller, and uses+a reference to the CPU OPP table to translate clock frequencies into+performance states. This allows the CPUs to use the standard cpufreq-dt+mechanism for frequency scaling.++properties:+compatible:+items:+-enum:+-apple,t8103-cluster-clk+-const:apple,cluster-clk++reg:+maxItems:1++'#clock-cells':+const:0++operating-points-v2:+$ref:/schemas/types.yaml#/definitions/phandle-array+description:+A reference to the OPP table used for the CPU cluster controlled by this+device instance. The table should contain an `opp-level` property for+every OPP, which represents the p-state index used by the hardware to+represent this performance level.++OPPs may also have a `required-opps` property (see power-domains).++power-domains:+maxItems:1+description:+An optional reference to a power domain provider that links its+performance state to the CPU cluster performance state. This is typically+a memory controller. If set, the `required-opps` property in the CPU+frequency OPP nodes will be used to change the performance state of this+provider state in tandem with CPU frequency changes.++required:+-compatible+-reg+-'#clock-cells'+-operating-points-v2++additionalProperties:false++
The opp-level thing wasn't designed to work this way, though it may
work just fine. It was designed as a unique key for power-domains,
which don't have opp-hz. The OPP core currently looks at 3 different
values, which can act as a unique key to identify the OPP. clk-rate,
bandwidth and level.
I think this is the first platform which has both hz and level in the
CPUs OPP table. What exactly is level in this case ?
Again, it may work fine, I just don't know where it may end up
breaking :)
--
viresh
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
@@ -0,0 +1,115 @@+# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)+%YAML1.2+---+$id:http://devicetree.org/schemas/clock/apple,cluster-clk.yaml#+$schema:http://devicetree.org/meta-schemas/core.yaml#++title:CPU cluster frequency scaling for Apple SoCs++maintainers:+-Hector Martin <marcan@marcan.st>++description:|+Apple SoCs control CPU cluster frequencies by using a performance state+index. This node represents the feature as a clock controller, and uses+a reference to the CPU OPP table to translate clock frequencies into+performance states. This allows the CPUs to use the standard cpufreq-dt+mechanism for frequency scaling.++properties:+compatible:+items:+-enum:+-apple,t8103-cluster-clk+-const:apple,cluster-clk
Is the generic compatible going to be used? The typical approach is to
only have SoC specific compatible strings, especially if the generic
compatible can't actually do anything besides generally identify a
device.
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
From: Hector Martin <hidden> Date: 2021-10-11 16:57:58
When required-opps is used in CPU OPP tables, there is no parent power
domain to drive it. Squelch this error, to allow a clock driver to
handle this directly instead.
Signed-off-by: Hector Martin <redacted>
---
drivers/opp/core.c | 5 +++--
1 file changed, 3 insertions(+), 2 deletions(-)
@@ -873,12 +873,13 @@ static int _set_required_opp(struct device *dev, struct device *pd_dev,return0;ret=dev_pm_genpd_set_performance_state(pd_dev,pstate);-if(ret){+if(ret&&ret!=-ENODEV){dev_err(dev,"Failed to set performance rate of %s: %d (%d)\n",dev_name(pd_dev),pstate,ret);+returnret;}-returnret;+return0;}/* This is only called for PM domain for now */
--
2.33.0
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
When required-opps is used in CPU OPP tables, there is no parent power
domain to drive it. Squelch this error, to allow a clock driver to
handle this directly instead.
Signed-off-by: Hector Martin <redacted>
---
drivers/opp/core.c | 5 +++--
1 file changed, 3 insertions(+), 2 deletions(-)
@@ -873,12 +873,13 @@ static int _set_required_opp(struct device *dev, struct device *pd_dev,return0;ret=dev_pm_genpd_set_performance_state(pd_dev,pstate);-if(ret){+if(ret&&ret!=-ENODEV){dev_err(dev,"Failed to set performance rate of %s: %d (%d)\n",dev_name(pd_dev),pstate,ret);+returnret;}-returnret;+return0;}/* This is only called for PM domain for now */
I am not sure why you need this, since _set_required_opps() has this check:
if (unlikely(!required_opp_tables[0]->is_genpd)) {
dev_err(dev, "required-opps don't belong to a genpd\n");
return -ENOENT;
}
--
viresh
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
From: Hector Martin <hidden> Date: 2021-10-12 05:34:29
On 12/10/2021 12.21, Viresh Kumar wrote:
I am not sure why you need this, since _set_required_opps() has this check:
if (unlikely(!required_opp_tables[0]->is_genpd)) {
dev_err(dev, "required-opps don't belong to a genpd\n");
return -ENOENT;
}
The table *is* assigned to a genpd (the memory controller), it's just
that that genpd isn't actually a parent of the CPU device. Without the
patch you end up with:
[ 3.040060] cpu cpu4: Failed to set performance rate of cpu4: 0 (-19)
[ 3.042881] cpu cpu4: Failed to set required opps: -19
[ 3.045508] cpufreq: __target_index: Failed to change cpu frequency: -19
--
Hector Martin (marcan@marcan.st)
Public Key: https://mrcn.st/pub
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
The table *is* assigned to a genpd (the memory controller), it's just that
that genpd isn't actually a parent of the CPU device. Without the patch you
end up with:
[ 3.040060] cpu cpu4: Failed to set performance rate of cpu4: 0 (-19)
[ 3.042881] cpu cpu4: Failed to set required opps: -19
[ 3.045508] cpufreq: __target_index: Failed to change cpu frequency: -19
Hmm, Saravana and Sibi were working on a similar problem earlier and decided to
solve this using devfreq instead. Don't remember the exact series which got
merged for this, Sibi ?
If this part fails, how do you actually set the performance state of the memory
controller's genpd ?
--
viresh
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
From: Hector Martin <hidden> Date: 2021-10-12 05:58:04
On 12/10/2021 14.51, Viresh Kumar wrote:
On 12-10-21, 14:34, Hector Martin wrote:
quoted
The table *is* assigned to a genpd (the memory controller), it's just that
that genpd isn't actually a parent of the CPU device. Without the patch you
end up with:
[ 3.040060] cpu cpu4: Failed to set performance rate of cpu4: 0 (-19)
[ 3.042881] cpu cpu4: Failed to set required opps: -19
[ 3.045508] cpufreq: __target_index: Failed to change cpu frequency: -19
Hmm, Saravana and Sibi were working on a similar problem earlier and decided to
solve this using devfreq instead. Don't remember the exact series which got
merged for this, Sibi ?
If this part fails, how do you actually set the performance state of the memory
controller's genpd ?
The clock controller has the genpd as an actual power-domain parent, so
it does it instead. From patch #7:
+ if (cluster->has_pd)
+ dev_pm_genpd_set_performance_state(cluster->dev,
+ dev_pm_opp_get_required_pstate(opp, 0));
+
This is arguably not entirely representative of how the hardware works,
since technically the cluster switching couldn't care less what the
memory controller is doing; it's a soft dependency, states that should
be switched together but are not interdependent (in fact, the clock code
does this unconditionally after the CPU p-state change, regardless of
whether we're shifting up or down; this is, FWIW, the same order macOS
uses, and it clearly doesn't matter which way you do it).
--
Hector Martin (marcan@marcan.st)
Public Key: https://mrcn.st/pub
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
The table *is* assigned to a genpd (the memory controller), it's just that
that genpd isn't actually a parent of the CPU device. Without the patch you
end up with:
[ 3.040060] cpu cpu4: Failed to set performance rate of cpu4: 0 (-19)
[ 3.042881] cpu cpu4: Failed to set required opps: -19
[ 3.045508] cpufreq: __target_index: Failed to change cpu frequency: -19
Hmm, Saravana and Sibi were working on a similar problem earlier and decided to
solve this using devfreq instead. Don't remember the exact series which got
merged for this, Sibi ?
If this part fails, how do you actually set the performance state of the memory
controller's genpd ?
The clock controller has the genpd as an actual power-domain parent, so it
does it instead. From patch #7:
quoted
+ if (cluster->has_pd)
+ dev_pm_genpd_set_performance_state(cluster->dev,
+ dev_pm_opp_get_required_pstate(opp, 0));
+
This is arguably not entirely representative of how the hardware works,
since technically the cluster switching couldn't care less what the memory
controller is doing; it's a soft dependency, states that should be switched
together but are not interdependent (in fact, the clock code does this
unconditionally after the CPU p-state change, regardless of whether we're
shifting up or down; this is, FWIW, the same order macOS uses, and it
clearly doesn't matter which way you do it).
Yeah, I understand what you are doing. But the current patch is
incorrect in the sense that it can cause a bug on other platforms. To
make this work, you should rather set this genpd as parent of CPU
devices (which are doing anyway since you are updating them with CPU's
DVFS). With that the clk driver won't be required to do the magic
behind the scene.
--
viresh
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
From: Hector Martin "marcan" <hidden> Date: 2021-10-12 09:31:32
On 2021年10月12日 18:26:03 JST, Viresh Kumar [off-list ref] wrote:
On 12-10-21, 14:57, Hector Martin wrote:
quoted
This is arguably not entirely representative of how the hardware works,
since technically the cluster switching couldn't care less what the memory
controller is doing; it's a soft dependency, states that should be switched
together but are not interdependent (in fact, the clock code does this
unconditionally after the CPU p-state change, regardless of whether we're
shifting up or down; this is, FWIW, the same order macOS uses, and it
clearly doesn't matter which way you do it).
Yeah, I understand what you are doing. But the current patch is
incorrect in the sense that it can cause a bug on other platforms. To
make this work, you should rather set this genpd as parent of CPU
devices (which are doing anyway since you are updating them with CPU's
DVFS). With that the clk driver won't be required to do the magic
behind the scene.
That doesn't work, though, because the CPUs aren't normal devices with runtime-pm. That was the first thing I tried :).
If you think this *should* be made to work instead then I can try that.
--
Hector Martin "marcan" (marcan@marcan.st)
Public key: https://mrcn.st/pub
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
From: Hector Martin <hidden> Date: 2021-10-14 06:53:06
On 12/10/2021 18.32, Viresh Kumar wrote:
On 12-10-21, 18:31, Hector Martin "marcan" wrote:
quoted
That doesn't work, though, because the CPUs aren't normal devices
with runtime-pm. That was the first thing I tried :).
What's the exact problem with runtime PM here ?
The CPU devices aren't attached to their genpd, so the required OPP
transition fails with the same error.
However, this was easier to fix than I expected. With this patch to
cpufreq-dt, it all works properly, and I can drop the parent genpd
from the clock node and related handling. Thoughts?
commit c4f88743374c1f4678ee7f17fb6cae30ded9ed59
Author: Hector Martin [off-list ref]
Date: Thu Oct 14 15:47:45 2021 +0900
cpufreq: dt: Attach CPU devices to power domains
This allows the required-opps mechanism to work for CPU OPP tables,
triggering specific OPP levels in a parent power domain.
Signed-off-by: Hector Martin [off-list ref]
@@ -264,6 +265,16 @@ static int dt_cpufreq_early_init(struct device *dev, int cpu)gotoout;}+/*+*AttachtheCPUdevicetoitsgenpddomain(ifany),toallowOPP+*dependenciestobesatisfied.+*/+ret=genpd_dev_pm_attach(cpu_dev);+if(ret<=0){+dev_err(cpu_dev,"Failed to attach CPU device to genpd\n");+gotoout;+}+/**TheOPPtablemustbeinitialized,staticallyordynamically,bythis*point.
--
Hector Martin (marcan@marcan.st)
Public Key: https://mrcn.st/pub
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
The CPU devices aren't attached to their genpd, so the required OPP
transition fails with the same error.
However, this was easier to fix than I expected. With this patch to
cpufreq-dt, it all works properly, and I can drop the parent genpd
from the clock node and related handling. Thoughts?
commit c4f88743374c1f4678ee7f17fb6cae30ded9ed59
Author: Hector Martin [off-list ref]
Date: Thu Oct 14 15:47:45 2021 +0900
cpufreq: dt: Attach CPU devices to power domains
This allows the required-opps mechanism to work for CPU OPP tables,
triggering specific OPP levels in a parent power domain.
Signed-off-by: Hector Martin [off-list ref]
@@ -264,6 +265,16 @@ static int dt_cpufreq_early_init(struct device *dev, int cpu)gotoout;}+/*+*AttachtheCPUdevicetoitsgenpddomain(ifany),toallowOPP+*dependenciestobesatisfied.+*/+ret=genpd_dev_pm_attach(cpu_dev);+if(ret<=0){+dev_err(cpu_dev,"Failed to attach CPU device to genpd\n");+gotoout;+}+
Other platform do this from some other place I think.
Ulf, where should this code be moved ? cpu-clk driver ?
--
viresh
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
From: Hector Martin <hidden> Date: 2021-10-14 07:03:48
On 14/10/2021 15.56, Viresh Kumar wrote:
quoted
+ /*
+ * Attach the CPU device to its genpd domain (if any), to allow OPP
+ * dependencies to be satisfied.
+ */
+ ret = genpd_dev_pm_attach(cpu_dev);
+ if (ret <= 0) {
+ dev_err(cpu_dev, "Failed to attach CPU device to genpd\n");
+ goto out;
+ }
+
Other platform do this from some other place I think.
Ulf, where should this code be moved ? cpu-clk driver ?
I see one driver that does this is drivers/clk/qcom/apcs-sdx55.c (via
dev_pm_domain_attach). Though it only does it for CPU#0; we need to do
it for all CPUs.
--
Hector Martin (marcan@marcan.st)
Public Key: https://mrcn.st/pub
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
+ /*
+ * Attach the CPU device to its genpd domain (if any), to allow OPP
+ * dependencies to be satisfied.
+ */
+ ret = genpd_dev_pm_attach(cpu_dev);
+ if (ret <= 0) {
+ dev_err(cpu_dev, "Failed to attach CPU device to genpd\n");
+ goto out;
+ }
+
Other platform do this from some other place I think.
Ulf, where should this code be moved ? cpu-clk driver ?
I see one driver that does this is drivers/clk/qcom/apcs-sdx55.c (via
dev_pm_domain_attach).
That may be a good place since you are already adding it and it is related to
CPU clk.
Though it only does it for CPU#0; we need to do it
for all CPUs.
From: Hector Martin <hidden> Date: 2021-10-14 07:23:32
On 14/10/2021 16.03, Hector Martin wrote:
On 14/10/2021 15.56, Viresh Kumar wrote:
quoted
quoted
+ /*
+ * Attach the CPU device to its genpd domain (if any), to allow OPP
+ * dependencies to be satisfied.
+ */
+ ret = genpd_dev_pm_attach(cpu_dev);
+ if (ret <= 0) {
+ dev_err(cpu_dev, "Failed to attach CPU device to genpd\n");
+ goto out;
+ }
+
Other platform do this from some other place I think.
Ulf, where should this code be moved ? cpu-clk driver ?
I see one driver that does this is drivers/clk/qcom/apcs-sdx55.c (via
dev_pm_domain_attach). Though it only does it for CPU#0; we need to do
it for all CPUs.
Looking into this further, I'm not sure I like the idea of doing this in
the clocks driver. There might be locking issues since it gets
instantiated twice and yet doesn't really itself know what subset of
CPUs it applies to.
There's another driver that does this:
drivers/cpuidle/cpuidle-psci-domain.c. That one specifically looks for a
power domain called "psci". Perhaps it would make sense to make this
generic in cpufreq-dt as per my prior patch, but explicitly request a
"cpufreq" domain? That way only devicetrees that opt in to having this
handled by cpufreq by naming it that way would get this behavior.
--
Hector Martin (marcan@marcan.st)
Public Key: https://mrcn.st/pub
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
On Thu, 14 Oct 2021 at 09:23, Hector Martin [off-list ref] wrote:
On 14/10/2021 16.03, Hector Martin wrote:
quoted
On 14/10/2021 15.56, Viresh Kumar wrote:
quoted
quoted
+ /*
+ * Attach the CPU device to its genpd domain (if any), to allow OPP
+ * dependencies to be satisfied.
+ */
+ ret = genpd_dev_pm_attach(cpu_dev);
+ if (ret <= 0) {
+ dev_err(cpu_dev, "Failed to attach CPU device to genpd\n");
+ goto out;
+ }
+
Other platform do this from some other place I think.
Ulf, where should this code be moved ? cpu-clk driver ?
I see one driver that does this is drivers/clk/qcom/apcs-sdx55.c (via
dev_pm_domain_attach). Though it only does it for CPU#0; we need to do
it for all CPUs.
Looking into this further, I'm not sure I like the idea of doing this in
the clocks driver. There might be locking issues since it gets
instantiated twice and yet doesn't really itself know what subset of
CPUs it applies to.
I agree. I suggest you look into using a genpd provider and hook up
all CPU's devices to it. I think that is what Viresh also suggested
earlier - and this makes most sense to me.
As a reference you may have a look at some Qcom platforms that already use this:
arch/arm64/boot/dts/qcom/qcs404.dtsi
drivers/cpufreq/qcom-cpufreq-nvmem.c:
To hook up CPU devices to their PM domains (genpds) - it calls
dev_pm_opp_attach_genpd(), which is a kind of wrapper for
dev_pm_domain_attach_by_name().
drivers/soc/qcom/cpr.c
Registers the genpd provider that is capable of dealing with
performance states/OPPs for CPUs.
There's another driver that does this:
drivers/cpuidle/cpuidle-psci-domain.c. That one specifically looks for a
power domain called "psci". Perhaps it would make sense to make this
generic in cpufreq-dt as per my prior patch, but explicitly request a
"cpufreq" domain? That way only devicetrees that opt in to having this
handled by cpufreq by naming it that way would get this behavior.
That sounds like an idea that is worth exploring. In this way, the
only thing that needs to be implemented for new cases would be the
genpd provider driver.
BTW, as you will figure out by looking at the above references, for
the qcom case we are using "cpr" as the domain name for cpufreq. Of
course, that doesn't mean we can use "cpufreq" (or whatever name that
makes sense) going forward for new cases.
On Tue, 12 Oct 2021 at 07:57, Hector Martin [off-list ref] wrote:
On 12/10/2021 14.51, Viresh Kumar wrote:
quoted
On 12-10-21, 14:34, Hector Martin wrote:
quoted
The table *is* assigned to a genpd (the memory controller), it's just that
that genpd isn't actually a parent of the CPU device. Without the patch you
end up with:
[ 3.040060] cpu cpu4: Failed to set performance rate of cpu4: 0 (-19)
[ 3.042881] cpu cpu4: Failed to set required opps: -19
[ 3.045508] cpufreq: __target_index: Failed to change cpu frequency: -19
Hmm, Saravana and Sibi were working on a similar problem earlier and decided to
solve this using devfreq instead. Don't remember the exact series which got
merged for this, Sibi ?
If this part fails, how do you actually set the performance state of the memory
controller's genpd ?
The clock controller has the genpd as an actual power-domain parent, so
it does it instead. From patch #7:
quoted
+ if (cluster->has_pd)
+ dev_pm_genpd_set_performance_state(cluster->dev,
+ dev_pm_opp_get_required_pstate(opp, 0));
+
This is arguably not entirely representative of how the hardware works,
since technically the cluster switching couldn't care less what the
memory controller is doing; it's a soft dependency, states that should
be switched together but are not interdependent (in fact, the clock code
does this unconditionally after the CPU p-state change, regardless of
whether we're shifting up or down; this is, FWIW, the same order macOS
uses, and it clearly doesn't matter which way you do it).
Yes, this sounds like you should move away from modeling the memory
part as a parent genpd for the CPUs' genpd.
As Viresh pointed out, a devfreq driver seems like a better way to do
this. As a matter of fact, there are already devfreq drivers that do
this, unless I am mistaken.
It looks like devfreq providers are listening to opp/cpufreq
notifiers, as to get an indication of when it could make sense to
change a performance state.
In some cases the devfreq provider is also modeled as an interconnect
provider, allowing consumers to specify memory bandwidth constraints,
which may trigger a new performance state to be set for the memory
controller.
In the tegra case, the memory controller is modelled as an
interconnect provider and the devfreq node is modelled as an
interconnect-consumer of the memory controller. Perhaps this can work
for apple SoCs too?
That said, perhaps as an option to move forward, we can try to get the
cpufreq pieces solved first. Then as a step on top, add the
performance scaling for the memory controller?
From: Hector Martin <hidden> Date: 2021-10-14 11:44:04
On 14/10/2021 18.55, Ulf Hansson wrote:
Yes, this sounds like you should move away from modeling the memory
part as a parent genpd for the CPUs' genpd.
As Viresh pointed out, a devfreq driver seems like a better way to do
this. As a matter of fact, there are already devfreq drivers that do
this, unless I am mistaken.
It looks like devfreq providers are listening to opp/cpufreq
notifiers, as to get an indication of when it could make sense to
change a performance state.
In some cases the devfreq provider is also modeled as an interconnect
provider, allowing consumers to specify memory bandwidth constraints,
which may trigger a new performance state to be set for the memory
controller.
In the tegra case, the memory controller is modelled as an
interconnect provider and the devfreq node is modelled as an
interconnect-consumer of the memory controller. Perhaps this can work
for apple SoCs too?
I was poking around and noticed the OPP core can already integrate with
interconnect requirements, so perhaps the memory controller can be an
interconnect provider, and the CPU nodes can directly reference it as a
consumer? This seems like a more accurate model of what the hardware
does, and I think I saw some devices doing this already.
(only problem is I have no idea of the actual bandwidth numbers involved
here... I'll have to run some benchmarks to make sure this isn't just
completely dummy data)
That said, perhaps as an option to move forward, we can try to get the
cpufreq pieces solved first. Then as a step on top, add the
performance scaling for the memory controller?
Sure; that's a pretty much independent part of this patchset, though I'm
thinking I might as well try some things out for v2 anyway; if it looks
like it'll take longer we can split it out and do just the cpufreq side.
--
Hector Martin (marcan@marcan.st)
Public Key: https://mrcn.st/pub
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
On Thu, 14 Oct 2021 at 13:43, Hector Martin [off-list ref] wrote:
On 14/10/2021 18.55, Ulf Hansson wrote:
quoted
Yes, this sounds like you should move away from modeling the memory
part as a parent genpd for the CPUs' genpd.
As Viresh pointed out, a devfreq driver seems like a better way to do
this. As a matter of fact, there are already devfreq drivers that do
this, unless I am mistaken.
It looks like devfreq providers are listening to opp/cpufreq
notifiers, as to get an indication of when it could make sense to
change a performance state.
In some cases the devfreq provider is also modeled as an interconnect
provider, allowing consumers to specify memory bandwidth constraints,
which may trigger a new performance state to be set for the memory
controller.
In the tegra case, the memory controller is modelled as an
interconnect provider and the devfreq node is modelled as an
interconnect-consumer of the memory controller. Perhaps this can work
for apple SoCs too?
I was poking around and noticed the OPP core can already integrate with
interconnect requirements, so perhaps the memory controller can be an
interconnect provider, and the CPU nodes can directly reference it as a
consumer? This seems like a more accurate model of what the hardware
does, and I think I saw some devices doing this already.
Yeah, that could work too. And, yes, I agree, it may be a better
description of the HW.
(only problem is I have no idea of the actual bandwidth numbers involved
here... I'll have to run some benchmarks to make sure this isn't just
completely dummy data)
quoted
That said, perhaps as an option to move forward, we can try to get the
cpufreq pieces solved first. Then as a step on top, add the
performance scaling for the memory controller?
Sure; that's a pretty much independent part of this patchset, though I'm
thinking I might as well try some things out for v2 anyway; if it looks
like it'll take longer we can split it out and do just the cpufreq side.
In any case, I do my best to help with review.
Kind regards
Uffe
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
From: Hector Martin <hidden> Date: 2021-10-14 17:02:32
On 14/10/2021 21.55, Ulf Hansson wrote:
On Thu, 14 Oct 2021 at 13:43, Hector Martin [off-list ref] wrote:
quoted
I was poking around and noticed the OPP core can already integrate with
interconnect requirements, so perhaps the memory controller can be an
interconnect provider, and the CPU nodes can directly reference it as a
consumer? This seems like a more accurate model of what the hardware
does, and I think I saw some devices doing this already.
Yeah, that could work too. And, yes, I agree, it may be a better
description of the HW.
quoted
(only problem is I have no idea of the actual bandwidth numbers involved
here... I'll have to run some benchmarks to make sure this isn't just
completely dummy data)
So... I tried getting bandwidth numbers and failed. It seems these
registers don't actually affect peak performance in any measurable way.
I'm also getting almost the same GeekBench scores on macOS with and
without this mechanism enabled, although there is one subtest that seems
to show a measurable difference.
My current guess is this is something more subtle (latencies? idle
timers and such?) than a performance state. If that is the case, do you
have any ideas as to the best way to model it in Linux? Should we even
bother if it mostly has a minimal performance gain for typical workloads?
I'll try to do some latency tests, see if I can make sense of what it's
actually doing.
--
Hector Martin (marcan@marcan.st)
Public Key: https://mrcn.st/pub
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
On Thu, 14 Oct 2021 at 19:02, Hector Martin [off-list ref] wrote:
On 14/10/2021 21.55, Ulf Hansson wrote:
quoted
On Thu, 14 Oct 2021 at 13:43, Hector Martin [off-list ref] wrote:
quoted
I was poking around and noticed the OPP core can already integrate with
interconnect requirements, so perhaps the memory controller can be an
interconnect provider, and the CPU nodes can directly reference it as a
consumer? This seems like a more accurate model of what the hardware
does, and I think I saw some devices doing this already.
Yeah, that could work too. And, yes, I agree, it may be a better
description of the HW.
quoted
(only problem is I have no idea of the actual bandwidth numbers involved
here... I'll have to run some benchmarks to make sure this isn't just
completely dummy data)
So... I tried getting bandwidth numbers and failed. It seems these
registers don't actually affect peak performance in any measurable way.
I'm also getting almost the same GeekBench scores on macOS with and
without this mechanism enabled, although there is one subtest that seems
to show a measurable difference.
My current guess is this is something more subtle (latencies? idle
timers and such?) than a performance state. If that is the case, do you
have any ideas as to the best way to model it in Linux? Should we even
bother if it mostly has a minimal performance gain for typical workloads?
For latency constraints, we have dev_pm_qos. This will make the genpd
governor, to prevent deeper idle states for the device and its
corresponding PM domain (genpd). But that doesn't sound like a good
fit here.
If you are right, it rather sounds like there is some kind of
quiescence mode of the memory controller that can be prevented. But I
have no clue, of course. :-)
I'll try to do some latency tests, see if I can make sense of what it's
actually doing.
@@ -2247,7 +2243,10 @@ int of_genpd_add_provider_simple(struct device_node *np,/* Parse genpd OPP table */if(genpd->set_performance_state){-ret=dev_pm_opp_of_add_table(&genpd->dev);+if(getclk)+ret=dev_pm_opp_of_add_table(&genpd->dev);+else+ret=dev_pm_opp_of_add_table_noclk(&genpd->dev,0);if(ret){if(ret!=-EPROBE_DEFER)dev_err(&genpd->dev,"Failed to add OPP table: %d\n",
@@ -2278,8 +2277,32 @@ int of_genpd_add_provider_simple(struct device_node *np,return0;}++/**+*of_genpd_add_provider_simple()-RegisterasimplePMdomainprovider+*@np:DevicenodepointerassociatedwiththePMdomainprovider.+*@genpd:PointertoPMdomainassociatedwiththePMdomainprovider.+*/+intof_genpd_add_provider_simple(structdevice_node*np,+structgeneric_pm_domain*genpd)+{+return_of_genpd_add_provider_simple(np,genpd,true);+}EXPORT_SYMBOL_GPL(of_genpd_add_provider_simple);+/**+*of_genpd_add_provider_simple_noclk()-Registerasimpleclockless+*PMdomainprovider+*@np:DevicenodepointerassociatedwiththePMdomainprovider.+*@genpd:PointertoPMdomainassociatedwiththePMdomainprovider.+*/+intof_genpd_add_provider_simple_noclk(structdevice_node*np,+structgeneric_pm_domain*genpd)+{+return_of_genpd_add_provider_simple(np,genpd,false);+}+EXPORT_SYMBOL_GPL(of_genpd_add_provider_simple_noclk);+/***of_genpd_add_provider_onecell()-RegisteraonecellPMdomainprovider*@np:DevicenodepointerassociatedwiththePMdomainprovider.
From: Hector Martin <hidden> Date: 2021-10-11 16:59:00
This driver binds to the memory controller hardware in Apple SoCs such
as the Apple M1, and provides a power domain that downstream devices can
use to change the performance state of the memory controller.
Signed-off-by: Hector Martin <redacted>
---
drivers/memory/Kconfig | 9 +++
drivers/memory/Makefile | 1 +
drivers/memory/apple-mcc.c | 130 +++++++++++++++++++++++++++++++++++++
3 files changed, 140 insertions(+)
create mode 100644 drivers/memory/apple-mcc.c
From: Krzysztof Kozlowski <hidden> Date: 2021-10-12 09:19:20
On 11/10/2021 18:57, Hector Martin wrote:
quoted hunk
This driver binds to the memory controller hardware in Apple SoCs such
as the Apple M1, and provides a power domain that downstream devices can
use to change the performance state of the memory controller.
Signed-off-by: Hector Martin <redacted>
---
drivers/memory/Kconfig | 9 +++
drivers/memory/Makefile | 1 +
drivers/memory/apple-mcc.c | 130 +++++++++++++++++++++++++++++++++++++
3 files changed, 140 insertions(+)
create mode 100644 drivers/memory/apple-mcc.c
From: Hector Martin <hidden> Date: 2021-10-14 06:59:37
On 12/10/2021 18.19, Krzysztof Kozlowski wrote:
quoted
+// SPDX-License-Identifier: GPL-2.0-only OR MIT
+/*
+ * Apple SoC MCC memory controller performance control driver
+ *
+ * Copyright The Asahi Linux Contributors
+ mcc->reg_base = devm_platform_ioremap_resource(pdev, 0);
+ if (IS_ERR(mcc->reg_base))
+ return PTR_ERR(mcc->reg_base);
+
+ if (of_property_read_u32(node, "apple,num-channels", &mcc->num_channels)) {
Don't you have a limit of supported channels? It cannot be any uint32...
Today, it's max 8. But if come Monday we find out Apple's new chips have
16 channels and otherwise the same register layout, I'd much rather not
have to change the driver...
+
+MODULE_AUTHOR("Hector Martin [off-list ref]");
+MODULE_DESCRIPTION("MCC memory controller performance tuning driver for Apple SoCs");
+MODULE_LICENSE("GPL v2");
I think this will be "Dual MIT/GPL", based on your SPDX.
Ah, I didn't realize that was a valid option for MODULE_LICENSE. I guess
anything containing "GPL" works with EXPORT_SYMBOL_GPL?
Thanks for the review!
--
Hector Martin (marcan@marcan.st)
Public Key: https://mrcn.st/pub
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
From: Krzysztof Kozlowski <hidden> Date: 2021-10-14 07:36:31
On 14/10/2021 08:59, Hector Martin wrote:
On 12/10/2021 18.19, Krzysztof Kozlowski wrote:
quoted
quoted
+// SPDX-License-Identifier: GPL-2.0-only OR MIT
+/*
+ * Apple SoC MCC memory controller performance control driver
+ *
+ * Copyright The Asahi Linux Contributors
+ mcc->reg_base = devm_platform_ioremap_resource(pdev, 0);
+ if (IS_ERR(mcc->reg_base))
+ return PTR_ERR(mcc->reg_base);
+
+ if (of_property_read_u32(node, "apple,num-channels", &mcc->num_channels)) {
Don't you have a limit of supported channels? It cannot be any uint32...
Today, it's max 8. But if come Monday we find out Apple's new chips have
16 channels and otherwise the same register layout, I'd much rather not
have to change the driver...
OK, however if the driver ever receives different DT with a different
value, it will accept it unconditionally and go via address space. I am
just saying that being conservative on received values is safer, but I
am fine with skipping this problem. At the end we trust DT that it will
always match the kernel, don't we? Oh wait, someone can use DT from
other kernel in this one...
+
+MODULE_AUTHOR("Hector Martin [off-list ref]");
+MODULE_DESCRIPTION("MCC memory controller performance tuning driver for Apple SoCs");
+MODULE_LICENSE("GPL v2");
I think this will be "Dual MIT/GPL", based on your SPDX.
Ah, I didn't realize that was a valid option for MODULE_LICENSE. I guess
anything containing "GPL" works with EXPORT_SYMBOL_GPL?
I don't think exporting symbols is related to how you license your code.
Best regards,
Krzysztof
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
From: Hector Martin <hidden> Date: 2021-10-14 07:52:15
On 14/10/2021 16.36, Krzysztof Kozlowski wrote:
On 14/10/2021 08:59, Hector Martin wrote:
quoted
quoted
Don't you have a limit of supported channels? It cannot be any uint32...
Today, it's max 8. But if come Monday we find out Apple's new chips have
16 channels and otherwise the same register layout, I'd much rather not
have to change the driver...
OK, however if the driver ever receives different DT with a different
value, it will accept it unconditionally and go via address space. I am
just saying that being conservative on received values is safer, but I
am fine with skipping this problem. At the end we trust DT that it will
always match the kernel, don't we? Oh wait, someone can use DT from
other kernel in this one...
DTs using these compatibles should have the same register layout, and
should work with this driver; if a new chip comes out that has a
different register layout we will change the compatibles (both) and
therefore older kernels won't bind at all. If it has the same layout
we'll keep the base compatible, `reg` will grow as needed to accomodate
the extra channels, and e.g. num-channels=16 will then just work on
older kernels with no changes.
Obviously a broken DT with an insane value here would crash the driver,
but so would any other number of crazy DT things; however, I don't
expect that to ever happen.
There's also the case where we end up with multiple memory controllers
at discrete offsets (e.g. rumored multi-die configurations); in that
case we'll end up with multiple genpd parents and have to add code to
support that, and in the meantime older kernels will just have broken
cpufreq on the p-cores. But I think that is ~acceptable as long as the
system boots; we don't expect to be able to *fully* support newer SoCs
on older kernels with no code changes. What I'm aiming for is just
making the system work, hopefully with NVMe and USB and a dumb
framebuffer, so that distro installers can run and then users can later
install a proper up to date kernel will full support for the new SoC.
quoted
Ah, I didn't realize that was a valid option for MODULE_LICENSE. I guess
anything containing "GPL" works with EXPORT_SYMBOL_GPL?
I don't think exporting symbols is related to how you license your code.
It is; only modules with a GPL-compatible MODULE_LICENSE get to use
symbols exported via EXPORT_SYMBOL_GPL.
See kernel/module.c for the symbol lookup logic and
include/linux/license.h for the logic to check the string (seems like
"Dual MIT/GPL" is explicitly whitelisted there).
Of course, this is a futile effort, as ~every time I see a proprietary
module in some embedded device, it either falsely declares itself to be
GPL, or they have a shim module that re-exports GPL symbols as non-GPL.
--
Hector Martin (marcan@marcan.st)
Public Key: https://mrcn.st/pub
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
From: Krzysztof Kozlowski <hidden> Date: 2021-10-14 08:04:58
On 14/10/2021 09:52, Hector Martin wrote:
On 14/10/2021 16.36, Krzysztof Kozlowski wrote:
(...)
quoted
quoted
Ah, I didn't realize that was a valid option for MODULE_LICENSE. I guess
anything containing "GPL" works with EXPORT_SYMBOL_GPL?
I don't think exporting symbols is related to how you license your code.
It is; only modules with a GPL-compatible MODULE_LICENSE get to use
symbols exported via EXPORT_SYMBOL_GPL.
Although there might be such correlation but it's not a rule. You can
have a GPL module exporting symbols without GPL requirement
(EXPORT_SYMBOLS). You can have a GPL+MIT module exporting symbols as
GPL. Obviously you cannot have a non-GPL module, as we do not accept
these and there is no such choice.
So answering your question that "GPL" works with EXPORT_SYMBOL_GPL -
everything is GPL but it works with both EXPORT_SYMBOL and
EXPORT_SYMBOL_GPL.
See kernel/module.c for the symbol lookup logic and
include/linux/license.h for the logic to check the string (seems like
"Dual MIT/GPL" is explicitly whitelisted there).
Not related to export symbol. It is used for determining the tainted
kernel via other licenses.
Of course, this is a futile effort, as ~every time I see a proprietary
module in some embedded device, it either falsely declares itself to be
GPL, or they have a shim module that re-exports GPL symbols as non-GPL.
This is being removed soon (or already).
Best regards,
Krzysztof
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
From: Hector Martin <hidden> Date: 2021-10-14 08:31:37
On 14/10/2021 17.04, Krzysztof Kozlowski wrote:
quoted
It is; only modules with a GPL-compatible MODULE_LICENSE get to use
symbols exported via EXPORT_SYMBOL_GPL.
Although there might be such correlation but it's not a rule. You can
have a GPL module exporting symbols without GPL requirement
(EXPORT_SYMBOLS). You can have a GPL+MIT module exporting symbols as
GPL. Obviously you cannot have a non-GPL module, as we do not accept
these and there is no such choice.
What I mean is that modules can only import GPL symbols if they
themselves are GPL compatible. What I didn't know is that "Dual MIT/GPL"
is a valid string for MODULE_LICENSE to qualify as such.
quoted
See kernel/module.c for the symbol lookup logic and
include/linux/license.h for the logic to check the string (seems like
"Dual MIT/GPL" is explicitly whitelisted there).
Not related to export symbol. It is used for determining the tainted
kernel via other licenses.
Not just that; that module taint is used as a filter so that
non-GPL-compatible modules are technically prevented from resolving
EXPORT_SYMBOL_GPL symbols.
quoted
Of course, this is a futile effort, as ~every time I see a proprietary
module in some embedded device, it either falsely declares itself to be
GPL, or they have a shim module that re-exports GPL symbols as non-GPL.
This is being removed soon (or already).
? Good luck getting proprietary embedded vendors to start following
licenses... :)
--
Hector Martin (marcan@marcan.st)
Public Key: https://mrcn.st/pub
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
From: Hector Martin <hidden> Date: 2021-10-11 16:59:17
This driver exposes the CPU performance state switching hardware in
Apple SoCs as a clock controller that can be used together with the
generic cpufreq-dt mechanism to implement cpufreq support. It also
supports binding to an apple-mcc instance, to increase memory controller
performance when the CPUs are in the highest performance states.
Signed-off-by: Hector Martin <redacted>
---
drivers/clk/Kconfig | 9 ++
drivers/clk/Makefile | 1 +
drivers/clk/clk-apple-cluster.c | 184 ++++++++++++++++++++++++++++++++
3 files changed, 194 insertions(+)
create mode 100644 drivers/clk/clk-apple-cluster.c
@@ -390,6 +390,15 @@ config COMMON_CLK_K210helpSupportfortheCanaanKendryteK210RISC-VSoCclocks.+configCOMMON_CLK_APPLE_CLUSTER+bool"Clock driver for Apple SoC CPU clusters"+depends onARCH_APPLE||COMPILE_TEST+selectCPUFREQ_DT+defaultARCH_APPLE+help+ThisdriversupportsCPUclusterfrequencyswitchingonAppleSoC+platforms.+source"drivers/clk/actions/Kconfig"source"drivers/clk/analogbits/Kconfig"source"drivers/clk/baikal-t1/Kconfig"
@@ -0,0 +1,184 @@+// SPDX-License-Identifier: GPL-2.0-only OR MIT+/*+*AppleSoCCPUclusterperformancestatedriver+*+*CopyrightTheAsahiLinuxContributors+*/++#include<linux/bitfield.h>+#include<linux/bitops.h>+#include<linux/clk-provider.h>+#include<linux/delay.h>+#include<linux/err.h>+#include<linux/io.h>+#include<linux/iopoll.h>+#include<linux/module.h>+#include<linux/of.h>+#include<linux/platform_device.h>+#include<linux/pm_domain.h>+#include<linux/pm_opp.h>++#define APPLE_CLUSTER_PSTATE 0x20+#define APPLE_CLUSTER_PSTATE_BUSY BIT(31)+#define APPLE_CLUSTER_PSTATE_SET BIT(25)+#define APPLE_CLUSTER_PSTATE_DESIRED2 GENMASK(15, 12)+#define APPLE_CLUSTER_PSTATE_DESIRED1 GENMASK(3, 0)++structapple_cluster_clk{+structclk_hwhw;+structdevice*dev;+void__iomem*reg_base;+boolhas_pd;+};++#define to_apple_cluster_clk(_hw) container_of(_hw, struct apple_cluster_clk, hw)++#define APPLE_CLUSTER_SWITCH_TIMEOUT 100++staticintapple_cluster_clk_set_rate(structclk_hw*hw,unsignedlongrate,+unsignedlongparent_rate)+{+structapple_cluster_clk*cluster=to_apple_cluster_clk(hw);+structdev_pm_opp*opp;+unsignedintlevel;+u64reg;++opp=dev_pm_opp_find_freq_floor(cluster->dev,&rate);++if(IS_ERR(opp))+returnPTR_ERR(opp);++level=dev_pm_opp_get_level(opp);++dev_dbg(cluster->dev,"set_rate: %ld -> %d\n",rate,level);++if(readq_poll_timeout(cluster->reg_base+APPLE_CLUSTER_PSTATE,reg,+!(reg&APPLE_CLUSTER_PSTATE_BUSY),2,+APPLE_CLUSTER_SWITCH_TIMEOUT)){+dev_err(cluster->dev,"timed out waiting for busy flag\n");+return-EIO;+}++reg&=~(APPLE_CLUSTER_PSTATE_DESIRED1|APPLE_CLUSTER_PSTATE_DESIRED2);+reg|=FIELD_PREP(APPLE_CLUSTER_PSTATE_DESIRED1,level);+reg|=FIELD_PREP(APPLE_CLUSTER_PSTATE_DESIRED2,level);+reg|=APPLE_CLUSTER_PSTATE_SET;++writeq_relaxed(reg,cluster->reg_base+APPLE_CLUSTER_PSTATE);++if(cluster->has_pd)+dev_pm_genpd_set_performance_state(cluster->dev,+dev_pm_opp_get_required_pstate(opp,0));++return0;+}++staticunsignedlongapple_cluster_clk_recalc_rate(structclk_hw*hw,unsignedlongparent_rate)+{+structapple_cluster_clk*cluster=to_apple_cluster_clk(hw);+structdev_pm_opp*opp;+u64reg;++reg=readq_relaxed(cluster->reg_base+APPLE_CLUSTER_PSTATE);++opp=dev_pm_opp_find_level_exact(cluster->dev,+FIELD_GET(APPLE_CLUSTER_PSTATE_DESIRED1,reg));++if(IS_ERR(opp)){+dev_err(cluster->dev,"failed to find level: 0x%llx (%ld)\n",reg,PTR_ERR(opp));+return0;+}++returndev_pm_opp_get_freq(opp);+}++staticlongapple_cluster_clk_round_rate(structclk_hw*hw,unsignedlongrate,+unsignedlong*parent_rate)+{+structapple_cluster_clk*cluster=to_apple_cluster_clk(hw);+structdev_pm_opp*opp;++opp=dev_pm_opp_find_freq_floor(cluster->dev,&rate);++if(IS_ERR(opp)){+dev_err(cluster->dev,"failed to find rate: %ld (%ld)\n",rate,PTR_ERR(opp));+returnPTR_ERR(opp);+}++returnrate;+}++staticconststructclk_opsapple_cluster_clk_ops={+.set_rate=apple_cluster_clk_set_rate,+.recalc_rate=apple_cluster_clk_recalc_rate,+.round_rate=apple_cluster_clk_round_rate,+};++staticintapple_cluster_clk_probe(structplatform_device*pdev)+{+structdevice*dev=&pdev->dev;+structdevice_node*node=dev->of_node;+structapple_cluster_clk*cluster;+structclk_hw*hw;+structclk_init_datainit;+intret;++memset(&init,0,sizeof(init));+cluster=devm_kzalloc(dev,sizeof(*cluster),GFP_KERNEL);+if(!cluster)+return-ENOMEM;++cluster->dev=dev;+cluster->reg_base=devm_platform_ioremap_resource(pdev,0);+if(IS_ERR(cluster->reg_base))+returnPTR_ERR(cluster->reg_base);++hw=&cluster->hw;+hw->init=&init;++init.name=pdev->name;+init.num_parents=0;+init.ops=&apple_cluster_clk_ops;+init.flags=0;++ret=dev_pm_opp_of_add_table_noclk(dev,0);+if(ret<0){+dev_err(dev,"failed to get opp table\n");+returnret;+}++cluster->has_pd=of_property_read_bool(node,"power-domains");++ret=devm_of_clk_add_hw_provider(dev,of_clk_hw_simple_get,hw);+if(ret<0)+returnret;++ret=devm_clk_hw_register(dev,hw);+if(ret){+dev_err(dev,"failed to register clock\n");+returnret;+}++return0;+}++staticconststructof_device_idapple_cluster_clk_of_match[]={+{.compatible="apple,cluster-clk"},+{}+};++MODULE_DEVICE_TABLE(of,apple_cluster_clk_of_match);++staticstructplatform_driverapple_cluster_clk_driver={+.probe=apple_cluster_clk_probe,+.driver={+.name="apple-cluster-clk",+.of_match_table=apple_cluster_clk_of_match,+},+};++MODULE_AUTHOR("Hector Martin <marcan@marcan.st>");+MODULE_DESCRIPTION("CPU cluster performance state driver for Apple SoCs");+MODULE_LICENSE("GPL v2");++module_platform_driver(apple_cluster_clk_driver);
--
2.33.0
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
Please place in alphabetical sort order of config name
quoted hunk
+ bool "Clock driver for Apple SoC CPU clusters"
+ depends on ARCH_APPLE || COMPILE_TEST
+ select CPUFREQ_DT
+ default ARCH_APPLE
+ help
+ This driver supports CPU cluster frequency switching on Apple SoC
+ platforms.
+
source "drivers/clk/actions/Kconfig"
source "drivers/clk/analogbits/Kconfig"
source "drivers/clk/baikal-t1/Kconfig"
This looks bad from a locking perspective. How is lockdep holding up
with this driver? We're underneath the prepare lock here and we're
setting a couple level registers which is all good but now we're calling
into genpd code and who knows what's going to happen locking wise.
I don't actually see anything in here that indicates this is supposed to
be a clk provider. Is it being modeled as a clk so that it can use
cpufreq-dt? If it was a clk provider I'd expect it to be looking at
parent clk rates, and reading hardware to calculate frequencies based on
dividers and multipliers, etc. None of that is happening here.
Why not write a cpufreq driver, similar to qcom-cpufreq-hw.c that looks
through the OPP table and then writes the value into the pstate
registers? The registers in here look awfully similar to the qcom
hardware. I don't know what the DESIRED1 and DESIRED2 registers are for
though. Maybe they're so that one or the other frequency can be used if
available? Like a min/max?
Either way, writing this as a cpufreq driver avoids the clk framework
entirely which is super great for me :) It also avoids locking headaches
from the clk prepare lock, and it also lets you support lockless cpufreq
transitions by implementing the fast_switch function. I don't see any
downsides to the cpufreq driver approach.
From: Hector Martin <hidden> Date: 2021-10-17 09:16:44
On 15/10/2021 07.07, Stephen Boyd wrote:
This looks bad from a locking perspective. How is lockdep holding up
with this driver? We're underneath the prepare lock here and we're
setting a couple level registers which is all good but now we're calling
into genpd code and who knows what's going to happen locking wise.
It seems this is all going away given the other discussion threads point
towards handling this directly via OPP in the cpufreq-dt driver. I'll
run whatever I end up with for v2 through lockdep though, good call!
I don't actually see anything in here that indicates this is supposed to
be a clk provider. Is it being modeled as a clk so that it can use
cpufreq-dt? If it was a clk provider I'd expect it to be looking at
parent clk rates, and reading hardware to calculate frequencies based on
dividers and multipliers, etc. None of that is happening here.
Why not write a cpufreq driver, similar to qcom-cpufreq-hw.c that looks
through the OPP table and then writes the value into the pstate
registers? The registers in here look awfully similar to the qcom
hardware. I don't know what the DESIRED1 and DESIRED2 registers are for
though. Maybe they're so that one or the other frequency can be used if
available? Like a min/max?
Either way, writing this as a cpufreq driver avoids the clk framework
entirely which is super great for me :) It also avoids locking headaches
from the clk prepare lock, and it also lets you support lockless cpufreq
transitions by implementing the fast_switch function. I don't see any
downsides to the cpufreq driver approach.
I wasn't too sure about this approach; I thought using a clk provider
would end up simplifying things since I could use the cpufreq-dt
machinery to take care of all the OPP stuff, and a lot of SoCs seemed to
be going that way, but it seems cpufreq might be a better approach for
this SoC?
There can only be one cpufreq driver instance, while I used two clock
controllers to model the two clusters. So in the cpufreq case, the
driver itself would have to deal with all potential CPU cluster
instances/combinations itself. Not sure how much more code that will be,
hopefully not too much...
I see qcom-cpufreq-hw uses a qcom,freq-domain prop to link CPUs to the
cpufreq domains. cpufreq-dt and vexpress-spc-cpufreq instead use
dev_pm_opp_get_sharing_cpus to look for shared OPP tables. Is there a
reason not to do it that way and avoid the vendor prop? I guess the prop
is more explicit while the sharing approach would have an implicit order
dependency (i.e. CPUs are always grouped by cluster and clusters are
listed in /cpus in the same order as in the cpufreq node)...
(Ack on the other comments, but if this becomes a cpufreq driver most of
it is going to end up rewritten... :))
For the cpufreq case, do you have any suggestions as to how to relate it
to the memory controller configuration tweaks? Ideally this would go
through the OPP tables so it can be customized for future SoCs without
stuff hardcoded in the driver... it seems the configuration affects
power saving behavior / latencies, so it doesn't quite match the
interconnect framework bandwidth request stuff. I'm also not sure how
this would affect fast_switch, since going through those frameworks
might imply locks... we might even find ourselves with a situation in
the near future where multiple cpufreq policies can request memory
controller latency reduction independently; I can come up with how to do
this locklessly using atomics, but I can't imagine that being workable
with higher-level frameworks, it would have to be a vendor-specific
mechanism at that point...
--
Hector Martin (marcan@marcan.st)
Public Key: https://mrcn.st/pub
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
From: Hector Martin <hidden> Date: 2021-10-11 16:59:19
This driver is for the memory controller and users should always have
it; it is also a requirement with our devicetree configs for cpufreq to
work properly if that driver is available.
Signed-off-by: Hector Martin <redacted>
---
arch/arm64/Kconfig.platforms | 2 ++
1 file changed, 2 insertions(+)
From: Hector Martin <hidden> Date: 2021-10-11 16:59:23
- Proper CPU topology
- CPU capacities
* The 714 value is based on the CoreMark benchmark [1]. This is
roughly in line with other real-world test cases, like gzip. For
some reason, Dhrystone gives a wildly different value of 326, but
this doesn't seem representative of real-world workloads. This might
be adjusted in the future.
- MCC instance to control memory controller performance
- MCC OPP for t8103 using config values from hardware/ADT
- E-Cluster and P-Cluster clock controllers for CPU frequency switching
- Cluster OPP tables, including latency values determined
experimentally.
This relies on the generic cpufreq-dt driver to implement the cpufreq
side. It also assumes the bootloader did any required init (iBoot does
everything on firmware 12.0 and later; for 11.x we will have m1n1 do
what's missing).
[1] https://github.com/kdrag0n/linux-m1/commit/05c296604a42189cb61a0f7e8665566de192cbe9
Signed-off-by: Hector Martin <redacted>
---
arch/arm64/boot/dts/apple/t8103.dtsi | 255 ++++++++++++++++++++++++++-
1 file changed, 247 insertions(+), 8 deletions(-)
@@ -20,68 +20,284 @@ cpus {#address-cells=<2>;#size-cells=<0>;-cpu0:cpu@0{+cpu-map{+cluster0{+core0{+cpu=<&cpu_e0>;+};+core1{+cpu=<&cpu_e1>;+};+core2{+cpu=<&cpu_e2>;+};+core3{+cpu=<&cpu_e3>;+};+};++cluster1{+core0{+cpu=<&cpu_p0>;+};+core1{+cpu=<&cpu_p1>;+};+core2{+cpu=<&cpu_p2>;+};+core3{+cpu=<&cpu_p3>;+};+};+};++cpu_e0:cpu@0{compatible="apple,icestorm";device_type="cpu";reg=<0x00x0>;enable-method="spin-table";cpu-release-addr=<00>;/* To be filled by loader */+clocks=<&clk_ecluster>;+operating-points-v2=<&ecluster_opp>;+capacity-dmips-mhz=<714>;};-cpu1:cpu@1{+cpu_e1:cpu@1{compatible="apple,icestorm";device_type="cpu";reg=<0x00x1>;enable-method="spin-table";cpu-release-addr=<00>;/* To be filled by loader */+clocks=<&clk_ecluster>;+operating-points-v2=<&ecluster_opp>;+capacity-dmips-mhz=<714>;};-cpu2:cpu@2{+cpu_e2:cpu@2{compatible="apple,icestorm";device_type="cpu";reg=<0x00x2>;enable-method="spin-table";cpu-release-addr=<00>;/* To be filled by loader */+clocks=<&clk_ecluster>;+operating-points-v2=<&ecluster_opp>;+capacity-dmips-mhz=<714>;};-cpu3:cpu@3{+cpu_e3:cpu@3{compatible="apple,icestorm";device_type="cpu";reg=<0x00x3>;enable-method="spin-table";cpu-release-addr=<00>;/* To be filled by loader */+clocks=<&clk_ecluster>;+operating-points-v2=<&ecluster_opp>;+capacity-dmips-mhz=<714>;};-cpu4:cpu@10100{+cpu_p0:cpu@10100{compatible="apple,firestorm";device_type="cpu";reg=<0x00x10100>;enable-method="spin-table";cpu-release-addr=<00>;/* To be filled by loader */+clocks=<&clk_pcluster>;+operating-points-v2=<&pcluster_opp>;+capacity-dmips-mhz=<1024>;};-cpu5:cpu@10101{+cpu_p1:cpu@10101{compatible="apple,firestorm";device_type="cpu";reg=<0x00x10101>;enable-method="spin-table";cpu-release-addr=<00>;/* To be filled by loader */+clocks=<&clk_pcluster>;+operating-points-v2=<&pcluster_opp>;+capacity-dmips-mhz=<1024>;};-cpu6:cpu@10102{+cpu_p2:cpu@10102{compatible="apple,firestorm";device_type="cpu";reg=<0x00x10102>;enable-method="spin-table";cpu-release-addr=<00>;/* To be filled by loader */+clocks=<&clk_pcluster>;+operating-points-v2=<&pcluster_opp>;+capacity-dmips-mhz=<1024>;};-cpu7:cpu@10103{+cpu_p3:cpu@10103{compatible="apple,firestorm";device_type="cpu";reg=<0x00x10103>;enable-method="spin-table";cpu-release-addr=<00>;/* To be filled by loader */+clocks=<&clk_pcluster>;+operating-points-v2=<&pcluster_opp>;+capacity-dmips-mhz=<1024>;+};+};++ecluster_opp:opp-table-0{+compatible="operating-points-v2";+opp-shared;++opp01{+opp-hz=/bits/64<600000000>;+opp-microvolt=<559000>;+opp-level=<1>;+clock-latency-ns=<7500>;+};+opp02{+opp-hz=/bits/64<972000000>;+opp-microvolt=<628000>;+opp-level=<2>;+clock-latency-ns=<22000>;+};+opp03{+opp-hz=/bits/64<1332000000>;+opp-microvolt=<684000>;+opp-level=<3>;+clock-latency-ns=<27000>;+};+opp04{+opp-hz=/bits/64<1704000000>;+opp-microvolt=<765000>;+opp-level=<4>;+clock-latency-ns=<33000>;+};+opp05{+opp-hz=/bits/64<2064000000>;+opp-microvolt=<868000>;+opp-level=<5>;+clock-latency-ns=<50000>;+};+};++pcluster_opp:opp-table-1{+compatible="operating-points-v2";+opp-shared;++opp01{+opp-hz=/bits/64<600000000>;+opp-microvolt=<781000>;+opp-level=<1>;+clock-latency-ns=<8000>;+required-opps=<&mcc_lowperf>;+};+opp02{+opp-hz=/bits/64<828000000>;+opp-microvolt=<781000>;+opp-level=<2>;+clock-latency-ns=<19000>;+required-opps=<&mcc_lowperf>;+};+opp03{+opp-hz=/bits/64<1056000000>;+opp-microvolt=<781000>;+opp-level=<3>;+clock-latency-ns=<21000>;+required-opps=<&mcc_lowperf>;+};+opp04{+opp-hz=/bits/64<1284000000>;+opp-microvolt=<800000>;+opp-level=<4>;+clock-latency-ns=<23000>;+required-opps=<&mcc_lowperf>;+};+opp05{+opp-hz=/bits/64<1500000000>;+opp-microvolt=<821000>;+opp-level=<5>;+clock-latency-ns=<24000>;+required-opps=<&mcc_lowperf>;+};+opp06{+opp-hz=/bits/64<1728000000>;+opp-microvolt=<831000>;+opp-level=<6>;+clock-latency-ns=<29000>;+required-opps=<&mcc_lowperf>;+};+opp07{+opp-hz=/bits/64<1956000000>;+opp-microvolt=<865000>;+opp-level=<7>;+clock-latency-ns=<31000>;+required-opps=<&mcc_lowperf>;+};+opp08{+opp-hz=/bits/64<2184000000>;+opp-microvolt=<909000>;+opp-level=<8>;+clock-latency-ns=<34000>;+required-opps=<&mcc_highperf>;+};+opp09{+opp-hz=/bits/64<2388000000>;+opp-microvolt=<953000>;+opp-level=<9>;+clock-latency-ns=<36000>;+required-opps=<&mcc_highperf>;+};+opp10{+opp-hz=/bits/64<2592000000>;+opp-microvolt=<1003000>;+opp-level=<10>;+clock-latency-ns=<51000>;+required-opps=<&mcc_highperf>;+};+opp11{+opp-hz=/bits/64<2772000000>;+opp-microvolt=<1053000>;+opp-level=<11>;+clock-latency-ns=<54000>;+required-opps=<&mcc_highperf>;+};+opp12{+opp-hz=/bits/64<2988000000>;+opp-microvolt=<1081000>;+opp-level=<12>;+clock-latency-ns=<55000>;+required-opps=<&mcc_highperf>;+};+opp13{+opp-hz=/bits/64<3096000000>;+opp-microvolt=<1081000>;+opp-level=<13>;+clock-latency-ns=<55000>;+required-opps=<&mcc_highperf>;+};+opp14{+opp-hz=/bits/64<3144000000>;+opp-microvolt=<1081000>;+opp-level=<14>;+clock-latency-ns=<56000>;+required-opps=<&mcc_highperf>;+};+opp15{+opp-hz=/bits/64<3204000000>;+opp-microvolt=<1081000>;+opp-level=<15>;+clock-latency-ns=<56000>;+required-opps=<&mcc_highperf>;+};+};++mcc_opp:opp-table-2{+compatible="apple,mcc-operating-points";++mcc_lowperf:opp0{+opp-level=<0>;+apple,memory-perf-config=<0x813057f0x1800180>;+};+mcc_highperf:opp1{+opp-level=<1>;+apple,memory-perf-config=<0x1330x55555340>;};};