@@ -860,6 +860,7 @@ void __init smp_prepare_cpus(unsigned int max_cpus)GFP_KERNEL,cpu_to_node(cpu));zalloc_cpumask_var_node(&per_cpu(cpu_core_map,cpu),GFP_KERNEL,cpu_to_node(cpu));+#ifdef CONFIG_NEED_MULTIPLE_NODES/**numa_node_id()worksafterthis.*/
@@ -868,6 +869,7 @@ void __init smp_prepare_cpus(unsigned int max_cpus)set_cpu_numa_mem(cpu,local_memory_node(numa_cpu_lookup_table[cpu]));}+#endif}/* Init the cpumasks so the boot CPU is related to itself */
A new sched_domain_topology_level was added just for Power9. However the
same can be achieved by merging powerpc_topology with power9_topology
and makes the code more simpler especially when adding a new sched
domain.
Cc: linuxppc-dev <redacted>
Cc: LKML <redacted>
Cc: Michael Ellerman <mpe@ellerman.id.au>
Cc: Nicholas Piggin <npiggin@gmail.com>
Cc: Anton Blanchard <redacted>
Cc: Oliver O'Halloran <oohall@gmail.com>
Cc: Nathan Lynch <redacted>
Cc: Michael Neuling <redacted>
Cc: Gautham R Shenoy <redacted>
Cc: Ingo Molnar <mingo@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Valentin Schneider <redacted>
Cc: Jordan Niethe <redacted>
Reviewed-by: Gautham R. Shenoy <redacted>
Signed-off-by: Srikar Dronamraju <redacted>
---
Changelog v1 -> v2:
Replaced a reference to cpu_smt_mask with per_cpu(cpu_sibling_map, cpu)
since cpu_smt_mask is only defined under CONFIG_SCHED_SMT
arch/powerpc/kernel/smp.c | 33 ++++++++++-----------------------
1 file changed, 10 insertions(+), 23 deletions(-)
@@ -1313,7 +1313,7 @@ int setup_profiling_timer(unsigned int multiplier)}#ifdef CONFIG_SCHED_SMT-/* cpumask of CPUs with asymetric SMT dependancy */+/* cpumask of CPUs with asymmetric SMT dependency */staticintpowerpc_smt_flags(void){intflags=SD_SHARE_CPUCAPACITY|SD_SHARE_PKG_RESOURCES;
@@ -1326,14 +1326,6 @@ static int powerpc_smt_flags(void)}#endif-staticstructsched_domain_topology_levelpowerpc_topology[]={-#ifdef CONFIG_SCHED_SMT-{cpu_smt_mask,powerpc_smt_flags,SD_INIT_NAME(SMT)},-#endif-{cpu_cpu_mask,SD_INIT_NAME(DIE)},-{NULL,},-};-/**P9hasaslightlyoddarchitecturewherepairsofcoresshareanL2cache.*Thistopologymakesit*much*cheapertomigratetasksbetweenadjacentcores
@@ -1351,7 +1343,13 @@ static int powerpc_shared_cache_flags(void)*/staticconststructcpumask*shared_cache_mask(intcpu){-returncpu_l2_cache_mask(cpu);+if(shared_caches)+returncpu_l2_cache_mask(cpu);++if(has_big_cores)+returncpu_smallcore_mask(cpu);++returnper_cpu(cpu_sibling_map,cpu);}#ifdef CONFIG_SCHED_SMT
@@ -1386,21 +1384,10 @@ void __init smp_cpus_done(unsigned int max_cpus)#ifdef CONFIG_SCHED_SMTif(has_big_cores){pr_info("Big cores detected but using small core scheduling\n");-power9_topology[0].mask=smallcore_smt_mask;powerpc_topology[0].mask=smallcore_smt_mask;}#endif-/*-*IfanyCPUdetectsthatit'ssharingacachewithanotherCPUthen-*usethedeepertopologythatisawareofthissharing.-*/-if(shared_caches){-pr_info("Using shared cache scheduler topology\n");-set_sched_topology(power9_topology);-}else{-pr_info("Using standard scheduler topology\n");-set_sched_topology(powerpc_topology);-}+set_sched_topology(powerpc_topology);}#ifdef CONFIG_HOTPLUG_CPU
Just moving the powerpc_topology description above.
This will help in using functions in this file and avoid declarations.
No other functional changes
Cc: linuxppc-dev <redacted>
Cc: LKML <redacted>
Cc: Michael Ellerman <mpe@ellerman.id.au>
Cc: Nicholas Piggin <npiggin@gmail.com>
Cc: Anton Blanchard <redacted>
Cc: Oliver O'Halloran <oohall@gmail.com>
Cc: Nathan Lynch <redacted>
Cc: Michael Neuling <redacted>
Cc: Gautham R Shenoy <redacted>
Cc: Ingo Molnar <mingo@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Valentin Schneider <redacted>
Cc: Jordan Niethe <redacted>
Reviewed-by: Gautham R. Shenoy <redacted>
Signed-off-by: Srikar Dronamraju <redacted>
---
arch/powerpc/kernel/smp.c | 116 +++++++++++++++++++-------------------
1 file changed, 58 insertions(+), 58 deletions(-)
Move topology fixup based on the platform attributes into its own
function which is called just before set_sched_topology.
Cc: linuxppc-dev <redacted>
Cc: LKML <redacted>
Cc: Michael Ellerman <mpe@ellerman.id.au>
Cc: Nicholas Piggin <npiggin@gmail.com>
Cc: Anton Blanchard <redacted>
Cc: Oliver O'Halloran <oohall@gmail.com>
Cc: Nathan Lynch <redacted>
Cc: Michael Neuling <redacted>
Cc: Gautham R Shenoy <redacted>
Cc: Ingo Molnar <mingo@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Valentin Schneider <redacted>
Cc: Jordan Niethe <redacted>
Reviewed-by: Gautham R. Shenoy <redacted>
Signed-off-by: Srikar Dronamraju <redacted>
---
Changelog v2 -> v3:
Rewrote changelog (Gautham)
Renamed to powerpc/smp: Move topology fixups into a new function
arch/powerpc/kernel/smp.c | 17 +++++++++++------
1 file changed, 11 insertions(+), 6 deletions(-)
@@ -1368,6 +1368,16 @@ int setup_profiling_timer(unsigned int multiplier)return0;}+staticvoidfixup_topology(void)+{+#ifdef CONFIG_SCHED_SMT+if(has_big_cores){+pr_info("Big cores detected but using small core scheduling\n");+powerpc_topology[0].mask=smallcore_smt_mask;+}+#endif+}+void__initsmp_cpus_done(unsignedintmax_cpus){/*
@@ -1381,12 +1391,7 @@ void __init smp_cpus_done(unsigned int max_cpus)dump_numa_cpu_topology();-#ifdef CONFIG_SCHED_SMT-if(has_big_cores){-pr_info("Big cores detected but using small core scheduling\n");-powerpc_topology[0].mask=smallcore_smt_mask;-}-#endif+fixup_topology();set_sched_topology(powerpc_topology);}
Current code assumes that cpumask of cpus sharing a l2-cache mask will
always be a superset of cpu_sibling_mask.
Lets stop that assumption. cpu_l2_cache_mask is a superset of
cpu_sibling_mask if and only if shared_caches is set.
Cc: linuxppc-dev <redacted>
Cc: LKML <redacted>
Cc: Michael Ellerman <mpe@ellerman.id.au>
Cc: Nicholas Piggin <npiggin@gmail.com>
Cc: Anton Blanchard <redacted>
Cc: Oliver O'Halloran <oohall@gmail.com>
Cc: Nathan Lynch <redacted>
Cc: Michael Neuling <redacted>
Cc: Gautham R Shenoy <redacted>
Cc: Ingo Molnar <mingo@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Valentin Schneider <redacted>
Cc: Jordan Niethe <redacted>
Reviewed-by: Gautham R. Shenoy <redacted>
Signed-off-by: Srikar Dronamraju <redacted>
---
Changelog v1 -> v2:
Set cpumask after verifying l2-cache. (Gautham)
arch/powerpc/kernel/smp.c | 28 +++++++++++++++-------------
1 file changed, 15 insertions(+), 13 deletions(-)
Currently "CACHE" domain happens to be the 2nd sched domain as per
powerpc_topology. This domain will collapse if cpumask of l2-cache is
same as SMT domain. However we could generalize this domain such that it
could mean either be a "CACHE" domain or a "BIGCORE" domain.
While setting up the "CACHE" domain, check if shared_cache is already
set.
Cc: linuxppc-dev <redacted>
Cc: LKML <redacted>
Cc: Michael Ellerman <mpe@ellerman.id.au>
Cc: Nicholas Piggin <npiggin@gmail.com>
Cc: Anton Blanchard <redacted>
Cc: Oliver O'Halloran <oohall@gmail.com>
Cc: Nathan Lynch <redacted>
Cc: Michael Neuling <redacted>
Cc: Gautham R Shenoy <redacted>
Cc: Ingo Molnar <mingo@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Valentin Schneider <redacted>
Cc: Jordan Niethe <redacted>
Signed-off-by: Srikar Dronamraju <redacted>
---
Changelog v1 -> v2:
Moved shared_cache topology fixup to fixup_topology (Gautham)
arch/powerpc/kernel/smp.c | 48 +++++++++++++++++++++++++++------------
1 file changed, 34 insertions(+), 14 deletions(-)
Add support for grouping cores based on the device-tree classification.
- The last domain in the associativity domains always refers to the
core.
- If primary reference domain happens to be the penultimate domain in
the associativity domains device-tree property, then there are no
coregroups. However if its not a penultimate domain, then there are
coregroups. There can be more than one coregroup. For now we would be
interested in the last or the smallest coregroups.
Cc: linuxppc-dev <redacted>
Cc: LKML <redacted>
Cc: Michael Ellerman <mpe@ellerman.id.au>
Cc: Nicholas Piggin <npiggin@gmail.com>
Cc: Anton Blanchard <redacted>
Cc: Oliver O'Halloran <oohall@gmail.com>
Cc: Nathan Lynch <redacted>
Cc: Michael Neuling <redacted>
Cc: Gautham R Shenoy <redacted>
Cc: Ingo Molnar <mingo@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Valentin Schneider <redacted>
Cc: Jordan Niethe <redacted>
Reviewed-by: Gautham R. Shenoy <redacted>
Signed-off-by: Srikar Dronamraju <redacted>
---
Changelog v1 -> v2:
Explained Coregroup in commit msg (Michael Ellerman)
arch/powerpc/include/asm/smp.h | 1 +
arch/powerpc/kernel/smp.c | 1 +
arch/powerpc/mm/numa.c | 34 +++++++++++++++++++++-------------
3 files changed, 23 insertions(+), 13 deletions(-)
If allocated earlier and the search fails, then cpumask need to be
freed. However cpu_l1_cache_map can be allocated after we search thread
group.
Cc: linuxppc-dev <redacted>
Cc: LKML <redacted>
Cc: Michael Ellerman <mpe@ellerman.id.au>
Cc: Nicholas Piggin <npiggin@gmail.com>
Cc: Anton Blanchard <redacted>
Cc: Oliver O'Halloran <oohall@gmail.com>
Cc: Nathan Lynch <redacted>
Cc: Michael Neuling <redacted>
Cc: Gautham R Shenoy <redacted>
Cc: Ingo Molnar <mingo@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Valentin Schneider <redacted>
Cc: Jordan Niethe <redacted>
Reviewed-by: Gautham R. Shenoy <redacted>
Signed-off-by: Srikar Dronamraju <redacted>
---
arch/powerpc/kernel/smp.c | 7 +++----
1 file changed, 3 insertions(+), 4 deletions(-)
Lookup the coregroup id from the associativity array.
If unable to detect the coregroup id, fallback on the core id.
This way, ensure sched_domain degenerates and an extra sched domain is
not created.
Ideally this function should have been implemented in
arch/powerpc/kernel/smp.c. However if its implemented in mm/numa.c, we
don't need to find the primary domain again.
If the device-tree mentions more than one coregroup, then kernel
implements only the last or the smallest coregroup, which currently
corresponds to the penultimate domain in the device-tree.
Cc: linuxppc-dev <redacted>
Cc: LKML <redacted>
Cc: Michael Ellerman <mpe@ellerman.id.au>
Cc: Nicholas Piggin <npiggin@gmail.com>
Cc: Anton Blanchard <redacted>
Cc: Oliver O'Halloran <oohall@gmail.com>
Cc: Nathan Lynch <redacted>
Cc: Michael Neuling <redacted>
Cc: Gautham R Shenoy <redacted>
Cc: Ingo Molnar <mingo@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Valentin Schneider <redacted>
Cc: Jordan Niethe <redacted>
Reviewed-by : Gautham R. Shenoy [off-list ref]
Signed-off-by: Srikar Dronamraju <redacted>
---
Changelog v1 -> v2:
Move coregroup_enabled before getting associativity (Gautham)
arch/powerpc/mm/numa.c | 20 ++++++++++++++++++++
1 file changed, 20 insertions(+)
From: Gautham R Shenoy <hidden> Date: 2020-07-27 18:53:14
Hi Srikar,
On Mon, Jul 27, 2020 at 11:02:29AM +0530, Srikar Dronamraju wrote:
Add percpu coregroup maps and masks to create coregroup domain.
If a coregroup doesn't exist, the coregroup domain will be degenerated
in favour of SMT/CACHE domain.
Cc: linuxppc-dev <redacted>
Cc: LKML <redacted>
Cc: Michael Ellerman <mpe@ellerman.id.au>
Cc: Nicholas Piggin <npiggin@gmail.com>
Cc: Anton Blanchard <redacted>
Cc: Oliver O'Halloran <oohall@gmail.com>
Cc: Nathan Lynch <redacted>
Cc: Michael Neuling <redacted>
Cc: Gautham R Shenoy <redacted>
Cc: Ingo Molnar <mingo@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Valentin Schneider <redacted>
Cc: Jordan Niethe <redacted>
Signed-off-by: Srikar Dronamraju <redacted>
This version looks good to me.
Reviewed-by: Gautham R. Shenoy <redacted>
quoted hunk
---
Changelog v3 ->v4:
if coregroup_support doesn't exist, update MC mask to the next
smaller domain mask.
Changelog v2 -> v3:
Add optimization for mask updation under coregroup_support
Changelog v1 -> v2:
Moved coregroup topology fixup to fixup_topology (Gautham)
arch/powerpc/include/asm/topology.h | 10 +++++++
arch/powerpc/kernel/smp.c | 44 +++++++++++++++++++++++++++++
arch/powerpc/mm/numa.c | 5 ++++
3 files changed, 59 insertions(+)
Add percpu coregroup maps and masks to create coregroup domain.
If a coregroup doesn't exist, the coregroup domain will be degenerated
in favour of SMT/CACHE domain.
So there's at least one arm64 platform out there with the same "pairs of
cores share L2" thing (Ampere eMAG), and that lives quite happily with the
default scheduler topology (SMT/MC/DIE). Each pair of core gets its MC
domain, and the whole system is covered by DIE.
Now arguably it's not a perfect representation; DIE doesn't have
SD_SHARE_PKG_RESOURCES so the highest level sd_llc can point to is MC. That
will impact all callsites using cpus_share_cache(): in the eMAG case, only
pairs of cores will be seen as sharing cache, even though *all* cores share
the same L3.
I'm trying to paint a picture of what the P9 topology looks like (the one
you showcase in your cover letter) to see if there are any similarities;
from what I gather in [1], wikichips and your cover letter, with P9 you can
have something like this in a single DIE (somewhat unsure about L3 setup;
it looks to be distributed?)
+---------------------------------------------------------------------+
| L3 |
+---------------+-+---------------+-+---------------+-+---------------+
| L2 | | L2 | | L2 | | L2 |
+------+-+------+ +------+-+------+ +------+-+------+ +------+-+------+
| L1 | | L1 | | L1 | | L1 | | L1 | | L1 | | L1 | | L1 |
+------+ +------+ +------+ +------+ +------+ +------+ +------+ +------+
|4 CPUs| |4 CPUs| |4 CPUs| |4 CPUs| |4 CPUs| |4 CPUs| |4 CPUs| |4 CPUs|
+------+ +------+ +------+ +------+ +------+ +------+ +------+ +------+
Which would lead to (ignoring the whole SMT CPU numbering shenanigans)
NUMA [ ...
DIE [ ]
MC [ ] [ ] [ ] [ ]
BIGCORE [ ] [ ] [ ] [ ]
SMT [ ] [ ] [ ] [ ] [ ] [ ] [ ] [ ]
00-03 04-07 08-11 12-15 16-19 20-23 24-27 28-31 <other node here>
This however has MC == BIGCORE; what makes it you can have different spans
for these two domains? If it's not too much to ask, I'd love to have a P9
topology diagram.
[1]: 20200722081822.GG9290@linux.vnet.ibm.com
* Valentin Schneider [off-list ref] [2020-07-28 16:03:11]:
Hi Valentin,
Thanks for looking into the patches.
On 27/07/20 06:32, Srikar Dronamraju wrote:
quoted
Add percpu coregroup maps and masks to create coregroup domain.
If a coregroup doesn't exist, the coregroup domain will be degenerated
in favour of SMT/CACHE domain.
So there's at least one arm64 platform out there with the same "pairs of
cores share L2" thing (Ampere eMAG), and that lives quite happily with the
default scheduler topology (SMT/MC/DIE). Each pair of core gets its MC
domain, and the whole system is covered by DIE.
Now arguably it's not a perfect representation; DIE doesn't have
SD_SHARE_PKG_RESOURCES so the highest level sd_llc can point to is MC. That
will impact all callsites using cpus_share_cache(): in the eMAG case, only
pairs of cores will be seen as sharing cache, even though *all* cores share
the same L3.
Okay, Its good to know that we have a chip which is similar to P9 in
topology.
I'm trying to paint a picture of what the P9 topology looks like (the one
you showcase in your cover letter) to see if there are any similarities;
from what I gather in [1], wikichips and your cover letter, with P9 you can
have something like this in a single DIE (somewhat unsure about L3 setup;
it looks to be distributed?)
+---------------------------------------------------------------------+
| L3 |
+---------------+-+---------------+-+---------------+-+---------------+
| L2 | | L2 | | L2 | | L2 |
+------+-+------+ +------+-+------+ +------+-+------+ +------+-+------+
| L1 | | L1 | | L1 | | L1 | | L1 | | L1 | | L1 | | L1 |
+------+ +------+ +------+ +------+ +------+ +------+ +------+ +------+
|4 CPUs| |4 CPUs| |4 CPUs| |4 CPUs| |4 CPUs| |4 CPUs| |4 CPUs| |4 CPUs|
+------+ +------+ +------+ +------+ +------+ +------+ +------+ +------+
Which would lead to (ignoring the whole SMT CPU numbering shenanigans)
NUMA [ ...
DIE [ ]
MC [ ] [ ] [ ] [ ]
BIGCORE [ ] [ ] [ ] [ ]
SMT [ ] [ ] [ ] [ ] [ ] [ ] [ ] [ ]
00-03 04-07 08-11 12-15 16-19 20-23 24-27 28-31 <other node here>
What you have summed up is perfectly what a P9 topology looks like. I dont
think I could have explained it better than this.
This however has MC == BIGCORE; what makes it you can have different spans
for these two domains? If it's not too much to ask, I'd love to have a P9
topology diagram.
[1]: 20200722081822.GG9290@linux.vnet.ibm.com
At this time the current topology would be good enough i.e BIGCORE would
always be equal to a MC. However in future we could have chips that can have
lesser/larger number of CPUs in llc than in a BIGCORE or we could have
granular or split L3 caches within a DIE. In such a case BIGCORE != MC.
Also in the current P9 itself, two neighbouring core-pairs form a quad.
Cache latency within a quad is better than a latency to a distant core-pair.
Cache latency within a core pair is way better than latency within a quad.
So if we have only 4 threads running on a DIE all of them accessing the same
cache-lines, then we could probably benefit if all the tasks were to run
within the quad aka MC/Coregroup.
I have found some benchmarks which are latency sensitive to benefit by
having a grouping a quad level (using kernel hacks and not backed by
firmware changes). Gautham also found similar results in his experiments
but he only used binding within the stock kernel.
I am not setting SD_SHARE_PKG_RESOURCES in MC/Coregroup sd_flags as in MC
domain need not be LLC domain for Power.
--
Thanks and Regards
Srikar Dronamraju
From: Gautham R Shenoy <hidden> Date: 2020-07-30 05:57:20
On Mon, Jul 27, 2020 at 11:02:26AM +0530, Srikar Dronamraju wrote:
Currently "CACHE" domain happens to be the 2nd sched domain as per
powerpc_topology. This domain will collapse if cpumask of l2-cache is
same as SMT domain. However we could generalize this domain such that it
could mean either be a "CACHE" domain or a "BIGCORE" domain.
While setting up the "CACHE" domain, check if shared_cache is already
set.
Cc: linuxppc-dev <redacted>
Cc: LKML <redacted>
Cc: Michael Ellerman <mpe@ellerman.id.au>
Cc: Nicholas Piggin <npiggin@gmail.com>
Cc: Anton Blanchard <redacted>
Cc: Oliver O'Halloran <oohall@gmail.com>
Cc: Nathan Lynch <redacted>
Cc: Michael Neuling <redacted>
Cc: Gautham R Shenoy <redacted>
Cc: Ingo Molnar <mingo@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Valentin Schneider <redacted>
Cc: Jordan Niethe <redacted>
Signed-off-by: Srikar Dronamraju <redacted>
(+Cc Morten)
On 29/07/20 07:13, Srikar Dronamraju wrote:
* Valentin Schneider [off-list ref] [2020-07-28 16:03:11]:
Hi Valentin,
Thanks for looking into the patches.
quoted
On 27/07/20 06:32, Srikar Dronamraju wrote:
quoted
Add percpu coregroup maps and masks to create coregroup domain.
If a coregroup doesn't exist, the coregroup domain will be degenerated
in favour of SMT/CACHE domain.
So there's at least one arm64 platform out there with the same "pairs of
cores share L2" thing (Ampere eMAG), and that lives quite happily with the
default scheduler topology (SMT/MC/DIE). Each pair of core gets its MC
domain, and the whole system is covered by DIE.
Now arguably it's not a perfect representation; DIE doesn't have
SD_SHARE_PKG_RESOURCES so the highest level sd_llc can point to is MC. That
will impact all callsites using cpus_share_cache(): in the eMAG case, only
pairs of cores will be seen as sharing cache, even though *all* cores share
the same L3.
Okay, Its good to know that we have a chip which is similar to P9 in
topology.
quoted
I'm trying to paint a picture of what the P9 topology looks like (the one
you showcase in your cover letter) to see if there are any similarities;
from what I gather in [1], wikichips and your cover letter, with P9 you can
have something like this in a single DIE (somewhat unsure about L3 setup;
it looks to be distributed?)
+---------------------------------------------------------------------+
| L3 |
+---------------+-+---------------+-+---------------+-+---------------+
| L2 | | L2 | | L2 | | L2 |
+------+-+------+ +------+-+------+ +------+-+------+ +------+-+------+
| L1 | | L1 | | L1 | | L1 | | L1 | | L1 | | L1 | | L1 |
+------+ +------+ +------+ +------+ +------+ +------+ +------+ +------+
|4 CPUs| |4 CPUs| |4 CPUs| |4 CPUs| |4 CPUs| |4 CPUs| |4 CPUs| |4 CPUs|
+------+ +------+ +------+ +------+ +------+ +------+ +------+ +------+
Which would lead to (ignoring the whole SMT CPU numbering shenanigans)
NUMA [ ...
DIE [ ]
MC [ ] [ ] [ ] [ ]
BIGCORE [ ] [ ] [ ] [ ]
SMT [ ] [ ] [ ] [ ] [ ] [ ] [ ] [ ]
00-03 04-07 08-11 12-15 16-19 20-23 24-27 28-31 <other node here>
What you have summed up is perfectly what a P9 topology looks like. I dont
think I could have explained it better than this.
Yay!
quoted
This however has MC == BIGCORE; what makes it you can have different spans
for these two domains? If it's not too much to ask, I'd love to have a P9
topology diagram.
[1]: 20200722081822.GG9290@linux.vnet.ibm.com
At this time the current topology would be good enough i.e BIGCORE would
always be equal to a MC. However in future we could have chips that can have
lesser/larger number of CPUs in llc than in a BIGCORE or we could have
granular or split L3 caches within a DIE. In such a case BIGCORE != MC.
Right, that one's fair enough.
Also in the current P9 itself, two neighbouring core-pairs form a quad.
Cache latency within a quad is better than a latency to a distant core-pair.
Cache latency within a core pair is way better than latency within a quad.
So if we have only 4 threads running on a DIE all of them accessing the same
cache-lines, then we could probably benefit if all the tasks were to run
within the quad aka MC/Coregroup.
Did you test this? WRT load balance we do try to balance "load" over the
different domain spans, so if you represent quads as their own MC domain,
you would AFAICT end up spreading tasks over the quads (rather than packing
them) when balancing at e.g. DIE level. The desired behaviour might be
hackable with some more ASYM_PACKING, but I'm not sure I should be
suggesting that :-)
I have found some benchmarks which are latency sensitive to benefit by
having a grouping a quad level (using kernel hacks and not backed by
firmware changes). Gautham also found similar results in his experiments
but he only used binding within the stock kernel.
IIUC you reflect this "fabric quirk" (i.e. coregroups) using this DT
binding thing.
That's also where things get interesting (for me) because I experienced
something similar on another arm64 platform (ThunderX1). This was more
about cache bandwidth than cache latency, but IMO it's in the same bag of
fabric quirks. I blabbered a bit about this at last LPC [1], but kind of
gave up on it given the TX1 was the only (arm64) platform where I could get
both significant and reproducible results.
Now, if you folks are seeing this on completely different hardware and have
"real" workloads that truly benefit from this kind of domain partitioning,
this might be another incentive to try and sort of generalize this. That's
outside the scope of your series, but your findings give me some hope!
I think what I had in mind back then was that if enough folks cared about
it, we might get some bits added to the ACPI spec; something along the
lines of proximity domains for the caches described in PPTT, IOW a cache
distance matrix. I don't really know what it'll take to get there, but I
figured I'd dump this in case someone's listening :-)
I am not setting SD_SHARE_PKG_RESOURCES in MC/Coregroup sd_flags as in MC
domain need not be LLC domain for Power.
From what I understood your MC domain does seem to map to LLC; but in any
case, shouldn't you set that flag at least for BIGCORE (i.e. L2)? AIUI with
your changes your sd_llc is gonna be SMT, and that's not going to be a very
big mask. IMO you do want to correctly reflect your LLC situation via this
flag to make cpus_share_cache() work properly.
[1]: https://linuxplumbersconf.org/event/4/contributions/484/
At this time the current topology would be good enough i.e BIGCORE would
always be equal to a MC. However in future we could have chips that can have
lesser/larger number of CPUs in llc than in a BIGCORE or we could have
granular or split L3 caches within a DIE. In such a case BIGCORE != MC.
Also in the current P9 itself, two neighbouring core-pairs form a quad.
Cache latency within a quad is better than a latency to a distant core-pair.
Cache latency within a core pair is way better than latency within a quad.
So if we have only 4 threads running on a DIE all of them accessing the same
cache-lines, then we could probably benefit if all the tasks were to run
within the quad aka MC/Coregroup.
I have found some benchmarks which are latency sensitive to benefit by
having a grouping a quad level (using kernel hacks and not backed by
firmware changes). Gautham also found similar results in his experiments
but he only used binding within the stock kernel.
I am not setting SD_SHARE_PKG_RESOURCES in MC/Coregroup sd_flags as in MC
domain need not be LLC domain for Power.
I am observing that SD_SHARE_PKG_RESOURCES at L2 provides the best
results for POWER9 in terms of cache-benefits during wakeup. On a
POWER9 Boston machine, running a producer-consumer test case
(https://github.com/gautshen/misc/blob/master/producer_consumer/producer_consumer.c)
The test case creates two threads, one Producer and another
Consumer. Both work on a fairly large shared array of size 64M. In an
interation the Producer performs stores to 1024 random locations and
wakes up the Consumer. In the Consumer's iteration, loads from those
exact 1024 locations.
We measure the number of Consumer iterations per second and the
average time for each Consumer iteration. The smaller the time, the
better it is.
The following results are when I pinned the Producer and Consumer to
different combinations of CPUs to cover Small core , Big-core,
Neighbouring Big-core, Far off core within the same chip, and across
chips. There is a also a case where they are not affined anywhere, and
we let the scheduler wake them up correctly.
We find the best results when the Producer and Consumer are within the
same L2 domain. These numbers are also close to the numbers that we
get when we let the Scheduler wake them up (where LLC is L2).
## Same Small core (4 threads: Shares L1, L2, L3, Frequency Domain)
Consumer affined to CPU 3
Producer affined to CPU 1
4698 iterations, avg time: 20034 ns
4951 iterations, avg time: 20012 ns
4957 iterations, avg time: 19971 ns
4968 iterations, avg time: 19985 ns
4970 iterations, avg time: 19977 ns
## Same Big Core (8 threads: Shares L2, L3, Frequency Domain)
Consumer affined to CPU 7
Producer affined to CPU 1
4580 iterations, avg time: 19403 ns
4851 iterations, avg time: 19373 ns
4849 iterations, avg time: 19394 ns
4856 iterations, avg time: 19394 ns
4867 iterations, avg time: 19353 ns
## Neighbouring Big-core (Faster data-snooping from L2. Shares L3, Frequency Domain)
Producer affined to CPU 1
Consumer affined to CPU 11
4270 iterations, avg time: 24158 ns
4491 iterations, avg time: 24157 ns
4500 iterations, avg time: 24148 ns
4516 iterations, avg time: 24164 ns
4518 iterations, avg time: 24165 ns
## Any other Big-core from Same Chip (Shares L3)
Producer affined to CPU 1
Consumer affined to CPU 87
4176 iterations, avg time: 27953 ns
4417 iterations, avg time: 27925 ns
4415 iterations, avg time: 27934 ns
4417 iterations, avg time: 27983 ns
4430 iterations, avg time: 27958 ns
## Different Chips (No cache-sharing)
Consumer affined to CPU 175
Producer affined to CPU 1
3277 iterations, avg time: 50786 ns
3063 iterations, avg time: 50732 ns
2831 iterations, avg time: 50737 ns
2859 iterations, avg time: 50688 ns
2849 iterations, avg time: 50722 ns
## Without affining them (Let Scheduler wake-them up appropriately)
Consumer affined to CPU 0-175
Producer affined to CPU 0-175
4821 iterations, avg time: 19412 ns
4863 iterations, avg time: 19435 ns
4855 iterations, avg time: 19381 ns
4811 iterations, avg time: 19458 ns
4892 iterations, avg time: 19429 ns
--
Thanks and Regards
gautham.
From: Michael Ellerman <mpe@ellerman.id.au> Date: 2020-07-31 07:45:47
Srikar Dronamraju [off-list ref] writes:
Currently "CACHE" domain happens to be the 2nd sched domain as per
powerpc_topology. This domain will collapse if cpumask of l2-cache is
same as SMT domain. However we could generalize this domain such that it
could mean either be a "CACHE" domain or a "BIGCORE" domain.
While setting up the "CACHE" domain, check if shared_cache is already
set.
PeterZ asked for some overview of what you're doing and why, you
responded to his mail, but I was expecting to see that text incorporated
here somewhere.
He also asked for some comments, which I would also like to see.
I'm also not clear why we want to rename it to "bigcore", that's not a
commonly understood term, I don't think it's clear to new readers what
it means.
Leaving it as the shared cache domain, and having a comment mentioning
that "bigcores" share a cache, would be clearer I think.
cheers
quoted hunk
Cc: linuxppc-dev <redacted>
Cc: LKML <redacted>
Cc: Michael Ellerman <mpe@ellerman.id.au>
Cc: Nicholas Piggin <npiggin@gmail.com>
Cc: Anton Blanchard <redacted>
Cc: Oliver O'Halloran <oohall@gmail.com>
Cc: Nathan Lynch <redacted>
Cc: Michael Neuling <redacted>
Cc: Gautham R Shenoy <redacted>
Cc: Ingo Molnar <mingo@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Valentin Schneider <redacted>
Cc: Jordan Niethe <redacted>
Signed-off-by: Srikar Dronamraju <redacted>
---
Changelog v1 -> v2:
Moved shared_cache topology fixup to fixup_topology (Gautham)
arch/powerpc/kernel/smp.c | 48 +++++++++++++++++++++++++++------------
1 file changed, 34 insertions(+), 14 deletions(-)
From: Michael Ellerman <mpe@ellerman.id.au> Date: 2020-07-31 07:49:59
Srikar Dronamraju [off-list ref] writes:
Add support for grouping cores based on the device-tree classification.
- The last domain in the associativity domains always refers to the
core.
- If primary reference domain happens to be the penultimate domain in
the associativity domains device-tree property, then there are no
coregroups. However if its not a penultimate domain, then there are
coregroups. There can be more than one coregroup. For now we would be
interested in the last or the smallest coregroups.
This still doesn't tell me what a coregroup actually represents.
I get that it's a grouping of cores, and that the device tree specifies
it for us, but grouping based on what?
I think the answer is we aren't being told by firmware, it's just a
grouping based on some opaque performance characteristic and we just
have to take that as given.
But please explain that clearly in the change log and the code comments.
cheers
quoted hunk
Cc: linuxppc-dev <redacted>
Cc: LKML <redacted>
Cc: Michael Ellerman <mpe@ellerman.id.au>
Cc: Nicholas Piggin <npiggin@gmail.com>
Cc: Anton Blanchard <redacted>
Cc: Oliver O'Halloran <oohall@gmail.com>
Cc: Nathan Lynch <redacted>
Cc: Michael Neuling <redacted>
Cc: Gautham R Shenoy <redacted>
Cc: Ingo Molnar <mingo@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Valentin Schneider <redacted>
Cc: Jordan Niethe <redacted>
Reviewed-by: Gautham R. Shenoy <redacted>
Signed-off-by: Srikar Dronamraju <redacted>
---
Changelog v1 -> v2:
Explained Coregroup in commit msg (Michael Ellerman)
arch/powerpc/include/asm/smp.h | 1 +
arch/powerpc/kernel/smp.c | 1 +
arch/powerpc/mm/numa.c | 34 +++++++++++++++++++++-------------
3 files changed, 23 insertions(+), 13 deletions(-)
From: Michael Ellerman <mpe@ellerman.id.au> Date: 2020-07-31 08:02:28
Srikar Dronamraju [off-list ref] writes:
Lookup the coregroup id from the associativity array.
It's slightly strange that this is called in patch 9, but only properly
implemented here in patch 10.
I'm not saying you have to squash them together, but it would be good if
the change log for patch 9 mentioned that a subsequent commit will
complete the implementation and how that affects the behaviour.
cheers
quoted hunk
If unable to detect the coregroup id, fallback on the core id.
This way, ensure sched_domain degenerates and an extra sched domain is
not created.
Ideally this function should have been implemented in
arch/powerpc/kernel/smp.c. However if its implemented in mm/numa.c, we
don't need to find the primary domain again.
If the device-tree mentions more than one coregroup, then kernel
implements only the last or the smallest coregroup, which currently
corresponds to the penultimate domain in the device-tree.
Cc: linuxppc-dev <redacted>
Cc: LKML <redacted>
Cc: Michael Ellerman <mpe@ellerman.id.au>
Cc: Nicholas Piggin <npiggin@gmail.com>
Cc: Anton Blanchard <redacted>
Cc: Oliver O'Halloran <oohall@gmail.com>
Cc: Nathan Lynch <redacted>
Cc: Michael Neuling <redacted>
Cc: Gautham R Shenoy <redacted>
Cc: Ingo Molnar <mingo@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Valentin Schneider <redacted>
Cc: Jordan Niethe <redacted>
Reviewed-by : Gautham R. Shenoy [off-list ref]
Signed-off-by: Srikar Dronamraju <redacted>
---
Changelog v1 -> v2:
Move coregroup_enabled before getting associativity (Gautham)
arch/powerpc/mm/numa.c | 20 ++++++++++++++++++++
1 file changed, 20 insertions(+)
* Michael Ellerman [off-list ref] [2020-07-31 17:49:55]:
Srikar Dronamraju [off-list ref] writes:
quoted
Add support for grouping cores based on the device-tree classification.
- The last domain in the associativity domains always refers to the
core.
- If primary reference domain happens to be the penultimate domain in
the associativity domains device-tree property, then there are no
coregroups. However if its not a penultimate domain, then there are
coregroups. There can be more than one coregroup. For now we would be
interested in the last or the smallest coregroups.
This still doesn't tell me what a coregroup actually represents.
I get that it's a grouping of cores, and that the device tree specifies
it for us, but grouping based on what?
We have just abstracted the fact that we are creating a sub-group of cores
within a DIE. We are limiting to one sub-group per core. However this would
allow the firmware the flexibility to vary the grouping. Once the firmware
starts using this group, we could add more code to detect the type of
grouping and adjust the sd domain flags accordingly.
I think the answer is we aren't being told by firmware, it's just a
grouping based on some opaque performance characteristic and we just
have to take that as given.
This is partially true. At this time, we dont have firmwares that can
exploit this code. Once the firmwares start using this grouping, we could
add more code to align the grouping to the scheduler topology.
But please explain that clearly in the change log and the code comments.
* Michael Ellerman [off-list ref] [2020-07-31 17:45:37]:
Srikar Dronamraju [off-list ref] writes:
quoted
Currently "CACHE" domain happens to be the 2nd sched domain as per
powerpc_topology. This domain will collapse if cpumask of l2-cache is
same as SMT domain. However we could generalize this domain such that it
could mean either be a "CACHE" domain or a "BIGCORE" domain.
While setting up the "CACHE" domain, check if shared_cache is already
set.
PeterZ asked for some overview of what you're doing and why, you
responded to his mail, but I was expecting to see that text incorporated
here somewhere.
Okay, do you want that as part of the code or documentation dir or the
changelog?
He also asked for some comments, which I would also like to see.
I'm also not clear why we want to rename it to "bigcore", that's not a
commonly understood term, I don't think it's clear to new readers what
it means.
Leaving it as the shared cache domain, and having a comment mentioning
that "bigcores" share a cache, would be clearer I think.
Today, Shared cache is equal to Big Core. However in not too distant future,
Shared cache domain and Big Core may not be the same. For example lets
assume that L2 cache were to Shrink per small core with the firmware
exposing the core as a bigcore. Then with the current design, we have a SMT
== SHARED CACHE, and a DIE. We would not have any domain at the publicised 8
thread level. Keeping the Bigcore as a domain and mapping the shared
cache, (I am resetting the domain name as CACHE if BIGCORE==SHARED_CACHE),
helps us in this scenario.
* Michael Ellerman [off-list ref] [2020-07-31 17:52:15]:
Srikar Dronamraju [off-list ref] writes:
quoted
If allocated earlier and the search fails, then cpumask need to be
freed. However cpu_l1_cache_map can be allocated after we search thread
group.
It's not freed anywhere AFAICS?
Yes, its never freed. Infact we are never checking if
zalloc_cpumask_var_node fails. Its not just this cpumask, but historically
all the other existing cpumasks in arch/powerpc/kernel/smp.c are never
freed/checked. I did dig into this a bit and it appears that ..
(Please do correct me if I am wrong!! )
Powerpc using cpumask_var_t for all of the percpu variables. And it dont seem
to enable CONFIG_CPUMASK_OFFSTACK even from the MAXSMP config.
So from include/linux/cpumask.h
typedef struct cpumask cpumask_var_t[1];
and
zalloc_cpumask_var_node ends up being cpumask_clear
So I think we are historically we seem to assume we are always
!CPUMASK_OFFSTACK and hence we dont need to check for return as well as
free..
I would look forward to your comments on how we should handle this going
forward. But I would keep this the same for this patchset.
One of the questions that I have is if we most likely are to be in
!CONFIG_CPUMASK_OFFSTACK, then should be migrate to cpumask_t for percpu
variables.
The reason being we end up using NR_CPU cpumask for each percpu cpumask
variable instead of using NR_CPU cpumask_t pointer.
And even after this change there's still an error path that doesn't free
it, isn't there?
cheers
quoted
Cc: linuxppc-dev <redacted>
Cc: LKML <redacted>
Cc: Michael Ellerman <mpe@ellerman.id.au>
Cc: Nicholas Piggin <npiggin@gmail.com>
Cc: Anton Blanchard <redacted>
Cc: Oliver O'Halloran <oohall@gmail.com>
Cc: Nathan Lynch <redacted>
Cc: Michael Neuling <redacted>
Cc: Gautham R Shenoy <redacted>
Cc: Ingo Molnar <mingo@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Valentin Schneider <redacted>
Cc: Jordan Niethe <redacted>
Reviewed-by: Gautham R. Shenoy <redacted>
Signed-off-by: Srikar Dronamraju <redacted>
---
arch/powerpc/kernel/smp.c | 7 +++----
1 file changed, 3 insertions(+), 4 deletions(-)
* Michael Ellerman [off-list ref] [2020-07-31 18:02:21]:
Srikar Dronamraju [off-list ref] writes:
quoted
Lookup the coregroup id from the associativity array.
Thanks Michael for all your comments and inputs.
It's slightly strange that this is called in patch 9, but only properly
implemented here in patch 10.
I'm not saying you have to squash them together, but it would be good if
the change log for patch 9 mentioned that a subsequent commit will
complete the implementation and how that affects the behaviour.
I probably got influenced by few LKML community members who always add a
stub and implement the gory details in a subsequent patch. I will surely add
the change log in patch 9 about the subsequent patches.
From: Michael Ellerman <mpe@ellerman.id.au> Date: 2020-07-31 11:29:21
Srikar Dronamraju [off-list ref] writes:
* Michael Ellerman [off-list ref] [2020-07-31 18:02:21]:
quoted
Srikar Dronamraju [off-list ref] writes:
quoted
Lookup the coregroup id from the associativity array.
Thanks Michael for all your comments and inputs.
quoted
It's slightly strange that this is called in patch 9, but only properly
implemented here in patch 10.
I'm not saying you have to squash them together, but it would be good if
the change log for patch 9 mentioned that a subsequent commit will
complete the implementation and how that affects the behaviour.
I probably got influenced by few LKML community members who always add a
stub and implement the gory details in a subsequent patch. I will surely add
the change log in patch 9 about the subsequent patches.
That's OK, it's a valid way to do things, and can be good for keeping
the size of individual patches down to make them easier to review.
But yeah a mention in the change log of the preceeding patch is helpful
for anyone looking at that commit on its own in the future.
cheers
From: Michael Ellerman <mpe@ellerman.id.au> Date: 2020-07-31 11:31:18
Srikar Dronamraju [off-list ref] writes:
* Michael Ellerman [off-list ref] [2020-07-31 17:49:55]:
quoted
Srikar Dronamraju [off-list ref] writes:
quoted
Add support for grouping cores based on the device-tree classification.
- The last domain in the associativity domains always refers to the
core.
- If primary reference domain happens to be the penultimate domain in
the associativity domains device-tree property, then there are no
coregroups. However if its not a penultimate domain, then there are
coregroups. There can be more than one coregroup. For now we would be
interested in the last or the smallest coregroups.
This still doesn't tell me what a coregroup actually represents.
I get that it's a grouping of cores, and that the device tree specifies
it for us, but grouping based on what?
We have just abstracted the fact that we are creating a sub-group of cores
within a DIE. We are limiting to one sub-group per core. However this would
allow the firmware the flexibility to vary the grouping. Once the firmware
starts using this group, we could add more code to detect the type of
grouping and adjust the sd domain flags accordingly.
OK. That's good info to have in the change log.
quoted
I think the answer is we aren't being told by firmware, it's just a
grouping based on some opaque performance characteristic and we just
have to take that as given.
This is partially true. At this time, we dont have firmwares that can
exploit this code. Once the firmwares start using this grouping, we could
add more code to align the grouping to the scheduler topology.
quoted
But please explain that clearly in the change log and the code comments.
From: Michael Ellerman <mpe@ellerman.id.au> Date: 2020-07-31 12:14:19
Srikar Dronamraju [off-list ref] writes:
* Michael Ellerman [off-list ref] [2020-07-31 17:52:15]:
quoted
Srikar Dronamraju [off-list ref] writes:
quoted
If allocated earlier and the search fails, then cpumask need to be
freed. However cpu_l1_cache_map can be allocated after we search thread
group.
It's not freed anywhere AFAICS?
Yes, its never freed. Infact we are never checking if
zalloc_cpumask_var_node fails. Its not just this cpumask, but historically
all the other existing cpumasks in arch/powerpc/kernel/smp.c are never
freed/checked. I did dig into this a bit and it appears that ..
(Please do correct me if I am wrong!! )
That's correct.
Powerpc using cpumask_var_t for all of the percpu variables. And it dont seem
to enable CONFIG_CPUMASK_OFFSTACK even from the MAXSMP config.
I remember Rusty adding that code, but I don't know if we ever
considered enabling CPUMASK_OFFSTACK.
Probably we meant to but never got around to doing it.
So from include/linux/cpumask.h
typedef struct cpumask cpumask_var_t[1];
and
zalloc_cpumask_var_node ends up being cpumask_clear
So I think we are historically we seem to assume we are always
!CPUMASK_OFFSTACK and hence we dont need to check for return as well as
free..
Right.
I would look forward to your comments on how we should handle this going
forward. But I would keep this the same for this patchset.
Agreed, just clarify in the change log that it's not freed at the moment
because of CPU_MASK_OFFSTACK=n
One of the questions that I have is if we most likely are to be in
!CONFIG_CPUMASK_OFFSTACK, then should be migrate to cpumask_t for percpu
variables.
I don't think so, cpumask_t is semi-deprecated AIUI.
The reason being we end up using NR_CPU cpumask for each percpu cpumask
variable instead of using NR_CPU cpumask_t pointer.
Our current defconfigs have NR_CPUS=2048, which is probably just small
enough to continue using OFFSTACK=n.
But we allow configuring NR_CPUS up to 8192, which surely would need
OFFSTACK=y in order to work.
So I think we need to stick with cpumask_var_t, but we should test with
OFFSTACK=y, and should probably be a bit more careful with checking the
allocations succeed.
And then we should select OFFSTACK=y for NR_CPUS above some threshold.
cheers
From: Michael Ellerman <mpe@ellerman.id.au> Date: 2020-07-31 12:22:40
Srikar Dronamraju [off-list ref] writes:
* Michael Ellerman [off-list ref] [2020-07-31 17:45:37]:
quoted
Srikar Dronamraju [off-list ref] writes:
quoted
Currently "CACHE" domain happens to be the 2nd sched domain as per
powerpc_topology. This domain will collapse if cpumask of l2-cache is
same as SMT domain. However we could generalize this domain such that it
could mean either be a "CACHE" domain or a "BIGCORE" domain.
While setting up the "CACHE" domain, check if shared_cache is already
set.
PeterZ asked for some overview of what you're doing and why, you
responded to his mail, but I was expecting to see that text incorporated
here somewhere.
Okay, do you want that as part of the code or documentation dir or the
changelog?
I guess a comment is best, as that's most likely to be seen by people
looking at the code in future.
A little bit of overview in the change log is also good.
quoted
He also asked for some comments, which I would also like to see.
I'm also not clear why we want to rename it to "bigcore", that's not a
commonly understood term, I don't think it's clear to new readers what
it means.
Leaving it as the shared cache domain, and having a comment mentioning
that "bigcores" share a cache, would be clearer I think.
Today, Shared cache is equal to Big Core. However in not too distant future,
Shared cache domain and Big Core may not be the same. For example lets
assume that L2 cache were to Shrink per small core with the firmware
exposing the core as a bigcore. Then with the current design, we have a SMT
== SHARED CACHE, and a DIE. We would not have any domain at the publicised 8
thread level. Keeping the Bigcore as a domain and mapping the shared
cache, (I am resetting the domain name as CACHE if BIGCORE==SHARED_CACHE),
helps us in this scenario.
Yeah OK.
In that scenario it's not really clear what the 8 thread level domain
expresses, if the two "halves" of the bigcore have separate L2s. But
presumably there is still some benefit to exposing it.
cheers
Also in the current P9 itself, two neighbouring core-pairs form a quad.
Cache latency within a quad is better than a latency to a distant core-pair.
Cache latency within a core pair is way better than latency within a quad.
So if we have only 4 threads running on a DIE all of them accessing the same
cache-lines, then we could probably benefit if all the tasks were to run
within the quad aka MC/Coregroup.
Did you test this? WRT load balance we do try to balance "load" over the
different domain spans, so if you represent quads as their own MC domain,
you would AFAICT end up spreading tasks over the quads (rather than packing
them) when balancing at e.g. DIE level. The desired behaviour might be
hackable with some more ASYM_PACKING, but I'm not sure I should be
suggesting that :-)
Agree, load balance will try to spread the load across the quads. In my hack,
I was explicitly marking QUAD domains as !SD_PREFER_SIBLING + relaxing few
load spreading rules when SD_PREFER_SIBLING was not set. And this was on a
slightly older kernel (without recent Vincent's load balance overhaul).
quoted
I have found some benchmarks which are latency sensitive to benefit by
having a grouping a quad level (using kernel hacks and not backed by
firmware changes). Gautham also found similar results in his experiments
but he only used binding within the stock kernel.
IIUC you reflect this "fabric quirk" (i.e. coregroups) using this DT
binding thing.
That's also where things get interesting (for me) because I experienced
something similar on another arm64 platform (ThunderX1). This was more
about cache bandwidth than cache latency, but IMO it's in the same bag of
fabric quirks. I blabbered a bit about this at last LPC [1], but kind of
gave up on it given the TX1 was the only (arm64) platform where I could get
both significant and reproducible results.
Now, if you folks are seeing this on completely different hardware and have
"real" workloads that truly benefit from this kind of domain partitioning,
this might be another incentive to try and sort of generalize this. That's
outside the scope of your series, but your findings give me some hope!
I think what I had in mind back then was that if enough folks cared about
it, we might get some bits added to the ACPI spec; something along the
lines of proximity domains for the caches described in PPTT, IOW a cache
distance matrix. I don't really know what it'll take to get there, but I
figured I'd dump this in case someone's listening :-)
Very interesting.
quoted
I am not setting SD_SHARE_PKG_RESOURCES in MC/Coregroup sd_flags as in MC
domain need not be LLC domain for Power.
From what I understood your MC domain does seem to map to LLC; but in any
case, shouldn't you set that flag at least for BIGCORE (i.e. L2)? AIUI with
your changes your sd_llc is gonna be SMT, and that's not going to be a very
big mask. IMO you do want to correctly reflect your LLC situation via this
flag to make cpus_share_cache() work properly.
I detect if the LLC is shared at BIGCORE, and if they are shared at BIGCORE,
then I dynamically rename the DOMAIN as CACHE and enable
SD_SHARE_PKG_RESOURCES in that domain.