From: Benjamin Herrenschmidt <benh@kernel.crashing.org> Date: 2017-02-02 06:22:45
This adds AUX vectors for the L1I,D, L2 and L3 cache levels
providing for each cache level the size of the cache in bytes
and the geometry (line size and number of ways).
We chose to not use the existing alpha/sh definition which
packs all the information in a single entry per cache level as
it is too restricted to represent some of the geometries used
on POWER.
Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
---
v2. Remove some leftovers and fix 32-bit build
Paul, can you test & ack this please ? It replaces the last patch of
my previous series (the rest is identical).
arch/powerpc/include/asm/cache.h | 1 +
arch/powerpc/include/asm/elf.h | 20 ++++++++++++++++++++
arch/powerpc/include/uapi/asm/auxvec.h | 33 ++++++++++++++++++++++++++++++++-
arch/powerpc/kernel/setup_64.c | 4 ++++
4 files changed, 57 insertions(+), 1 deletion(-)
From: Paul Clarke <hidden> Date: 2017-03-01 20:54:14
On 02/02/2017 12:22 AM, Benjamin Herrenschmidt wrote:
This adds AUX vectors for the L1I,D, L2 and L3 cache levels
providing for each cache level the size of the cache in bytes
and the geometry (line size and number of ways).
We chose to not use the existing alpha/sh definition which
packs all the information in a single entry per cache level as
it is too restricted to represent some of the geometries used
on POWER.
Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
---
v2. Remove some leftovers and fix 32-bit build
Paul, can you test & ack this please ? It replaces the last patch of
my previous series (the rest is identical).
Sorry for the delay on this, I totally missed this e-mail until this week.
I tested this with the Power Functional Simulator (a.k.a. Mambo), with a fairly recent 4.9 kernel from the powerpc git tree.
Using Mambo-P8, I see what I expect for cache line size, but associativity is 0:
AT_L1I_CACHESIZE: 32768 (0x8000)
AT_L1I_CACHEGEOMETRY: associativity 0; line size 128
AT_L1D_CACHESIZE: 32768 (0x8000)
AT_L1D_CACHEGEOMETRY: associativity 0; line size 128
AT_L2_CACHESIZE: 524288 (0x80000)
AT_L2_CACHEGEOMETRY: associativity 0; line size 128
AT_L3_CACHESIZE: 8388608 (0x800000)
AT_L3_CACHEGEOMETRY: associativity 0; line size 128
...shouldn't associativity be 8 for all levels?
Using Mambo-P9, I don't get any cache sizes, and only L1 geometries, and no associativity:
AT_L1I_CACHEGEOMETRY: associativity 0; line size 128
AT_L1D_CACHEGEOMETRY: associativity 0; line size 128
PC
From: Michael Ellerman <mpe@ellerman.id.au> Date: 2017-03-02 06:33:57
Paul Clarke [off-list ref] writes:
On 02/02/2017 12:22 AM, Benjamin Herrenschmidt wrote:
quoted
This adds AUX vectors for the L1I,D, L2 and L3 cache levels
providing for each cache level the size of the cache in bytes
and the geometry (line size and number of ways).
We chose to not use the existing alpha/sh definition which
packs all the information in a single entry per cache level as
it is too restricted to represent some of the geometries used
on POWER.
Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
---
v2. Remove some leftovers and fix 32-bit build
Paul, can you test & ack this please ? It replaces the last patch of
my previous series (the rest is identical).
Sorry for the delay on this, I totally missed this e-mail until this week.
I tested this with the Power Functional Simulator (a.k.a. Mambo), with
a fairly recent 4.9 kernel from the powerpc git tree.
You mean 4.9 plus this patch?
Using Mambo-P8, I see what I expect for cache line size, but associativity is 0:
AT_L1I_CACHESIZE: 32768 (0x8000)
AT_L1I_CACHEGEOMETRY: associativity 0; line size 128
AT_L1D_CACHESIZE: 32768 (0x8000)
AT_L1D_CACHEGEOMETRY: associativity 0; line size 128
AT_L2_CACHESIZE: 524288 (0x80000)
AT_L2_CACHEGEOMETRY: associativity 0; line size 128
AT_L3_CACHESIZE: 8388608 (0x800000)
AT_L3_CACHEGEOMETRY: associativity 0; line size 128
...shouldn't associativity be 8 for all levels?
From: Paul Clarke <hidden> Date: 2017-03-02 23:46:42
On 03/02/2017 12:33 AM, Michael Ellerman wrote:
Paul Clarke [off-list ref] writes:
quoted
On 02/02/2017 12:22 AM, Benjamin Herrenschmidt wrote:
quoted
This adds AUX vectors for the L1I,D, L2 and L3 cache levels
providing for each cache level the size of the cache in bytes
and the geometry (line size and number of ways).
We chose to not use the existing alpha/sh definition which
packs all the information in a single entry per cache level as
it is too restricted to represent some of the geometries used
on POWER.
Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
---
v2. Remove some leftovers and fix 32-bit build
Paul, can you test & ack this please ? It replaces the last patch of
my previous series (the rest is identical).
Sorry for the delay on this, I totally missed this e-mail until this week.
I tested this with the Power Functional Simulator (a.k.a. Mambo), with
a fairly recent 4.9 kernel from the powerpc git tree.
You mean 4.9 plus this patch?
quoted
Using Mambo-P8, I see what I expect for cache line size, but associativity is 0:
AT_L1I_CACHESIZE: 32768 (0x8000)
AT_L1I_CACHEGEOMETRY: associativity 0; line size 128
AT_L1D_CACHESIZE: 32768 (0x8000)
AT_L1D_CACHEGEOMETRY: associativity 0; line size 128
AT_L2_CACHESIZE: 524288 (0x80000)
AT_L2_CACHEGEOMETRY: associativity 0; line size 128
AT_L3_CACHESIZE: 8388608 (0x800000)
AT_L3_CACHEGEOMETRY: associativity 0; line size 128
...shouldn't associativity be 8 for all levels?
It's not working for me, but it's also very likely that I could be doing something stupid. Just pulling from git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git, it built (without the "systemsim" patches"), but wouldn't boot in Mambo (not because it couldn't find root...something else not fully explored). I checked out tag "v4.10", and that built (with the "systemsim" patches, Ben's 1-8 and your 9v3) and booted. On Mambo-P8, the device tree looks correct (to me), but the associativity is still 0. On P9, the device tree isn't correct, so I'm not sure what to expect.
I added a bit more debug to my test:
AT_L1I_CACHESIZE: 32768 (0x8000)
getauxval(AT_L1I_CACHEGEOMETRY) = 0x0000000000000080
AT_L1I_CACHEGEOMETRY: associativity 0; line size 128
AT_L1D_CACHESIZE: 32768 (0x8000)
getauxval(AT_L1D_CACHEGEOMETRY) = 0x0000000000000080
AT_L1D_CACHEGEOMETRY: associativity 0; line size 128
AT_L2_CACHESIZE: 524288 (0x80000)
getauxval(AT_L2_CACHEGEOMETRY) = 0x0000000000000080
AT_L2_CACHEGEOMETRY: associativity 0; line size 128
AT_L3_CACHESIZE: 8388608 (0x800000)
getauxval(AT_L2_CACHEGEOMETRY) = 0x0000000000000080
AT_L3_CACHEGEOMETRY: associativity 0; line size 128
When I use gdb, and break at main:
(gdb) x/60gx ($r6)
0x3ffffffffca0: 0x0000000000000016 0x0000000000000016
0x3ffffffffcb0: 0x0000000000000016 0x0000000000000016
0x3ffffffffcc0: 0x0000000000000013 0x0000000000000080
0x3ffffffffcd0: 0x0000000000000014 0x0000000000000080
0x3ffffffffce0: 0x0000000000000015 0x0000000000000000
0x3ffffffffcf0: 0x0000000000000021 0x00003fffb7fa0000
0x3ffffffffd00: 0x0000000000000028 0x0000000000008000
0x3ffffffffd10: 0x0000000000000029 0x0000000000000080
0x3ffffffffd20: 0x000000000000002a 0x0000000000008000
0x3ffffffffd30: 0x000000000000002b 0x0000000000000080
0x3ffffffffd40: 0x000000000000002c 0x0000000000080000
0x3ffffffffd50: 0x000000000000002d 0x0000000000000080
0x3ffffffffd60: 0x000000000000002e 0x0000000000800000
0x3ffffffffd70: 0x000000000000002f 0x0000000000000080
and, looking at the device-tree:
# for i in /proc/device-tree/cpus/PowerPC@0/*cache*; do echo $i;
od -tx1 $i; done
/proc/device-tree/cpus/PowerPC@0/d-cache-block-size
0000000 00 00 00 80
/proc/device-tree/cpus/PowerPC@0/d-cache-line-size
0000000 00 00 00 80
/proc/device-tree/cpus/PowerPC@0/d-cache-sets
0000000 00 00 00 04
/proc/device-tree/cpus/PowerPC@0/d-cache-size
0000000 00 00 80 00
/proc/device-tree/cpus/PowerPC@0/i-cache-block-size
0000000 00 00 00 80
/proc/device-tree/cpus/PowerPC@0/i-cache-line-size
0000000 00 00 00 80
/proc/device-tree/cpus/PowerPC@0/i-cache-sets
0000000 00 00 00 02
/proc/device-tree/cpus/PowerPC@0/i-cache-size
0000000 00 00 80 00
/proc/device-tree/cpus/PowerPC@0/l2-cache-size
0000000 00 0a 00 00
From: Michael Ellerman <mpe@ellerman.id.au> Date: 2017-03-03 04:01:32
Paul Clarke [off-list ref] writes:
On 03/02/2017 12:33 AM, Michael Ellerman wrote:
quoted
Paul Clarke [off-list ref] writes:
quoted
On 02/02/2017 12:22 AM, Benjamin Herrenschmidt wrote:
quoted
This adds AUX vectors for the L1I,D, L2 and L3 cache levels
providing for each cache level the size of the cache in bytes
and the geometry (line size and number of ways).
We chose to not use the existing alpha/sh definition which
packs all the information in a single entry per cache level as
it is too restricted to represent some of the geometries used
on POWER.
Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
---
v2. Remove some leftovers and fix 32-bit build
Paul, can you test & ack this please ? It replaces the last patch of
my previous series (the rest is identical).
Sorry for the delay on this, I totally missed this e-mail until this week.
I tested this with the Power Functional Simulator (a.k.a. Mambo), with
a fairly recent 4.9 kernel from the powerpc git tree.
You mean 4.9 plus this patch?
quoted
Using Mambo-P8, I see what I expect for cache line size, but associativity is 0:
AT_L1I_CACHESIZE: 32768 (0x8000)
AT_L1I_CACHEGEOMETRY: associativity 0; line size 128
AT_L1D_CACHESIZE: 32768 (0x8000)
AT_L1D_CACHEGEOMETRY: associativity 0; line size 128
AT_L2_CACHESIZE: 524288 (0x80000)
AT_L2_CACHEGEOMETRY: associativity 0; line size 128
AT_L3_CACHESIZE: 8388608 (0x800000)
AT_L3_CACHEGEOMETRY: associativity 0; line size 128
...shouldn't associativity be 8 for all levels?
It's not working for me, but it's also very likely that I could be
doing something stupid. Just pulling from
git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git, it
built (without the "systemsim" patches"), but wouldn't boot in Mambo
(not because it couldn't find root...something else not fully
explored).
Hmm OK, it's booting for me, but might depend on your config/toolchain
etc. Linus' tree just before rc1 is a bit of a dangerous place to be.
I checked out tag "v4.10", and that built (with the
"systemsim" patches, Ben's 1-8 and your 9v3) and booted.
I think you need my entire v3, ignore Ben's series.
On Mambo-P8,
the device tree looks correct (to me), but the associativity is still
0.
OK, I think if you use the full v3 the associativity will be fixed.
Though it might still be wrong, because the kernel code hard codes the
values on P8 bare metal, because on real machines (running skiboot) the
values provided in the device tree are wrong.
Hopefully those hard coded values are also correct on p8 mambo, but we
should probably check.
See: https://git.kernel.org/torvalds/c/608b42140e96
On P9, the device tree isn't correct, so I'm not sure what to expect.
Junk :)
On P9 it all comes from the device tree so if that's wrong the AUX
vectors will definitely be wrong.
I couldn't get this working, either, so I'm probably doing something stupid.
Did you apply the first patch of the series also?
If you apply those on top of v4.10 and then do:
$ make TARGETS=powerpc ARCH=powerpc CROSS_COMPILE=ppc64le- -C tools/testing/selftests
You should end up with:
$ file tools/testing/selftests/powerpc/cache_shape/cache_shape
tools/testing/selftests/powerpc/cache_shape/cache_shape: ELF 64-bit LSB executable, 64-bit PowerPC or cisco 7500, version 1 (SYSV), dynamically linked, interpreter /lib64/ld64.so.2, for GNU/Linux 3.2.0, BuildID[sha1]=3535b7ab6ec854eceaa7ad23456a087111b51583, not stripped
cheers
From: Paul Clarke <hidden> Date: 2017-03-03 13:51:17
On 03/02/2017 10:01 PM, Michael Ellerman wrote:
Paul Clarke [off-list ref] writes:
quoted
On 03/02/2017 12:33 AM, Michael Ellerman wrote:
quoted
Paul Clarke [off-list ref] writes:
quoted
On 02/02/2017 12:22 AM, Benjamin Herrenschmidt wrote:
quoted
This adds AUX vectors for the L1I,D, L2 and L3 cache levels
providing for each cache level the size of the cache in bytes
and the geometry (line size and number of ways).
We chose to not use the existing alpha/sh definition which
packs all the information in a single entry per cache level as
it is too restricted to represent some of the geometries used
on POWER.
Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
---
v2. Remove some leftovers and fix 32-bit build
Paul, can you test & ack this please ? It replaces the last patch of
my previous series (the rest is identical).
Sorry for the delay on this, I totally missed this e-mail until this week.
I tested this with the Power Functional Simulator (a.k.a. Mambo), with
a fairly recent 4.9 kernel from the powerpc git tree.
You mean 4.9 plus this patch?
quoted
Using Mambo-P8, I see what I expect for cache line size, but associativity is 0:
AT_L1I_CACHESIZE: 32768 (0x8000)
AT_L1I_CACHEGEOMETRY: associativity 0; line size 128
AT_L1D_CACHESIZE: 32768 (0x8000)
AT_L1D_CACHEGEOMETRY: associativity 0; line size 128
AT_L2_CACHESIZE: 524288 (0x80000)
AT_L2_CACHEGEOMETRY: associativity 0; line size 128
AT_L3_CACHESIZE: 8388608 (0x800000)
AT_L3_CACHEGEOMETRY: associativity 0; line size 128
...shouldn't associativity be 8 for all levels?
It's not working for me, but it's also very likely that I could be
doing something stupid. Just pulling from
git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git, it
built (without the "systemsim" patches"), but wouldn't boot in Mambo
(not because it couldn't find root...something else not fully
explored).
Hmm OK, it's booting for me, but might depend on your config/toolchain
etc. Linus' tree just before rc1 is a bit of a dangerous place to be.
quoted
I checked out tag "v4.10", and that built (with the
"systemsim" patches, Ben's 1-8 and your 9v3) and booted.
I think you need my entire v3, ignore Ben's series.
quoted
On Mambo-P8,
the device tree looks correct (to me), but the associativity is still
0.
OK, I think if you use the full v3 the associativity will be fixed.
Though it might still be wrong, because the kernel code hard codes the
values on P8 bare metal, because on real machines (running skiboot) the
values provided in the device tree are wrong.
Hopefully those hard coded values are also correct on p8 mambo, but we
should probably check.
See: https://git.kernel.org/torvalds/c/608b42140e96
OK, thanks for your help. All is working as expected now. mambo-p8 looks fine:
--
AT_L1I_CACHESIZE: 32768 (0x8000)
getauxval(AT_L1I_CACHEGEOMETRY) = 0x0000000000080080
AT_L1I_CACHEGEOMETRY: associativity 8; line size 128
AT_L1D_CACHESIZE: 32768 (0x8000)
getauxval(AT_L1D_CACHEGEOMETRY) = 0x0000000000080080
AT_L1D_CACHEGEOMETRY: associativity 8; line size 128
AT_L2_CACHESIZE: 524288 (0x80000)
getauxval(AT_L2_CACHEGEOMETRY) = 0x0000000000080080
AT_L2_CACHEGEOMETRY: associativity 8; line size 128
AT_L3_CACHESIZE: 8388608 (0x800000)
getauxval(AT_L2_CACHEGEOMETRY) = 0x0000000000080080
AT_L3_CACHEGEOMETRY: associativity 8; line size 128
--
quoted
On P9, the device tree isn't correct, so I'm not sure what to expect.
Junk :)
On P9 it all comes from the device tree so if that's wrong the AUX
vectors will definitely be wrong.
mambo-p9 is still falling victim to the device-tree, but "working":
--
getauxval(AT_L1I_CACHEGEOMETRY) = 0x0000000000000080
AT_L1I_CACHEGEOMETRY: associativity 0; line size 128
getauxval(AT_L1D_CACHEGEOMETRY) = 0x0000000000000080
AT_L1D_CACHEGEOMETRY: associativity 0; line size 128
getauxval(AT_L2_CACHEGEOMETRY) = 0x0000000000000000
getauxval(AT_L2_CACHEGEOMETRY) = 0x0000000000000000
--
From: Benjamin Herrenschmidt <hidden> Date: 2017-03-05 23:13:00
On Fri, 2017-03-03 at 07:51 -0600, Paul Clarke wrote:
quoted
On P9 it all comes from the device tree so if that's wrong the AUX
vectors will definitely be wrong.
mambo-p9 is still falling victim to the device-tree, but "working":
--
getauxval(AT_L1I_CACHEGEOMETRY) = 0x0000000000000080
AT_L1I_CACHEGEOMETRY: associativity 0; line size 128
getauxval(AT_L1D_CACHEGEOMETRY) = 0x0000000000000080
AT_L1D_CACHEGEOMETRY: associativity 0; line size 128
getauxval(AT_L2_CACHEGEOMETRY) = 0x0000000000000000
getauxval(AT_L2_CACHEGEOMETRY) = 0x0000000000000000
--
The mambo device-tree comes from the mambo own internals afaik.
On real HW, the values come from HDAT. If they are wrong we need
the HostBoot guys to fix them.
Cheers,
Ben.
Hmm, which p9 mambo is this? I get:
test: cache_shape
tags: git_version:v4.10-10927-g0a55d1671c33
L1I cache size: 0x8000 32768B 32K
L1I line size: 0x80 128-way associative
L1D cache size: 0x8000 32768B 32K
L1D line size: 0x80 128-way associative
L2 cache size: 0 0B 0K
L2 line size: 0 fully associative
L3 cache size: 0 0B 0K
L3 line size: 0 fully associative
success: cache_shape
So that looks OK, other than that here is no L2/L3 info.
I'm using mambo commit e0c03c93c6d14c5f842840cc4ac4183cb25e123b.
Do you want me to Ack your v3 patch set in that thread?
No don't bother, it's already merged so your ack wouldn't get included
in the git history. This mail thread is record enough that it got
tested by someone other than me.
Thanks for testing.
cheers
On Mon, Mar 06, 2017 at 04:46:46PM +1100, Michael Ellerman wrote:
Hmm, which p9 mambo is this? I get:
test: cache_shape
tags: git_version:v4.10-10927-g0a55d1671c33
L1I cache size: 0x8000 32768B 32K
L1I line size: 0x80 128-way associative
L1D cache size: 0x8000 32768B 32K
L1D line size: 0x80 128-way associative
L2 cache size: 0 0B 0K
L2 line size: 0 fully associative
L3 cache size: 0 0B 0K
L3 line size: 0 fully associative
success: cache_shape
So that looks OK, other than that here is no L2/L3 info.