From: Michael Ellerman <mpe@ellerman.id.au> Date: 2016-07-27 14:18:39
Empty for now, but we'll add to it in the next patch.
Signed-off-by: Michael Ellerman <mpe@ellerman.id.au>
---
arch/powerpc/include/asm/book3s/64/mmu.h | 1 +
arch/powerpc/include/asm/mmu.h | 1 +
arch/powerpc/kernel/prom.c | 2 ++
arch/powerpc/mm/init_64.c | 6 ++++++
4 files changed, 10 insertions(+)
v3: Merged into this series.
@@ -750,6 +750,8 @@ void __init early_init_devtree(void *params)if(disable_radix)cur_cpu_spec->mmu_features&=~MMU_FTR_RADIX;+mmu_early_init_devtree();+#ifdef CONFIG_PPC_POWERNV/* Scan and build the list of machine check recoverable ranges */of_scan_flat_dt(early_init_dt_scan_recoverable_ranges,NULL);
From: Michael Ellerman <mpe@ellerman.id.au> Date: 2016-07-27 14:18:40
Move the handling of the disable_radix command line argument into the
newly created mmu_early_init_devtree().
It's an MMU option so it's preferable to have it in an mm related file,
and it also means platforms that don't support radix don't have to carry
the code.
Signed-off-by: Michael Ellerman <mpe@ellerman.id.au>
---
arch/powerpc/kernel/prom.c | 13 -------------
arch/powerpc/mm/init_64.c | 11 +++++++++++
2 files changed, 11 insertions(+), 13 deletions(-)
v3: Merged into this series.
Move the handling of the disable_radix command line argument into the
newly created mmu_early_init_devtree().
It's an MMU option so it's preferable to have it in an mm related file,
and it also means platforms that don't support radix don't have to carry
the code.
Signed-off-by: Michael Ellerman <mpe@ellerman.id.au>
---
Should patch 1 and 2 be squashed together? Ideally nothing should ever bisect at patch 1
Anyway,
Acked-by: Balbir Singh <bsingharora@gmail.com>
From: Michael Ellerman <mpe@ellerman.id.au> Date: 2016-07-27 14:18:41
Currently MMU initialisation (early_init_mmu()) consists of a mixture of
scanning the device tree, setting MMU feature bits, and then also doing
actual initialisation of MMU data structures.
We'd like to decouple the setting of the MMU features from the actual
setup. So split out the device tree scanning, and associated code, and
call it from mmu_init_early_devtree().
Signed-off-by: Michael Ellerman <mpe@ellerman.id.au>
---
arch/powerpc/include/asm/book3s/64/mmu.h | 1 +
arch/powerpc/mm/hash_utils_64.c | 15 +++++++++------
arch/powerpc/mm/init_64.c | 3 +++
3 files changed, 13 insertions(+), 6 deletions(-)
v3: Merged into this series.
From: Michael Ellerman <mpe@ellerman.id.au> Date: 2016-07-28 12:40:44
Currently MMU initialisation (early_init_mmu()) consists of a mixture of
scanning the device tree, setting MMU feature bits, and then also doing
actual initialisation of MMU data structures.
We'd like to decouple the setting of the MMU features from the actual
setup. So split out the device tree scanning, and associated code, and
call it from mmu_init_early_devtree().
Signed-off-by: Michael Ellerman <mpe@ellerman.id.au>
---
arch/powerpc/include/asm/book3s/64/mmu.h | 1 +
arch/powerpc/mm/hash_utils_64.c | 53 ++++++++++++++++----------------
arch/powerpc/mm/init_64.c | 3 ++
3 files changed, 31 insertions(+), 26 deletions(-)
v4: Split htab_init_page_sizes() into two parts, the bit that needs to scan the
device tree (now htab_scan_page_sizes()), and the rest.
Fold htab_init_seg_sizes() into hash__early_init_devtree().
@@ -363,11 +363,6 @@ static int __init htab_dt_scan_seg_sizes(unsigned long node,return0;}-staticvoid__inithtab_init_seg_sizes(void)-{-of_scan_flat_dt(htab_dt_scan_seg_sizes,NULL);-}-staticint__initget_idx_from_shift(unsignedintshift){intidx=-1;
From: Michael Ellerman <mpe@ellerman.id.au> Date: 2016-07-27 14:18:42
Like we just did for hash, split the device tree scanning parts out and
call them from mmu_early_init_devtree().
Signed-off-by: Michael Ellerman <mpe@ellerman.id.au>
---
arch/powerpc/include/asm/book3s/64/mmu.h | 1 +
arch/powerpc/mm/init_64.c | 4 +++-
arch/powerpc/mm/pgtable-radix.c | 3 +--
3 files changed, 5 insertions(+), 3 deletions(-)
v3: Merged into this series.
@@ -264,7 +264,7 @@ static int __init radix_dt_scan_page_sizes(unsigned long node,return1;}-staticvoid__initradix_init_page_sizes(void)+void__initradix__early_init_devtree(void){intrc;
Like we just did for hash, split the device tree scanning parts out and
call them from mmu_early_init_devtree().
Signed-off-by: Michael Ellerman <mpe@ellerman.id.au>
---
arch/powerpc/include/asm/book3s/64/mmu.h | 1 +
arch/powerpc/mm/init_64.c | 4 +++-
arch/powerpc/mm/pgtable-radix.c | 3 +--
3 files changed, 5 insertions(+), 3 deletions(-)
v3: Merged into this series.
@@ -264,7 +264,7 @@ static int __init radix_dt_scan_page_sizes(unsigned long node,return1;}-staticvoid__initradix_init_page_sizes(void)+void__initradix__early_init_devtree(void){intrc;
If I am reading this correctly, radix_init_page_sizes() has become
radix__early_init_devtree() where as hash__early_init_devtree() initializes
both segment and page sizes? I would still like to keep
mmu_early_init_devtree()
-> radix__early_init_devtree()
-> radix__init_page_sizes()
Balbir Singh.
If I am reading this correctly, radix_init_page_sizes() has become
radix__early_init_devtree() where as hash__early_init_devtree() initializes
both segment and page sizes? I would still like to keep
mmu_early_init_devtree()
-> radix__early_init_devtree()
-> radix__init_page_sizes()
But then radix__early_init_devtree() would just be:
void radix__early_init_devtree(void)
{
radix__init_page_sizes();
}
Which seems silly.
I'm doing a new version which splits the htab scanning from the page
init more, as Ben suggested.
cheers
If I am reading this correctly, radix_init_page_sizes() has become
radix__early_init_devtree() where as hash__early_init_devtree() initializes
both segment and page sizes? I would still like to keep
mmu_early_init_devtree()
-> radix__early_init_devtree()
-> radix__init_page_sizes()
But then radix__early_init_devtree() would just be:
void radix__early_init_devtree(void)
{
radix__init_page_sizes();
}
Which seems silly.
But for a person parsing both hpte and radix bits, the code seems
saner and the compiler will do the right thing
I'm doing a new version which splits the htab scanning from the page
init more, as Ben suggested.
From: Michael Ellerman <mpe@ellerman.id.au> Date: 2016-07-27 14:18:42
Up until now we needed to do the MMU init before feature patching,
because part of the MMU init was scanning the device tree and setting
and/or clearing some MMU feature bits.
Now that we have split that MMU feature modification out into routines
called from early_init_devtree() (called earlier) we can now do feature
patching before calling MMU init.
The advantage of this is it means the remainder of the MMU init runs
with the final set of features which will apply for the rest of the life
of the system. This means we don't have to special case anything called
from MMU init to deal with a changing set of feature bits.
Signed-off-by: Michael Ellerman <mpe@ellerman.id.au>
---
arch/powerpc/kernel/setup_64.c | 6 +++---
1 file changed, 3 insertions(+), 3 deletions(-)
v3: Merged into this series.
@@ -298,12 +298,12 @@ void __init early_setup(unsigned long dt_ptr)*/configure_exceptions();-/* Initialize the hash table or TLB handling */-early_init_mmu();-/* Apply all the dynamic patching */apply_feature_fixups();+/* Initialize the hash table or TLB handling */+early_init_mmu();+/**Atthispoint,wecanletinterruptsswitchtovirtualmode*(theMMUhasbeensetup),soadjusttheMSRinthePACAto
From: Michael Ellerman <mpe@ellerman.id.au> Date: 2016-07-27 14:18:44
Early in boot we binary patch some sections of code based on the CPU and
MMU feature bits. But it is a one-time patching, there is no facility
for repatching the code later if the set of features change.
It is a major bug if the set of features changes after we've done the
code patching - so add a check for it.
Signed-off-by: Michael Ellerman <mpe@ellerman.id.au>
---
arch/powerpc/lib/feature-fixups.c | 27 ++++++++++++++++++++++++++-
1 file changed, 26 insertions(+), 1 deletion(-)
v3: Merged into this series.
@@ -173,12 +182,28 @@ void apply_feature_fixups(void)PTRRELOC(&__stop___lwsync_fixup));#ifdef CONFIG_PPC64+saved_firmware_features=powerpc_firmware_features;do_feature_fixups(powerpc_firmware_features,&__start___fw_ftr_fixup,&__stop___fw_ftr_fixup);#endifdo_final_fixups();}+staticint__initcheck_features(void)+{+WARN(saved_cpu_features!=cur_cpu_spec->cpu_features,+"CPU features changed after feature patching!\n");+WARN(saved_mmu_features!=cur_cpu_spec->mmu_features,+"MMU features changed after feature patching!\n");+#ifdef CONFIG_PPC64+WARN(saved_firmware_features!=powerpc_firmware_features,+"Firmware features changed after feature patching!\n");+#endif++return0;+}+late_initcall(check_features);+#ifdef CONFIG_FTR_FIXUP_SELFTEST#define check(x) \
From: Michael Ellerman <mpe@ellerman.id.au> Date: 2016-07-27 14:18:45
From: "Aneesh Kumar K.V" <redacted>
MMU feature bits are defined such that we use the lower half to
present MMU family features. Remove the strict split of half and
also move Radix to a mmu family feature. Radix introduce a new MMU
model and strictly speaking it is a new MMU family. This also free
up bits which can be used for individual features later.
Signed-off-by: Aneesh Kumar K.V <redacted>
Signed-off-by: Michael Ellerman <mpe@ellerman.id.au>
---
arch/powerpc/include/asm/book3s/64/mmu.h | 2 +-
arch/powerpc/include/asm/mmu.h | 15 +++++++--------
arch/powerpc/kernel/entry_64.S | 2 +-
arch/powerpc/kernel/exceptions-64s.S | 8 ++++----
arch/powerpc/kernel/idle_book3s.S | 2 +-
arch/powerpc/kernel/prom.c | 2 +-
arch/powerpc/mm/init_64.c | 2 +-
7 files changed, 16 insertions(+), 17 deletions(-)
v3: Merged into this series.
From: Michael Ellerman <mpe@ellerman.id.au> Date: 2016-07-27 14:18:45
The intention is that the result is only used as a boolean, so enforce
that by changing the return type to bool.
Signed-off-by: Michael Ellerman <mpe@ellerman.id.au>
---
arch/powerpc/include/asm/mmu.h | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
v3: Split out.
From: Michael Ellerman <mpe@ellerman.id.au> Date: 2016-07-27 14:18:46
The intention is that the result is only used as a boolean, so enforce
that by changing the return type to bool.
Signed-off-by: Michael Ellerman <mpe@ellerman.id.au>
---
arch/powerpc/include/asm/cputable.h | 9 ++++-----
1 file changed, 4 insertions(+), 5 deletions(-)
v3: Split out.
From: Michael Ellerman <mpe@ellerman.id.au> Date: 2016-07-27 14:18:48
Currently we have radix_enabled() three times, twice in asm/book3s/64/mmu.h
and then a fallback in asm/mmu.h.
Consolidate them in asm/mmu.h. While we're at it convert them to be
static inlines, and change the fallback case to returning a bool, like
mmu_has_feature().
Signed-off-by: Michael Ellerman <mpe@ellerman.id.au>
---
arch/powerpc/include/asm/book3s/64/mmu.h | 7 -------
arch/powerpc/include/asm/mmu.h | 16 ++++++++++++----
2 files changed, 12 insertions(+), 11 deletions(-)
v3: New.
@@ -163,6 +163,18 @@ static inline void assert_pte_locked(struct mm_struct *mm, unsigned long addr)}#endif /* !CONFIG_DEBUG_VM */+#ifdef CONFIG_PPC_RADIX_MMU+staticinlineboolradix_enabled(void)+{+returnmmu_has_feature(MMU_FTR_TYPE_RADIX);+}+#else+staticinlineboolradix_enabled(void)+{+returnfalse;+}+#endif+#endif /* !__ASSEMBLY__ *//* The kernel use the constants below to index in the page sizes array.
From: Nicholas Piggin <npiggin@gmail.com> Date: 2016-07-28 07:46:22
On Thu, 28 Jul 2016 00:18:06 +1000
Michael Ellerman [off-list ref] wrote:
quoted hunk
Currently we have radix_enabled() three times, twice in
asm/book3s/64/mmu.h and then a fallback in asm/mmu.h.
Consolidate them in asm/mmu.h. While we're at it convert them to be
static inlines, and change the fallback case to returning a bool, like
mmu_has_feature().
Signed-off-by: Michael Ellerman <mpe@ellerman.id.au>
---
arch/powerpc/include/asm/book3s/64/mmu.h | 7 -------
arch/powerpc/include/asm/mmu.h | 16 ++++++++++++----
2 files changed, 12 insertions(+), 11 deletions(-)
v3: New.
From: Michael Ellerman <mpe@ellerman.id.au> Date: 2016-07-30 10:08:16
Balbir Singh [off-list ref] writes:
On Fri, 2016-07-29 at 21:42 +1000, Michael Ellerman wrote:
quoted
Nicholas Piggin [off-list ref] writes:
quoted
Won't MMU_FTRS_POSSIBLE just do the right thing when
!CONFIG_PPC_RADIX_MMU?
Yes it should.
I'll have to work out why Aneesh thought he needed to do it explicitly
and whether that is needed or not.
IIRC, If CONFIG_PPC_RADIX_MMU=3Dn MMU_FTR_RAIDX will not be present in the
MMU_FTRS_POSSIBLE mask=C2=A0
That's right.
The obvious thing to do would be to define MMU_FTR_RADIX to 0 when
RADIX=3Dn, but that doesn't work because it breaks the ASM FTR macros
(because they check (mmu_features & mask) =3D=3D mask).
So instead we just remove it from the possible mask.
cheers
From: Michael Ellerman <mpe@ellerman.id.au> Date: 2016-07-27 14:18:49
In later patches, we will be switching cpu and mmu feature checks to
use static keys. For checks in early boot before jump label is
initialized we need a variant of cpu/mmu_has_feature() that doesn't use
jump labels. So create those called, unimaginatively,
__cpu/__mmu_has_feature().
Signed-off-by: Aneesh Kumar K.V <redacted>
Signed-off-by: Michael Ellerman <mpe@ellerman.id.au>
---
arch/powerpc/include/asm/cputable.h | 7 ++++++-
arch/powerpc/include/asm/mmu.h | 17 ++++++++++++++++-
2 files changed, 22 insertions(+), 2 deletions(-)
v3: Don't change any logic.
Bool conversions were split out.
Don't convert any call sites in this patch.
From: Michael Ellerman <mpe@ellerman.id.au> Date: 2016-07-27 14:18:50
From: "Aneesh Kumar K.V" <redacted>
This switches early feature checks to use the non static key variant of
the function. In later patches we will be switching cpu_has_feature()
and mmu_has_feature() to use static keys and we can use them only after
static key/jump label is initialized. Any check for feature before jump
label init should be done using this new helper.
Signed-off-by: Aneesh Kumar K.V <redacted>
Signed-off-by: Michael Ellerman <mpe@ellerman.id.au>
---
arch/powerpc/include/asm/book3s/64/mmu.h | 4 ++--
arch/powerpc/kernel/paca.c | 2 +-
arch/powerpc/kernel/setup_64.c | 4 ++--
arch/powerpc/mm/hash_utils_64.c | 6 +++---
arch/powerpc/mm/init_64.c | 2 +-
5 files changed, 9 insertions(+), 9 deletions(-)
v3: Add/remove some sites now that we're rebased on the early MMU init series.
@@ -227,8 +227,8 @@ static void __init configure_exceptions(void)opal_configure_cores();/* Enable AIL if supported, and we are in hypervisor mode */-if(cpu_has_feature(CPU_FTR_HVMODE)&&-cpu_has_feature(CPU_FTR_ARCH_207S)){+if(__cpu_has_feature(CPU_FTR_HVMODE)&&+__cpu_has_feature(CPU_FTR_ARCH_207S)){unsignedlonglpcr=mfspr(SPRN_LPCR);mtspr(SPRN_LPCR,lpcr|LPCR_AIL_3);}
* we will never see an HEA ethernet device.
*/
#ifdef CONFIG_IBMEBUS
- return !cpu_has_feature(CPU_FTR_ARCH_207S) &&
+ return !__cpu_has_feature(CPU_FTR_ARCH_207S) &&
!firmware_has_feature(FW_FEATURE_SPLPAR);
#else
All these could go if that function was split. The part that reads the
DT stays in early_init_mmu_devtree (bastically up to "found:" and then
the bit at the end that scans the huge pages).
The rest, which just assigns the various mmu_*_psize can go into
eary_init_mmu(). That means the only conversion needed is the one
below:
* Not in the device-tree, let's fallback on known size
* list for 16M capable GP & GR
*/
- if (mmu_has_feature(MMU_FTR_16M_PAGE))
+ if (__mmu_has_feature(MMU_FTR_16M_PAGE))
memcpy(mmu_psize_defs, mmu_psize_defaults_gp,
sizeof(mmu_psize_defaults_gp));
found:
And the rest can remain.
quoted hunk
@@ -591,7 +591,7 @@ found:
mmu_vmalloc_psize = MMU_PAGE_64K;
if (mmu_linear_psize == MMU_PAGE_4K)
mmu_linear_psize = MMU_PAGE_64K;
- if (mmu_has_feature(MMU_FTR_CI_LARGE_PAGE)) {
+ if (__mmu_has_feature(MMU_FTR_CI_LARGE_PAGE)) {
/*
* When running on pSeries using 64k pages
for ioremap
* would stop us accessing the HEA ethernet.
So if we
From: Nicholas Piggin <npiggin@gmail.com> Date: 2016-07-28 07:49:40
On Thu, 28 Jul 2016 00:18:08 +1000
Michael Ellerman [off-list ref] wrote:
From: "Aneesh Kumar K.V" <redacted>
This switches early feature checks to use the non static key variant
of the function. In later patches we will be switching
cpu_has_feature() and mmu_has_feature() to use static keys and we can
use them only after static key/jump label is initialized. Any check
for feature before jump label init should be done using this new
helper.
Can't convince you to call it *_has_feature_early()?
Any point to a WARN_ON_ONCE() in these guys that trips if they are
used after the jump labels are set up?
Thanks,
Nick
From: Michael Ellerman <mpe@ellerman.id.au> Date: 2016-07-28 13:04:49
Nicholas Piggin [off-list ref] writes:
On Thu, 28 Jul 2016 00:18:08 +1000
Michael Ellerman [off-list ref] wrote:
quoted
From: "Aneesh Kumar K.V" <redacted>
This switches early feature checks to use the non static key variant
of the function. In later patches we will be switching
cpu_has_feature() and mmu_has_feature() to use static keys and we can
use them only after static key/jump label is initialized. Any check
for feature before jump label init should be done using this new
helper.
Can't convince you to call it *_has_feature_early()?
Hmmm, I'll go with early_cpu_has_feature().
Otherwise it reads "does the CPU have feature 'early'".
Any point to a WARN_ON_ONCE() in these guys that trips if they are
used after the jump labels are set up?
See patch 20. Though we can't use WARN() as it may be too early in boot
to WARN(), so it just prints() and dumps stack.
cheers
From: Michael Ellerman <mpe@ellerman.id.au> Date: 2016-07-27 14:18:51
From: Kevin Hao <redacted>
Some arches (powerpc at least) would like to invoke jump_label_init()
much earlier in boot. So check static_key_initialized in order to make
sure this function runs only once.
Signed-off-by: Kevin Hao <redacted>
Signed-off-by: Aneesh Kumar K.V <redacted>
Signed-off-by: Michael Ellerman <mpe@ellerman.id.au>
---
kernel/jump_label.c | 3 +++
1 file changed, 3 insertions(+)
v3: Updated change log.
From: Michael Ellerman <mpe@ellerman.id.au> Date: 2016-07-27 14:18:52
From: "Aneesh Kumar K.V" <redacted>
Call jump_label_init() early so that we can use static keys for CPU and
MMU feature checks.
Signed-off-by: Aneesh Kumar K.V <redacted>
Signed-off-by: Michael Ellerman <mpe@ellerman.id.au>
---
arch/powerpc/lib/feature-fixups.c | 8 ++++++++
1 file changed, 8 insertions(+)
v3: Updated comment.
From: Michael Ellerman <mpe@ellerman.id.au> Date: 2016-07-27 14:18:53
From: Kevin Hao <redacted>
This function is only used by get_vtb(). They are almost the same except
the reading from the real register. Move the mfspr() to get_vtb() and
kill the function mfvtb(). With this, we can eliminate the use of
cpu_has_feature() in very core header file like reg.h. This is a
preparation for the use of jump label for cpu_has_feature().
Signed-off-by: Kevin Hao <redacted>
Signed-off-by: Aneesh Kumar K.V <redacted>
Signed-off-by: Michael Ellerman <mpe@ellerman.id.au>
---
arch/powerpc/include/asm/reg.h | 9 ---------
arch/powerpc/include/asm/time.h | 2 +-
2 files changed, 1 insertion(+), 10 deletions(-)
v3: No change.
From: Michael Ellerman <mpe@ellerman.id.au> Date: 2016-07-27 14:18:54
From: Kevin Hao <redacted>
We plan to use jump label for cpu_has_feature(). In order to implement
this we need to include the linux/jump_label.h in asm/cputable.h.
Unfortunately if we do that it leads to an include loop. The root of the
problem seems to be that reg.h needs cputable.h (for CPU_FTRs), and then
cputable.h via jump_label.h eventually pulls in hw_irq.h which needs
reg.h (for MSR_EE).
So move cpu_has_feature() to a separate file on its own.
Signed-off-by: Kevin Hao <redacted>
Signed-off-by: Aneesh Kumar K.V <redacted>
Signed-off-by: Michael Ellerman <mpe@ellerman.id.au>
---
arch/powerpc/include/asm/book3s/64/mmu-hash.h | 1 +
arch/powerpc/include/asm/cacheflush.h | 1 +
arch/powerpc/include/asm/cpu_has_feature.h | 20 ++++++++++++++++++++
arch/powerpc/include/asm/cputable.h | 11 -----------
arch/powerpc/include/asm/cputime.h | 1 +
arch/powerpc/include/asm/dbell.h | 1 +
arch/powerpc/include/asm/dcr-native.h | 1 +
arch/powerpc/include/asm/mman.h | 1 +
arch/powerpc/include/asm/time.h | 1 +
arch/powerpc/include/asm/xor.h | 1 +
arch/powerpc/kernel/align.c | 1 +
arch/powerpc/kernel/irq.c | 1 +
arch/powerpc/kernel/process.c | 1 +
arch/powerpc/kernel/setup-common.c | 1 +
arch/powerpc/kernel/setup_32.c | 1 +
arch/powerpc/kernel/smp.c | 1 +
arch/powerpc/platforms/cell/pervasive.c | 1 +
arch/powerpc/xmon/ppc-dis.c | 1 +
18 files changed, 36 insertions(+), 11 deletions(-)
create mode 100644 arch/powerpc/include/asm/cpu_has_feature.h
v3: Change the header name, and flesh out change log.
@@ -20,6 +20,7 @@ along with this file; see the file COPYING. If not, write to the FreeSoftwareFoundation,51FranklinStreet-FifthFloor,Boston,MA02110-1301,USA.*/#include<asm/cputable.h>+#include<asm/cpu_has_feature.h>#include"nonstdio.h"#include"ansidecl.h"#include"ppc.h"
From: Michael Ellerman <mpe@ellerman.id.au> Date: 2016-07-27 14:18:55
Add a kconfig option to control whether we use jump label for the
cpu/mmu_has_feature() checks. Currently this does nothing, but we will
enabled it in the subsequent patches.
Signed-off-by: Michael Ellerman <mpe@ellerman.id.au>
---
arch/powerpc/Kconfig.debug | 9 +++++++++
1 file changed, 9 insertions(+)
v3: New.
@@ -60,6 +60,15 @@ config CODE_PATCHING_SELFTESTdepends onDEBUG_KERNELdefaultn+configJUMP_LABEL_FEATURE_CHECKS+bool"Enable use of jump label for cpu/mmu_has_feature()"+depends onJUMP_LABEL+defaulty+help+Selectingthisoptionsenablesuseofjumplabelsforsomeinternal+featurechecks.Thisshouldgeneratemoreoptimalcodeforthose+checks.+configFTR_FIXUP_SELFTESTbool"Run self-tests of the feature-fixup code"depends onDEBUG_KERNEL
From: Michael Ellerman <mpe@ellerman.id.au> Date: 2016-07-27 14:18:56
From: Kevin Hao <redacted>
We do binary patching of asm code using CPU features, which is a
one-time operation, done during early boot. However checks of CPU
features in C code are currently done at run time, even though the set
of CPU features can never change after boot.
We can optimise this by using jump labels to implement cpu_has_feature(),
meaning checks in C code are binary patched into a single nop or branch.
For a C sequence along the lines of:
if (cpu_has_feature(FOO))
return 2;
The generated code before is roughly:
ld r9,-27640(r2)
ld r9,0(r9)
lwz r9,32(r9)
cmpwi cr7,r9,0
bge cr7, 1f
li r3,2
blr
1: ...
After (true):
nop
li r3,2
blr
After (false):
b 1f
li r3,2
blr
1: ...
Signed-off-by: Kevin Hao <redacted>
Signed-off-by: Aneesh Kumar K.V <redacted>
Signed-off-by: Michael Ellerman <mpe@ellerman.id.au>
---
arch/powerpc/include/asm/cpu_has_feature.h | 22 ++++++++++++++++++++++
arch/powerpc/include/asm/cputable.h | 6 ++++++
arch/powerpc/kernel/cputable.c | 20 ++++++++++++++++++++
arch/powerpc/lib/feature-fixups.c | 1 +
4 files changed, 49 insertions(+)
v3: Rename MAX_CPU_FEATURES as we already have a #define with that name.
Define NUM_CPU_FTR_KEYS as a constant.
Rename the array to cpu_feature_keys.
Use the kconfig we added to guard it.
Rewrite the change log.
From: Nicholas Piggin <npiggin@gmail.com> Date: 2016-07-28 07:52:03
On Thu, 28 Jul 2016 00:18:14 +1000
Michael Ellerman [off-list ref] wrote:
quoted hunk
From: Kevin Hao <redacted>
We do binary patching of asm code using CPU features, which is a
one-time operation, done during early boot. However checks of CPU
features in C code are currently done at run time, even though the set
of CPU features can never change after boot.
We can optimise this by using jump labels to implement
cpu_has_feature(), meaning checks in C code are binary patched into a
single nop or branch.
For a C sequence along the lines of:
if (cpu_has_feature(FOO))
return 2;
The generated code before is roughly:
ld r9,-27640(r2)
ld r9,0(r9)
lwz r9,32(r9)
cmpwi cr7,r9,0
bge cr7, 1f
li r3,2
blr
1: ...
After (true):
nop
li r3,2
blr
After (false):
b 1f
li r3,2
blr
1: ...
Signed-off-by: Kevin Hao <redacted>
Signed-off-by: Aneesh Kumar K.V <redacted>
Signed-off-by: Michael Ellerman <mpe@ellerman.id.au>
---
arch/powerpc/include/asm/cpu_has_feature.h | 22
++++++++++++++++++++++ arch/powerpc/include/asm/cputable.h |
6 ++++++ arch/powerpc/kernel/cputable.c | 20
++++++++++++++++++++ arch/powerpc/lib/feature-fixups.c | 1 +
4 files changed, 49 insertions(+)
v3: Rename MAX_CPU_FEATURES as we already have a #define with that
name. Define NUM_CPU_FTR_KEYS as a constant.
Rename the array to cpu_feature_keys.
Use the kconfig we added to guard it.
Rewrite the change log.
From: Michael Ellerman <mpe@ellerman.id.au> Date: 2016-07-27 14:18:56
From: Kevin Hao <redacted>
As we just did for CPU features.
Signed-off-by: Kevin Hao <redacted>
Signed-off-by: Aneesh Kumar K.V <redacted>
Signed-off-by: Michael Ellerman <mpe@ellerman.id.au>
---
arch/powerpc/include/asm/mmu.h | 36 ++++++++++++++++++++++++++++++++++++
arch/powerpc/kernel/cputable.c | 17 +++++++++++++++++
arch/powerpc/lib/feature-fixups.c | 1 +
3 files changed, 54 insertions(+)
v3: Rename to mmu_feature_keys, and NUM_MMU_FTR_KEYS.
Use the kconfig.
From: Nicholas Piggin <npiggin@gmail.com> Date: 2016-07-28 07:53:04
On Thu, 28 Jul 2016 00:18:15 +1000
Michael Ellerman [off-list ref] wrote:
quoted hunk
From: Kevin Hao <redacted>
As we just did for CPU features.
Signed-off-by: Kevin Hao <redacted>
Signed-off-by: Aneesh Kumar K.V <redacted>
Signed-off-by: Michael Ellerman <mpe@ellerman.id.au>
---
arch/powerpc/include/asm/mmu.h | 36
++++++++++++++++++++++++++++++++++++
arch/powerpc/kernel/cputable.c | 17 +++++++++++++++++
arch/powerpc/lib/feature-fixups.c | 1 + 3 files changed, 54
insertions(+)
v3: Rename to mmu_feature_keys, and NUM_MMU_FTR_KEYS.
Use the kconfig.
diff --git a/arch/powerpc/include/asm/mmu.h
b/arch/powerpc/include/asm/mmu.h index e3eff365e55d..3900cb7fe7cf
100644 --- a/arch/powerpc/include/asm/mmu.h
From: Anton Blanchard <hidden> Date: 2016-08-08 07:35:18
Hi,
This patch causes an oops when building with the gold linker:
Found the problem. On binutils .meminit.text is within _stext/_etext:
[Nr] Name Type Address Off Size ES Flg Lk Inf Al
[ 3] .meminit.text PROGBITS c000000000989d14 999d14 00225c 00 AX 0 0 4
c000000000990000 R _etext
But on gold it is not:
c000000000970000 A _etext
[Nr] Name Type Address Off Size ES Flg Lk Inf Al
[ 3] .meminit.text PROGBITS c000000000970bcc 980bcc 002220 00 AX 0 0 4
As a result kernel_text_address() returns false, and
__jump_label_update() fails to update:
if (entry->code && kernel_text_address(entry->code))
arch_jump_label_transform(entry, jump_label_type(entry));
Seems like we get the correct layout on binutils by luck and we
need to explicitly handle .meminit.text.
Anton
From: Michael Ellerman <mpe@ellerman.id.au> Date: 2016-07-27 14:18:57
From: "Aneesh Kumar K.V" <redacted>
This allows us to catch incorrect usage of cpu_has_feature() and
mmu_has_feature() prior to jump labels being initialised.
Signed-off-by: Aneesh Kumar K.V <redacted>
Signed-off-by: Michael Ellerman <mpe@ellerman.id.au>
---
arch/powerpc/Kconfig.debug | 10 ++++++++++
arch/powerpc/include/asm/cpu_has_feature.h | 7 +++++++
arch/powerpc/include/asm/mmu.h | 14 ++++++++++++++
arch/powerpc/kernel/process.c | 2 +-
4 files changed, 32 insertions(+), 1 deletion(-)
v3: Use printk() and dump_stack() rather than WARN_ON(), because
WARN_ON() may not work this early in boot.
Rename the Kconfig.
@@ -69,6 +69,16 @@ config JUMP_LABEL_FEATURE_CHECKSfeaturechecks.Thisshouldgeneratemoreoptimalcodeforthosechecks.+configJUMP_LABEL_FEATURE_CHECK_DEBUG+bool"Do extra check on feature fixup calls"+depends onDEBUG_KERNEL&&JUMP_LABEL_FEATURE_CHECKS+defaultn+help+Thistriestocatchincorrectusageofcpu_has_feature()and+mmu_has_feature()inthecode.++Ifyoudon'tknowwhatthismeans,sayN.+configFTR_FIXUP_SELFTESTbool"Run self-tests of the feature-fixup code"depends onDEBUG_KERNEL
From: Michael Ellerman <mpe@ellerman.id.au> Date: 2016-07-27 14:18:58
Add a comment to the generated assembler for jump labels. This makes it
easier to identify them in asm listings (generated with $ make foo.s).
Signed-off-by: Michael Ellerman <mpe@ellerman.id.au>
---
arch/powerpc/include/asm/jump_label.h | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
v3: New.
From: Nicholas Piggin <npiggin@gmail.com> Date: 2016-07-28 07:57:07
On Thu, 28 Jul 2016 00:18:17 +1000
Michael Ellerman [off-list ref] wrote:
Add a comment to the generated assembler for jump labels. This makes
it easier to identify them in asm listings (generated with $ make
foo.s).
The series looks in much better shape now.
As a disclaimer I did not go through the shuffling around of the early
init code too closely, and I don't know that part of the arch very
well. But other than that it looks good.
Thanks,
Nick