Provide STRICT_KERNEL_RWX for PPC64/BOOK3S
These patches enable RX mappings of kernel text.
rodata is mapped RX as well as a trade-off, there
are more details in the patch description
As a prerequisite for R/O text, patch_instruction
is moved over to using a separate mapping that
allows write to kernel text. xmon/ftrace/kprobes
have been moved over to work with patch_instruction
There is a bug fix, the updatepp and updateboltedpp
(pseries) providers, did not use flags as described in
PAPR (patch 6). I would like to see that patch marked
to stable, I've not added a Cc:stable myself.
Another build failure was reported, because instead
of using ARCH_HAS_SET_MEMORY as a gate for set_memory.h
inclusion, some of the infrastructure in the core kernel
uses CONFIG_STRICT_KERNEL_RWX. I've sent a fix to the
fix the latter. It should be picked up by 4.13 at which
time we can remove the config dependency on !HIBERNATION
in arch/powerpc/Kconfig
This version received testing under CONFIG_RELOCATABLE_TEST.
CONFIG_STRICT_KERNEL_RWX does not work great with that config
and disables the feature when relocation is on and prints
a warning message. The radix variant of CONFIG_RELOCATABLE_TEST
did not boot with and without the config, I am investigating
the issue. I also suspect optprobes might not be compatible
with relocatable kernels (even without these patches).
After these changes go in we can get the PPC32 varaint
for the same feature based on patches already posted
by Christophe.
Changelog v4:
Multiple cleanups to patch_instruction() based on
review comments from Michael Ellerman
Changes to Kconfig to make the feature selectable
Changelog v3:
Support radix
Drop ptdump patch, already picked from v2
Changelog v2:
Support optprobes via patch_instruction
Balbir Singh (9):
powerpc/lib/code-patching: Use alternate map for patch_instruction()
powerpc/kprobes: Move kprobes over to patch_instruction
powerpc/kprobes/optprobes: Move over to patch_instruction
powerpc/xmon: Add patch_instruction() support for xmon
powerpc/vmlinux.lds: Align __init_begin to 16M
powerpc/platform/pseries/lpar: Fix updatepp and updateboltedpp
powerpc/mm/hash: Implement mark_rodata_ro() for hash
powerpc/mm/radix: Implement mark_rodata_ro() for radix
powerpc/Kconfig: Enable STRICT_KERNEL_RWX
arch/powerpc/Kconfig | 2 +
arch/powerpc/include/asm/book3s/64/hash.h | 3 +
arch/powerpc/include/asm/book3s/64/radix.h | 4 +
arch/powerpc/kernel/kprobes.c | 4 +-
arch/powerpc/kernel/optprobes.c | 58 +++++++----
arch/powerpc/kernel/vmlinux.lds.S | 10 +-
arch/powerpc/lib/code-patching.c | 160 ++++++++++++++++++++++++++++-
arch/powerpc/mm/pgtable-hash64.c | 41 ++++++++
arch/powerpc/mm/pgtable-radix.c | 73 ++++++++++++-
arch/powerpc/mm/pgtable_64.c | 9 ++
arch/powerpc/platforms/pseries/lpar.c | 21 +++-
arch/powerpc/xmon/xmon.c | 7 +-
12 files changed, 358 insertions(+), 34 deletions(-)
--
2.9.4
This patch creates the window using text_poke_area, allocated
via get_vm_area(). text_poke_area is per CPU to avoid locking.
text_poke_area for each cpu is setup using late_initcall, prior
to setup of these alternate mapping areas, we continue to use
direct write to change/modify kernel text. With the ability
to use alternate mappings to write to kernel text, it provides
us the freedom to then turn text read-only and implement
CONFIG_STRICT_KERNEL_RWX.
This code is CPU hotplug aware to ensure that the we have mappings
for any new cpus as they come online and tear down mappings for
any cpus that are offline.
Other arches do similar things, but use fixmaps. The reason
for not using fixmaps is to make use of any randomization in
the future.
Signed-off-by: Balbir Singh <bsingharora@gmail.com>
---
arch/powerpc/lib/code-patching.c | 160 ++++++++++++++++++++++++++++++++++++++-
1 file changed, 156 insertions(+), 4 deletions(-)
With text moving to read-only migrate optprobes to using
the patch_instruction infrastructure. Without this optprobes
will fail and complain.
Signed-off-by: Balbir Singh <bsingharora@gmail.com>
---
arch/powerpc/kernel/optprobes.c | 58 ++++++++++++++++++++++++++---------------
1 file changed, 37 insertions(+), 21 deletions(-)
@@ -231,8 +237,15 @@ int arch_prepare_optimized_kprobe(struct optimized_kprobe *op, struct kprobe *p)gotoerror;/* Setup template */-memcpy(buff,optprobe_template_entry,-TMPL_END_IDX*sizeof(kprobe_opcode_t));+/* We can optimize this via patch_instruction_window later */+size=(TMPL_END_IDX*sizeof(kprobe_opcode_t))/sizeof(int);+pr_devel("Copying template to %p, size %lu\n",buff,size);+for(i=0;i<size;i++){+rc=patch_instruction((unsignedint*)buff+i,+*((unsignedint*)(optprobe_template_entry)+i));+if(rc<0)+gotoerror;+}/**Fixupthetemplatewithinstructionsto:
For CONFIG_STRICT_KERNEL_RWX align __init_begin to 16M.
We use 16M since its the larger of 2M on radix and 16M
on hash for our linear mapping. The plan is to have
.text, .rodata and everything upto __init_begin marked
as RX. Note we still have executable read only data.
We could further align rodata to another 16M boundary.
I've used keeping text plus rodata as read-only-executable
as a trade-off to doing read-only-executable for text and
read-only for rodata.
We don't use multi PT_LOAD in PHDRS because we are
not sure if all bootloaders support them. This patch keeps
PHDRS in vmlinux.lds.S as the same they are with just one
PT_LOAD for all of the kernel marked as RWX (7).
Signed-off-by: Balbir Singh <bsingharora@gmail.com>
---
arch/powerpc/kernel/vmlinux.lds.S | 10 ++++++++--
1 file changed, 8 insertions(+), 2 deletions(-)
PAPR has pp0 in bit 55, currently we assumed that bit
pp0 is bit 0 (all bits in IBM order). This patch fixes
the pp0 bits for both these routines that use H_PROTECT.
(e58e87a powerpc/mm: Update _PAGE_KERNEL_RO)
Signed-off-by: Balbir Singh <bsingharora@gmail.com>
---
arch/powerpc/platforms/pseries/lpar.c | 21 +++++++++++++++++++--
1 file changed, 19 insertions(+), 2 deletions(-)
@@ -301,7 +301,7 @@ static long pSeries_lpar_hpte_updatepp(unsigned long slot,intssize,unsignedlonginv_flags){unsignedlonglpar_rc;-unsignedlongflags=(newpp&7)|H_AVPN;+unsignedlongflags;unsignedlongwant_v;want_v=hpte_encode_avpn(vpn,psize,ssize);
@@ -309,6 +309,15 @@ static long pSeries_lpar_hpte_updatepp(unsigned long slot,pr_devel(" update: avpnv=%016lx, hash=%016lx, f=%lx, psize: %d ...",want_v,slot,flags,psize);+/*+*Movepp0andsetthemask,pp0isbit55+*Weignorethekeysfornow.+*/+if(mmu_has_feature(MMU_FTR_KERNEL_RO))+flags=((newpp&HPTE_R_PP0)>>55)|(newpp&7)|H_AVPN;+else+flags=(newpp&7)|H_AVPN;+lpar_rc=plpar_pte_protect(flags,slot,want_v);if(lpar_rc==H_NOT_FOUND){
@@ -379,7 +388,15 @@ static void pSeries_lpar_hpte_updateboltedpp(unsigned long newpp,slot=pSeries_lpar_hpte_find(vpn,psize,ssize);BUG_ON(slot==-1);-flags=newpp&7;+/*+*Movepp0andsetthemask,pp0isbit55+*Weignorethekeysfornow.+*/+if(mmu_has_feature(MMU_FTR_KERNEL_RO))+flags=((newpp&HPTE_R_PP0)>>55)|(newpp&7);+else+flags=(newpp&7);+lpar_rc=plpar_pte_protect(flags,slot,0);BUG_ON(lpar_rc!=H_SUCCESS);
With hash we update the bolted pte to mark it read-only. We rely
on the MMU_FTR_KERNEL_RO to generate the correct permissions
for read-only text. The radix implementation just prints a warning
in this implementation
Signed-off-by: Balbir Singh <bsingharora@gmail.com>
---
arch/powerpc/include/asm/book3s/64/hash.h | 3 +++
arch/powerpc/include/asm/book3s/64/radix.h | 4 +++
arch/powerpc/mm/pgtable-hash64.c | 41 ++++++++++++++++++++++++++++++
arch/powerpc/mm/pgtable-radix.c | 7 +++++
arch/powerpc/mm/pgtable_64.c | 9 +++++++
5 files changed, 64 insertions(+)
@@ -342,3 +346,40 @@ int hash__has_transparent_hugepage(void)return1;}#endif /* CONFIG_TRANSPARENT_HUGEPAGE */++#ifdef CONFIG_STRICT_KERNEL_RWX+voidhash__mark_rodata_ro(void)+{+unsignedlongstart=(unsignedlong)_stext;+unsignedlongend=(unsignedlong)__init_begin;+unsignedlongidx;+unsignedintstep,shift;+unsignedlongnewpp=PP_RXXX;++if(!mmu_has_feature(MMU_FTR_KERNEL_RO)){+pr_info("R/O rodata not supported\n");+return;+}++shift=mmu_psize_defs[mmu_linear_psize].shift;+step=1<<shift;++start=((start+step-1)>>shift)<<shift;+end=(end>>shift)<<shift;++pr_devel("marking ro start %lx, end %lx, step %x\n",+start,end,step);++if(start==end){+pr_warn("could not set rodata ro, relocate the start"+" of the kernel to a 0x%x boundary\n",step);+return;+}++for(idx=start;idx<end;idx+=step)+/* Not sure if we can do much with the return value */+mmu_hash_ops.hpte_updateboltedpp(newpp,idx,mmu_linear_psize,+mmu_kernel_ssize);++}+#endif
@@ -109,6 +109,13 @@ int radix__map_kernel_page(unsigned long ea, unsigned long pa,return0;}+#ifdef CONFIG_STRICT_KERNEL_RWX+voidradix__mark_rodata_ro(void)+{+pr_warn("Not yet implemented for radix\n");+}+#endif+staticinlinevoid__meminitprint_mapping(unsignedlongstart,unsignedlongend,unsignedlongsize)
The patch splits the linear page mapping such that
the ones with kernel text are mapped as 2M and others
are mapped with the largest possible size - 1G. The downside
of this is that we split a 1G mapping into 512 2M mappings
for the kernel, but in the absence of that we cannot support
R/O areas in 1G, the kernel size is much smaller and using
1G as the granularity will waste a lot of space at the cost
of optimizing the TLB. The text itself should fit into about
6-8 mappings, so the effect should not be all that bad.
Signed-off-by: Balbir Singh <bsingharora@gmail.com>
---
arch/powerpc/mm/pgtable-radix.c | 68 +++++++++++++++++++++++++++++++++++++++--
1 file changed, 66 insertions(+), 2 deletions(-)
@@ -112,7 +115,53 @@ int radix__map_kernel_page(unsigned long ea, unsigned long pa,#ifdef CONFIG_STRICT_KERNEL_RWXvoidradix__mark_rodata_ro(void){-pr_warn("Not yet implemented for radix\n");+unsignedlongstart=(unsignedlong)_stext;+unsignedlongend=(unsignedlong)__init_begin;+unsignedlongidx;+unsignedintstep,shift;+pgd_t*pgdp;+pud_t*pudp;+pmd_t*pmdp;+pte_t*ptep;++if(!mmu_has_feature(MMU_FTR_KERNEL_RO)){+pr_info("R/O rodata not supported\n");+return;+}++shift=mmu_psize_defs[mmu_radix_linear_psize].shift;+step=1<<shift;++start=((start+step-1)>>shift)<<shift;+end=(end>>shift)<<shift;++pr_devel("marking ro start %lx, end %lx, step %x\n",+start,end,step);++for(idx=start;idx<end;idx+=step){+pgdp=pgd_offset_k(idx);+pudp=pud_alloc(&init_mm,pgdp,idx);+if(!pudp)+continue;+if(pud_huge(*pudp)){+ptep=(pte_t*)pudp;+gotoupdate_the_pte;+}+pmdp=pmd_alloc(&init_mm,pudp,idx);+if(!pmdp)+continue;+if(pmd_huge(*pmdp)){+ptep=pmdp_ptep(pmdp);+gotoupdate_the_pte;+}+ptep=pte_alloc_kernel(pmdp,idx);+if(!ptep)+continue;+update_the_pte:+radix__pte_update(&init_mm,idx,ptep,_PAGE_WRITE,0,0);+}+radix__flush_tlb_kernel_range(start,end);+}#endif
@@ -131,6 +180,7 @@ static int __meminit create_physical_mapping(unsigned long start,{unsignedlongvaddr,addr,mapping_size=0;pgprot_tprot;+unsignedlongmax_mapping_size;start=_ALIGN_UP(start,PAGE_SIZE);for(addr=start;addr<end;addr+=mapping_size){
@@ -139,9 +189,12 @@ static int __meminit create_physical_mapping(unsigned long start,gap=end-addr;previous_size=mapping_size;+max_mapping_size=PUD_SIZE;+retry:if(IS_ALIGNED(addr,PUD_SIZE)&&gap>=PUD_SIZE&&-mmu_psize_defs[MMU_PAGE_1G].shift)+mmu_psize_defs[MMU_PAGE_1G].shift&&+PUD_SIZE<=max_mapping_size)mapping_size=PUD_SIZE;elseif(IS_ALIGNED(addr,PMD_SIZE)&&gap>=PMD_SIZE&&mmu_psize_defs[MMU_PAGE_2M].shift)
@@ -149,6 +202,17 @@ static int __meminit create_physical_mapping(unsigned long start,elsemapping_size=PAGE_SIZE;+if(mapping_size==PUD_SIZE&&+addr<=__pa_symbol(__init_begin)&&+(addr+mapping_size)>=__pa_symbol(_stext)){+max_mapping_size=PMD_SIZE;+gotoretry;+}++if(addr<=__pa_symbol(__init_begin)&&+(addr+mapping_size)>=__pa_symbol(_stext))+mmu_radix_linear_psize=mapping_size;+if(mapping_size!=previous_size){print_mapping(start,addr,previous_size);start=addr;
We have the basic support in the form of patching R/O
text sections, linker scripts for extending alignment
of text data. We've also got mark_rodata_ro()
NOTE: There is a temporary work-around for disabling
STRICT_KERNEL_RWX if CONFIG_HIBERNATION is enabled
Signed-off-by: Balbir Singh <bsingharora@gmail.com>
---
arch/powerpc/Kconfig | 2 ++
1 file changed, 2 insertions(+)
This patch creates the window using text_poke_area, allocated
via get_vm_area(). text_poke_area is per CPU to avoid locking.
text_poke_area for each cpu is setup using late_initcall, prior
to setup of these alternate mapping areas, we continue to use
direct write to change/modify kernel text. With the ability
to use alternate mappings to write to kernel text, it provides
us the freedom to then turn text read-only and implement
CONFIG_STRICT_KERNEL_RWX.
This code is CPU hotplug aware to ensure that the we have mappings
for any new cpus as they come online and tear down mappings for
any cpus that are offline.
Other arches do similar things, but use fixmaps. The reason
for not using fixmaps is to make use of any randomization in
the future.
Signed-off-by: Balbir Singh <bsingharora@gmail.com>
---
arch/powerpc/lib/code-patching.c | 160 ++++++++++++++++++++++++++++++++++++++-
1 file changed, 156 insertions(+), 4 deletions(-)
arch_arm/disarm_probe use direct assignment for copying
instructions, replace them with patch_instruction
Signed-off-by: Balbir Singh <bsingharora@gmail.com>
With text moving to read-only migrate optprobes to using
the patch_instruction infrastructure. Without this optprobes
will fail and complain.
Signed-off-by: Balbir Singh <bsingharora@gmail.com>
@@ -231,8 +237,15 @@ int arch_prepare_optimized_kprobe(struct optimized_kprobe *op, struct kprobe *p)gotoerror;/* Setup template */-memcpy(buff,optprobe_template_entry,-TMPL_END_IDX*sizeof(kprobe_opcode_t));+/* We can optimize this via patch_instruction_window later */+size=(TMPL_END_IDX*sizeof(kprobe_opcode_t))/sizeof(int);+pr_devel("Copying template to %p, size %lu\n",buff,size);+for(i=0;i<size;i++){+rc=patch_instruction((unsignedint*)buff+i,+*((unsignedint*)(optprobe_template_entry)+i));+if(rc<0)+gotoerror;+}/**Fixupthetemplatewithinstructionsto:
On Tue, 2017-06-27 at 10:34 +0200, Christophe LEROY wrote:
Le 27/06/2017 à 09:48, Balbir Singh a écrit :
quoted
With text moving to read-only migrate optprobes to using
the patch_instruction infrastructure. Without this optprobes
will fail and complain.
Signed-off-by: Balbir Singh <bsingharora@gmail.com>
Didn't Michael picked it up already ?
Yes, he did, I posted the entire series and I'll let him keep the
better versions, he has edited. I spoke to him, but I was not 100%
sure what was picked up, the email responses mentioned 3, but I thought
4 patches were picked up
Balbir Singh.
On Tue, 2017-06-27 at 10:32 +0200, Christophe LEROY wrote:
Le 27/06/2017 à 09:48, Balbir Singh a écrit :
quoted
This patch creates the window using text_poke_area, allocated
via get_vm_area(). text_poke_area is per CPU to avoid locking.
text_poke_area for each cpu is setup using late_initcall, prior
to setup of these alternate mapping areas, we continue to use
direct write to change/modify kernel text. With the ability
to use alternate mappings to write to kernel text, it provides
us the freedom to then turn text read-only and implement
CONFIG_STRICT_KERNEL_RWX.
This code is CPU hotplug aware to ensure that the we have mappings
for any new cpus as they come online and tear down mappings for
any cpus that are offline.
Other arches do similar things, but use fixmaps. The reason
for not using fixmaps is to make use of any randomization in
the future.
Signed-off-by: Balbir Singh <bsingharora@gmail.com>
---
arch/powerpc/lib/code-patching.c | 160 ++++++++++++++++++++++++++++++++++++++-
1 file changed, 156 insertions(+), 4 deletions(-)