Shadow Stack provides protection against function return address
corruption. It is active when the processor supports it, the kernel has
CONFIG_X86_CET enabled, and the application is built for the feature.
This is only implemented for the 64-bit kernel. When it is enabled, legacy
non-Shadow Stack applications continue to work, but without protection.
Signed-off-by: Yu-cheng Yu <redacted>
---
arch/x86/Kconfig | 22 ++++++++++++++++++++++
arch/x86/Kconfig.assembler | 5 +++++
2 files changed, 27 insertions(+)
A control-protection fault is triggered when a control-flow transfer
attempt violates Shadow Stack or Indirect Branch Tracking constraints.
For example, the return address for a RET instruction differs from the copy
on the shadow stack; or an indirect JMP instruction, without the NOTRACK
prefix, arrives at a non-ENDBR opcode.
The control-protection fault handler works in a similar way as the general
protection fault handler. It provides the si_code SEGV_CPERR to the signal
handler.
Signed-off-by: Yu-cheng Yu <redacted>
Cc: Michael Kerrisk <redacted>
---
arch/x86/include/asm/idtentry.h | 4 ++
arch/x86/kernel/idt.c | 4 ++
arch/x86/kernel/signal_compat.c | 2 +-
arch/x86/kernel/traps.c | 60 ++++++++++++++++++++++++++++++
include/uapi/asm-generic/siginfo.h | 3 +-
5 files changed, 71 insertions(+), 2 deletions(-)
@@ -574,6 +574,10 @@ DECLARE_IDTENTRY_ERRORCODE(X86_TRAP_SS, exc_stack_segment);DECLARE_IDTENTRY_ERRORCODE(X86_TRAP_GP,exc_general_protection);DECLARE_IDTENTRY_ERRORCODE(X86_TRAP_AC,exc_alignment_check);+#ifdef CONFIG_X86_CET+DECLARE_IDTENTRY_ERRORCODE(X86_TRAP_CP,exc_control_protection);+#endif+/* Raw exception entries which need extra work */DECLARE_IDTENTRY_RAW(X86_TRAP_UD,exc_invalid_op);DECLARE_IDTENTRY_RAW(X86_TRAP_BP,exc_int3);
@@ -606,6 +606,66 @@ DEFINE_IDTENTRY_ERRORCODE(exc_general_protection)cond_local_irq_disable(regs);}+#ifdef CONFIG_X86_CET+staticconstchar*constcontrol_protection_err[]={+"unknown",+"near-ret",+"far-ret/iret",+"endbranch",+"rstorssp",+"setssbsy",+};++/*+*Whenacontrolprotectionexceptionoccurs,sendasignaltotheresponsible+*application.Currently,controlprotectionisonlyenabledforusermode.+*Thisexceptionshouldnotcomefromkernelmode.+*/+DEFINE_IDTENTRY_ERRORCODE(exc_control_protection)+{+staticDEFINE_RATELIMIT_STATE(rs,DEFAULT_RATELIMIT_INTERVAL,+DEFAULT_RATELIMIT_BURST);+structtask_struct*tsk;++if(!user_mode(regs)){+pr_emerg("PANIC: unexpected kernel control protection fault\n");+die("kernel control protection fault",regs,error_code);+panic("Machine halted.");+}++cond_local_irq_enable(regs);++if(!boot_cpu_has(X86_FEATURE_CET))+WARN_ONCE(1,"Control protection fault with CET support disabled\n");++tsk=current;+tsk->thread.error_code=error_code;+tsk->thread.trap_nr=X86_TRAP_CP;++if(show_unhandled_signals&&unhandled_signal(tsk,SIGSEGV)&&+__ratelimit(&rs)){+unsignedintmax_err;+unsignedlongssp;++max_err=ARRAY_SIZE(control_protection_err)-1;+if(error_code<0||error_code>max_err)+error_code=0;++rdmsrl(MSR_IA32_PL3_SSP,ssp);+pr_emerg("%s[%d] control protection ip:%lx sp:%lx ssp:%lx error:%lx(%s)",+tsk->comm,task_pid_nr(tsk),+regs->ip,regs->sp,ssp,error_code,+control_protection_err[error_code]);+print_vma_addr(KERN_CONT" in ",regs->ip);+pr_cont("\n");+}++force_sig_fault(SIGSEGV,SEGV_CPERR,+(void__user*)uprobe_get_trap_addr(regs));+cond_local_irq_disable(regs);+}+#endif+staticbooldo_int3(structpt_regs*regs){intres;
The x86 family of processors do not directly create read-only and Dirty
PTEs. These PTEs are created by software. One such case is that kernel
read-only pages are historically setup as Dirty.
New processors that support Shadow Stack regard read-only and Dirty PTEs as
shadow stack pages. This results in ambiguity between shadow stack and
kernel read-only pages. To resolve this, removed Dirty from kernel read-
only pages.
Signed-off-by: Yu-cheng Yu <redacted>
Cc: "H. Peter Anvin" <hpa@zytor.com>
Cc: Kees Cook <redacted>
Cc: Thomas Gleixner <redacted>
Cc: Dave Hansen <dave.hansen@linux.intel.com>
Cc: Christoph Hellwig <hch@lst.de>
Cc: Andy Lutomirski <luto@kernel.org>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: Borislav Petkov <bp@alien8.de>
Cc: Peter Zijlstra <peterz@infradead.org>
---
arch/x86/include/asm/pgtable_types.h | 6 +++---
arch/x86/mm/pat/set_memory.c | 2 +-
2 files changed, 4 insertions(+), 4 deletions(-)
@@ -1932,7 +1932,7 @@ int set_memory_nx(unsigned long addr, int numpages)intset_memory_ro(unsignedlongaddr,intnumpages){-returnchange_page_attr_clear(&addr,numpages,__pgprot(_PAGE_RW),0);+returnchange_page_attr_clear(&addr,numpages,__pgprot(_PAGE_RW|_PAGE_DIRTY),0);}intset_memory_rw(unsignedlongaddr,intnumpages)
There is essentially no room left in the x86 hardware PTEs on some OSes
(not Linux). That left the hardware architects looking for a way to
represent a new memory type (shadow stack) within the existing bits.
They chose to repurpose a lightly-used state: Write=0, Dirty=1.
The reason it's lightly used is that Dirty=1 is normally set by hardware
and cannot normally be set by hardware on a Write=0 PTE. Software must
normally be involved to create one of these PTEs, so software can simply
opt to not create them.
In places where Linux normally creates Write=0, Dirty=1, it can use the
software-defined _PAGE_COW in place of the hardware _PAGE_DIRTY. In other
words, whenever Linux needs to create Write=0, Dirty=1, it instead creates
Write=0, Cow=1, except for shadow stack, which is Write=0, Dirty=1. This
clearly separates shadow stack from other data, and results in the
following:
(a) A modified, copy-on-write (COW) page: (Write=0, Cow=1)
(b) A R/O page that has been COW'ed: (Write=0, Cow=1)
The user page is in a R/O VMA, and get_user_pages() needs a writable
copy. The page fault handler creates a copy of the page and sets
the new copy's PTE as Write=0 and Cow=1.
(c) A shadow stack PTE: (Write=0, Dirty=1)
(d) A shared shadow stack PTE: (Write=0, Cow=1)
When a shadow stack page is being shared among processes (this happens
at fork()), its PTE is made Dirty=0, so the next shadow stack access
causes a fault, and the page is duplicated and Dirty=1 is set again.
This is the COW equivalent for shadow stack pages, even though it's
copy-on-access rather than copy-on-write.
(e) A page where the processor observed a Write=1 PTE, started a write, set
Dirty=1, but then observed a Write=0 PTE. That's possible today, but
will not happen on processors that support shadow stack.
Define _PAGE_COW and update pte_*() helpers and apply the same changes to
pmd and pud.
After this, there are six free bits left in the 64-bit PTE, and no more
free bits in the 32-bit PTE (except for PAE) and Shadow Stack is not
implemented for the 32-bit kernel.
Signed-off-by: Yu-cheng Yu <redacted>
---
arch/x86/include/asm/pgtable.h | 125 ++++++++++++++++++++++++---
arch/x86/include/asm/pgtable_types.h | 42 ++++++++-
2 files changed, 154 insertions(+), 13 deletions(-)
@@ -23,7 +23,8 @@#define _PAGE_BIT_SOFTW2 10 /* " */#define _PAGE_BIT_SOFTW3 11 /* " */#define _PAGE_BIT_PAT_LARGE 12 /* On 2MB or 1GB pages */-#define _PAGE_BIT_SOFTW4 58 /* available for programmer */+#define _PAGE_BIT_SOFTW4 57 /* available for programmer */+#define _PAGE_BIT_SOFTW5 58 /* available for programmer */#define _PAGE_BIT_PKEY_BIT0 59 /* Protection Keys, bit 1/4 */#define _PAGE_BIT_PKEY_BIT1 60 /* Protection Keys, bit 2/4 */#define _PAGE_BIT_PKEY_BIT2 61 /* Protection Keys, bit 3/4 */
@@ -36,6 +37,15 @@#define _PAGE_BIT_SOFT_DIRTY _PAGE_BIT_SOFTW3 /* software dirty tracking */#define _PAGE_BIT_DEVMAP _PAGE_BIT_SOFTW4+/*+*Indicatesacopy-on-writepage.+*/+#ifdef CONFIG_X86_CET+#define _PAGE_BIT_COW _PAGE_BIT_SOFTW5 /* copy-on-write */+#else+#define _PAGE_BIT_COW 0+#endif+/* If _PAGE_BIT_PRESENT is clear, we use these: *//* - if the user mapped it with PROT_NONE; pte_present gives true */#define _PAGE_BIT_PROTNONE _PAGE_BIT_GLOBAL
The read-only and Dirty PTE has been used to indicate copy-on-write pages.
However, newer x86 processors also regard a read-only and Dirty PTE as a
shadow stack page. In order to separate the two, the software-defined
_PAGE_COW is created to replace _PAGE_DIRTY for the copy-on-write case, and
pte_*() are updated.
Pte_modify() changes a PTE to 'newprot', but it doesn't use the pte_*().
Introduce fixup_dirty_pte(), which sets a dirty PTE, based on _PAGE_RW,
to either _PAGE_DIRTY or _PAGE_COW.
Apply the same changes to pmd_modify().
Signed-off-by: Yu-cheng Yu <redacted>
---
arch/x86/include/asm/pgtable.h | 33 +++++++++++++++++++++++++++++++++
1 file changed, 33 insertions(+)
Shadow stack accesses are those that are performed by the CPU where it
expects to encounter a shadow stack mapping. These accesses are performed
implicitly by CALL/RET at the site of the shadow stack pointer. These
accesses are made explicitly by shadow stack management instructions like
WRUSSQ.
Shadow stacks accesses to shadow-stack mapping can see faults in normal,
valid operation just like regular accesses to regular mappings. Shadow
stacks need some of the same features like delayed allocation, swap and
copy-on-write.
Shadow stack accesses can also result in errors, such as when a shadow
stack overflows, or if a shadow stack access occurs to a non-shadow-stack
mapping.
In handling a shadow stack page fault, verify it occurs within a shadow
stack mapping. It is always an error otherwise. For valid shadow stack
accesses, set FAULT_FLAG_WRITE to effect copy-on-write. Because clearing
_PAGE_DIRTY (vs. _PAGE_RW) is used to trigger the fault, shadow stack read
fault and shadow stack write fault are not differentiated and both are
handled as a write access.
Signed-off-by: Yu-cheng Yu <redacted>
Reviewed-by: Kees Cook <redacted>
---
arch/x86/include/asm/trap_pf.h | 2 ++
arch/x86/mm/fault.c | 19 +++++++++++++++++++
2 files changed, 21 insertions(+)
When Shadow Stack is introduced, [R/O + _PAGE_DIRTY] PTE is reserved for
shadow stack. Copy-on-write PTEs have [R/O + _PAGE_COW].
When a PTE goes from [R/W + _PAGE_DIRTY] to [R/O + _PAGE_COW], it could
become a transient shadow stack PTE in two cases:
The first case is that some processors can start a write but end up seeing
a read-only PTE by the time they get to the Dirty bit, creating a transient
shadow stack PTE. However, this will not occur on processors supporting
Shadow Stack, and a TLB flush is not necessary.
The second case is that when _PAGE_DIRTY is replaced with _PAGE_COW non-
atomically, a transient shadow stack PTE can be created as a result.
Thus, prevent that with cmpxchg.
Dave Hansen, Jann Horn, Andy Lutomirski, and Peter Zijlstra provided many
insights to the issue. Jann Horn provided the cmpxchg solution.
Signed-off-by: Yu-cheng Yu <redacted>
---
arch/x86/include/asm/pgtable.h | 36 ++++++++++++++++++++++++++++++++++
1 file changed, 36 insertions(+)
A shadow stack PTE must be read-only and have _PAGE_DIRTY set. However,
read-only and Dirty PTEs also exist for copy-on-write (COW) pages. These
two cases are handled differently for page faults. Introduce VM_SHSTK to
track shadow stack VMAs.
Signed-off-by: Yu-cheng Yu <redacted>
Reviewed-by: Kees Cook <redacted>
---
arch/x86/mm/mmap.c | 2 ++
fs/proc/task_mmu.c | 3 +++
include/linux/mm.h | 8 ++++++++
3 files changed, 13 insertions(+)
When serving a page fault, maybe_mkwrite() makes a PTE writable if it is in
a writable vma. A shadow stack vma is writable, but its PTEs need
_PAGE_DIRTY to be set to become writable. For this reason, maybe_mkwrite()
has been updated.
There are a few places that call pte_mkwrite() directly, but effect the
same result as from maybe_mkwrite(). These sites need to be updated for
shadow stack as well. Thus, change them to maybe_mkwrite():
- do_anonymous_page() and migrate_vma_insert_page() check VM_WRITE directly
and call pte_mkwrite(), which is the same as maybe_mkwrite(). Change
them to maybe_mkwrite().
- In do_numa_page(), if the numa entry 'was-writable', then pte_mkwrite()
is called directly. Fix it by doing maybe_mkwrite().
- In change_pte_range(), pte_mkwrite() is called directly. Replace it with
maybe_mkwrite().
A shadow stack vma is writable but has different vma
flags, and handled accordingly in maybe_mkwrite().
Signed-off-by: Yu-cheng Yu <redacted>
---
mm/memory.c | 5 ++---
mm/migrate.c | 3 +--
mm/mprotect.c | 2 +-
3 files changed, 4 insertions(+), 6 deletions(-)
There was no more caller passing vm_flags to do_mmap(), and vm_flags was
removed from the function's input by:
commit 45e55300f114 ("mm: remove unnecessary wrapper function do_mmap_pgoff()").
There is a new user now. Shadow stack allocation passes VM_SHSTK to
do_mmap(). Re-introduce vm_flags to do_mmap(), but without the old wrapper
do_mmap_pgoff(). Instead, make all callers of the wrapper pass a zero
vm_flags to do_mmap().
Signed-off-by: Yu-cheng Yu <redacted>
Reviewed-by: Peter Collingbourne <redacted>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Oleg Nesterov <oleg@redhat.com>
Cc: linux-mm@kvack.org
---
fs/aio.c | 2 +-
include/linux/mm.h | 3 ++-
ipc/shm.c | 2 +-
mm/mmap.c | 10 +++++-----
mm/nommu.c | 4 ++--
mm/util.c | 2 +-
6 files changed, 12 insertions(+), 11 deletions(-)
@@ -1071,6 +1071,7 @@ unsigned long do_mmap(struct file *file,unsignedlonglen,unsignedlongprot,unsignedlongflags,+vm_flags_tvm_flags,unsignedlongpgoff,unsignedlong*populate,structlist_head*uf)
@@ -1078,7 +1079,6 @@ unsigned long do_mmap(struct file *file,structvm_area_struct*vma;structvm_region*region;structrb_node*rb;-vm_flags_tvm_flags;unsignedlongcapabilities,result;intret;
@@ -1097,7 +1097,7 @@ unsigned long do_mmap(struct file *file,/* we've determined that we can make the mapping, now translate what we*nowknowintoVMAflags*/-vm_flags=determine_vm_flags(file,prot,flags,capabilities);+vm_flags|=determine_vm_flags(file,prot,flags,capabilities);/* we're going to need to record the mapping */region=kmem_cache_zalloc(vm_region_jar,GFP_KERNEL);
@@ -536,6 +537,10 @@ struct thread_struct {unsignedintsig_on_uaccess_err:1;+#ifdef CONFIG_X86_CET+structcet_statuscet;+#endif+/* Floating point and extended processor state */structfpufpu;/*
To deliver a signal, create a shadow stack restore token and put the token
and the signal restorer address on the shadow stack. For sigreturn, verify
the token and restore from it the shadow stack pointer.
A shadow stack restore token marks a restore point of the shadow stack.
The token is distinctively different from any shadow stack address.
In sigreturn, restoring from a token ensures the target address is the
location pointed by the token.
Introduce WRUSS, which is a kernel-mode instruction but writes directly to
user shadow stack. It is used to construct the user signal stack as
described above.
Currently there is no systematic facility for extending a signal context.
Introduce a signal context extension 'struct sc_ext', which is used to save
shadow stack restore token address and WAIT_ENDBR status. WAIT_ENDBR will
be introduced later in the Indirect Branch Tracking (IBT) series, but add
that into sc_ext now to keep the struct stable in case the IBT series is
applied later.
Signed-off-by: Yu-cheng Yu <redacted>
---
arch/x86/ia32/ia32_signal.c | 17 +++
arch/x86/include/asm/cet.h | 8 ++
arch/x86/include/asm/fpu/internal.h | 10 ++
arch/x86/include/asm/special_insns.h | 32 ++++++
arch/x86/include/uapi/asm/sigcontext.h | 9 ++
arch/x86/kernel/cet.c | 152 +++++++++++++++++++++++++
arch/x86/kernel/fpu/signal.c | 100 ++++++++++++++++
arch/x86/kernel/signal.c | 10 ++
8 files changed, 338 insertions(+)
@@ -196,6 +196,15 @@ struct _xstate {/* New processor state extensions go here: */};+/*+*Locatedattheendofsigcontext->fpstate,alignedto8.+*/+structsc_ext{+unsignedlongtotal_size;+unsignedlongssp;+unsignedlongwait_endbr;+};+/**The32-bitsignalframe:*/
@@ -72,6 +74,80 @@ static unsigned long alloc_shstk(unsigned long size, int flags)returnaddr;}+#define TOKEN_MODE_MASK 3UL+#define TOKEN_MODE_64 1UL+#define IS_TOKEN_64(token) (((token) & TOKEN_MODE_MASK) == TOKEN_MODE_64)+#define IS_TOKEN_32(token) (((token) & TOKEN_MODE_MASK) == 0)++/*+*Verifytherestoretokenattheaddressof'ssp'is+*validandthensetshadowstackpointeraccordingtothe+*token.+*/+intcet_verify_rstor_token(boolia32,unsignedlongssp,+unsignedlong*new_ssp)+{+unsignedlongtoken;++*new_ssp=0;++if(!IS_ALIGNED(ssp,8))+return-EINVAL;++if(get_user(token,(unsignedlong__user*)ssp))+return-EFAULT;++/* Is 64-bit mode flag correct? */+if(!ia32&&!IS_TOKEN_64(token))+return-EINVAL;+elseif(ia32&&!IS_TOKEN_32(token))+return-EINVAL;++token&=~TOKEN_MODE_MASK;++/*+*Restoreaddressproperlyaligned?+*/+if((!ia32&&!IS_ALIGNED(token,8))||!IS_ALIGNED(token,4))+return-EINVAL;++/*+*Tokenwasplacedproperly?+*/+if(((ALIGN_DOWN(token,8)-8)!=ssp)||(token>=TASK_SIZE_MAX))+return-EINVAL;++*new_ssp=token;+return0;+}++/*+*Createarestoretokenontheshadowstack.+*Atokenisalways8-byteandalignedto8.+*/+staticintcreate_rstor_token(boolia32,unsignedlongssp,+unsignedlong*new_ssp)+{+unsignedlongaddr;++*new_ssp=0;++if((!ia32&&!IS_ALIGNED(ssp,8))||!IS_ALIGNED(ssp,4))+return-EINVAL;++addr=ALIGN_DOWN(ssp,8)-8;++/* Is the token for 64-bit? */+if(!ia32)+ssp|=TOKEN_MODE_64;++if(write_user_shstk_64(addr,ssp))+return-EFAULT;++*new_ssp=addr;+return0;+}+intcet_setup_shstk(void){unsignedlongaddr,size;
There are three possible options to create a shadow stack allocation API:
an arch_prctl, a new syscall, or adding PROT_SHSTK to mmap()/mprotect().
Each has its advantages and compromises.
An arch_prctl() is the least intrusive. However, the existing x86
arch_prctl() takes only two parameters. Multiple parameters must be
passed in a memory buffer. There is a proposal to pass more parameters in
registers [1], but no active discussion on that.
A new syscall minimizes compatibility issues and offers an extensible frame
work to other architectures, but this will likely result in some overlap of
mmap()/mprotect().
The introduction of PROT_SHSTK to mmap()/mprotect() takes advantage of
existing APIs. The x86-specific PROT_SHSTK is translated to VM_SHSTK and
a shadow stack mapping is created without reinventing the wheel. There are
potential pitfalls though. The most obvious one would be using this as a
bypass to shadow stack protection. However, the attacker would have to get
to the syscall first.
Since arch_calc_vm_prot_bits() is modified, I have moved arch_vm_get_page
_prot() and arch_calc_vm_prot_bits() to x86/include/asm/mman.h.
This will be more consistent with other architectures.
[1] https://lore.kernel.org/lkml/20200828121624.108243-1-hjl.tools@gmail.com/
Signed-off-by: Yu-cheng Yu <redacted>
---
arch/x86/include/asm/mman.h | 85 ++++++++++++++++++++++++++++++++
arch/x86/include/uapi/asm/mman.h | 28 ++---------
include/linux/mm.h | 1 +
mm/mmap.c | 8 ++-
4 files changed, 97 insertions(+), 25 deletions(-)
create mode 100644 arch/x86/include/asm/mman.h
arch_prctl(ARCH_X86_CET_STATUS, u64 *args)
Get CET feature status.
The parameter 'args' is a pointer to a user buffer. The kernel returns
the following information:
*args = shadow stack/IBT status
*(args + 1) = shadow stack base address
*(args + 2) = shadow stack size
32-bit binaries use the same interface, but only lower 32-bits of each
item.
arch_prctl(ARCH_X86_CET_DISABLE, unsigned int features)
Disable CET features specified in 'features'. Return -EPERM if CET is
locked.
arch_prctl(ARCH_X86_CET_LOCK)
Lock in CET features.
Also change do_arch_prctl_common()'s parameter 'cpuid_enabled' to
'arg2', as it is now also passed to prctl_cet().
Signed-off-by: Yu-cheng Yu <redacted>
---
arch/x86/include/asm/cet.h | 3 ++
arch/x86/include/uapi/asm/prctl.h | 4 +++
arch/x86/kernel/Makefile | 2 +-
arch/x86/kernel/cet_prctl.c | 60 +++++++++++++++++++++++++++++++
arch/x86/kernel/process.c | 6 ++--
5 files changed, 71 insertions(+), 4 deletions(-)
create mode 100644 arch/x86/kernel/cet_prctl.c
@@ -25,6 +27,7 @@ int cet_verify_rstor_token(bool ia32, unsigned long ssp, unsigned long *new_ssp)voidcet_restore_signal(structsc_ext*sc);intcet_setup_signal(boolia32,unsignedlongrstor,structsc_ext*sc);#else+staticinlineintprctl_cet(intoption,u64arg2){return-EINVAL;}staticinlineintcet_setup_thread_shstk(structtask_struct*p,unsignedlongclone_flags){return0;}staticinlinevoidcet_disable_shstk(void){}
An ELF file's .note.gnu.property indicates arch features supported by the
file. These features are extracted by arch_parse_elf_property() and stored
in 'arch_elf_state'.
Introduce x86 feature definitions and arch_setup_elf_property(), which
enables such features. The first use-case of this function is Shadow
Stack.
ARM64 is the other arch that has ARCH_USE_GNU_PROPERTY and arch_parse_elf_
property(). Add arch_setup_elf_property() for it.
Signed-off-by: Yu-cheng Yu <redacted>
Cc: Mark Brown <broonie@kernel.org>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Dave Martin <Dave.Martin@arm.com>
---
arch/arm64/include/asm/elf.h | 5 +++++
arch/x86/Kconfig | 2 ++
arch/x86/include/asm/elf.h | 13 +++++++++++++
arch/x86/kernel/process_64.c | 32 ++++++++++++++++++++++++++++++++
fs/binfmt_elf.c | 4 ++++
include/linux/elf.h | 6 ++++++
include/uapi/linux/elf.h | 9 +++++++++
7 files changed, 71 insertions(+)
The kernel allocates (and frees on thread exit) a new shadow stack for a
pthread child.
It is possible for the kernel to complete the clone syscall and set the
child's shadow stack pointer to NULL and let the child thread allocate
a shadow stack for itself. There are two issues in this approach: It
is not compatible with existing code that does inline syscall and it
cannot handle signals before the child can successfully allocate a
shadow stack.
A 64-bit shadow stack has a size of min(RLIMIT_STACK, 4 GB). A compat-mode
thread shadow stack has a size of 1/4 min(RLIMIT_STACK, 4 GB). This allows
more threads to run in a 32-bit address space.
Signed-off-by: Yu-cheng Yu <redacted>
---
arch/x86/include/asm/cet.h | 3 ++
arch/x86/include/asm/mmu_context.h | 3 ++
arch/x86/kernel/cet.c | 44 ++++++++++++++++++++++++++++++
arch/x86/kernel/process.c | 8 ++++++
4 files changed, 58 insertions(+)
@@ -172,6 +172,50 @@ int cet_setup_shstk(void)return0;}+intcet_setup_thread_shstk(structtask_struct*tsk,unsignedlongclone_flags)+{+unsignedlongaddr,size;+structcet_user_state*state;+structcet_status*cet=&tsk->thread.cet;++if(!cet->shstk_size)+return0;++if((clone_flags&(CLONE_VFORK|CLONE_VM))!=CLONE_VM)+return0;++state=get_xsave_addr(&tsk->thread.fpu.state.xsave,+XFEATURE_CET_USER);++if(!state)+return-EINVAL;++/* Cap shadow stack size to 4 GB */+size=min(rlimit(RLIMIT_STACK),1UL<<32);++/*+*Compat-modepthreadssharealimitedaddressspace.+*Ifeachfunctioncalltakesanaverageoffourslots+*stackspace,allocate1/4ofstacksizeforshadowstack.+*/+if(in_compat_syscall())+size/=4;+size=round_up(size,PAGE_SIZE);+addr=alloc_shstk(size,0);++if(IS_ERR_VALUE(addr)){+cet->shstk_base=0;+cet->shstk_size=0;+returnPTR_ERR((void*)addr);+}++fpu__prepare_write(&tsk->thread.fpu);+state->user_ssp=(u64)(addr+size);+cet->shstk_base=addr;+cet->shstk_size=size;+return0;+}+voidcet_disable_shstk(void){structcet_status*cet=¤t->thread.cet;
@@ -181,6 +183,12 @@ int copy_thread(unsigned long clone_flags, unsigned long sp, unsigned long arg,if(clone_flags&CLONE_SETTLS)ret=set_new_tls(p,tls);+#ifdef CONFIG_X86_64+/* Allocate a new shadow stack for pthread */+if(!ret)+ret=cet_setup_thread_shstk(p,clone_flags);+#endif+if(!ret&&unlikely(test_tsk_thread_flag(current,TIF_IO_BITMAP)))io_bitmap_share(p);
Can_follow_write_pte() ensures a read-only page is COWed by checking the
FOLL_COW flag, and uses pte_dirty() to validate the flag is still valid.
Like a writable data page, a shadow stack page is writable, and becomes
read-only during copy-on-write, but it is always dirty. Thus, in the
can_follow_write_pte() check, it belongs to the writable page case and
should be excluded from the read-only page pte_dirty() check. Apply
the same changes to can_follow_write_pmd().
Signed-off-by: Yu-cheng Yu <redacted>
---
mm/gup.c | 8 +++++---
mm/huge_memory.c | 8 +++++---
2 files changed, 10 insertions(+), 6 deletions(-)
INCSSP(Q/D) increments shadow stack pointer and 'pops and discards' the
first and the last elements in the range, effectively touches those memory
areas.
The maximum moving distance by INCSSPQ is 255 * 8 = 2040 bytes and
255 * 4 = 1020 bytes by INCSSPD. Both ranges are far from PAGE_SIZE.
Thus, putting a gap page on both ends of a shadow stack prevents INCSSP,
CALL, and RET from going beyond.
Signed-off-by: Yu-cheng Yu <redacted>
---
arch/x86/include/asm/page_64_types.h | 10 ++++++++++
include/linux/mm.h | 24 ++++++++++++++++++++----
2 files changed, 30 insertions(+), 4 deletions(-)
When serving a page fault, maybe_mkwrite() makes a PTE writable if its vma
has VM_WRITE.
A shadow stack vma has VM_SHSTK. Its PTEs have _PAGE_DIRTY, but not
_PAGE_WRITE. In fork(), _PAGE_DIRTY is cleared to effect copy-on-write,
and in page fault, _PAGE_DIRTY is restored and the shadow stack page is
writable again.
Update maybe_mkwrite() by introducing arch_maybe_mkwrite(), which sets
_PAGE_DIRTY for a shadow stack PTE.
Apply the same changes to maybe_pmd_mkwrite().
Signed-off-by: Yu-cheng Yu <redacted>
---
arch/x86/Kconfig | 4 ++++
arch/x86/mm/pgtable.c | 18 ++++++++++++++++++
include/linux/mm.h | 2 ++
include/linux/pgtable.h | 24 ++++++++++++++++++++++++
mm/huge_memory.c | 2 ++
5 files changed, 50 insertions(+)
After the introduction of _PAGE_COW, a modified page's PTE can have either
_PAGE_DIRTY or _PAGE_COW. Change _PAGE_DIRTY to _PAGE_DIRTY_BITS.
Signed-off-by: Yu-cheng Yu <redacted>
Reviewed-by: Kees Cook <redacted>
Cc: David Airlie <redacted>
Cc: Joonas Lahtinen <joonas.lahtinen@linux.intel.com>
Cc: Jani Nikula <jani.nikula@linux.intel.com>
Cc: Daniel Vetter <redacted>
Cc: Rodrigo Vivi <rodrigo.vivi@intel.com>
Cc: Zhenyu Wang <redacted>
Cc: Zhi Wang <redacted>
---
drivers/gpu/drm/i915/gvt/gtt.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
Control-flow Enforcement Technology (CET) introduces these MSRs:
MSR_IA32_U_CET (user-mode CET settings),
MSR_IA32_PL3_SSP (user-mode shadow stack pointer),
MSR_IA32_PL0_SSP (kernel-mode shadow stack pointer),
MSR_IA32_PL1_SSP (Privilege Level 1 shadow stack pointer),
MSR_IA32_PL2_SSP (Privilege Level 2 shadow stack pointer),
MSR_IA32_S_CET (kernel-mode CET settings),
MSR_IA32_INT_SSP_TAB (exception shadow stack table).
The two user-mode MSRs belong to XFEATURE_CET_USER. The first three of
kernel-mode MSRs belong to XFEATURE_CET_KERNEL. Both XSAVES states are
supervisor states. This means that there is no direct, unprivileged access
to these states, making it harder for an attacker to subvert CET.
For sigreturn and future ptrace() support, shadow stack address and MSR
reserved bits are checked before written to the supervisor states.
Signed-off-by: Yu-cheng Yu <redacted>
---
arch/x86/include/asm/fpu/types.h | 23 +++++++++++++++++++++--
arch/x86/include/asm/fpu/xstate.h | 6 ++++--
arch/x86/include/asm/msr-index.h | 19 +++++++++++++++++++
arch/x86/kernel/fpu/xstate.c | 10 +++++++++-
4 files changed, 53 insertions(+), 5 deletions(-)
@@ -35,7 +35,8 @@XFEATURE_MASK_BNDCSR)/* All currently supported supervisor features */-#define XFEATURE_MASK_SUPERVISOR_SUPPORTED (XFEATURE_MASK_PASID)+#define XFEATURE_MASK_SUPERVISOR_SUPPORTED (XFEATURE_MASK_PASID | \+XFEATURE_MASK_CET_USER)/**Asupervisorstatecomponentmaynotalwayscontainvaluableinformation,
@@ -62,7 +63,8 @@*Unsupportedsupervisorfeatures.Whenasupervisorfeatureinthismaskis*supportedinthefuture,moveittothesupportedsupervisorfeaturemask.*/-#define XFEATURE_MASK_SUPERVISOR_UNSUPPORTED (XFEATURE_MASK_PT)+#define XFEATURE_MASK_SUPERVISOR_UNSUPPORTED (XFEATURE_MASK_PT | \+XFEATURE_MASK_CET_KERNEL)/* All supervisor states including supported and unsupported states. */#define XFEATURE_MASK_SUPERVISOR_ALL (XFEATURE_MASK_SUPERVISOR_SUPPORTED | \
@@ -0,0 +1,136 @@+.. SPDX-License-Identifier: GPL-2.0++=========================================+Control-flow Enforcement Technology (CET)+=========================================++[1] Overview+============++Control-flow Enforcement Technology (CET) is an Intel processor feature+that provides protection against return/jump-oriented programming (ROP)+attacks. It can be set up to protect both applications and the kernel.+Only user-mode protection is implemented in the 64-bit kernel, including+support for running legacy 32-bit applications.++CET introduces Shadow Stack and Indirect Branch Tracking. Shadow stack is+a secondary stack allocated from memory and cannot be directly modified by+applications. When executing a CALL instruction, the processor pushes the+return address to both the normal stack and the shadow stack. Upon+function return, the processor pops the shadow stack copy and compares it+to the normal stack copy. If the two differ, the processor raises a+control-protection fault. Indirect branch tracking verifies indirect+CALL/JMP targets are intended as marked by the compiler with 'ENDBR'+opcodes.++There is a Kconfig option:++ X86_CET.++To build a CET-enabled kernel, Binutils v2.31 and GCC v8.1 or LLVM v10.0.1+or later are required. To build a CET-enabled application, GLIBC v2.28 or+later is also required.++There are two command-line options for disabling CET features::++ no_user_shstk - disables user shadow stack, and+ no_user_ibt - disables user indirect branch tracking.++At run time, /proc/cpuinfo shows CET features if the processor supports+CET.++[2] Application Enabling+========================++An application's CET capability is marked in its ELF header and can be+verified from readelf/llvm-readelf output:++ readelf -n <application> | grep -a SHSTK+ properties: x86 feature: IBT, SHSTK++If an application supports CET and is statically linked, it will run with+CET protection. If the application needs any shared libraries, the loader+checks all dependencies and enables CET when all requirements are met.++[3] Backward Compatibility+==========================++GLIBC provides a few CET tunables via the GLIBC_TUNABLES environment+variable:++GLIBC_TUNABLES=glibc.tune.hwcaps=-SHSTK,-IBT+ Turn off SHSTK/IBT.++GLIBC_TUNABLES=glibc.tune.x86_shstk=<on, permissive>+ This controls how dlopen() handles SHSTK legacy libraries::++ on - continue with SHSTK enabled;+ permissive - continue with SHSTK off.++Details can be found in the GLIBC manual pages.++[4] CET arch_prctl()'s+======================++Several arch_prctl()'s have been added for CET:++arch_prctl(ARCH_X86_CET_STATUS, u64 *addr)+ Return CET feature status.++ The parameter 'addr' is a pointer to a user buffer.+ On returning to the caller, the kernel fills the following+ information::++ *addr = shadow stack/indirect branch tracking status+ *(addr + 1) = shadow stack base address+ *(addr + 2) = shadow stack size++arch_prctl(ARCH_X86_CET_DISABLE, unsigned int features)+ Disable shadow stack and/or indirect branch tracking as specified in+ 'features'. Return -EPERM if CET is locked.++arch_prctl(ARCH_X86_CET_LOCK)+ Lock in all CET features. They cannot be turned off afterwards.++Note:+ There is no CET-enabling arch_prctl function. By design, CET is enabled+ automatically if the binary and the system can support it.++[5] The implementation of the Shadow Stack+==========================================++Shadow Stack size+-----------------++A task's shadow stack is allocated from memory to a fixed size of+MIN(RLIMIT_STACK, 4 GB). In other words, the shadow stack is allocated to+the maximum size of the normal stack, but capped to 4 GB. However,+a compat-mode application's address space is smaller, each of its thread's+shadow stack size is MIN(1/4 RLIMIT_STACK, 4 GB).++Signal+------++The main program and its signal handlers use the same shadow stack.+Because the shadow stack stores only return addresses, a large shadow+stack covers the condition that both the program stack and the signal+alternate stack run out.++The kernel creates a restore token for the shadow stack restoring address+and verifies that token when restoring from the signal handler.++Fork+----++The shadow stack's vma has VM_SHSTK flag set; its PTEs are required to be+read-only and dirty. When a shadow stack PTE is not RO and dirty, a+shadow access triggers a page fault with the shadow stack access bit set+in the page fault error code.++When a task forks a child, its shadow stack PTEs are copied and both the+parent's and the child's shadow stack PTEs are cleared of the dirty bit.+Upon the next shadow stack access, the resulting shadow stack page fault+is handled by page copy/re-use.++When a pthread child is created, the kernel allocates a new shadow stack+for the new thread.
On Wed, Feb 03, 2021 at 02:55:23PM -0800, Yu-cheng Yu wrote:
Explain no_user_shstk/no_user_ibt kernel parameters, and introduce a new
document on Control-flow Enforcement Technology (CET).
Signed-off-by: Yu-cheng Yu <redacted>
On Wed, Feb 03, 2021 at 02:55:27PM -0800, Yu-cheng Yu wrote:
Control-flow Enforcement Technology (CET) introduces these MSRs:
MSR_IA32_U_CET (user-mode CET settings),
MSR_IA32_PL3_SSP (user-mode shadow stack pointer),
MSR_IA32_PL0_SSP (kernel-mode shadow stack pointer),
MSR_IA32_PL1_SSP (Privilege Level 1 shadow stack pointer),
MSR_IA32_PL2_SSP (Privilege Level 2 shadow stack pointer),
MSR_IA32_S_CET (kernel-mode CET settings),
MSR_IA32_INT_SSP_TAB (exception shadow stack table).
The two user-mode MSRs belong to XFEATURE_CET_USER. The first three of
kernel-mode MSRs belong to XFEATURE_CET_KERNEL. Both XSAVES states are
supervisor states. This means that there is no direct, unprivileged access
to these states, making it harder for an attacker to subvert CET.
For sigreturn and future ptrace() support, shadow stack address and MSR
reserved bits are checked before written to the supervisor states.
Signed-off-by: Yu-cheng Yu <redacted>
On Wed, Feb 03, 2021 at 02:55:26PM -0800, Yu-cheng Yu wrote:
Introduce a software-defined X86_FEATURE_CET, which indicates either Shadow
Stack or Indirect Branch Tracking (or both) is present. Also introduce
related cpu init/setup functions.
Signed-off-by: Yu-cheng Yu <redacted>
On Wed, Feb 03, 2021 at 02:55:23PM -0800, Yu-cheng Yu wrote:
quoted
Explain no_user_shstk/no_user_ibt kernel parameters, and introduce a new
document on Control-flow Enforcement Technology (CET).
Signed-off-by: Yu-cheng Yu <redacted>
On Wed, Feb 03, 2021 at 02:55:24PM -0800, Yu-cheng Yu wrote:
quoted hunk
Shadow Stack provides protection against function return address
corruption. It is active when the processor supports it, the kernel has
CONFIG_X86_CET enabled, and the application is built for the feature.
This is only implemented for the 64-bit kernel. When it is enabled, legacy
non-Shadow Stack applications continue to work, but without protection.
Signed-off-by: Yu-cheng Yu <redacted>
---
arch/x86/Kconfig | 22 ++++++++++++++++++++++
arch/x86/Kconfig.assembler | 5 +++++
2 files changed, 27 insertions(+)
This seems backwards to me? Shouldn't 'config X86_64' do the 'select
ARCH_HAS_SHADOW_STACK' and 'config X86_CET' do a 'depends on
ARCH_HAS_SHADOW_STACK' instead?
quoted hunk
+ help
+ Control-flow protection is a set of hardware features which place
+ additional restrictions on indirect branches. These help
+ mitigate ROP attacks. Applications must be enabled to use it,
+ and old userspace does not get protection "for free".
+ Support for this feature is present on Tiger Lake family of
+ processors released in 2020 or later. Enabling this feature
+ increases kernel text size by 3.7 KB.
+ See Documentation/x86/intel_cet.rst for more information.
+
+ If unsure, say N.
+
config EFI
bool "EFI runtime service support"
depends on ACPI
On Wed, Feb 03, 2021 at 02:55:29PM -0800, Yu-cheng Yu wrote:
The x86 family of processors do not directly create read-only and Dirty
PTEs. These PTEs are created by software. One such case is that kernel
read-only pages are historically setup as Dirty.
New processors that support Shadow Stack regard read-only and Dirty PTEs as
shadow stack pages. This results in ambiguity between shadow stack and
kernel read-only pages. To resolve this, removed Dirty from kernel read-
only pages.
Does this have any other side-effects?
Otherwise, sure:
Reviewed-by: Kees Cook <redacted>
-Kees
quoted hunk
Signed-off-by: Yu-cheng Yu <redacted>
Cc: "H. Peter Anvin" <hpa@zytor.com>
Cc: Kees Cook <redacted>
Cc: Thomas Gleixner <redacted>
Cc: Dave Hansen <dave.hansen@linux.intel.com>
Cc: Christoph Hellwig <hch@lst.de>
Cc: Andy Lutomirski <luto@kernel.org>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: Borislav Petkov <bp@alien8.de>
Cc: Peter Zijlstra <peterz@infradead.org>
---
arch/x86/include/asm/pgtable_types.h | 6 +++---
arch/x86/mm/pat/set_memory.c | 2 +-
2 files changed, 4 insertions(+), 4 deletions(-)
@@ -1932,7 +1932,7 @@ int set_memory_nx(unsigned long addr, int numpages)intset_memory_ro(unsignedlongaddr,intnumpages){-returnchange_page_attr_clear(&addr,numpages,__pgprot(_PAGE_RW),0);+returnchange_page_attr_clear(&addr,numpages,__pgprot(_PAGE_RW|_PAGE_DIRTY),0);}intset_memory_rw(unsignedlongaddr,intnumpages)
On Wed, Feb 03, 2021 at 02:55:30PM -0800, Yu-cheng Yu wrote:
There is essentially no room left in the x86 hardware PTEs on some OSes
(not Linux). That left the hardware architects looking for a way to
represent a new memory type (shadow stack) within the existing bits.
They chose to repurpose a lightly-used state: Write=0, Dirty=1.
The reason it's lightly used is that Dirty=1 is normally set by hardware
and cannot normally be set by hardware on a Write=0 PTE. Software must
normally be involved to create one of these PTEs, so software can simply
opt to not create them.
In places where Linux normally creates Write=0, Dirty=1, it can use the
software-defined _PAGE_COW in place of the hardware _PAGE_DIRTY. In other
words, whenever Linux needs to create Write=0, Dirty=1, it instead creates
Write=0, Cow=1, except for shadow stack, which is Write=0, Dirty=1. This
clearly separates shadow stack from other data, and results in the
following:
(a) A modified, copy-on-write (COW) page: (Write=0, Cow=1)
(b) A R/O page that has been COW'ed: (Write=0, Cow=1)
The user page is in a R/O VMA, and get_user_pages() needs a writable
copy. The page fault handler creates a copy of the page and sets
the new copy's PTE as Write=0 and Cow=1.
(c) A shadow stack PTE: (Write=0, Dirty=1)
(d) A shared shadow stack PTE: (Write=0, Cow=1)
When a shadow stack page is being shared among processes (this happens
at fork()), its PTE is made Dirty=0, so the next shadow stack access
causes a fault, and the page is duplicated and Dirty=1 is set again.
This is the COW equivalent for shadow stack pages, even though it's
copy-on-access rather than copy-on-write.
(e) A page where the processor observed a Write=1 PTE, started a write, set
Dirty=1, but then observed a Write=0 PTE. That's possible today, but
will not happen on processors that support shadow stack.
What happens for "e" with/without CET? It sounds like direct writes to
such pages will be (correctly) rejected by the MMU?
Define _PAGE_COW and update pte_*() helpers and apply the same changes to
pmd and pud.
After this, there are six free bits left in the 64-bit PTE, and no more
free bits in the 32-bit PTE (except for PAE) and Shadow Stack is not
implemented for the 32-bit kernel.
Are there selftests to validate this change?
I think it might be useful to more clearly describe what is considered
"dirty" and "writeable" in comments above the pte_helpers.
-Kees
@@ -23,7 +23,8 @@#define _PAGE_BIT_SOFTW2 10 /* " */#define _PAGE_BIT_SOFTW3 11 /* " */#define _PAGE_BIT_PAT_LARGE 12 /* On 2MB or 1GB pages */-#define _PAGE_BIT_SOFTW4 58 /* available for programmer */+#define _PAGE_BIT_SOFTW4 57 /* available for programmer */+#define _PAGE_BIT_SOFTW5 58 /* available for programmer */#define _PAGE_BIT_PKEY_BIT0 59 /* Protection Keys, bit 1/4 */#define _PAGE_BIT_PKEY_BIT1 60 /* Protection Keys, bit 2/4 */#define _PAGE_BIT_PKEY_BIT2 61 /* Protection Keys, bit 3/4 */
@@ -36,6 +37,15 @@#define _PAGE_BIT_SOFT_DIRTY _PAGE_BIT_SOFTW3 /* software dirty tracking */#define _PAGE_BIT_DEVMAP _PAGE_BIT_SOFTW4+/*+*Indicatesacopy-on-writepage.+*/+#ifdef CONFIG_X86_CET+#define _PAGE_BIT_COW _PAGE_BIT_SOFTW5 /* copy-on-write */+#else+#define _PAGE_BIT_COW 0+#endif+/* If _PAGE_BIT_PRESENT is clear, we use these: *//* - if the user mapped it with PROT_NONE; pte_present gives true */#define _PAGE_BIT_PROTNONE _PAGE_BIT_GLOBAL
On Wed, Feb 03, 2021 at 02:55:32PM -0800, Yu-cheng Yu wrote:
The read-only and Dirty PTE has been used to indicate copy-on-write pages.
However, newer x86 processors also regard a read-only and Dirty PTE as a
shadow stack page. In order to separate the two, the software-defined
_PAGE_COW is created to replace _PAGE_DIRTY for the copy-on-write case, and
pte_*() are updated.
Pte_modify() changes a PTE to 'newprot', but it doesn't use the pte_*().
Introduce fixup_dirty_pte(), which sets a dirty PTE, based on _PAGE_RW,
to either _PAGE_DIRTY or _PAGE_COW.
Apply the same changes to pmd_modify().
Signed-off-by: Yu-cheng Yu <redacted>
On Wed, Feb 03, 2021 at 02:55:33PM -0800, Yu-cheng Yu wrote:
When Shadow Stack is introduced, [R/O + _PAGE_DIRTY] PTE is reserved for
shadow stack. Copy-on-write PTEs have [R/O + _PAGE_COW].
When a PTE goes from [R/W + _PAGE_DIRTY] to [R/O + _PAGE_COW], it could
become a transient shadow stack PTE in two cases:
The first case is that some processors can start a write but end up seeing
a read-only PTE by the time they get to the Dirty bit, creating a transient
shadow stack PTE. However, this will not occur on processors supporting
Shadow Stack, and a TLB flush is not necessary.
The second case is that when _PAGE_DIRTY is replaced with _PAGE_COW non-
atomically, a transient shadow stack PTE can be created as a result.
Thus, prevent that with cmpxchg.
Dave Hansen, Jann Horn, Andy Lutomirski, and Peter Zijlstra provided many
insights to the issue. Jann Horn provided the cmpxchg solution.
Signed-off-by: Yu-cheng Yu <redacted>
On Wed, Feb 03, 2021 at 02:55:36PM -0800, Yu-cheng Yu wrote:
When serving a page fault, maybe_mkwrite() makes a PTE writable if its vma
has VM_WRITE.
A shadow stack vma has VM_SHSTK. Its PTEs have _PAGE_DIRTY, but not
_PAGE_WRITE. In fork(), _PAGE_DIRTY is cleared to effect copy-on-write,
and in page fault, _PAGE_DIRTY is restored and the shadow stack page is
writable again.
Update maybe_mkwrite() by introducing arch_maybe_mkwrite(), which sets
_PAGE_DIRTY for a shadow stack PTE.
Apply the same changes to maybe_pmd_mkwrite().
Signed-off-by: Yu-cheng Yu <redacted>
On Wed, Feb 03, 2021 at 02:55:28PM -0800, Yu-cheng Yu wrote:
quoted hunk
A control-protection fault is triggered when a control-flow transfer
attempt violates Shadow Stack or Indirect Branch Tracking constraints.
For example, the return address for a RET instruction differs from the copy
on the shadow stack; or an indirect JMP instruction, without the NOTRACK
prefix, arrives at a non-ENDBR opcode.
The control-protection fault handler works in a similar way as the general
protection fault handler. It provides the si_code SEGV_CPERR to the signal
handler.
Signed-off-by: Yu-cheng Yu <redacted>
Cc: Michael Kerrisk <redacted>
---
arch/x86/include/asm/idtentry.h | 4 ++
arch/x86/kernel/idt.c | 4 ++
arch/x86/kernel/signal_compat.c | 2 +-
arch/x86/kernel/traps.c | 60 ++++++++++++++++++++++++++++++
include/uapi/asm-generic/siginfo.h | 3 +-
5 files changed, 71 insertions(+), 2 deletions(-)
@@ -574,6 +574,10 @@ DECLARE_IDTENTRY_ERRORCODE(X86_TRAP_SS, exc_stack_segment);DECLARE_IDTENTRY_ERRORCODE(X86_TRAP_GP,exc_general_protection);DECLARE_IDTENTRY_ERRORCODE(X86_TRAP_AC,exc_alignment_check);+#ifdef CONFIG_X86_CET+DECLARE_IDTENTRY_ERRORCODE(X86_TRAP_CP,exc_control_protection);+#endif+/* Raw exception entries which need extra work */DECLARE_IDTENTRY_RAW(X86_TRAP_UD,exc_invalid_op);DECLARE_IDTENTRY_RAW(X86_TRAP_BP,exc_int3);
@@ -606,6 +606,66 @@ DEFINE_IDTENTRY_ERRORCODE(exc_general_protection)cond_local_irq_disable(regs);}+#ifdef CONFIG_X86_CET+staticconstchar*constcontrol_protection_err[]={+"unknown",+"near-ret",+"far-ret/iret",+"endbranch",+"rstorssp",+"setssbsy",+};++/*+*Whenacontrolprotectionexceptionoccurs,sendasignaltotheresponsible+*application.Currently,controlprotectionisonlyenabledforusermode.+*Thisexceptionshouldnotcomefromkernelmode.+*/+DEFINE_IDTENTRY_ERRORCODE(exc_control_protection)+{+staticDEFINE_RATELIMIT_STATE(rs,DEFAULT_RATELIMIT_INTERVAL,+DEFAULT_RATELIMIT_BURST);+structtask_struct*tsk;++if(!user_mode(regs)){+pr_emerg("PANIC: unexpected kernel control protection fault\n");+die("kernel control protection fault",regs,error_code);+panic("Machine halted.");+}++cond_local_irq_enable(regs);++if(!boot_cpu_has(X86_FEATURE_CET))+WARN_ONCE(1,"Control protection fault with CET support disabled\n");++tsk=current;+tsk->thread.error_code=error_code;+tsk->thread.trap_nr=X86_TRAP_CP;++if(show_unhandled_signals&&unhandled_signal(tsk,SIGSEGV)&&+__ratelimit(&rs)){+unsignedintmax_err;+unsignedlongssp;++max_err=ARRAY_SIZE(control_protection_err)-1;+if(error_code<0||error_code>max_err)+error_code=0;
Do you want to mask the error_code here before printing its value?
Instead, you could clamp error_code to ARRAY_SIZE(control_protection_err),
and add another "unknown" to the end of the strings:
control_protection_err[
array_index_nospec(error_code,
ARRAY_SIZE(control_protection_err))]
Everything else looks good.
On Wed, Feb 03, 2021 at 02:55:37PM -0800, Yu-cheng Yu wrote:
When serving a page fault, maybe_mkwrite() makes a PTE writable if it is in
a writable vma. A shadow stack vma is writable, but its PTEs need
_PAGE_DIRTY to be set to become writable. For this reason, maybe_mkwrite()
has been updated.
There are a few places that call pte_mkwrite() directly, but effect the
same result as from maybe_mkwrite(). These sites need to be updated for
shadow stack as well. Thus, change them to maybe_mkwrite():
- do_anonymous_page() and migrate_vma_insert_page() check VM_WRITE directly
and call pte_mkwrite(), which is the same as maybe_mkwrite(). Change
them to maybe_mkwrite().
- In do_numa_page(), if the numa entry 'was-writable', then pte_mkwrite()
is called directly. Fix it by doing maybe_mkwrite().
- In change_pte_range(), pte_mkwrite() is called directly. Replace it with
maybe_mkwrite().
A shadow stack vma is writable but has different vma
flags, and handled accordingly in maybe_mkwrite().
Signed-off-by: Yu-cheng Yu <redacted>
On Wed, Feb 03, 2021 at 02:55:38PM -0800, Yu-cheng Yu wrote:
INCSSP(Q/D) increments shadow stack pointer and 'pops and discards' the
first and the last elements in the range, effectively touches those memory
areas.
The maximum moving distance by INCSSPQ is 255 * 8 = 2040 bytes and
255 * 4 = 1020 bytes by INCSSPD. Both ranges are far from PAGE_SIZE.
Thus, putting a gap page on both ends of a shadow stack prevents INCSSP,
CALL, and RET from going beyond.
Signed-off-by: Yu-cheng Yu <redacted>
On Wed, Feb 03, 2021 at 02:55:40PM -0800, Yu-cheng Yu wrote:
Can_follow_write_pte() ensures a read-only page is COWed by checking the
FOLL_COW flag, and uses pte_dirty() to validate the flag is still valid.
Like a writable data page, a shadow stack page is writable, and becomes
read-only during copy-on-write, but it is always dirty. Thus, in the
can_follow_write_pte() check, it belongs to the writable page case and
should be excluded from the read-only page pte_dirty() check. Apply
the same changes to can_follow_write_pmd().
Does this need the vma passed down? Should it just pass vm_flags? I
suppose it doesn't really matter, though.
Reviewed-by: Kees Cook <redacted>
-Kees
From: Dave Hansen <hidden> Date: 2021-02-04 20:30:56
On 2/4/21 12:19 PM, Kees Cook wrote:
quoted
(e) A page where the processor observed a Write=1 PTE, started a write, set
Dirty=1, but then observed a Write=0 PTE. That's possible today, but
will not happen on processors that support shadow stack.
What happens for "e" with/without CET? It sounds like direct writes to
such pages will be (correctly) rejected by the MMU?
A page fault would be generated regardless of CET support.
If CET were not around, the fault would be reported as a present, write
fault.
If this happened and CET were around (which shouldn't happen in
practice, it means we have a hardware issue) a page fault exception is
generated. Yu-cheng, I'm not sure there's enough debugging around to
tell us if this happens. Would we even notice?
On Wed, Feb 03, 2021 at 02:55:41PM -0800, Yu-cheng Yu wrote:
There was no more caller passing vm_flags to do_mmap(), and vm_flags was
removed from the function's input by:
commit 45e55300f114 ("mm: remove unnecessary wrapper function do_mmap_pgoff()").
There is a new user now. Shadow stack allocation passes VM_SHSTK to
do_mmap(). Re-introduce vm_flags to do_mmap(), but without the old wrapper
do_mmap_pgoff(). Instead, make all callers of the wrapper pass a zero
vm_flags to do_mmap().
Signed-off-by: Yu-cheng Yu <redacted>
On Wed, Feb 03, 2021 at 02:55:42PM -0800, Yu-cheng Yu wrote:
Introduce basic shadow stack enabling/disabling/allocation routines.
A task's shadow stack is allocated from memory with VM_SHSTK flag and has
a fixed size of min(RLIMIT_STACK, 4GB).
Signed-off-by: Yu-cheng Yu <redacted>
On Wed, Feb 03, 2021 at 02:55:44PM -0800, Yu-cheng Yu wrote:
An ELF file's .note.gnu.property indicates arch features supported by the
file. These features are extracted by arch_parse_elf_property() and stored
in 'arch_elf_state'.
Introduce x86 feature definitions and arch_setup_elf_property(), which
enables such features. The first use-case of this function is Shadow
Stack.
ARM64 is the other arch that has ARCH_USE_GNU_PROPERTY and arch_parse_elf_
property(). Add arch_setup_elf_property() for it.
Signed-off-by: Yu-cheng Yu <redacted>
On Wed, Feb 03, 2021 at 02:55:46PM -0800, Yu-cheng Yu wrote:
arch_prctl(ARCH_X86_CET_STATUS, u64 *args)
Get CET feature status.
The parameter 'args' is a pointer to a user buffer. The kernel returns
the following information:
*args = shadow stack/IBT status
*(args + 1) = shadow stack base address
*(args + 2) = shadow stack size
What happens if this needs to grow in the future? Should the first u64
contain the array size?
Otherwise, looks sensible.
-Kees
quoted hunk
32-bit binaries use the same interface, but only lower 32-bits of each
item.
arch_prctl(ARCH_X86_CET_DISABLE, unsigned int features)
Disable CET features specified in 'features'. Return -EPERM if CET is
locked.
arch_prctl(ARCH_X86_CET_LOCK)
Lock in CET features.
Also change do_arch_prctl_common()'s parameter 'cpuid_enabled' to
'arg2', as it is now also passed to prctl_cet().
Signed-off-by: Yu-cheng Yu <redacted>
---
arch/x86/include/asm/cet.h | 3 ++
arch/x86/include/uapi/asm/prctl.h | 4 +++
arch/x86/kernel/Makefile | 2 +-
arch/x86/kernel/cet_prctl.c | 60 +++++++++++++++++++++++++++++++
arch/x86/kernel/process.c | 6 ++--
5 files changed, 71 insertions(+), 4 deletions(-)
create mode 100644 arch/x86/kernel/cet_prctl.c
@@ -25,6 +27,7 @@ int cet_verify_rstor_token(bool ia32, unsigned long ssp, unsigned long *new_ssp)voidcet_restore_signal(structsc_ext*sc);intcet_setup_signal(boolia32,unsignedlongrstor,structsc_ext*sc);#else+staticinlineintprctl_cet(intoption,u64arg2){return-EINVAL;}staticinlineintcet_setup_thread_shstk(structtask_struct*p,unsignedlongclone_flags){return0;}staticinlinevoidcet_disable_shstk(void){}
IIRC we've these abbreviations explained in documentaion
(proc.rst file). Could you please update it once time
permit? I think it can be done on top of the series.
IIRC we've these abbreviations explained in documentaion
(proc.rst file). Could you please update it once time
permit? I think it can be done on top of the series.
On Wed, Feb 03, 2021 at 02:55:46PM -0800, Yu-cheng Yu wrote:
quoted
arch_prctl(ARCH_X86_CET_STATUS, u64 *args)
Get CET feature status.
The parameter 'args' is a pointer to a user buffer. The kernel returns
the following information:
*args = shadow stack/IBT status
*(args + 1) = shadow stack base address
*(args + 2) = shadow stack size
What happens if this needs to grow in the future? Should the first u64
contain the array size?
Otherwise, looks sensible.
-Kees
The first item is a bitmap, and there are two possible bits. Should
there be a need, we can then do things about it. My thought at the
moment is, we may not meet the situation. Can we keep this for now?
--
Yu-cheng
On Wed, Feb 03, 2021 at 02:55:24PM -0800, Yu-cheng Yu wrote:
quoted
Shadow Stack provides protection against function return address
corruption. It is active when the processor supports it, the kernel has
CONFIG_X86_CET enabled, and the application is built for the feature.
This is only implemented for the 64-bit kernel. When it is enabled, legacy
non-Shadow Stack applications continue to work, but without protection.
Signed-off-by: Yu-cheng Yu <redacted>
---
arch/x86/Kconfig | 22 ++++++++++++++++++++++
arch/x86/Kconfig.assembler | 5 +++++
2 files changed, 27 insertions(+)
This seems backwards to me? Shouldn't 'config X86_64' do the 'select
ARCH_HAS_SHADOW_STACK' and 'config X86_CET' do a 'depends on
ARCH_HAS_SHADOW_STACK' instead?
@@ -606,6 +606,66 @@ DEFINE_IDTENTRY_ERRORCODE(exc_general_protection)cond_local_irq_disable(regs);}+#ifdef CONFIG_X86_CET+staticconstchar*constcontrol_protection_err[]={+"unknown",+"near-ret",+"far-ret/iret",+"endbranch",+"rstorssp",+"setssbsy",+};++/*+*Whenacontrolprotectionexceptionoccurs,sendasignaltotheresponsible+*application.Currently,controlprotectionisonlyenabledforusermode.+*Thisexceptionshouldnotcomefromkernelmode.+*/+DEFINE_IDTENTRY_ERRORCODE(exc_control_protection)+{+staticDEFINE_RATELIMIT_STATE(rs,DEFAULT_RATELIMIT_INTERVAL,+DEFAULT_RATELIMIT_BURST);+structtask_struct*tsk;++if(!user_mode(regs)){+pr_emerg("PANIC: unexpected kernel control protection fault\n");+die("kernel control protection fault",regs,error_code);+panic("Machine halted.");+}++cond_local_irq_enable(regs);++if(!boot_cpu_has(X86_FEATURE_CET))+WARN_ONCE(1,"Control protection fault with CET support disabled\n");++tsk=current;+tsk->thread.error_code=error_code;+tsk->thread.trap_nr=X86_TRAP_CP;++if(show_unhandled_signals&&unhandled_signal(tsk,SIGSEGV)&&+__ratelimit(&rs)){+unsignedintmax_err;+unsignedlongssp;++max_err=ARRAY_SIZE(control_protection_err)-1;+if(error_code<0||error_code>max_err)+error_code=0;
Do you want to mask the error_code here before printing its value?
Instead, you could clamp error_code to ARRAY_SIZE(control_protection_err),
and add another "unknown" to the end of the strings:
control_protection_err[
array_index_nospec(error_code,
ARRAY_SIZE(control_protection_err))]
Everything else looks good.
On Wed, Feb 03, 2021 at 02:55:28PM -0800, Yu-cheng Yu wrote:
quoted
+DEFINE_IDTENTRY_ERRORCODE(exc_control_protection)
+{
+ static DEFINE_RATELIMIT_STATE(rs, DEFAULT_RATELIMIT_INTERVAL,
+ DEFAULT_RATELIMIT_BURST);
+ struct task_struct *tsk;
+
+ if (!user_mode(regs)) {
+ pr_emerg("PANIC: unexpected kernel control protection fault\n");
+ die("kernel control protection fault", regs, error_code);
+ panic("Machine halted.");
+ }
+
+ cond_local_irq_enable(regs);
+
+ if (!boot_cpu_has(X86_FEATURE_CET))
+ WARN_ONCE(1, "Control protection fault with CET support disabled\n");
+
+ tsk = current;
+ tsk->thread.error_code = error_code;
+ tsk->thread.trap_nr = X86_TRAP_CP;
+
+ if (show_unhandled_signals && unhandled_signal(tsk, SIGSEGV) &&
+ __ratelimit(&rs)) {
I can't find it written down anywhere why the ratelimiting is needed at
all?
The ratelimit here is only for #CP, and its rate is not counted together
with other types of faults. If a task gets here, it will exit. The
only condition the ratelimit will trigger is when multiple tasks hit #CP
at once, which is unlikely. Are you suggesting that we do not need the
ratelimit here?
Thanks!
--
Yu-cheng
On Thu, Feb 04, 2021 at 03:41:59PM -0800, Yu, Yu-cheng wrote:
On 2/4/2021 12:35 PM, Kees Cook wrote:
quoted
On Wed, Feb 03, 2021 at 02:55:46PM -0800, Yu-cheng Yu wrote:
quoted
arch_prctl(ARCH_X86_CET_STATUS, u64 *args)
Get CET feature status.
The parameter 'args' is a pointer to a user buffer. The kernel returns
the following information:
*args = shadow stack/IBT status
*(args + 1) = shadow stack base address
*(args + 2) = shadow stack size
What happens if this needs to grow in the future? Should the first u64
contain the array size?
Otherwise, looks sensible.
-Kees
The first item is a bitmap, and there are two possible bits. Should there
be a need, we can then do things about it. My thought at the moment is, we
may not meet the situation. Can we keep this for now?
Ah, good point. Yes, since that's a bitmap it ends up describing what
follows. This is fine as-is. Thanks!
Reviewed-by: Kees Cook <redacted>
--
Kees Cook
On Fri, Feb 05, 2021 at 10:00:21AM -0800, Yu, Yu-cheng wrote:
On 2/5/2021 5:59 AM, Borislav Petkov wrote:
quoted
On Wed, Feb 03, 2021 at 02:55:28PM -0800, Yu-cheng Yu wrote:
quoted
+DEFINE_IDTENTRY_ERRORCODE(exc_control_protection)
+{
+ static DEFINE_RATELIMIT_STATE(rs, DEFAULT_RATELIMIT_INTERVAL,
+ DEFAULT_RATELIMIT_BURST);
+ struct task_struct *tsk;
+
+ if (!user_mode(regs)) {
+ pr_emerg("PANIC: unexpected kernel control protection fault\n");
+ die("kernel control protection fault", regs, error_code);
+ panic("Machine halted.");
+ }
+
+ cond_local_irq_enable(regs);
+
+ if (!boot_cpu_has(X86_FEATURE_CET))
+ WARN_ONCE(1, "Control protection fault with CET support disabled\n");
+
+ tsk = current;
+ tsk->thread.error_code = error_code;
+ tsk->thread.trap_nr = X86_TRAP_CP;
+
+ if (show_unhandled_signals && unhandled_signal(tsk, SIGSEGV) &&
+ __ratelimit(&rs)) {
I can't find it written down anywhere why the ratelimiting is needed at
all?
The ratelimit here is only for #CP, and its rate is not counted together
with other types of faults. If a task gets here, it will exit. The only
condition the ratelimit will trigger is when multiple tasks hit #CP at once,
which is unlikely. Are you suggesting that we do not need the ratelimit
here?
Since this is a potentially unprivileged-userspace-triggerable
condition, I tend to prefer having a ratelimit. I don't feel _strongly_
about it, but I find it better to be defensive against log spamming
(whether malicious or accidental).
--
Kees Cook
(e) A page where the processor observed a Write=1 PTE, started a write, set
Dirty=1, but then observed a Write=0 PTE. That's possible today, but
will not happen on processors that support shadow stack.
What happens for "e" with/without CET? It sounds like direct writes to
such pages will be (correctly) rejected by the MMU?
A page fault would be generated regardless of CET support.
If CET were not around, the fault would be reported as a present, write
fault.
If this happened and CET were around (which shouldn't happen in
practice, it means we have a hardware issue) a page fault exception is
generated.
Thanks for the clarification. With or without CET, direct write to
Write=0, Dirty=1 PTE triggers page fault.
Yu-cheng, I'm not sure there's enough debugging around to
tell us if this happens. Would we even notice?
That potential hardware issue is, on a CET-capable system, a processor
writes to a Write=1, Dirty=0 page, and then observes the PTE is Write=0,
Dirty=1. Let me think about it...
Thanks!
--
Yu-cheng
On Wed, Feb 03, 2021 at 02:55:30PM -0800, Yu-cheng Yu wrote:
quoted
There is essentially no room left in the x86 hardware PTEs on some OSes
(not Linux). That left the hardware architects looking for a way to
represent a new memory type (shadow stack) within the existing bits.
They chose to repurpose a lightly-used state: Write=0, Dirty=1.
The reason it's lightly used is that Dirty=1 is normally set by hardware
and cannot normally be set by hardware on a Write=0 PTE. Software must
normally be involved to create one of these PTEs, so software can simply
opt to not create them.
In places where Linux normally creates Write=0, Dirty=1, it can use the
software-defined _PAGE_COW in place of the hardware _PAGE_DIRTY. In other
words, whenever Linux needs to create Write=0, Dirty=1, it instead creates
Write=0, Cow=1, except for shadow stack, which is Write=0, Dirty=1. This
clearly separates shadow stack from other data, and results in the
following:
(a) A modified, copy-on-write (COW) page: (Write=0, Cow=1)
(b) A R/O page that has been COW'ed: (Write=0, Cow=1)
The user page is in a R/O VMA, and get_user_pages() needs a writable
copy. The page fault handler creates a copy of the page and sets
the new copy's PTE as Write=0 and Cow=1.
(c) A shadow stack PTE: (Write=0, Dirty=1)
(d) A shared shadow stack PTE: (Write=0, Cow=1)
When a shadow stack page is being shared among processes (this happens
at fork()), its PTE is made Dirty=0, so the next shadow stack access
causes a fault, and the page is duplicated and Dirty=1 is set again.
This is the COW equivalent for shadow stack pages, even though it's
copy-on-access rather than copy-on-write.
(e) A page where the processor observed a Write=1 PTE, started a write, set
Dirty=1, but then observed a Write=0 PTE. That's possible today, but
will not happen on processors that support shadow stack.
What happens for "e" with/without CET? It sounds like direct writes to
such pages will be (correctly) rejected by the MMU?
quoted
Define _PAGE_COW and update pte_*() helpers and apply the same changes to
pmd and pud.
After this, there are six free bits left in the 64-bit PTE, and no more
free bits in the 32-bit PTE (except for PAE) and Shadow Stack is not
implemented for the 32-bit kernel.
Are there selftests to validate this change?
I have some tests to verify, for example,
- After clone(), shadow stack pages are indeed copy-on-write,
- Shadow stack pages (i.e. Write=0, Dirty=1) cannot be directly written to,
- Shadow stack guard pages exist.
These tests are now on github, but kind of messy. I can gradually clean
up them and submit as selftests separately.
If you are asking for the detection of the potential hardware issue
(that Dave Hansen talked about), then maybe we need to detect it from
the kernel.
I think it might be useful to more clearly describe what is considered
"dirty" and "writeable" in comments above the pte_helpers.
On Fri, Feb 05, 2021 at 10:00:21AM -0800, Yu, Yu-cheng wrote:
The ratelimit here is only for #CP, and its rate is not counted together
with other types of faults. If a task gets here, it will exit. The only
condition the ratelimit will trigger is when multiple tasks hit #CP at once,
which is unlikely. Are you suggesting that we do not need the ratelimit
here?
I'm trying to first find out why is it there.
Is this something you've hit during testing and thought, oh well, this
needs a ratelimit or was it added just because?
--
Regards/Gruss,
Boris.
https://people.kernel.org/tglx/notes-about-netiquette
On Fri, Feb 05, 2021 at 10:00:21AM -0800, Yu, Yu-cheng wrote:
quoted
The ratelimit here is only for #CP, and its rate is not counted together
with other types of faults. If a task gets here, it will exit. The only
condition the ratelimit will trigger is when multiple tasks hit #CP at once,
which is unlikely. Are you suggesting that we do not need the ratelimit
here?
I'm trying to first find out why is it there.
Is this something you've hit during testing and thought, oh well, this
needs a ratelimit or was it added just because?
I have not run into the situation. Initially it was there because other
faults have it. When you asked, I went through it and put out my
reasoning. I think it still makes sense to keep it.
--
Yu-cheng
On Mon, Feb 08, 2021 at 10:50:07AM -0800, Yu, Yu-cheng wrote:
quoted
I have not run into the situation. Initially it was there because other
faults have it.
Which other faults?
exc_general_protection() and do_trap() both call show_signal(), which
then calls printk_ratelimit().
quoted
When you asked, I went through it and put out my reasoning.
What does that mean?
I went through my patch and check if ratelimit is necessary, and then
describe the finding.
quoted
I think it still makes sense to keep it.
Because you have a hunch or you actually have an objective reason why?
For example, if a shell script, in a loop re-starts an app when it
exits, and the app is causing control-protection fault. The log
messages should be rate limited.
On Mon, Feb 08, 2021 at 11:23:18AM -0800, Yu, Yu-cheng wrote:
exc_general_protection() and do_trap() both call show_signal(), which
then calls printk_ratelimit().
You could've done some git archeology and could've found
abd4f7505baf ("x86: i386-show-unhandled-signals-v3")
which explains why that ratelimiting is needed.
For example, if a shell script, in a loop re-starts an app when it
exits, and the app is causing control-protection fault. The log
messages should be rate limited.
I think you should be able to get where I'm going with this, by now:
please put a comment over the ratelimiting to explain why it is there,
just like the above commit explains.
Thx.
--
Regards/Gruss,
Boris.
https://people.kernel.org/tglx/notes-about-netiquette
On Mon, Feb 08, 2021 at 11:23:18AM -0800, Yu, Yu-cheng wrote:
quoted
exc_general_protection() and do_trap() both call show_signal(), which
then calls printk_ratelimit().
You could've done some git archeology and could've found
abd4f7505baf ("x86: i386-show-unhandled-signals-v3")
which explains why that ratelimiting is needed.
quoted
For example, if a shell script, in a loop re-starts an app when it
exits, and the app is causing control-protection fault. The log
messages should be rate limited.
I think you should be able to get where I'm going with this, by now:
please put a comment over the ratelimiting to explain why it is there,
just like the above commit explains.
(e) A page where the processor observed a Write=1 PTE, started a
write, set
Dirty=1, but then observed a Write=0 PTE. That's possible
today, but
will not happen on processors that support shadow stack.
What happens for "e" with/without CET? It sounds like direct writes to
such pages will be (correctly) rejected by the MMU?
A page fault would be generated regardless of CET support.
If CET were not around, the fault would be reported as a present, write
fault.
If this happened and CET were around (which shouldn't happen in
practice, it means we have a hardware issue) a page fault exception is
generated.
Thanks for the clarification. With or without CET, direct write to
Write=0, Dirty=1 PTE triggers page fault.
quoted
Yu-cheng, I'm not sure there's enough debugging around to
tell us if this happens. Would we even notice?
That potential hardware issue is, on a CET-capable system, a processor
writes to a Write=1, Dirty=0 page, and then observes the PTE is Write=0,
Dirty=1. Let me think about it...
One way to detect the potential issue is adding a check when a
non-shadow stack page's PTE goes from RW=0 to RW=1, like the following...
I run all my routine stress tests with the changes, and do not see any
warning triggered. If this change is desirable, we can probably add
#ifdef CONFIG_DEBUG_VM around it and make it a separate patch.
--
Yu-cheng