On Mon, 2020-09-14 at 11:31 -0700, Andy Lutomirski wrote:

quoted

On Sep 14, 2020, at 7:50 AM, Dave Hansen [off-list ref]
wrote:

On 9/11/20 3:59 PM, Yu-cheng Yu wrote:
...

quoted

Here are the changes if we take the mprotect(PROT_SHSTK)
approach.
Any comments/suggestions?

I still don't like it. :)

I'll also be much happier when there's a proper changelog to
accompany
this which also spells out the alternatives any why they suck so
much.

Let’s take a step back here. Ignoring the precise API, what exactly
is
a shadow stack from the perspective of a Linux user program?

The simplest answer is that it’s just memory that happens to have
certain protections.  This enables all kinds of shenanigans.  A
program could map a memfd twice, once as shadow stack and once as
non-shadow-stack, and change its control flow.  Similarly, a program
could mprotect its shadow stack, modify it, and mprotect it back.  In
some threat models, though could be seen as a WRSS bypass.  (Although
if an attacker can coerce a process to call mprotect(), the game is
likely mostly over anyway.)

But we could be more restrictive, or perhaps we could allow user code
to opt into more restrictions.  For example, we could have shadow
stacks be special memory that cannot be written from usermode by any
means other than ptrace() and friends, WRSS, and actual shadow stack
usage.

What is the goal?

No matter what we do, the effects of calling vfork() are going to be
a
bit odd with SHSTK enabled.  I suppose we could disallow this, but
that seems likely to cause its own issues.

Hi,

Resurrecting this old thread to highlight a consequence of the design
change that came out of it. I am going to be taking over this series
from Yu-cheng, and wanted to check if people would be interested in re-
visiting this interface.

The consequence I wanted to highlight, is that making userspace be
responsible for mapping memory as shadow stack, also requires moving
the writing of the restore token to userspace for glibc ucontext
operations. Since these operations involve creating/pivoting to new
stacks in userspace, ucontext cet support involves also creating a new
shadow stack. For normal thread stacks, the kernel has always done the
shadow stack allocation and so it is never writable (in the normal
sense) from userspace. But after this change makecontext() now first
has to mmap() writable memory, then write the restore token, then
mprotect() it as shadow stack. See the glibc changes to support
PROT_SHADOW_STACK here[0].

The writable window leaves an opening for an attacker to create an
arbitrary shadow stack that could be pivoted to later by tweaking the
ucontext_t structure. To try to see how much this matters, we have done
a small test that uses this window to ROP from writes in another
thread during the makecontext()/setcontext() window. (offensive work
credit to Joao on CC). This would require a real app to already to be
using ucontext in the course of normal runtime.

The original prctl solution prevents this case since the kernel did the
allocation and restore token setup, but of course it had other issues.
The other ideas discussed previously were a new syscall, or some sort
of new madvise() operation that could be involved in setting up shadow
stack, such that it is never writable in userspace. Or, simpler but
uglier, tweaking the existing PROT_SHADOW_STACK based solution by
making core mmap code write a restore token.

Those are still probably not enough to stop all ROP given extreme
threat models, as Andy alluded. But to me, this one seemed maybe worth
trying to improve. Especially thinking to avoid future ABI changes if
it becomes a problem. Would anyone like to see attempts to tighten this
up by revisiting this shadow stack allocation interface?

Thanks,

Rick

[0] 
https://gitlab.com/x86-glibc/glibc/-/commit/98eba0b829a1884c7d7bf76b0b39725919a9b6af#92c86763df6845c5b29664ac2ff40277678fd2c5_0_1