On Tue, Aug 25, 2020 at 04:34:27PM -0700, Yu, Yu-cheng wrote:
quoted
On 8/25/2020 4:20 PM, Dave Hansen wrote:
quoted
On 8/25/20 2:04 PM, Yu, Yu-cheng wrote:
quoted
quoted
quoted
I think this is more arch-specific. Even if it becomes a new syscall,
we still need to pass the same parameters.
Right, but without the copying in and out of memory.
Linux-api is already on the Cc list. Do we need to add more people to
get some agreements for the syscall?
What kind of agreement are you looking for? I'd suggest just coding it
up and posting the patches. Adding syscalls really is really pretty
straightforward and isn't much code at all.
Sure, I will do that.
Alternatively, would a regular prctl() work here?
Is this something appliation code has to call, or just the dynamic
loader?
prctl in glibc is a variadic function, so if there's a mismatch between
the kernel/userspace syscall convention and the userspace calling
convention (for variadic functions) for specific types, it can't be made
to work in a generic way.
The loader can use inline assembly for system calls and does not have
this issue, but applications would be implcated by it.
Thanks,
Florian
From: Andy Lutomirski <luto@kernel.org> Date: 2020-08-26 17:05:15
On Wed, Aug 26, 2020 at 9:52 AM Florian Weimer [off-list ref] wrote:
* Dave Martin:
quoted
On Tue, Aug 25, 2020 at 04:34:27PM -0700, Yu, Yu-cheng wrote:
quoted
On 8/25/2020 4:20 PM, Dave Hansen wrote:
quoted
On 8/25/20 2:04 PM, Yu, Yu-cheng wrote:
quoted
quoted
quoted
I think this is more arch-specific. Even if it becomes a new syscall,
we still need to pass the same parameters.
Right, but without the copying in and out of memory.
Linux-api is already on the Cc list. Do we need to add more people to
get some agreements for the syscall?
What kind of agreement are you looking for? I'd suggest just coding it
up and posting the patches. Adding syscalls really is really pretty
straightforward and isn't much code at all.
Sure, I will do that.
Alternatively, would a regular prctl() work here?
Is this something appliation code has to call, or just the dynamic
loader?
prctl in glibc is a variadic function, so if there's a mismatch between
the kernel/userspace syscall convention and the userspace calling
convention (for variadic functions) for specific types, it can't be made
to work in a generic way.
The loader can use inline assembly for system calls and does not have
this issue, but applications would be implcated by it.
I would expect things like Go and various JITs to call it directly.
If we wanted to be fancy and add a potentially more widely useful
syscall, how about:
mmap_special(void *addr, size_t length, int prot, int flags, int type);
Where type is something like MMAP_SPECIAL_X86_SHSTK. Fundamentally,
this is really just mmap() except that we want to map something a bit
magical, and we don't want to require opening a device node to do it.
--Andy
From: Dave Martin <Dave.Martin@arm.com> Date: 2020-08-26 17:08:53
On Wed, Aug 26, 2020 at 06:51:48PM +0200, Florian Weimer wrote:
* Dave Martin:
quoted
On Tue, Aug 25, 2020 at 04:34:27PM -0700, Yu, Yu-cheng wrote:
quoted
On 8/25/2020 4:20 PM, Dave Hansen wrote:
quoted
On 8/25/20 2:04 PM, Yu, Yu-cheng wrote:
quoted
quoted
quoted
I think this is more arch-specific. Even if it becomes a new syscall,
we still need to pass the same parameters.
Right, but without the copying in and out of memory.
Linux-api is already on the Cc list. Do we need to add more people to
get some agreements for the syscall?
What kind of agreement are you looking for? I'd suggest just coding it
up and posting the patches. Adding syscalls really is really pretty
straightforward and isn't much code at all.
Sure, I will do that.
Alternatively, would a regular prctl() work here?
Is this something appliation code has to call, or just the dynamic
loader?
prctl in glibc is a variadic function, so if there's a mismatch between
the kernel/userspace syscall convention and the userspace calling
convention (for variadic functions) for specific types, it can't be made
to work in a generic way.
The loader can use inline assembly for system calls and does not have
this issue, but applications would be implcated by it.
To the extent that this is a problem, libc's prctl() wrapper has to
handle it already. New prctl() calls tend to demand precisely 4
arguments and require unused arguments to be 0, but this is more down to
policy rather than because anything breaks otherwise.
You're right that this has implications: for i386, libc probably pulls
more arguments off the stack than are really there in some situations.
This isn't a new problem though. There are already generic prctls with
fewer than 4 args that are used on x86.
Merging the actual prctl() and arch_prctl() syscalls doesn't acutally
stop libc from retaining separate wrappers if they have different
argument marshaling requirements in some corner cases.
There might be some underlying reason by x86 has its own call and nobody
else followed the same model, but I don't know what it is.
Cheers
---Dave
On Wed, Aug 26, 2020 at 9:52 AM Florian Weimer [off-list ref] wrote:
quoted
* Dave Martin:
quoted
On Tue, Aug 25, 2020 at 04:34:27PM -0700, Yu, Yu-cheng wrote:
quoted
On 8/25/2020 4:20 PM, Dave Hansen wrote:
quoted
On 8/25/20 2:04 PM, Yu, Yu-cheng wrote:
quoted
quoted
quoted
I think this is more arch-specific. Even if it becomes a new syscall,
we still need to pass the same parameters.
Right, but without the copying in and out of memory.
Linux-api is already on the Cc list. Do we need to add more people to
get some agreements for the syscall?
What kind of agreement are you looking for? I'd suggest just coding it
up and posting the patches. Adding syscalls really is really pretty
straightforward and isn't much code at all.
Sure, I will do that.
Alternatively, would a regular prctl() work here?
Is this something appliation code has to call, or just the dynamic
loader?
prctl in glibc is a variadic function, so if there's a mismatch between
the kernel/userspace syscall convention and the userspace calling
convention (for variadic functions) for specific types, it can't be made
to work in a generic way.
The loader can use inline assembly for system calls and does not have
this issue, but applications would be implcated by it.
I would expect things like Go and various JITs to call it directly.
If we wanted to be fancy and add a potentially more widely useful
syscall, how about:
mmap_special(void *addr, size_t length, int prot, int flags, int type);
Where type is something like MMAP_SPECIAL_X86_SHSTK. Fundamentally,
this is really just mmap() except that we want to map something a bit
magical, and we don't want to require opening a device node to do it.
One benefit of MMAP_SPECIAL_* is there are more free bits than MAP_*.
Does ARM have similar needs for memory mapping, Dave?
Yu-cheng
On Wed, Aug 26, 2020 at 11:49 AM Yu, Yu-cheng [off-list ref] wrote:
On 8/26/2020 10:04 AM, Andy Lutomirski wrote:
quoted
On Wed, Aug 26, 2020 at 9:52 AM Florian Weimer [off-list ref] wrote:
quoted
* Dave Martin:
quoted
On Tue, Aug 25, 2020 at 04:34:27PM -0700, Yu, Yu-cheng wrote:
quoted
On 8/25/2020 4:20 PM, Dave Hansen wrote:
quoted
On 8/25/20 2:04 PM, Yu, Yu-cheng wrote:
quoted
quoted
quoted
I think this is more arch-specific. Even if it becomes a new syscall,
we still need to pass the same parameters.
Right, but without the copying in and out of memory.
Linux-api is already on the Cc list. Do we need to add more people to
get some agreements for the syscall?
What kind of agreement are you looking for? I'd suggest just coding it
up and posting the patches. Adding syscalls really is really pretty
straightforward and isn't much code at all.
Sure, I will do that.
Alternatively, would a regular prctl() work here?
Is this something appliation code has to call, or just the dynamic
loader?
prctl in glibc is a variadic function, so if there's a mismatch between
the kernel/userspace syscall convention and the userspace calling
convention (for variadic functions) for specific types, it can't be made
to work in a generic way.
The loader can use inline assembly for system calls and does not have
this issue, but applications would be implcated by it.
I would expect things like Go and various JITs to call it directly.
If we wanted to be fancy and add a potentially more widely useful
syscall, how about:
mmap_special(void *addr, size_t length, int prot, int flags, int type);
Where type is something like MMAP_SPECIAL_X86_SHSTK. Fundamentally,
this is really just mmap() except that we want to map something a bit
magical, and we don't want to require opening a device node to do it.
One benefit of MMAP_SPECIAL_* is there are more free bits than MAP_*.
Does ARM have similar needs for memory mapping, Dave?
From: Dave Hansen <hidden> Date: 2020-08-26 19:57:44
On 8/26/20 11:49 AM, Yu, Yu-cheng wrote:
quoted
I would expect things like Go and various JITs to call it directly.
If we wanted to be fancy and add a potentially more widely useful
syscall, how about:
mmap_special(void *addr, size_t length, int prot, int flags, int type);
Where type is something like MMAP_SPECIAL_X86_SHSTK. Fundamentally,
this is really just mmap() except that we want to map something a bit
magical, and we don't want to require opening a device node to do it.
One benefit of MMAP_SPECIAL_* is there are more free bits than MAP_*.
Does ARM have similar needs for memory mapping, Dave?
No idea.
But, mmap_special() is *basically* mmap2() with extra-big flags space.
I suspect it will grow some more uses on top of shadow stacks. It could
have, for instance, been used to allocate MPX bounds tables.
On Wed, Aug 26, 2020 at 12:57 PM Dave Hansen [off-list ref] wrote:
On 8/26/20 11:49 AM, Yu, Yu-cheng wrote:
quoted
quoted
I would expect things like Go and various JITs to call it directly.
If we wanted to be fancy and add a potentially more widely useful
syscall, how about:
mmap_special(void *addr, size_t length, int prot, int flags, int type);
Where type is something like MMAP_SPECIAL_X86_SHSTK. Fundamentally,
this is really just mmap() except that we want to map something a bit
magical, and we don't want to require opening a device node to do it.
One benefit of MMAP_SPECIAL_* is there are more free bits than MAP_*.
Does ARM have similar needs for memory mapping, Dave?
No idea.
But, mmap_special() is *basically* mmap2() with extra-big flags space.
I suspect it will grow some more uses on top of shadow stacks. It could
have, for instance, been used to allocate MPX bounds tables.
There is no reason we can't use
long arch_prctl (int, unsigned long, unsigned long, unsigned long, ..);
for ARCH_X86_CET_MMAP_SHSTK. We just need to use
syscall (SYS_arch_prctl, ARCH_X86_CET_MMAP_SHSTK, ...);
--
H.J.
From: Dave Martin <Dave.Martin@arm.com> Date: 2020-09-01 10:28:17
On Thu, Aug 27, 2020 at 06:26:11AM -0700, H.J. Lu wrote:
On Wed, Aug 26, 2020 at 12:57 PM Dave Hansen [off-list ref] wrote:
quoted
On 8/26/20 11:49 AM, Yu, Yu-cheng wrote:
quoted
quoted
I would expect things like Go and various JITs to call it directly.
If we wanted to be fancy and add a potentially more widely useful
syscall, how about:
mmap_special(void *addr, size_t length, int prot, int flags, int type);
Where type is something like MMAP_SPECIAL_X86_SHSTK. Fundamentally,
this is really just mmap() except that we want to map something a bit
magical, and we don't want to require opening a device node to do it.
One benefit of MMAP_SPECIAL_* is there are more free bits than MAP_*.
Does ARM have similar needs for memory mapping, Dave?
No idea.
But, mmap_special() is *basically* mmap2() with extra-big flags space.
I suspect it will grow some more uses on top of shadow stacks. It could
have, for instance, been used to allocate MPX bounds tables.
There is no reason we can't use
long arch_prctl (int, unsigned long, unsigned long, unsigned long, ..);
for ARCH_X86_CET_MMAP_SHSTK. We just need to use
syscall (SYS_arch_prctl, ARCH_X86_CET_MMAP_SHSTK, ...);
For arm64 (and sparc etc.) we continue to use the regular mmap/mprotect
family of calls. One or two additional arch-specific mmap flags are
sufficient for now.
Is x86 definitely not going to fit within those calls?
For now, I can't see what arg[2] is used for (and hence the type
argument of mmap_special()), but I haven't dug through the whole series.
Cheers
---Dave
On Thu, Aug 27, 2020 at 06:26:11AM -0700, H.J. Lu wrote:
quoted
On Wed, Aug 26, 2020 at 12:57 PM Dave Hansen [off-list ref] wrote:
quoted
On 8/26/20 11:49 AM, Yu, Yu-cheng wrote:
quoted
quoted
I would expect things like Go and various JITs to call it directly.
If we wanted to be fancy and add a potentially more widely useful
syscall, how about:
mmap_special(void *addr, size_t length, int prot, int flags, int type);
Where type is something like MMAP_SPECIAL_X86_SHSTK. Fundamentally,
this is really just mmap() except that we want to map something a bit
magical, and we don't want to require opening a device node to do it.
One benefit of MMAP_SPECIAL_* is there are more free bits than MAP_*.
Does ARM have similar needs for memory mapping, Dave?
No idea.
But, mmap_special() is *basically* mmap2() with extra-big flags space.
I suspect it will grow some more uses on top of shadow stacks. It could
have, for instance, been used to allocate MPX bounds tables.
There is no reason we can't use
long arch_prctl (int, unsigned long, unsigned long, unsigned long, ..);
for ARCH_X86_CET_MMAP_SHSTK. We just need to use
syscall (SYS_arch_prctl, ARCH_X86_CET_MMAP_SHSTK, ...);
For arm64 (and sparc etc.) we continue to use the regular mmap/mprotect
family of calls. One or two additional arch-specific mmap flags are
sufficient for now.
Is x86 definitely not going to fit within those calls?
That can work for x86. Andy, what if we create PROT_SHSTK, which can
been seen only from the user. Once in kernel, it is translated to
VM_SHSTK. One question for mremap/mprotect is, do we allow a normal
data area to become shadow stack?
For now, I can't see what arg[2] is used for (and hence the type
argument of mmap_special()), but I haven't dug through the whole series.
If we use the approach above, then we don't need arch_prctl changes.
Thanks,
Yu-cheng
From: Andy Lutomirski <luto@kernel.org> Date: 2020-09-01 17:46:15
On Tue, Sep 1, 2020 at 10:23 AM Yu, Yu-cheng [off-list ref] wrote:
On 9/1/2020 3:28 AM, Dave Martin wrote:
quoted
On Thu, Aug 27, 2020 at 06:26:11AM -0700, H.J. Lu wrote:
quoted
On Wed, Aug 26, 2020 at 12:57 PM Dave Hansen [off-list ref] wrote:
quoted
On 8/26/20 11:49 AM, Yu, Yu-cheng wrote:
quoted
quoted
I would expect things like Go and various JITs to call it directly.
If we wanted to be fancy and add a potentially more widely useful
syscall, how about:
mmap_special(void *addr, size_t length, int prot, int flags, int type);
Where type is something like MMAP_SPECIAL_X86_SHSTK. Fundamentally,
this is really just mmap() except that we want to map something a bit
magical, and we don't want to require opening a device node to do it.
One benefit of MMAP_SPECIAL_* is there are more free bits than MAP_*.
Does ARM have similar needs for memory mapping, Dave?
No idea.
But, mmap_special() is *basically* mmap2() with extra-big flags space.
I suspect it will grow some more uses on top of shadow stacks. It could
have, for instance, been used to allocate MPX bounds tables.
There is no reason we can't use
long arch_prctl (int, unsigned long, unsigned long, unsigned long, ..);
for ARCH_X86_CET_MMAP_SHSTK. We just need to use
syscall (SYS_arch_prctl, ARCH_X86_CET_MMAP_SHSTK, ...);
For arm64 (and sparc etc.) we continue to use the regular mmap/mprotect
family of calls. One or two additional arch-specific mmap flags are
sufficient for now.
Is x86 definitely not going to fit within those calls?
That can work for x86. Andy, what if we create PROT_SHSTK, which can
been seen only from the user. Once in kernel, it is translated to
VM_SHSTK. One question for mremap/mprotect is, do we allow a normal
data area to become shadow stack?
I'm unconvinced that we want to use a somewhat precious PROT_ or VM_
bit for this. Using a flag bit makes sense if we expect anyone to
ever map an fd or similar as a shadow stack, but that seems a bit odd
in the first place. To me, it seems more logical for a shadow stack
to be a special sort of mapping with a special vm_ops, not a normal
mapping with a special flag set. Although I realize that we want
shadow stacks to work like anonymous memory with respect to fork().
Dave?
--Andy
From: Dave Hansen <hidden> Date: 2020-09-01 18:11:45
On 9/1/20 10:45 AM, Andy Lutomirski wrote:
quoted
quoted
For arm64 (and sparc etc.) we continue to use the regular mmap/mprotect
family of calls. One or two additional arch-specific mmap flags are
sufficient for now.
Is x86 definitely not going to fit within those calls?
That can work for x86. Andy, what if we create PROT_SHSTK, which can
been seen only from the user. Once in kernel, it is translated to
VM_SHSTK. One question for mremap/mprotect is, do we allow a normal
data area to become shadow stack?
I'm unconvinced that we want to use a somewhat precious PROT_ or VM_
bit for this. Using a flag bit makes sense if we expect anyone to
ever map an fd or similar as a shadow stack, but that seems a bit odd
in the first place. To me, it seems more logical for a shadow stack
to be a special sort of mapping with a special vm_ops, not a normal
mapping with a special flag set. Although I realize that we want
shadow stacks to work like anonymous memory with respect to fork().
Dave?
I actually don't like the idea of *creating* mappings much.
I think the pkey model has worked out pretty well where we separate
creating the mapping from doing something *to* it, like changing
protections. For instance, it would be nice if we could preserve things
like using hugetlbfs or heck even doing KSM for shadow stacks.
If we're *creating* mappings, we've pretty much ruled out things like
hugetlbfs.
Something like mprotect_shstk() would allow an implementation today that
only works on anonymous memory *and* sets up a special vm_ops. But, the
same exact ABI could do wonky stuff in the future if we decided we
wanted to do shadow stacks on DAX or hugetlbfs or whatever.
I don't really like the idea of PROT_SHSTK those are plumbed into a
bunch of interfaces. But, I also can't deny that it seems to be working
fine for the arm64 folks.
From: Dave Martin <Dave.Martin@arm.com> Date: 2020-09-02 14:11:48
On Tue, Sep 01, 2020 at 11:11:37AM -0700, Dave Hansen wrote:
On 9/1/20 10:45 AM, Andy Lutomirski wrote:
quoted
quoted
quoted
For arm64 (and sparc etc.) we continue to use the regular mmap/mprotect
family of calls. One or two additional arch-specific mmap flags are
sufficient for now.
Is x86 definitely not going to fit within those calls?
That can work for x86. Andy, what if we create PROT_SHSTK, which can
been seen only from the user. Once in kernel, it is translated to
VM_SHSTK. One question for mremap/mprotect is, do we allow a normal
data area to become shadow stack?
I'm unconvinced that we want to use a somewhat precious PROT_ or VM_
bit for this. Using a flag bit makes sense if we expect anyone to
ever map an fd or similar as a shadow stack, but that seems a bit odd
in the first place. To me, it seems more logical for a shadow stack
to be a special sort of mapping with a special vm_ops, not a normal
mapping with a special flag set. Although I realize that we want
shadow stacks to work like anonymous memory with respect to fork().
Dave?
I actually don't like the idea of *creating* mappings much.
I think the pkey model has worked out pretty well where we separate
creating the mapping from doing something *to* it, like changing
protections. For instance, it would be nice if we could preserve things
like using hugetlbfs or heck even doing KSM for shadow stacks.
If we're *creating* mappings, we've pretty much ruled out things like
hugetlbfs.
Something like mprotect_shstk() would allow an implementation today that
only works on anonymous memory *and* sets up a special vm_ops. But, the
same exact ABI could do wonky stuff in the future if we decided we
wanted to do shadow stacks on DAX or hugetlbfs or whatever.
I don't really like the idea of PROT_SHSTK those are plumbed into a
bunch of interfaces. But, I also can't deny that it seems to be working
fine for the arm64 folks.
Note, there are some rough edges, such as what happens when someone
calls mprotect() on memory marked with PROT_BTI. Unless the caller
knows whether PROT_BTI should be set for that page, the flag may get
unintentionally cleared. Since the flag only applies to text pages
though, it's not _that_ much of a concern. Software that deals with
writable text pages is also usually involved in generating the code and
so will know about PROT_BTI. That's was the theory anyway.
In the longer term, it might be preferable to have a mprotect2() that
can leave some flags unmodified, and that doesn't silently ignore
unknown flags (at least one of mmap or mprotect does; I don't recall
which). We attempt didn't go this far, for now.
For arm64 it seemed fairly natural for the BTI flag to be a PROT_ flag,
but I don't know enough detail about x86 shstk to know whether it's a
natural fit there.
Cheers
---Dave