On Tue, May 18, 2021 at 6:03 AM Arnd Bergmann [off-list ref] wrote:

On Tue, May 18, 2021 at 11:06 AM Amanieu d'Antras [off-list ref] wrote:

quoted

Setting bit 31 in x8 when performing a syscall will do the following:
- The remainder of x8 is treated as a compat syscall number and is used
  to index the compat syscall table.
- in_compat_syscall will return true for the duration of the syscall.
- VM allocations performed by the syscall will be located in the lower
  4G of the address space.
- Interrupted syscalls are properly restarted as compat syscalls.
- Seccomp will treats the syscall as having AUDIT_ARCH_ARM instead of
  AUDIT_ARCH_AARCH64. This affects the arch value seen by seccomp
  filters and reported by SIGSYS.
- PTRACE_GET_SYSCALL_INFO also treats the syscall as having
  AUDIT_ARCH_ARM. Recent versions of strace will correctly report the
  system call name and parameters when an AArch64 task mixes 32-bit and
  64-bit syscalls.

Previously, setting bit 31 of the syscall number would always cause the
sygscall to return ENOSYS. This allows user programs to reliably detect
kernel support for compat syscall by trying a simple syscall such as
getpid.

The AArch32-private compat syscalls (__ARM_NR_compat_*) are not exposed
through this interface. These syscalls do not make sense in the context
of an AArch64 task.

Signed-off-by: Amanieu d'Antras <redacted>
Co-developed-by: Ryan Houdek <redacted>
Signed-off-by: Ryan Houdek <redacted>

I'm still undecided about this approach. It is an easy way to expose the 32-bit
ABIs, it mostly copies what x86-64 already does with 32-bit syscalls and
it doesn't expose a lot of attack surface that isn't already exposed to normal
32-bit tasks running compat mode.

On the other hand, exposing the entire aarch32 syscall set seems both
too broad and not broad enough: Half of the system calls behave the
exact same way in native and compat mode, so they wouldn't need to
be exposed like this, a lot of others are trivially emulated in user space
by calling the native versions. The syscalls that are actually hard to do
such as ioctl() or the signal handling will work for aarch32 emulation, but
they are still insufficient to correctly emulate other 32-bit architectures
that have a slightly different ABI. This means the interface is a fairly good
fit for Tango, but much less so for FEX.

You are correct here. This meshes perfectly for Tango's use case. Where
the syscalls will match perfectly for their aarch32->aarch64->compat syscall
path.

For FEX's use case, we still need to deal with any data structure that
doesn't match between the 32-bit x86 to compat syscall boundary. While
x86->compat will require significantly less fixups than x86->aarch64, it
is still likely to have some structure differences that need fixing.

It's also worth pointing out that this approach has a few things in common
with Yury's ilp32 tree at https://github.com/norov/linux/tree/ilp32-5.2
Unlike the x86 x32 mode, that port however does not allow calling compat
syscalls from normal 64-bit tasks but rather keys the syscall entry point
off the executable format., which wouldn't work here. It also uses the
asm-generic system call numbers instead of the arm32 syscall numbers.

I assume you have already considered or tried the alternative approach of
only adding a minimal set of syscalls that are needed for the emulation.
Having a way to limit the address space for mmap() and similar
system calls sounds like a generally useful addition, and having an
extended variant of ioctl() that lets you pick the target ABI (arm32, x86-32,
...) on supported drivers would probably be better for FEX. Can you
explain the tradeoffs that led you towards duplicating the syscall
entry points instead?

I'm likely to not be very concise here. There are many paper cuts for
any route taken here.
For me, this one is the best route because of its ability to future proof
for any upcoming additions to syscalls.

If we were wanting to take a path of duplicating a bunch of compat
syscalls to work from the 64-bit side. We would first need to start with
around nine syscalls that are causing immediate problems.
mmap/mmap2, mremap, shmat, ioctl, recvmsg, recvmmsg, getdents,
and getdents64.
So we could carve those out, adding effectively the same memory
handling code that is being added here[1]. Do the ML dance to upstream.
We now have nine-ish syscalls that are added specifically for userspace
compatibility layers.
That's already beginning to have a bad smell.

Next step is a couple months down the line, someone adds a super cool
syscall that say, allocates memory that is secure over infiniband and flushes
to persistence on hibernate. Neato. Oops, this is allocating memory, and
since FEX is tracking very close to upstream kernel syscall support, we now
need to add yet another syscall that handles the compat version in a
64-bit space.
Or maybe it appends to a linked list of secure memory regions. Only visible as
the head of the list (Hello robust futexes).

See what I mean? Exposing the 32-bit compat syscalls removes the burden of
now needing to think about every syscall in a context of 32-bit,
64-bit, 32-bit on 64-bit.
Also removes the burden that I then need to come back and pester the ML
every single time with new patchsets adding syscalls only for compat layers.

And I'm all about removing unnecessary burden

[1]Side grade, personality flags won't be pretty here, FEX lives in a
mixed syscall world and doesn't want only one or the other working.
FEX does a bunch of stuff in the background and a personality flag
would be hard to work around whenever we need to do some
memory allocations, or file system handling, or its own 64-bit ioctl
handling. Just not very versatile.
FEX is already allocating all 48/52-bit VA, breaking ASLR and stack
growing, as a partial workaround here.

         Arnd

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel