On Mon, Mar 9, 2020 at 4:59 PM Andy Lutomirski [off-list ref] wrote:

On Mon, Mar 9, 2020 at 4:52 PM H.J. Lu [off-list ref] wrote:

quoted

On Mon, Mar 9, 2020 at 4:21 PM Dave Hansen [off-list ref] wrote:

quoted

On 3/9/20 4:11 PM, H.J. Lu wrote:

quoted

A threaded application is loaded from disk.  The object file on disk is
either CET enabled or not CET enabled.

Huh.  Are you saying that all instructions executed on userspace on
Linux come off of object files on the disk?  That's an interesting
assertion.  You might want to go take a look at the processes on your
systems.  Here's my browser for example:

# for p in $(ps aux | grep chromium | awk '{print $2}' ); do cat
/proc/$p/maps; done | grep ' r-xp 00000000 00:00 0'
...
202f00082000-202f000bf000 r-xp 00000000 00:00 0
202f000c2000-202f000c3000 r-xp 00000000 00:00 0
202f00102000-202f00103000 r-xp 00000000 00:00 0
202f00142000-202f00143000 r-xp 00000000 00:00 0
202f00182000-202f001bf000 r-xp 00000000 00:00 0

Lots of funny looking memory areas which are anonymous and executable!
Those didn't come off the disk.  Same thing in firefox.  Weird.  Any
idea what those are?

One guess: https://en.wikipedia.org/wiki/Just-in-time_compilation

jitted code belongs to a process loaded from disk.  Enable CET in
an application which uses JIT engine means to also enable CET in
JIT engine.  Take git as an example, "git grep" crashed for me on Tiger
Lake.   It turned out that git itself was compiled with -fcf-protection and
git was linked against libpcre2-8.so.0 also compiled with -fcf-protection,
which has a JIT, sljit, which was not CET enabled.  git crashed in the
jitted codes due to missing ENDBR.  I had to enable CET in sljit to make
git working on CET enabled Tiger Lake.  So we need to enable CET in
JIT engine before enabling CET in applications which use JIT engine.

This could presumably have been fixed by having libpcre or sljit
disable IBT before calling into JIT code or by running the JIT code in
another thread.  In the other direction, a non-CET libpcre build could
build IBT-capable JITted code and enable JIT (by syscall if we allow
that or by creating a thread?) when calling it.  And IBT has this

This is not how thread in user space works.

fancy legacy bitmap to allow non-instrumented code to run with IBT on,
although SHSTK doesn't have hardware support for a similar feature.

All these changes are called CET enabing.

So, sure, the glibc-linked ELF ecosystem needs some degree of CET
coordination, but it is absolutely not the case that a process MUST
have all CET or no CET.  Let's please support the complicated cases in
the kernel and the ABI too.  If glibc wants to make it annoying to do
complicated things, so be it.  People work behind glibc's back all the
time.

CET is no different from NX in this regard.


-- 
H.J.