On Mon, 2019-06-10 at 12:52 -0700, Dave Hansen wrote:

On 6/10/19 12:38 PM, Yu-cheng Yu wrote:

quoted

quoted

quoted

When an application starts, its highest stack address is determined.
It uses that as the maximum the bitmap needs to cover.

Huh, I didn't think we ran code from the stack. ;)

Especially given the way that we implemented the new 5-level-paging
address space, I don't think that expecting code to be below the stack
is a good universal expectation.

Yes, you make a good point.  However, allowing the application manage the
bitmap
is the most efficient and flexible.  If the loader finds a legacy lib is
beyond
the bitmap can cover, it can deal with the problem by moving the lib to a
lower
address; or re-allocate the bitmap.

How could the loader reallocate the bitmap and coordinate with other
users of the bitmap?

Assuming the loader actually chooses to re-allocate, it can copy the old bitmap
over to the new before doing the switch.  But, I agree, the other choice is
easier; the loader can simply put the lib at lower address.  AFAIK, the loader
does not request high address in mmap().

quoted

If the loader cannot allocate a big bitmap to cover all 5-level
address space (the bitmap will be large), it can put all legacy lib's
at lower address.  We cannot do these easily in the kernel.

This is actually an argument to do it in the kernel.  The kernel can
always allocate the virtual space however it wants, no matter how large.
 If we hide the bitmap behind a kernel API then we can put it at high
5-level user addresses because we also don't have to worry about the
high bits confusing userspace.

We actually tried this.  The kernel needs to reserve the bitmap space in the
beginning for every CET-enabled app, regardless of actual needs.  On each memory
request, the kernel then must consider a percentage of allocated space in its
calculation, and on systems with less memory this quickly becomes a problem.