On Mon, Jun 5, 2023 at 3:09 AM Mark Rutland [off-list ref] wrote:

[...]

quoted

quoted

quoted

Can you give more detail on what parameters you need? If the only extra
parameter is just "does this allocation need to live close to kernel
text", that's not that big of a deal.

My thinking was that we at least need the start + end for each caller. That
might be it, tbh.

Do you mean that modules will have something like

      jit_text_alloc(size, MODULES_START, MODULES_END);

and kprobes will have

      jit_text_alloc(size, KPROBES_START, KPROBES_END);
?

Yes.

How about we start with two APIs:
     jit_text_alloc(size);
     jit_text_alloc_range(size, start, end);

AFAICT, arm64 is the only arch that requires the latter API. And TBH, I am
not quite convinced it is needed.

quoted

It sill can be achieved with a single jit_alloc_arch_params(), just by
adding enum jit_type parameter to jit_text_alloc().

That feels backwards to me; it centralizes a bunch of information about
distinct users to be able to shove that into a static array, when the callsites
can pass that information.

I think we only two type of users: module and everything else (ftrace, kprobe,
bpf stuff). The key differences are:

  1. module uses text and data; while everything else only uses text.
  2. module code is generated by the compiler, and thus has stronger
  requirements in address ranges; everything else are generated via some
  JIT or manual written assembly, so they are more flexible with address
  ranges (in JIT, we can avoid using instructions that requires a specific
  address range).

The next question is, can we have the two types of users share the same
address ranges? If not, we can reserve the preferred range for modules,
and let everything else use the other range. I don't see reasons to further
separate users in the "everything else" group.

What's *actually* common after separating out the ranges? Is it just the
permissions?

I believe permission is the key, as we need the hardware to enforce
permission.

If we want this to be able to share allocations and so on, why can't we do this
like a kmem_cache, and have the callsite pass a pointer to the allocator data?
That would make it easy for callsites to share an allocator or use a distinct
one.

Sharing among different call sites will give us more benefit (in TLB
misses rate,
etc.). For example, a 2MB page may host text of two kernel modules, 4 kprobes,
6 ftrace trampolines, and 10 BPF programs. All of these only require one entry
in the iTLB.

Thanks,
Song