Re: [PATCH bpf-next] libbpf: support module BTF for BPF_TYPE_ID_TARGET CO-RE relocation

On Tue, 8 Dec 2020, Alexei Starovoitov wrote:

On Tue, Dec 08, 2020 at 10:13:35PM +0000, Alan Maguire wrote:

quoted

Does this approach prevent more complex run-time specification of BTF 
object fd though?  For example, I've been working on a simple tracer 
focused on kernel debugging; it uses a BPF map entry for each kernel 
function that is traced. User-space populates the map entry with BTF type 
ids for the function arguments/return value, and when the BPF program 
runs it uses the instruction pointer to look up the map entry for that
function, and uses bpf_snprintf_btf() to write the string representations 
of the function arguments/return values.  I'll send out an RFC soon, 
but longer-term I was hoping to extend it to support module-specific 
types.  Would a dynamic case like that - where the BTF module fd is looked 
up in a map entry during program execution (rather than derived via 
__btf_builtin_type_id()) work too? Thanks!

fd has to be resolved in the process context. bpf prog can read fd
number from the map, but that number is meaningless.
Say we allow using btf_obj_id+btf_id, how user space will know these
two numbers? Some new libbpf api that searches for it?
An extension to libbpf_find_vmlinux_btf_id() ? I was hoping that this api
will stay semi-internal. But say it's extended.
The user space will store a pair of numbers into a map and
what program are going to do with it?
If it's printing struct veth_stats contents it should have attached to
a corresponding function in the veth module via fentry or something.
The prog has hard coded logic in C with specific pointer to print.
The prog has its type right there. Why would the prog take a pointer
from one place, but it's type_id from the map? That's not realistic.
Where it would potentially make sense is what I think you're descring
where single kprobe style prog attached to many places and args of
those places are stored in a map and the prog selects them with
map_lookup with key=PT_REGS_IP ?

Right, that's exactly it.  A pair of generic tracing BPF programs are
used, and they attach to kprobe/kretprobes, and when they run they use 
the arguments plus the map details about BTF ids of those arguments to 
run bpf_snprintf_btf(), and send perf events to userspace containing
the results.

And passes pointers into bpf_snprintf_btf() from PT_REGS_PARM1() ?

Exactly.

I see why that is useful, but it's so racy. By the time the map
is populated those btf_obj_id+btf_id could be invalid.
I think instead of doing this in user space the program needs an access
to vmlinux+mods BTFs. Sort-of like proposed bpf helper to return ksym
based on IP there could be a helper to figure out btf_id+btf_obj_POINTER
based on IP. Then there will no need for external map to populate.
Would that solve your use case?

That would be fantastic! We could do that from the context passed into a
kprobe program as the IP in struct pt_regs points at the function.  
kretprobes seems a bit trickier as in that case the IP in struct pt_regs 
is actually set to kretprobe_trampoline rather than the function we're
returning from due to how kretprobes work; maybe there's another way to 
get it in that case though..

Alan