Re: [PATCH perf,bpf 0/5] reveal invisible bpf programs

On Nov 26, 2018, at 7:27 AM, Arnaldo Carvalho de Melo [off-list ref] wrote:

Em Mon, Nov 26, 2018 at 03:50:04PM +0100, Peter Zijlstra escreveu:

quoted

Now, I'm not saying this patch set is useless; but I'm saying most
people should not need this, and it is massive overkill for the needs of
most people.

So, the comparision is sort of with kernel modules, that can come and go
while you're profiling/tracing, if that happens, then samples, in post
processing, are not resolvable, and that is the case for kernel modules
right now. Sure, you're right, that doesn't happen so frequently, so
nobody hollered (thankfully that is now verbotten ;-)) at us so far.

You need to have the load-kernel-bin/unload-kernel-bin events recorded,
and you need to somehow match those addresses to some symtab/src(for
people that want to have src mixed up with assembly) and you need that
jitted code, with timestamps of when it was loaded and it was unloaded.

People doing post processing analysis of weird problems need all those
details.

Now I don't know how frequently those binary blobs gets loaded/unloaded
in the brave new world of eBPF, but for completeness sake, we need those
load/unload events and we need to grab a copy of the raw jitted binary,
etc.

BPF programs get loaded/unloaded more often than kernel modules. This is 
because BPF verifier makes sure BPF program will never crash the kernel, 
thus BPF programs could often be debugged as user space apps. 

Typical debug/tune process is like:

    perf record -- ./test_work
    perf report

    # make changes to bpf program and repeat

or even:

    perf record -o perf_1.data -- ./test_v1
    perf record -o perf_2.data -- ./test_v2
    perf record -o perf_3.data -- ./test_v3

    # perf report in 3 terminals, and compare the output

In such use cases, including all details in the perf.data is very helpful, 
as /proc/kcore doesn't have details of the program when perf-report runs. 

Thanks,
Song