Re: [PATCH 12/14] bpf: Add trampolines to kallsyms

On Wed, Feb 12, 2020 at 3:10 AM Jiri Olsa [off-list ref] wrote:

On Tue, Feb 11, 2020 at 10:51:27AM -0800, Andrii Nakryiko wrote:

quoted

On Sat, Feb 8, 2020 at 7:43 AM Jiri Olsa [off-list ref] wrote:

quoted

Adding trampolines to kallsyms. It's displayed as
  bpf_trampoline_<ID> [bpf]

where ID is the BTF id of the trampoline function.

Signed-off-by: Jiri Olsa <jolsa@kernel.org>
---
 include/linux/bpf.h     |  2 ++
 kernel/bpf/trampoline.c | 23 +++++++++++++++++++++++
 2 files changed, 25 insertions(+)

diff --git a/include/linux/bpf.h b/include/linux/bpf.h
index 7a4626c8e747..b91bac10d3ea 100644
--- a/include/linux/bpf.h
+++ b/include/linux/bpf.h

@@ -502,6 +502,7 @@ struct bpf_trampoline {
        /* Executable image of trampoline */
        void *image;
        u64 selector;
+       struct bpf_ksym ksym;
 };

 #define BPF_DISPATCHER_MAX 48 /* Fits in 2048B */

@@ -573,6 +574,7 @@ struct bpf_image {
 #define BPF_IMAGE_SIZE (PAGE_SIZE - sizeof(struct bpf_image))
 bool is_bpf_image_address(unsigned long address);
 void *bpf_image_alloc(void);
+void bpf_image_ksym_add(void *data, struct bpf_ksym *ksym);
 /* Called only from code, so there's no need for stubs. */
 void bpf_ksym_add(struct bpf_ksym *ksym);
 void bpf_ksym_del(struct bpf_ksym *ksym);

diff --git a/kernel/bpf/trampoline.c b/kernel/bpf/trampoline.c
index 6b264a92064b..1ee29907cbe5 100644
--- a/kernel/bpf/trampoline.c
+++ b/kernel/bpf/trampoline.c

@@ -96,6 +96,15 @@ bool is_bpf_image_address(unsigned long addr)
        return ret;
 }

+void bpf_image_ksym_add(void *data, struct bpf_ksym *ksym)
+{
+       struct bpf_image *image = container_of(data, struct bpf_image, data);
+
+       ksym->start = (unsigned long) image;
+       ksym->end = ksym->start + PAGE_SIZE;

this seems wrong, use BPF_IMAGE_SIZE instead of PAGE_SIZE?

BPF_IMAGE_SIZE is the size of the data portion of the image,
which is PAGE_SIZE - sizeof(struct bpf_image)

here we want to account the whole size = data + tree node (struct bpf_image)

Why? Seems like the main use case for this is resolve IP to symbol
(function, dispatcher, trampoline, bpf program, etc). For this
purpose, you only need part of trampoline actually containing
executable code?

Also, for bpf_dispatcher in later patch, you are not including struct
bpf_dispatcher itself, you only include image, so if the idea is to
include all the code and supporting data structures, that already
failed for bpf_dispatcher (and can't even work for that case, due to
dispatcher and image not being part of the same blob of memory, so
you'll need two symbols).

So I guess it would be good to be clear on why we include these
symbols and not mix data and executable parts.

quoted

quoted

+       bpf_ksym_add(ksym);
+}
+
 struct bpf_trampoline *bpf_trampoline_lookup(u64 key)
 {
        struct bpf_trampoline *tr;

@@ -131,6 +140,7 @@ struct bpf_trampoline *bpf_trampoline_lookup(u64 key)
        for (i = 0; i < BPF_TRAMP_MAX; i++)
                INIT_HLIST_HEAD(&tr->progs_hlist[i]);
        tr->image = image;
+       INIT_LIST_HEAD_RCU(&tr->ksym.lnode);
 out:
        mutex_unlock(&trampoline_mutex);
        return tr;

@@ -267,6 +277,15 @@ static enum bpf_tramp_prog_type bpf_attach_type_to_tramp(enum bpf_attach_type t)
        }
 }

+static void bpf_trampoline_kallsyms_add(struct bpf_trampoline *tr)
+{
+       struct bpf_ksym *ksym = &tr->ksym;
+
+       snprintf(ksym->name, KSYM_NAME_LEN, "bpf_trampoline_%llu",
+                tr->key & ((u64) (1LU << 32) - 1));

why the 32-bit truncation? also, wouldn't it be more trivial as (u32)tr->key?

tr->key can have the target prog id in upper 32 bits,

True, but not clear why it's bad? It's not a security concern, because
those IDs are already exposed (you can dump them from bpftool). On the
other hand, by cutting out part of key, you make symbols potentially
ambiguous, with different trampolines marked with the same name in
kallsyms, which is just going to be confusing to users/tools.

I'll try to use the casting as you suggest

quoted

quoted

+       bpf_image_ksym_add(tr->image, &tr->ksym);
+}
+
 int bpf_trampoline_link_prog(struct bpf_prog *prog)
 {
        enum bpf_tramp_prog_type kind;

@@ -311,6 +330,8 @@ int bpf_trampoline_link_prog(struct bpf_prog *prog)
        if (err) {
                hlist_del(&prog->aux->tramp_hlist);
                tr->progs_cnt[kind]--;
+       } else if (cnt == 0) {
+               bpf_trampoline_kallsyms_add(tr);

You didn't handle BPF_TRAMP_REPLACE case above.

ugh, right.. will add

quoted

Also this if (err) { ... } else if (cnt == 0) { } pattern is a bit
convoluted. How about:

if (err) {
   ... whatever ...
   goto out;
}
if (cnt == 0) { ... }

yep, that's better

quoted

quoted

        }
 out:
        mutex_unlock(&tr->mutex);

@@ -336,6 +357,8 @@ int bpf_trampoline_unlink_prog(struct bpf_prog *prog)
        }
        hlist_del(&prog->aux->tramp_hlist);
        tr->progs_cnt[kind]--;
+       if (!(tr->progs_cnt[BPF_TRAMP_FENTRY] + tr->progs_cnt[BPF_TRAMP_FEXIT]))
+               bpf_ksym_del(&tr->ksym);

same, BPF_TRAMP_REPLACE case. I'd also introduce cnt for consistency
with bpf_trampoline_link_prog?

ok, thanks a lot for comments

sure, you are welcome :)

jirka