Thread (28 messages) 28 messages, 4 authors, 2020-06-02

Re: [PATCH 7/9] bpf: Compile the BTF id whitelist data in vmlinux

From: Alexei Starovoitov <hidden>
Date: 2020-05-13 18:29:45
Also in: bpf

On Wed, May 06, 2020 at 03:29:44PM +0200, Jiri Olsa wrote:
quoted hunk ↗ jump to hunk
Squeezing in the BTF id whitelist data into vmlinux object
with BTF section compiled in, with following steps:

  - generate whitelist data with bpfwl
    $ bpfwl .tmp_vmlinux.btf kernel/bpf/helpers-whitelist > ${whitelist}.c

  - compile whitelist.c
    $ gcc -c -o ${whitelist}.o ${whitelist}.c

  - keep only the whitelist data in ${whitelist}.o using objcopy

  - link .tmp_vmlinux.btf and ${whitelist}.o into $btf_vmlinux_bin_o}
    $ ld -r -o ${btf_vmlinux_bin_o} .tmp_vmlinux.btf ${whitelist}.o

Signed-off-by: Jiri Olsa <jolsa@kernel.org>
---
 Makefile                |  3 ++-
 scripts/link-vmlinux.sh | 20 +++++++++++++++-----
 2 files changed, 17 insertions(+), 6 deletions(-)
diff --git a/Makefile b/Makefile
index b0537af523dc..3bb995245592 100644
--- a/Makefile
+++ b/Makefile
@@ -437,6 +437,7 @@ OBJSIZE		= $(CROSS_COMPILE)size
 STRIP		= $(CROSS_COMPILE)strip
 endif
 PAHOLE		= pahole
+BPFWL		= $(srctree)/tools/bpf/bpfwl/bpfwl
 LEX		= flex
 YACC		= bison
 AWK		= awk
@@ -493,7 +494,7 @@ GCC_PLUGINS_CFLAGS :=
 CLANG_FLAGS :=
 
 export ARCH SRCARCH CONFIG_SHELL BASH HOSTCC KBUILD_HOSTCFLAGS CROSS_COMPILE LD CC
-export CPP AR NM STRIP OBJCOPY OBJDUMP OBJSIZE READELF PAHOLE LEX YACC AWK INSTALLKERNEL
+export CPP AR NM STRIP OBJCOPY OBJDUMP OBJSIZE READELF PAHOLE BPFWL LEX YACC AWK INSTALLKERNEL
 export PERL PYTHON PYTHON3 CHECK CHECKFLAGS MAKE UTS_MACHINE HOSTCXX
 export KBUILD_HOSTCXXFLAGS KBUILD_HOSTLDFLAGS KBUILD_HOSTLDLIBS LDFLAGS_MODULE
 
diff --git a/scripts/link-vmlinux.sh b/scripts/link-vmlinux.sh
index d09ab4afbda4..dee91c6bf450 100755
--- a/scripts/link-vmlinux.sh
+++ b/scripts/link-vmlinux.sh
@@ -130,16 +130,26 @@ gen_btf()
 	info "BTF" ${2}
 	LLVM_OBJCOPY=${OBJCOPY} ${PAHOLE} -J ${1}
 
-	# Create ${2} which contains just .BTF section but no symbols. Add
+	# Create object which contains just .BTF section but no symbols. Add
 	# SHF_ALLOC because .BTF will be part of the vmlinux image. --strip-all
 	# deletes all symbols including __start_BTF and __stop_BTF, which will
 	# be redefined in the linker script. Add 2>/dev/null to suppress GNU
 	# objcopy warnings: "empty loadable segment detected at ..."
 	${OBJCOPY} --only-section=.BTF --set-section-flags .BTF=alloc,readonly \
-		--strip-all ${1} ${2} 2>/dev/null
-	# Change e_type to ET_REL so that it can be used to link final vmlinux.
-	# Unlike GNU ld, lld does not allow an ET_EXEC input.
-	printf '\1' | dd of=${2} conv=notrunc bs=1 seek=16 status=none
+		--strip-all ${1} 2>/dev/null
+
+	# Create object that contains just .BTF_whitelist_* sections generated
+	# by bpfwl. Same as BTF section, BTF_whitelist_* data will be part of
+	# the vmlinux image, hence SHF_ALLOC.
+	whitelist=.btf.vmlinux.whitelist
+
+	${BPFWL} ${1} kernel/bpf/helpers-whitelist > ${whitelist}.c
+	${CC} -c -o ${whitelist}.o ${whitelist}.c
+	${OBJCOPY} --only-section=.BTF_whitelist* --set-section-flags .BTF=alloc,readonly \
+                --strip-all ${whitelist}.o 2>/dev/null
+
+	# Link BTF and BTF_whitelist objects together
+	${LD} -r -o ${2} ${1} ${whitelist}.o
Thank you for working on it!
Looks great to me overall. In the next rev please drop RFC tag.

My only concern is this extra linking step. How many extra seconds does it add?

Also in patch 3:
+               func = func__find(str);
+               if (func)
+                       func->id = id;
which means that if somebody mistyped the name or that kernel function
got renamed there will be no warnings or errors.
I think it needs to fail the build instead.

If additional linking step takes another 20 seconds it could be a reason
to move the search to run-time.
We already have that with struct bpf_func_proto->btf_id[].
Whitelist could be something similar.
I think this mechanism will be reused for unstable helpers and other
func->btf_id mappings, so 'bpfwl' name would change eventually.
It's not white list specific. It generates a mapping of names to btf_ids.
Doing it at build time vs run-time is a trade off and it doesn't have
an obvious answer.
Keyboard shortcuts
hback out one level
jnext message in thread
kprevious message in thread
ldrill in
Escclose help / fold thread tree
?toggle this help