Re: [PATCH bpf-next v4 0/3] Annotate kfuncs in .BTF_ids section
From: Manu Bretelle <hidden>
Date: 2024-02-02 23:09:09
Also in:
bpf, cgroups, linux-arm-kernel, linux-doc, linux-input, linux-kselftest, lkml, netdev, netfilter-devel
Subsystem:
bpf [general] (safe dynamic programs and tools), the rest · Maintainers:
Alexei Starovoitov, Daniel Borkmann, Andrii Nakryiko, Eduard Zingerman, Kumar Kartikeya Dwivedi, Linus Torvalds
On Sun, Jan 28, 2024 at 06:24:05PM -0700, Daniel Xu wrote:
=== Description === This is a bpf-treewide change that annotates all kfuncs as such inside .BTF_ids. This annotation eventually allows us to automatically generate kfunc prototypes from bpftool. We store this metadata inside a yet-unused flags field inside struct btf_id_set8 (thanks Kumar!). pahole will be taught where to look. More details about the full chain of events are available in commit 3's description. The accompanying pahole and bpftool changes can be viewed here on these "frozen" branches [0][1]. [0]: https://github.com/danobi/pahole/tree/kfunc_btf-v3-mailed [1]: https://github.com/danobi/linux/tree/kfunc_bpftool-mailed
I hit a similar issue to [0] on master
943b043aeecc ("selftests/bpf: Fix bench runner SIGSEGV")
when cross-compiling on x86_64 (LE) to s390x (BE).
I do have CONFIG_DEBUG_INFO_BTF enable and the issue would not trigger if
I disabled CONFIG_DEBUG_INFO_BTF (and with the fix mentioned in [0]).
What seems to happen is that `tools/resolve_btfids` is ran in the context of the
host endianess and if I printk before the WARN_ON:diff --git a/kernel/bpf/btf.c b/kernel/bpf/btf.c
index ef380e546952..a9ed7a1a4936 100644
--- a/kernel/bpf/btf.c
+++ b/kernel/bpf/btf.c
@@ -8128,6 +8128,7 @@ int register_btf_kfunc_id_set(enum bpf_prog_type prog_type,
* WARN() for initcall registrations that do not check errors.
*/
if (!(kset->set->flags & BTF_SET8_KFUNCS)) {
+ printk("Flag 0x%08X, expected 0x%08X\n", kset->set->flags, BTF_SET8_KFUNCS);
WARN_ON(!kset->owner);
return -EINVAL;
}
the boot logs would show:
Flag 0x01000000, expected 0x00000001
The issue did not happen prior to
6f3189f38a3e ("bpf: treewide: Annotate BPF kfuncs in BTF")
has only 0 was written before.
It seems [1] will be addressing cross-compilation, but it did not fix it as is
by just applying on top of master, so probably some of the changes will also need
to be ported to `tools/include/linux/btf_ids.h`?
A hacky workaround to cross-compilation I have is to apply:
diff --git a/tools/bpf/resolve_btfids/Makefile b/tools/bpf/resolve_btfids/Makefile
index 4b8079f294f6..b706e7ab066f 100644
--- a/tools/bpf/resolve_btfids/Makefile
+++ b/tools/bpf/resolve_btfids/Makefile
@@ -22,10 +22,10 @@ HOST_OVERRIDES := AR="$(HOSTAR)" CC="$(HOSTCC)" LD="$(HOSTLD)" ARCH="$(HOSTARCH)
CROSS_COMPILE="" EXTRA_CFLAGS="$(HOSTCFLAGS)"
RM ?= rm
-HOSTCC ?= gcc
-HOSTLD ?= ld
-HOSTAR ?= ar
-CROSS_COMPILE =
+HOSTCC = $(CC)
+HOSTLD = $(LD)
+HOSTAR = $(AR)
+#CROSS_COMPILE =
OUTPUT ?= $(srctree)/tools/bpf/resolve_btfids/
@@ -56,16 +56,16 @@ $(OUTPUT) $(OUTPUT)/libsubcmd $(LIBBPF_OUT):
$(SUBCMDOBJ): fixdep FORCE | $(OUTPUT)/libsubcmd
$(Q)$(MAKE) -C $(SUBCMD_SRC) OUTPUT=$(SUBCMD_OUT) \
- DESTDIR=$(SUBCMD_DESTDIR) $(HOST_OVERRIDES) prefix= subdir= \
+ DESTDIR=$(SUBCMD_DESTDIR) prefix= subdir= \
$(abspath $@) install_headers
$(BPFOBJ): $(wildcard $(LIBBPF_SRC)/*.[ch] $(LIBBPF_SRC)/Makefile) | $(LIBBPF_OUT)
$(Q)$(MAKE) $(submake_extras) -C $(LIBBPF_SRC) OUTPUT=$(LIBBPF_OUT) \
- DESTDIR=$(LIBBPF_DESTDIR) $(HOST_OVERRIDES) prefix= subdir= \
+ DESTDIR=$(LIBBPF_DESTDIR) prefix= subdir= \
$(abspath $@) install_headers
-LIBELF_FLAGS := $(shell $(HOSTPKG_CONFIG) libelf --cflags 2>/dev/null)
-LIBELF_LIBS := $(shell $(HOSTPKG_CONFIG) libelf --libs 2>/dev/null || echo -lelf)
+LIBELF_FLAGS := $(shell $(PKG_CONFIG) libelf --cflags 2>/dev/null)
+LIBELF_LIBS := $(shell $(PKG_CONFIG) libelf --libs 2>/dev/null || echo -lelf)
HOSTCFLAGS_resolve_btfids += -g \
-I$(srctree)/tools/include \
@@ -84,7 +84,7 @@ $(BINARY_IN): fixdep FORCE prepare | $(OUTPUT)
$(BINARY): $(BPFOBJ) $(SUBCMDOBJ) $(BINARY_IN)
$(call msg,LINK,$@)
- $(Q)$(HOSTCC) $(BINARY_IN) $(KBUILD_HOSTLDFLAGS) -o $@ $(BPFOBJ) $(SUBCMDOBJ) $(LIBS)
+ $(Q)$(CC) $(BINARY_IN) $(KBUILD_HOSTLDFLAGS) -o $@ $(BPFOBJ) $(SUBCMDOBJ) $(LIBS)
clean_objects := $(wildcard $(OUTPUT)/*.o \
$(OUTPUT)/.*.o.cmd \
diff --git a/tools/testing/selftests/bpf/Makefile b/tools/testing/selftests/bpf/Makefile
index a38a3001527c..5cd193c04448 100644
--- a/tools/testing/selftests/bpf/Makefile
+++ b/tools/testing/selftests/bpf/Makefile
@@ -171,7 +171,7 @@ INCLUDE_DIR := $(SCRATCH_DIR)/include
BPFOBJ := $(BUILD_DIR)/libbpf/libbpf.a
ifneq ($(CROSS_COMPILE),)
HOST_BUILD_DIR := $(BUILD_DIR)/host
-HOST_SCRATCH_DIR := $(OUTPUT)/host-tools
+HOST_SCRATCH_DIR := $(SCRATCH_DIR)
HOST_INCLUDE_DIR := $(HOST_SCRATCH_DIR)/include
else
HOST_BUILD_DIR := $(BUILD_DIR)
This causes `resolve_btfids` to be compiled in the target endianess and gets
magically run provided that the hosts has `qemu-s390x-static` and a functional
binfmt_misc [2] on the host, but having this using host architecture per [1]
is likely better.
Here are steps to reproduce the issue on Ubuntu 23.10 and assuming
danobi/vmtest [3] is installed:
XPLATFORM="s390x"
XARCH="s390"
# Set up repo for s390x
cat <<EOF >> /etc/apt/sources.list.d/s390x.list
deb [arch=s390x] http://ports.ubuntu.com/ubuntu-ports mantic main restricted
deb [arch=s390x] http://ports.ubuntu.com/ubuntu-ports mantic-updates main restricted
EOF
sudo dpkg --add-architecture s390x
apt install qemu-system-s390x qemu-user-static g{cc,++}-"${XARCH}-linux-gnu" {libelf-dev,libssl-dev,pkgconf}:s390x
KBUILD_OUTPUT_DIR="/tmp/kbuild-${XPLATFORM}"
mkdir "${KBUILD_OUTPUT_DIR}"
cat tools/testing/selftests/bpf/config{,.vm,.${XPLATFORM}} > ${KBUILD_OUTPUT_DIR}/.config
make ARCH="${XARCH}" CROSS_COMPILE="${XPLATFORM}-linux-gnu-" O="${KBUILD_OUTPUT_DIR}" -j$((4 * $(nproc))) olddefconfig
make ARCH="${XARCH}" CROSS_COMPILE="${XPLATFORM}-linux-gnu-" O="${KBUILD_OUTPUT_DIR}" -j$((4 * $(nproc))) all
# No need for a s390x ubuntu 23.10 rootfs, we only care about booting the kernel
vmtest -k "${KBUILD_OUTPUT_DIR}/arch/s390/boot/bzImage" -a s390x "uname -m" | cat
For the chroot route, see [4].
[0] https://lore.kernel.org/linux-kernel/20240201155339.2b5936be@canb.auug.org.au/T/ (local)
[1] https://lore.kernel.org/bpf/cover.1706717857.git.vmalik@redhat.com/ (local)
[2] https://en.wikipedia.org/wiki/Binfmt_misc
[3] https://github.com/danobi/vmtest
[4] https://chantra.github.io/bpfcitools/bpf-cross-compile.html
Manu
=== Changelog === Changes from v3: * Rebase to bpf-next and add missing annotation on new kfunc Changes from v2: * Only WARN() for vmlinux kfuncs Changes from v1: * Move WARN_ON() up a call level * Also return error when kfunc set is not properly tagged * Use BTF_KFUNCS_START/END instead of flags * Rename BTF_SET8_KFUNC to BTF_SET8_KFUNCS Daniel Xu (3): bpf: btf: Support flags for BTF_SET8 sets bpf: btf: Add BTF_KFUNCS_START/END macro pair bpf: treewide: Annotate BPF kfuncs in BTF Documentation/bpf/kfuncs.rst | 8 +++---- drivers/hid/bpf/hid_bpf_dispatch.c | 8 +++---- fs/verity/measure.c | 4 ++-- include/linux/btf_ids.h | 21 +++++++++++++++---- kernel/bpf/btf.c | 8 +++++++ kernel/bpf/cpumask.c | 4 ++-- kernel/bpf/helpers.c | 8 +++---- kernel/bpf/map_iter.c | 4 ++-- kernel/cgroup/rstat.c | 4 ++-- kernel/trace/bpf_trace.c | 8 +++---- net/bpf/test_run.c | 8 +++---- net/core/filter.c | 20 +++++++++--------- net/core/xdp.c | 4 ++-- net/ipv4/bpf_tcp_ca.c | 4 ++-- net/ipv4/fou_bpf.c | 4 ++-- net/ipv4/tcp_bbr.c | 4 ++-- net/ipv4/tcp_cubic.c | 4 ++-- net/ipv4/tcp_dctcp.c | 4 ++-- net/netfilter/nf_conntrack_bpf.c | 4 ++-- net/netfilter/nf_nat_bpf.c | 4 ++-- net/xfrm/xfrm_interface_bpf.c | 4 ++-- net/xfrm/xfrm_state_bpf.c | 4 ++-- .../selftests/bpf/bpf_testmod/bpf_testmod.c | 8 +++---- 23 files changed, 87 insertions(+), 66 deletions(-) -- 2.42.1 _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel