Re: [PATCH] bpf: fix build for BPF preload when $(O) points to a relative path

2021-01-25 16:32 UTC-0800 ~ Andrii Nakryiko [off-list ref]

On Mon, Jan 25, 2021 at 7:49 AM Quentin Monnet [off-list ref] wrote:

quoted

Building the kernel with CONFIG_BPF_PRELOAD, and by providing a relative
path for the output directory, may fail with the following error:

  $ make O=build bindeb-pkg
  ...
  /.../linux/tools/scripts/Makefile.include:5: *** O=build does not exist.  Stop.
  make[7]: *** [/.../linux/kernel/bpf/preload/Makefile:9: kernel/bpf/preload/libbpf.a] Error 2
  make[6]: *** [/.../linux/scripts/Makefile.build:500: kernel/bpf/preload] Error 2
  make[5]: *** [/.../linux/scripts/Makefile.build:500: kernel/bpf] Error 2
  make[4]: *** [/.../linux/Makefile:1799: kernel] Error 2
  make[4]: *** Waiting for unfinished jobs....

In the case above, for the "bindeb-pkg" target, the error is produced by
the "dummy" check in Makefile.include, called from libbpf's Makefile.
This check changes directory to $(PWD) before checking for the existence
of $(O). But at this step we have $(PWD) pointing to "/.../linux/build",
and $(O) pointing to "build". So the Makefile.include tries in fact to
assert the existence of a directory named "/.../linux/build/build",
which does not exist.

By contrast, other tools called from the main Linux Makefile get the
variable set to $(abspath $(objtree)), where $(objtree) is ".". We can
update the Makefile for kernel/bpf/preload to set $(O) to the same
value, to permit compiling with a relative path for output. Note that
apart from the Makefile.include, the variable $(O) is not used in
libbpf's build system.

Note that the error does not occur for all make targets and
architectures combinations.

- On x86, "make O=build vmlinux" appears to work fine.
  $(PWD) points to "/.../linux/tools", but $(O) points to the absolute
  path "/.../linux/build" and the test succeeds.
- On UML, it has been reported to fail with a message similar to the
  above (see [0]).
- On x86, "make O=build bindeb-pkg" fails, as described above.

It is unsure where the different values for $(O) and $(PWD) come from
(likely some recursive make with different arguments at some point), and
because several targets are broken, it feels safer to fix the $(O) value
passed to libbpf rather than to hunt down all changes to the variable.

David Gow previously posted a slightly different version of this patch
as a RFC [0], two months ago or so.

[0] https://lore.kernel.org/bpf/20201119085022.3606135-1-davidgow@google.com/t/#u (local)

Cc: Andrii Nakryiko <redacted>
Cc: Brendan Higgins <redacted>
Cc: David Gow <redacted>
Reported-by: David Gow <redacted>
Signed-off-by: Quentin Monnet <redacted>
---

I still think it would benefit everyone to figure out where this is
breaking (given Linux Makefile explicitly tries to handle such
relative path situation for O=, I believe), but this is trivial
enough, so:

Acked-by: Andrii Nakryiko <andrii@kernel.org>

Agreed, I'll try to spend a bit more time on this when I can. But it
would be nice to have the fix in the meantime. Thanks for the review and
ack.

BTW, you haven't specified which tree you intended it for.

Oops! I _knew_ I was missing something, sorry. This build issue is here
since eBPF preload was introduced, so I meant to send to the *bpf* tree.

Because it does not concern the major build targets, I was not sure if a
"Fixes:" tag would be appropriate. If we want one, it should be for
d71fa5c9763c ("bpf: Add kernel module with user mode driver that
populates bpffs.")

quoted

 kernel/bpf/preload/Makefile | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/kernel/bpf/preload/Makefile b/kernel/bpf/preload/Makefile
index 23ee310b6eb4..11b9896424c0 100644
--- a/kernel/bpf/preload/Makefile
+++ b/kernel/bpf/preload/Makefile

@@ -4,8 +4,11 @@ LIBBPF_SRCS = $(srctree)/tools/lib/bpf/
 LIBBPF_A = $(obj)/libbpf.a
 LIBBPF_OUT = $(abspath $(obj))

+# Set $(O) so that the "dummy" test in tools/scripts/Makefile.include, called
+# by libbpf's Makefile, succeeds when building the kernel with $(O) pointing to
+# a relative path, as in "make O=build bindeb-pkg".
 $(LIBBPF_A):
-       $(Q)$(MAKE) -C $(LIBBPF_SRCS) OUTPUT=$(LIBBPF_OUT)/ $(LIBBPF_OUT)/libbpf.a
+       $(Q)$(MAKE) -C $(LIBBPF_SRCS) O=$(abspath .) OUTPUT=$(LIBBPF_OUT)/ $(LIBBPF_OUT)/libbpf.a

why not O=$(LIBBPF_OUT), btw?

$(LIBBPF_OUT) points to /.../linux/ZZ_BUILD/build/kernel/bpf/preload.
This is an absolute path so the "dummy" check should work, too. I
preferred to align the value on the root Makefile, which has
"O=$(abspath $(objtree))" for target "tools/%", but no strong opinion
here. David would simply empty the variable in his patch. I'm fine with
any of the three versions. Would you prefer me to resend with $(LIBBPF_OUT)?

Thanks,
Quentin