Re: [PATCH bpf-next v2 0/9] install libbpf headers when using the library

2021-10-02 21:40 UTC+0100 ~ Quentin Monnet [off-list ref]

On Sat, 2 Oct 2021 at 00:05, Andrii Nakryiko [off-list ref] wrote:

quoted

On Fri, Oct 1, 2021 at 4:09 AM Quentin Monnet [off-list ref] wrote:

quoted

Libbpf is used at several locations in the repository. Most of the time,
the tools relying on it build the library in its own directory, and include
the headers from there. This works, but this is not the cleanest approach.
It generates objects outside of the directory of the tool which is being
built, and it also increases the risk that developers include a header file
internal to libbpf, which is not supposed to be exposed to user
applications.

This set adjusts all involved Makefiles to make sure that libbpf is built
locally (with respect to the tool's directory or provided build directory),
and by ensuring that "make install_headers" is run from libbpf's Makefile
to export user headers properly.

This comes at a cost: given that the libbpf was so far mostly compiled in
its own directory by the different components using it, compiling it once
would be enough for all those components. With the new approach, each
component compiles its own version. To mitigate this cost, efforts were
made to reuse the compiled library when possible:

- Make the bpftool version in samples/bpf reuse the library previously
  compiled for the selftests.
- Make the bpftool version in BPF selftests reuse the library previously
  compiled for the selftests.
- Similarly, make resolve_btfids in BPF selftests reuse the same compiled
  library.
- Similarly, make runqslower in BPF selftests reuse the same compiled
  library; and make it rely on the bpftool version also compiled from the
  selftests (instead of compiling its own version).
- runqslower, when compiled independently, needs its own version of
  bpftool: make them share the same compiled libbpf.

As a result:

- Compiling the samples/bpf should compile libbpf just once.
- Compiling the BPF selftests should compile libbpf just once.
- Compiling the kernel (with BTF support) should now lead to compiling
  libbpf twice: one for resolve_btfids, one for kernel/bpf/preload.
- Compiling runqslower individually should compile libbpf just once. Same
  thing for bpftool, resolve_btfids, and kernel/bpf/preload/iterators.

The whole sharing of libbpf build artifacts is great, I just want to
point out that it's also dangerous if those multiple Makefiles aren't
ordered properly. E.g., if you build runqslower and the rest of
selftests in parallel without making sure that libbpf already
completed its build, you might end up building libbpf in parallel in
two independent make instances and subsequently corrupting generated
object files. I haven't looked through all the changes (and I'll
confess that it's super hard to reason about dependencies and ordering
in Makefile) and I'll keep this in mind, but wanted to bring this up.

I'm not sure how Makefile handles this exactly, I don't know if it can
possibly build the two in parallel or if it's smart enough to realise
that the libbpf.a is the same object in both cases and should be built
only once. Same as you, I didn't hit any issue of this kind when
testing the patches.

Testing further with make, I don't think it's smart enough to avoid
building the object twice in the same directory. This probably didn't
show in practice because the parallel build tends to build the different
object files in parallel rather than libbpf and bpftool in parallel. But
to remain on the safe side, I'll just add a dependency on libbpf for
bpftool/runqslower etc. to make sure they are not all built together
(most locations had the dependency already). v3 is coming.

Quentin