Re: [PATCH v2 bpf-next 12/17] libbpf: support extern resolution for BTF-defined maps in .maps section

On Thu, Apr 22, 2021 at 3:56 PM Yonghong Song [off-list ref] wrote:

On 4/16/21 1:23 PM, Andrii Nakryiko wrote:

quoted

Add extra logic to handle map externs (only BTF-defined maps are supported for
linking). Re-use the map parsing logic used during bpf_object__open(). Map
externs are currently restricted to always match complete map definition. So
all the specified attributes will be compared (down to pining, map_flags,
numa_node, etc). In the future this restriction might be relaxed with no
backwards compatibility issues. If any attribute is mismatched between extern
and actual map definition, linker will report an error, pointing out which one
mismatches.

The original intent was to allow for extern to specify attributes that matters
(to user) to enforce. E.g., if you specify just key information and omit
value, then any value fits. Similarly, it should have been possible to enforce
map_flags, pinning, and any other possible map attribute. Unfortunately, that
means that multiple externs can be only partially overlapping with each other,
which means linker would need to combine their type definitions to end up with
the most restrictive and fullest map definition. This requires an extra amount
of BTF manipulation which at this time was deemed unnecessary and would
require further extending generic BTF writer APIs. So that is left for future
follow ups, if there will be demand for that. But the idea seems intresting
and useful, so I want to document it here.

Weak definitions are also supported, but are pretty strict as well, just
like externs: all weak map definitions have to match exactly. In the follow up
patches this most probably will be relaxed, with __weak map definitions being
able to differ between each other (with non-weak definition always winning, of
course).

Signed-off-by: Andrii Nakryiko <andrii@kernel.org>

I think strict enforcement of extern/global map definitions is good.
If library want people will use its maps, it may put the map definition
into one of its headers and application can include and have
exact the same definition.

In a lot of cases yes. But imagine I, as BPF library creator, started
out with just a typical hashmap definition, and then decided to add
pinning and maybe map_flags BPF_F_NO_PREALLOC. Why would that change
necessitate extern definition? But as you said, library provider can
(and should) provide extern definition that will be kept 100% in sync,
so this is not something that I urgently want to change.

Acked-by: Yonghong Song <redacted>