Add BPF static linker support for extern resolution of global variables,
functions, and BTF-defined maps.
This patch set consists of 4 parts:
- few patches are extending bpftool to simplify working with BTF dump;
- libbpf object loading logic is extended to support __hidden functions and
overriden (unused) __weak functions; also BTF-defined map parsing logic is
refactored to be re-used by linker;
- the crux of the patch set is BPF static linker logic extension to perform
extern resolution for three categories: global variables, BPF
sub-programs, BTF-defined maps;
- a set of selftests that validate that all the combinations of
extern/weak/__hidden are working as expected.
See respective patches for more details.
One aspect hasn't been addressed yet and is going to be resolved in the next
patch set, but is worth mentioning. With BPF static linking of multiple .o
files, dealing with static everything becomes more problematic for BPF
skeleton and in general for any by name look up APIs. This is due to static
entities are allowed to have non-unique name. Historically this was never
a problem due to BPF programs were always confined to a single C file. That
changes now and needs to be addressed. The thinking so far is for BPF static
linker to prepend filename to each static variable and static map (which is
currently not supported by libbpf, btw), so that they can be unambiguously
resolved by (mostly) unique name. Mostly, because even filenames can be
duplicated, but that should be rare and easy to address by wiser choice of
filenames by users. Fortunately, static BPF subprograms don't suffer from this
issues, as they are not independent entities and are neither exposed in BPF
skeleton, nor is lookup-able by any of libbpf APIs (and there is little reason
to do that anyways).
This and few other things will be the topic of the next set of patches.
Some tests rely on Clang fix ([0]), so need latest Clang built from main.
[0] https://reviews.llvm.org/D100362
v1->v2:
- make map externs support full attribute list, adjust linked_maps selftest
to demonstrate that typedef works now (though no shared header file was
added to simplicity sake) (Alexei);
- remove commented out parts from selftests and fix few minor code style
issues;
- special __weak map definition semantics not yet implemented and will be
addressed in a follow up.
Andrii Nakryiko (17):
bpftool: support dumping BTF VAR's "extern" linkage
bpftool: dump more info about DATASEC members
libbpf: suppress compiler warning when using SEC() macro with externs
libbpf: mark BPF subprogs with hidden visibility as static for BPF
verifier
libbpf: allow gaps in BPF program sections to support overriden weak
functions
libbpf: refactor BTF map definition parsing
libbpf: factor out symtab and relos sanity checks
libbpf: make few internal helpers available outside of libbpf.c
libbpf: extend sanity checking ELF symbols with externs validation
libbpf: tighten BTF type ID rewriting with error checking
libbpf: add linker extern resolution support for functions and global
variables
libbpf: support extern resolution for BTF-defined maps in .maps
section
selftests/bpf: use -O0 instead of -Og in selftests builds
selftests/bpf: omit skeleton generation for multi-linked BPF object
files
selftests/bpf: add function linking selftest
selftests/bpf: add global variables linking selftest
sleftests/bpf: add map linking selftest
tools/bpf/bpftool/btf.c | 30 +-
tools/lib/bpf/bpf_helpers.h | 19 +-
tools/lib/bpf/btf.c | 5 -
tools/lib/bpf/libbpf.c | 370 +++--
tools/lib/bpf/libbpf_internal.h | 45 +
tools/lib/bpf/linker.c | 1261 ++++++++++++++---
tools/testing/selftests/bpf/Makefile | 18 +-
.../selftests/bpf/prog_tests/linked_funcs.c | 42 +
.../selftests/bpf/prog_tests/linked_maps.c | 30 +
.../selftests/bpf/prog_tests/linked_vars.c | 43 +
.../selftests/bpf/progs/linked_funcs1.c | 73 +
.../selftests/bpf/progs/linked_funcs2.c | 73 +
.../selftests/bpf/progs/linked_maps1.c | 82 ++
.../selftests/bpf/progs/linked_maps2.c | 76 +
.../selftests/bpf/progs/linked_vars1.c | 54 +
.../selftests/bpf/progs/linked_vars2.c | 55 +
16 files changed, 1922 insertions(+), 354 deletions(-)
create mode 100644 tools/testing/selftests/bpf/prog_tests/linked_funcs.c
create mode 100644 tools/testing/selftests/bpf/prog_tests/linked_maps.c
create mode 100644 tools/testing/selftests/bpf/prog_tests/linked_vars.c
create mode 100644 tools/testing/selftests/bpf/progs/linked_funcs1.c
create mode 100644 tools/testing/selftests/bpf/progs/linked_funcs2.c
create mode 100644 tools/testing/selftests/bpf/progs/linked_maps1.c
create mode 100644 tools/testing/selftests/bpf/progs/linked_maps2.c
create mode 100644 tools/testing/selftests/bpf/progs/linked_vars1.c
create mode 100644 tools/testing/selftests/bpf/progs/linked_vars2.c
--
2.30.2
Dump succinct information for each member of DATASEC: its kinds and name. This
is extremely helpful to see at a quick glance what is inside each DATASEC of
a given BTF. Without this, one has to jump around BTF data to just find out
the name of a VAR or FUNC. DATASEC's var_secinfo member is special in that
regard because it doesn't itself contain the name of the member, delegating
that to the referenced VAR and FUNC kinds. Other kinds, like
STRUCT/UNION/FUNC/ENUM, encode member names directly and thus are clearly
identifiable in BTF dump.
The new output looks like this:
[35] DATASEC '.bss' size=0 vlen=6
type_id=8 offset=0 size=4 (VAR 'input_bss1')
type_id=13 offset=0 size=4 (VAR 'input_bss_weak')
type_id=16 offset=0 size=4 (VAR 'output_bss1')
type_id=17 offset=0 size=4 (VAR 'output_data1')
type_id=18 offset=0 size=4 (VAR 'output_rodata1')
type_id=20 offset=0 size=8 (VAR 'output_sink1')
[36] DATASEC '.data' size=0 vlen=2
type_id=9 offset=0 size=4 (VAR 'input_data1')
type_id=14 offset=0 size=4 (VAR 'input_data_weak')
[37] DATASEC '.kconfig' size=0 vlen=2
type_id=25 offset=0 size=4 (VAR 'LINUX_KERNEL_VERSION')
type_id=28 offset=0 size=1 (VAR 'CONFIG_BPF_SYSCALL')
[38] DATASEC '.ksyms' size=0 vlen=1
type_id=30 offset=0 size=1 (VAR 'bpf_link_fops')
[39] DATASEC '.rodata' size=0 vlen=2
type_id=12 offset=0 size=4 (VAR 'input_rodata1')
type_id=15 offset=0 size=4 (VAR 'input_rodata_weak')
[40] DATASEC 'license' size=0 vlen=1
type_id=24 offset=0 size=4 (VAR 'LICENSE')
Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
---
tools/bpf/bpftool/btf.c | 26 ++++++++++++++++++--------
1 file changed, 18 insertions(+), 8 deletions(-)
When used on externs SEC() macro will trigger compilation warning about
inapplicable `__attribute__((used))`. That's expected for extern declarations,
so suppress it with the corresponding _Pragma.
Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
---
tools/lib/bpf/bpf_helpers.h | 11 +++++++++--
1 file changed, 9 insertions(+), 2 deletions(-)
Add dumping of "extern" linkage for BTF VAR kind. Also shorten
"global-allocated" to "global" to be in line with FUNC's "global".
Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
---
tools/bpf/bpftool/btf.c | 4 +++-
1 file changed, 3 insertions(+), 1 deletion(-)
Define __hidden helper macro in bpf_helpers.h, which is a short-hand for
__attribute__((visibility("hidden"))). Add libbpf support to mark BPF
subprograms marked with __hidden as static in BTF information to enforce BPF
verifier's static function validation algorithm, which takes more information
(caller's context) into account during a subprogram validation.
Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
---
tools/lib/bpf/bpf_helpers.h | 8 ++++++
tools/lib/bpf/btf.c | 5 ----
tools/lib/bpf/libbpf.c | 45 ++++++++++++++++++++++++++++++++-
tools/lib/bpf/libbpf_internal.h | 6 +++++
4 files changed, 58 insertions(+), 6 deletions(-)
@@ -698,6 +700,15 @@ bpf_object__add_programs(struct bpf_object *obj, Elf_Data *sec_data,if(err)returnerr;+/* if function is a global/weak symbol, but has hidden+*visibility(oranynon-defaultone),markitsBTFFUNCas+*statictoenablemorepermissiveBPFverificationmodewith+*moreoutsidecontextavailabletoBPFverifier+*/+if(GELF_ST_BIND(sym.st_info)!=STB_LOCAL+&&GELF_ST_VISIBILITY(sym.st_other)!=STV_DEFAULT)+prog->mark_btf_static=true;+nr_progs++;obj->nr_programs=nr_progs;
@@ -2618,7 +2629,7 @@ static int bpf_object__sanitize_and_load_btf(struct bpf_object *obj){structbtf*kern_btf=obj->btf;boolbtf_mandatory,sanitize;-interr=0;+inti,err=0;if(!obj->btf)return0;
@@ -2632,6 +2643,38 @@ static int bpf_object__sanitize_and_load_btf(struct bpf_object *obj)return0;}+/* Even though some subprogs are global/weak, user might prefer more+*permissiveBPFverificationprocessthatBPFverifierperformsfor+*staticfunctions,takingintoaccountmorecontextfromthecaller+*functions.Insuchcase,theyneedtomarksuchsubprogswith+*__attribute__((visibility("hidden")))andlibbpfwilladjust+*correspondingFUNCBTFtypetobemarkedasstaticandtriggermore+*involvedBPFverificationprocess.+*/+for(i=0;i<obj->nr_programs;i++){+structbpf_program*prog=&obj->programs[i];+structbtf_type*t;+constchar*name;+intj,n;++if(!prog->mark_btf_static||!prog_is_subprog(obj,prog))+continue;++n=btf__get_nr_types(obj->btf);+for(j=1;j<=n;j++){+t=btf_type_by_id(obj->btf,j);+if(!btf_is_func(t)||btf_func_linkage(t)!=BTF_FUNC_GLOBAL)+continue;++name=btf__str_by_offset(obj->btf,t->name_off);+if(strcmp(name,prog->name)!=0)+continue;++t->info=btf_type_info(BTF_KIND_FUNC,BTF_FUNC_STATIC,0);+break;+}+}+sanitize=btf_needs_sanitization(obj);if(sanitize){constvoid*raw_data;
Currently libbpf is very strict about parsing BPF program isnstruction
sections. No gaps are allowed between sequential BPF programs within a given
ELF section. Libbpf enforced that by keeping track of the next section offset
that should start a new BPF (sub)program and cross-checks that by searching for
a corresponding STT_FUNC ELF symbol.
But this is too restrictive once we allow to have weak BPF programs and link
together two or more BPF object files. In such case, some weak BPF programs
might be "overriden" by either non-weak BPF program with the same name and
signature, or even by another weak BPF program that just happened to be linked
first. That, in turn, leaves BPF instructions of the "lost" BPF (sub)program
intact, but there is no corresponding ELF symbol, because no one is going to
be referencing it.
Libbpf already correctly handles such cases in the sense that it won't append
such dead code to actual BPF programs loaded into kernel. So the only change
that needs to be done is to relax the logic of parsing BPF instruction
sections. Instead of assuming next BPF (sub)program section offset, iterate
available STT_FUNC ELF symbols to discover all available BPF subprograms and
programs.
Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
---
tools/lib/bpf/libbpf.c | 56 ++++++++++++++++--------------------------
1 file changed, 21 insertions(+), 35 deletions(-)
@@ -655,14 +655,16 @@ bpf_object__add_programs(struct bpf_object *obj, Elf_Data *sec_data,nr_progs=obj->nr_programs;sec_off=0;-while(sec_off<sec_sz){-if(elf_sym_by_sec_off(obj,sec_idx,sec_off,STT_FUNC,&sym)){-pr_warn("sec '%s': failed to find program symbol at offset %zu\n",-sec_name,sec_off);-return-LIBBPF_ERRNO__FORMAT;-}+for(i=0;i<n;i++){+if(!gelf_getsym(symbols,i,&sym))+continue;+if(sym.st_shndx!=sec_idx)+continue;+if(GELF_ST_TYPE(sym.st_info)!=STT_FUNC)+continue;prog_sz=sym.st_size;+sec_off=sym.st_value;name=elf_sym_str(obj,sym.st_name);if(!name){
@@ -2825,26 +2825,6 @@ static Elf_Data *elf_sec_data(const struct bpf_object *obj, Elf_Scn *scn)returndata;}-staticintelf_sym_by_sec_off(conststructbpf_object*obj,size_tsec_idx,-size_toff,__u32sym_type,GElf_Sym*sym)-{-Elf_Data*symbols=obj->efile.symbols;-size_tn=symbols->d_size/sizeof(GElf_Sym);-inti;--for(i=0;i<n;i++){-if(!gelf_getsym(symbols,i,sym))-continue;-if(sym->st_shndx!=sec_idx||sym->st_value!=off)-continue;-if(GELF_ST_TYPE(sym->st_info)!=sym_type)-continue;-return0;-}--return-ENOENT;-}-staticboolis_sec_name_dwarf(constchar*name){/* approximation, but the actual list is too long */
@@ -3769,9 +3755,9 @@ bpf_object__collect_prog_relos(struct bpf_object *obj, GElf_Shdr *shdr, Elf_Dataprog=find_prog_by_sec_insn(obj,sec_idx,insn_idx);if(!prog){-pr_warn("sec '%s': relo #%d: program not found in section '%s' for insn #%u\n",+pr_debug("sec '%s': relo #%d: couldn't find program in section '%s' for insn #%u, probably overridden weak function, skipping...\n",relo_sec_name,i,sec_name,insn_idx);-return-LIBBPF_ERRNO__RELOC;+continue;}relos=libbpf_reallocarray(prog->reloc_desc,
Refactor BTF-defined maps parsing logic to allow it to be nicely reused by BPF
static linker. Further, at least for BPF static linker, it's important to know
which attributes of a BPF map were defined explicitly, so provide a bit set
for each known portion of BTF map definition. This allows BPF static linker to
do a simple check when dealing with extern map declarations.
The same capabilities allow to distinguish attributes explicitly set to zero
(e.g., __uint(max_entries, 0)) vs the case of not specifying it at all (no
max_entries attribute at all). Libbpf is currently not utilizing that, but it
could be useful for backwards compatibility reasons later.
Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
---
tools/lib/bpf/libbpf.c | 256 ++++++++++++++++++--------------
tools/lib/bpf/libbpf_internal.h | 32 ++++
2 files changed, 177 insertions(+), 111 deletions(-)
@@ -2025,255 +2025,262 @@ static int build_map_pin_path(struct bpf_map *map, const char *path)returnbpf_map__set_pin_path(map,buf);}--staticintparse_btf_map_def(structbpf_object*obj,-structbpf_map*map,-conststructbtf_type*def,-boolstrict,boolis_inner,-constchar*pin_root_path)+intparse_btf_map_def(constchar*map_name,structbtf*btf,+conststructbtf_type*def_t,boolstrict,+structbtf_map_def*map_def,structbtf_map_def*inner_def){conststructbtf_type*t;conststructbtf_member*m;+boolis_inner=inner_def==NULL;intvlen,i;-vlen=btf_vlen(def);-m=btf_members(def);+vlen=btf_vlen(def_t);+m=btf_members(def_t);for(i=0;i<vlen;i++,m++){-constchar*name=btf__name_by_offset(obj->btf,m->name_off);+constchar*name=btf__name_by_offset(btf,m->name_off);if(!name){-pr_warn("map '%s': invalid field #%d.\n",map->name,i);+pr_warn("map '%s': invalid field #%d.\n",map_name,i);return-EINVAL;}if(strcmp(name,"type")==0){-if(!get_map_field_int(map->name,obj->btf,m,-&map->def.type))+if(!get_map_field_int(map_name,btf,m,&map_def->map_type))return-EINVAL;-pr_debug("map '%s': found type = %u.\n",-map->name,map->def.type);+map_def->parts|=MAP_DEF_MAP_TYPE;}elseif(strcmp(name,"max_entries")==0){-if(!get_map_field_int(map->name,obj->btf,m,-&map->def.max_entries))+if(!get_map_field_int(map_name,btf,m,&map_def->max_entries))return-EINVAL;-pr_debug("map '%s': found max_entries = %u.\n",-map->name,map->def.max_entries);+map_def->parts|=MAP_DEF_MAX_ENTRIES;}elseif(strcmp(name,"map_flags")==0){-if(!get_map_field_int(map->name,obj->btf,m,-&map->def.map_flags))+if(!get_map_field_int(map_name,btf,m,&map_def->map_flags))return-EINVAL;-pr_debug("map '%s': found map_flags = %u.\n",-map->name,map->def.map_flags);+map_def->parts|=MAP_DEF_MAP_FLAGS;}elseif(strcmp(name,"numa_node")==0){-if(!get_map_field_int(map->name,obj->btf,m,&map->numa_node))+if(!get_map_field_int(map_name,btf,m,&map_def->numa_node))return-EINVAL;-pr_debug("map '%s': found numa_node = %u.\n",map->name,map->numa_node);+map_def->parts|=MAP_DEF_NUMA_NODE;}elseif(strcmp(name,"key_size")==0){__u32sz;-if(!get_map_field_int(map->name,obj->btf,m,&sz))+if(!get_map_field_int(map_name,btf,m,&sz))return-EINVAL;-pr_debug("map '%s': found key_size = %u.\n",-map->name,sz);-if(map->def.key_size&&map->def.key_size!=sz){+if(map_def->key_size&&map_def->key_size!=sz){pr_warn("map '%s': conflicting key size %u != %u.\n",-map->name,map->def.key_size,sz);+map_name,map_def->key_size,sz);return-EINVAL;}-map->def.key_size=sz;+map_def->key_size=sz;+map_def->parts|=MAP_DEF_KEY_SIZE;}elseif(strcmp(name,"key")==0){__s64sz;-t=btf__type_by_id(obj->btf,m->type);+t=btf__type_by_id(btf,m->type);if(!t){pr_warn("map '%s': key type [%d] not found.\n",-map->name,m->type);+map_name,m->type);return-EINVAL;}if(!btf_is_ptr(t)){pr_warn("map '%s': key spec is not PTR: %s.\n",-map->name,btf_kind_str(t));+map_name,btf_kind_str(t));return-EINVAL;}-sz=btf__resolve_size(obj->btf,t->type);+sz=btf__resolve_size(btf,t->type);if(sz<0){pr_warn("map '%s': can't determine key size for type [%u]: %zd.\n",-map->name,t->type,(ssize_t)sz);+map_name,t->type,(ssize_t)sz);returnsz;}-pr_debug("map '%s': found key [%u], sz = %zd.\n",-map->name,t->type,(ssize_t)sz);-if(map->def.key_size&&map->def.key_size!=sz){+if(map_def->key_size&&map_def->key_size!=sz){pr_warn("map '%s': conflicting key size %u != %zd.\n",-map->name,map->def.key_size,(ssize_t)sz);+map_name,map_def->key_size,(ssize_t)sz);return-EINVAL;}-map->def.key_size=sz;-map->btf_key_type_id=t->type;+map_def->key_size=sz;+map_def->key_type_id=t->type;+map_def->parts|=MAP_DEF_KEY_SIZE|MAP_DEF_KEY_TYPE;}elseif(strcmp(name,"value_size")==0){__u32sz;-if(!get_map_field_int(map->name,obj->btf,m,&sz))+if(!get_map_field_int(map_name,btf,m,&sz))return-EINVAL;-pr_debug("map '%s': found value_size = %u.\n",-map->name,sz);-if(map->def.value_size&&map->def.value_size!=sz){+if(map_def->value_size&&map_def->value_size!=sz){pr_warn("map '%s': conflicting value size %u != %u.\n",-map->name,map->def.value_size,sz);+map_name,map_def->value_size,sz);return-EINVAL;}-map->def.value_size=sz;+map_def->value_size=sz;+map_def->parts|=MAP_DEF_VALUE_SIZE;}elseif(strcmp(name,"value")==0){__s64sz;-t=btf__type_by_id(obj->btf,m->type);+t=btf__type_by_id(btf,m->type);if(!t){pr_warn("map '%s': value type [%d] not found.\n",-map->name,m->type);+map_name,m->type);return-EINVAL;}if(!btf_is_ptr(t)){pr_warn("map '%s': value spec is not PTR: %s.\n",-map->name,btf_kind_str(t));+map_name,btf_kind_str(t));return-EINVAL;}-sz=btf__resolve_size(obj->btf,t->type);+sz=btf__resolve_size(btf,t->type);if(sz<0){pr_warn("map '%s': can't determine value size for type [%u]: %zd.\n",-map->name,t->type,(ssize_t)sz);+map_name,t->type,(ssize_t)sz);returnsz;}-pr_debug("map '%s': found value [%u], sz = %zd.\n",-map->name,t->type,(ssize_t)sz);-if(map->def.value_size&&map->def.value_size!=sz){+if(map_def->value_size&&map_def->value_size!=sz){pr_warn("map '%s': conflicting value size %u != %zd.\n",-map->name,map->def.value_size,(ssize_t)sz);+map_name,map_def->value_size,(ssize_t)sz);return-EINVAL;}-map->def.value_size=sz;-map->btf_value_type_id=t->type;+map_def->value_size=sz;+map_def->value_type_id=t->type;+map_def->parts|=MAP_DEF_VALUE_SIZE|MAP_DEF_VALUE_TYPE;}elseif(strcmp(name,"values")==0){+charinner_map_name[128];interr;if(is_inner){pr_warn("map '%s': multi-level inner maps not supported.\n",-map->name);+map_name);return-ENOTSUP;}if(i!=vlen-1){pr_warn("map '%s': '%s' member should be last.\n",-map->name,name);+map_name,name);return-EINVAL;}-if(!bpf_map_type__is_map_in_map(map->def.type)){+if(!bpf_map_type__is_map_in_map(map_def->map_type)){pr_warn("map '%s': should be map-in-map.\n",-map->name);+map_name);return-ENOTSUP;}-if(map->def.value_size&&map->def.value_size!=4){+if(map_def->value_size&&map_def->value_size!=4){pr_warn("map '%s': conflicting value size %u != 4.\n",-map->name,map->def.value_size);+map_name,map_def->value_size);return-EINVAL;}-map->def.value_size=4;-t=btf__type_by_id(obj->btf,m->type);+map_def->value_size=4;+t=btf__type_by_id(btf,m->type);if(!t){pr_warn("map '%s': map-in-map inner type [%d] not found.\n",-map->name,m->type);+map_name,m->type);return-EINVAL;}if(!btf_is_array(t)||btf_array(t)->nelems){pr_warn("map '%s': map-in-map inner spec is not a zero-sized array.\n",-map->name);+map_name);return-EINVAL;}-t=skip_mods_and_typedefs(obj->btf,btf_array(t)->type,-NULL);+t=skip_mods_and_typedefs(btf,btf_array(t)->type,NULL);if(!btf_is_ptr(t)){pr_warn("map '%s': map-in-map inner def is of unexpected kind %s.\n",-map->name,btf_kind_str(t));+map_name,btf_kind_str(t));return-EINVAL;}-t=skip_mods_and_typedefs(obj->btf,t->type,NULL);+t=skip_mods_and_typedefs(btf,t->type,NULL);if(!btf_is_struct(t)){pr_warn("map '%s': map-in-map inner def is of unexpected kind %s.\n",-map->name,btf_kind_str(t));+map_name,btf_kind_str(t));return-EINVAL;}-map->inner_map=calloc(1,sizeof(*map->inner_map));-if(!map->inner_map)-return-ENOMEM;-map->inner_map->fd=-1;-map->inner_map->sec_idx=obj->efile.btf_maps_shndx;-map->inner_map->name=malloc(strlen(map->name)+-sizeof(".inner")+1);-if(!map->inner_map->name)-return-ENOMEM;-sprintf(map->inner_map->name,"%s.inner",map->name);--err=parse_btf_map_def(obj,map->inner_map,t,strict,-true/* is_inner */,NULL);+snprintf(inner_map_name,sizeof(inner_map_name),"%s.inner",map_name);+err=parse_btf_map_def(inner_map_name,btf,t,strict,inner_def,NULL);if(err)returnerr;++map_def->parts|=MAP_DEF_INNER_MAP;}elseif(strcmp(name,"pinning")==0){__u32val;-interr;if(is_inner){-pr_debug("map '%s': inner def can't be pinned.\n",-map->name);+pr_warn("map '%s': inner def can't be pinned.\n",map_name);return-EINVAL;}-if(!get_map_field_int(map->name,obj->btf,m,&val))+if(!get_map_field_int(map_name,btf,m,&val))return-EINVAL;-pr_debug("map '%s': found pinning = %u.\n",-map->name,val);--if(val!=LIBBPF_PIN_NONE&&-val!=LIBBPF_PIN_BY_NAME){+if(val!=LIBBPF_PIN_NONE&&val!=LIBBPF_PIN_BY_NAME){pr_warn("map '%s': invalid pinning value %u.\n",-map->name,val);+map_name,val);return-EINVAL;}-if(val==LIBBPF_PIN_BY_NAME){-err=build_map_pin_path(map,pin_root_path);-if(err){-pr_warn("map '%s': couldn't build pin path.\n",-map->name);-returnerr;-}-}+map_def->pinning=val;+map_def->parts|=MAP_DEF_PINNING;}else{if(strict){-pr_warn("map '%s': unknown field '%s'.\n",-map->name,name);+pr_warn("map '%s': unknown field '%s'.\n",map_name,name);return-ENOTSUP;}-pr_debug("map '%s': ignoring unknown field '%s'.\n",-map->name,name);+pr_debug("map '%s': ignoring unknown field '%s'.\n",map_name,name);}}-if(map->def.type==BPF_MAP_TYPE_UNSPEC){-pr_warn("map '%s': map type isn't specified.\n",map->name);+if(map_def->map_type==BPF_MAP_TYPE_UNSPEC){+pr_warn("map '%s': map type isn't specified.\n",map_name);return-EINVAL;}return0;}+staticvoidfill_map_from_def(structbpf_map*map,conststructbtf_map_def*def)+{+map->def.type=def->map_type;+map->def.key_size=def->key_size;+map->def.value_size=def->value_size;+map->def.max_entries=def->max_entries;+map->def.map_flags=def->map_flags;++map->numa_node=def->numa_node;+map->btf_key_type_id=def->key_type_id;+map->btf_value_type_id=def->value_type_id;++if(def->parts&MAP_DEF_MAP_TYPE)+pr_debug("map '%s': found type = %u.\n",map->name,def->map_type);++if(def->parts&MAP_DEF_KEY_TYPE)+pr_debug("map '%s': found key [%u], sz = %u.\n",+map->name,def->key_type_id,def->key_size);+elseif(def->parts&MAP_DEF_KEY_SIZE)+pr_debug("map '%s': found key_size = %u.\n",map->name,def->key_size);++if(def->parts&MAP_DEF_VALUE_TYPE)+pr_debug("map '%s': found value [%u], sz = %u.\n",+map->name,def->value_type_id,def->value_size);+elseif(def->parts&MAP_DEF_VALUE_SIZE)+pr_debug("map '%s': found value_size = %u.\n",map->name,def->value_size);++if(def->parts&MAP_DEF_MAX_ENTRIES)+pr_debug("map '%s': found max_entries = %u.\n",map->name,def->max_entries);+if(def->parts&MAP_DEF_MAP_FLAGS)+pr_debug("map '%s': found map_flags = %u.\n",map->name,def->map_flags);+if(def->parts&MAP_DEF_PINNING)+pr_debug("map '%s': found pinning = %u.\n",map->name,def->pinning);+if(def->parts&MAP_DEF_NUMA_NODE)+pr_debug("map '%s': found numa_node = %u.\n",map->name,def->numa_node);++if(def->parts&MAP_DEF_INNER_MAP)+pr_debug("map '%s': found inner map definition.\n",map->name);+}+staticintbpf_object__init_user_btf_map(structbpf_object*obj,conststructbtf_type*sec,intvar_idx,intsec_idx,constElf_Data*data,boolstrict,constchar*pin_root_path){+structbtf_map_defmap_def={},inner_def={};conststructbtf_type*var,*def;conststructbtf_var_secinfo*vi;conststructbtf_var*var_extra;constchar*map_name;structbpf_map*map;+interr;vi=btf_var_secinfos(sec)+var_idx;var=btf__type_by_id(obj->btf,vi->type);
@@ -2327,7 +2334,34 @@ static int bpf_object__init_user_btf_map(struct bpf_object *obj,pr_debug("map '%s': at sec_idx %d, offset %zu.\n",map_name,map->sec_idx,map->sec_offset);-returnparse_btf_map_def(obj,map,def,strict,false,pin_root_path);+err=parse_btf_map_def(map->name,obj->btf,def,strict,&map_def,&inner_def);+if(err)+returnerr;++fill_map_from_def(map,&map_def);++if(map_def.pinning==LIBBPF_PIN_BY_NAME){+err=build_map_pin_path(map,pin_root_path);+if(err){+pr_warn("map '%s': couldn't build pin path.\n",map->name);+returnerr;+}+}++if(map_def.parts&MAP_DEF_INNER_MAP){+map->inner_map=calloc(1,sizeof(*map->inner_map));+if(!map->inner_map)+return-ENOMEM;+map->inner_map->fd=-1;+map->inner_map->name=malloc(strlen(map_name)+sizeof(".inner")+1);+if(!map->inner_map->name)+return-ENOMEM;+sprintf(map->inner_map->name,"%s.inner",map_name);++fill_map_from_def(map->inner_map,&inner_def);+}++return0;}staticintbpf_object__init_user_btf_maps(structbpf_object*obj,boolstrict,
@@ -138,6 +138,38 @@ static inline __u32 btf_type_info(int kind, int vlen, int kflag)return(kflag<<31)|(kind<<24)|vlen;}+enummap_def_parts{+MAP_DEF_MAP_TYPE=0x001,+MAP_DEF_KEY_TYPE=0x002,+MAP_DEF_KEY_SIZE=0x004,+MAP_DEF_VALUE_TYPE=0x008,+MAP_DEF_VALUE_SIZE=0x010,+MAP_DEF_MAX_ENTRIES=0x020,+MAP_DEF_MAP_FLAGS=0x040,+MAP_DEF_NUMA_NODE=0x080,+MAP_DEF_PINNING=0x100,+MAP_DEF_INNER_MAP=0x200,++MAP_DEF_ALL=0x3ff,/* combination of all above */+};++structbtf_map_def{+enummap_def_partsparts;+__u32map_type;+__u32key_type_id;+__u32key_size;+__u32value_type_id;+__u32value_size;+__u32max_entries;+__u32map_flags;+__u32numa_node;+__u32pinning;+};++intparse_btf_map_def(constchar*map_name,structbtf*btf,+conststructbtf_type*def_t,boolstrict,+structbtf_map_def*map_def,structbtf_map_def*inner_def);+void*libbpf_add_mem(void**data,size_t*cap_cnt,size_telem_sz,size_tcur_cnt,size_tmax_cnt,size_tadd_cnt);intlibbpf_ensure_mem(void**data,size_t*cap_cnt,size_telem_sz,size_tneed_cnt);
Add logic to validate extern symbols, plus some other minor extra checks, like
ELF symbol #0 validation, general symbol visibility and binding validations.
Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
---
tools/lib/bpf/linker.c | 43 +++++++++++++++++++++++++++++++++---------
1 file changed, 34 insertions(+), 9 deletions(-)
@@ -750,14 +750,39 @@ static int linker_sanity_check_elf_symtab(struct src_obj *obj, struct src_sec *sn=sec->shdr->sh_size/sec->shdr->sh_entsize;sym=sec->data->d_buf;for(i=0;i<n;i++,sym++){-if(sym->st_shndx-&&sym->st_shndx<SHN_LORESERVE-&&sym->st_shndx>=obj->sec_cnt){+intsym_type=ELF64_ST_TYPE(sym->st_info);+intsym_bind=ELF64_ST_BIND(sym->st_info);++if(i==0){+if(sym->st_name!=0||sym->st_info!=0+||sym->st_other!=0||sym->st_shndx!=0+||sym->st_value!=0||sym->st_size!=0){+pr_warn("ELF sym #0 is invalid in %s\n",obj->filename);+return-EINVAL;+}+continue;+}+if(sym_bind!=STB_LOCAL&&sym_bind!=STB_GLOBAL&&sym_bind!=STB_WEAK){+pr_warn("ELF sym #%d is section #%zu has unsupported symbol binding %d\n",+i,sec->sec_idx,sym_bind);+return-EINVAL;+}+if(sym->st_shndx==0){+if(sym_type!=STT_NOTYPE||sym_bind==STB_LOCAL+||sym->st_value!=0||sym->st_size!=0){+pr_warn("ELF sym #%d is invalid extern symbol in %s\n",+i,obj->filename);++return-EINVAL;+}+continue;+}+if(sym->st_shndx<SHN_LORESERVE&&sym->st_shndx>=obj->sec_cnt){pr_warn("ELF sym #%d in section #%zu points to missing section #%zu in %s\n",i,sec->sec_idx,(size_t)sym->st_shndx,obj->filename);return-EINVAL;}-if(ELF64_ST_TYPE(sym->st_info)==STT_SECTION){+if(sym_type==STT_SECTION){if(sym->st_value!=0)return-EINVAL;continue;
@@ -1135,16 +1160,16 @@ static int linker_append_elf_syms(struct bpf_linker *linker, struct src_obj *objsize_tdst_sym_idx;intname_off;-/* we already have all-zero initial symbol */-if(sym->st_name==0&&sym->st_info==0&&-sym->st_other==0&&sym->st_shndx==SHN_UNDEF&&-sym->st_value==0&&sym->st_size==0)+/* We already validated all-zero symbol #0 and we already+*appendeditpreventivelytothefinalSYMTAB,soskipit.+*/+if(i==0)continue;sym_name=elf_strptr(obj->elf,str_sec_idx,sym->st_name);if(!sym_name){pr_warn("can't fetch symbol name for symbol #%d in '%s'\n",i,obj->filename);-return-1;+return-EINVAL;}if(sym->st_shndx&&sym->st_shndx<SHN_LORESERVE){
Add BPF static linker logic to resolve extern variables and functions across
multiple linked together BPF object files.
For that, linker maintains a separate list of struct glob_sym structures,
which keeps track of few pieces of metadata (is it extern or resolved global,
is it a weak symbol, which ELF section it belongs to, etc) and ties together
BTF type info and ELF symbol information and keeps them in sync.
With adding support for extern variables/funcs, it's now possible for some
sections to contain both extern and non-extern definitions. This means that
some sections may start out as ephemeral (if only externs are present and thus
there is not corresponding ELF section), but will be "upgraded" to actual ELF
section as symbols are resolved or new non-extern definitions are appended.
Additional care is taken to not duplicate extern entries in sections like
.kconfig and .ksyms.
Given libbpf requires BTF type to always be present for .kconfig/.ksym
externs, linker extends this requirement to all the externs, even those that
are supposed to be resolved during static linking and which won't be visible
to libbpf. With BTF information always present, static linker will check not
just ELF symbol matches, but entire BTF type signature match as well. That
logic is stricter that BPF CO-RE checks. It probably should be re-used by
.ksym resolution logic in libbpf as well, but that's left for follow up
patches.
To make it unnecessary to rewrite ELF symbols and minimize BTF type
rewriting/removal, ELF symbols that correspond to externs initially will be
updated in place once they are resolved. Similarly for BTF type info, VAR/FUNC
and var_secinfo's (sec_vars in struct bpf_linker) are staying stable, but
types they point to might get replaced when extern is resolved. This might
leave some left-over types (even though we try to minimize this for common
cases of having extern funcs with not argument names vs concrete function with
names properly specified). That can be addresses later with a generic BTF
garbage collection. That's left for a follow up as well.
Given BTF type appending phase is separate from ELF symbol
appending/resolution, special struct glob_sym->underlying_btf_id variable is
used to communicate resolution and rewrite decisions. 0 means
underlying_btf_id needs to be appended (it's not yet in final linker->btf), <0
values are used for temporary storage of source BTF type ID (not yet
rewritten), so -glob_sym->underlying_btf_id is BTF type id in obj-btf. But by
the end of linker_append_btf() phase, that underlying_btf_id will be remapped
and will always be > 0. This is the uglies part of the whole process, but
keeps the other parts much simpler due to stability of sec_var and VAR/FUNC
types, as well as ELF symbol, so please keep that in mind while reviewing.
BTF-defined maps require some extra custom logic and is addressed separate in
the next patch, so that to keep this one smaller and easier to review.
Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
---
tools/lib/bpf/linker.c | 844 ++++++++++++++++++++++++++++++++++++++---
1 file changed, 785 insertions(+), 59 deletions(-)
@@ -22,6 +22,8 @@#include"libbpf_internal.h"#include"strset.h"+#define BTF_EXTERN_SEC ".extern"+structsrc_sec{constchar*sec_name;/* positional (not necessarily ELF) index in an array of sections */
@@ -74,11 +76,36 @@ struct btf_ext_sec_data {void*recs;};+structglob_sym{+/* ELF symbol index */+intsym_idx;+/* associated section id for .ksyms, .kconfig, etc, but not .extern */+intsec_id;+/* extern name offset in STRTAB */+intname_off;+/* optional associated BTF type ID */+intbtf_id;+/* BTF type ID to which VAR/FUNC type is pointing to; used for+*rewritingtypeswhenexternVAR/FUNCisresolvedtoaconcrete+*definition+*/+intunderlying_btf_id;+/* sec_var index in the corresponding dst_sec, if exists */+intvar_idx;++/* extern or resolved/global symbol */+boolis_extern;+/* weak or strong symbol, never goes back from strong to weak */+boolis_weak;+};+structdst_sec{char*sec_name;/* positional (not necessarily ELF) index in an array of sections */intid;+boolephemeral;+/* ELF info */size_tsec_idx;Elf_Scn*scn;
@@ -136,6 +167,8 @@ static int linker_sanity_check_btf_ext(struct src_obj *obj);staticintlinker_fixup_btf(structsrc_obj*obj);staticintlinker_append_sec_data(structbpf_linker*linker,structsrc_obj*obj);staticintlinker_append_elf_syms(structbpf_linker*linker,structsrc_obj*obj);+staticintlinker_append_elf_sym(structbpf_linker*linker,structsrc_obj*obj,+Elf64_Sym*sym,constchar*sym_name,intsrc_sym_idx);staticintlinker_append_elf_relos(structbpf_linker*linker,structsrc_obj*obj);staticintlinker_append_btf(structbpf_linker*linker,structsrc_obj*obj);staticintlinker_append_btf_ext(structbpf_linker*linker,structsrc_obj*obj);
@@ -941,6 +974,7 @@ static int init_sec(struct bpf_linker *linker, struct dst_sec *dst_sec, struct sdst_sec->sec_sz=0;dst_sec->sec_idx=0;+dst_sec->ephemeral=src_sec->ephemeral;/* ephemeral sections are just thin section shells lacking most parts */if(src_sec->ephemeral)
@@ -1029,13 +1066,33 @@ static bool sec_content_is_same(struct dst_sec *dst_sec, struct src_sec *src_secreturntrue;}-staticintextend_sec(structdst_sec*dst,structsrc_sec*src)+staticintextend_sec(structbpf_linker*linker,structdst_sec*dst,structsrc_sec*src){void*tmp;-size_tdst_align=dst->shdr->sh_addralign;-size_tsrc_align=src->shdr->sh_addralign;+size_tdst_align,src_align;size_tdst_align_sz,dst_final_sz;+interr;++/* Ephemeral source section doesn't contribute anything to ELF+*sectiondata.+*/+if(src->ephemeral)+return0;++/* Some sections (like .maps) can contain both externs (and thus be+*ephemeral)andnon-externs(mapdefinitions).Soit'spossiblethat+*ithastobe"upgraded"fromephemeraltonon-ephemeralwhenthe+*firstnon-ephemeralentityappears.Insuchcase,weaddELF+*section,data,etc.+*/+if(dst->ephemeral){+err=init_sec(linker,dst,src);+if(err)+returnerr;+}+dst_align=dst->shdr->sh_addralign;+src_align=src->shdr->sh_addralign;if(dst_align==0)dst_align=1;if(dst_align<src_align)
@@ -1131,10 +1188,7 @@ static int linker_append_sec_data(struct bpf_linker *linker, struct src_obj *obj/* record mapped section index */src_sec->dst_id=dst_sec->id;-if(src_sec->ephemeral)-continue;--err=extend_sec(dst_sec,src_sec);+err=extend_sec(linker,dst_sec,src_sec);if(err)returnerr;}
@@ -1145,21 +1199,16 @@ static int linker_append_sec_data(struct bpf_linker *linker, struct src_obj *objstaticintlinker_append_elf_syms(structbpf_linker*linker,structsrc_obj*obj){structsrc_sec*symtab=&obj->secs[obj->symtab_sec_idx];-Elf64_Sym*sym=symtab->data->d_buf,*dst_sym;-inti,n=symtab->shdr->sh_size/symtab->shdr->sh_entsize;+Elf64_Sym*sym=symtab->data->d_buf;+inti,n=symtab->shdr->sh_size/symtab->shdr->sh_entsize,err;intstr_sec_idx=symtab->shdr->sh_link;+constchar*sym_name;obj->sym_map=calloc(n+1,sizeof(*obj->sym_map));if(!obj->sym_map)return-ENOMEM;for(i=0;i<n;i++,sym++){-structsrc_sec*src_sec=NULL;-structdst_sec*dst_sec=NULL;-constchar*sym_name;-size_tdst_sym_idx;-intname_off;-/* We already validated all-zero symbol #0 and we already*appendeditpreventivelytothefinalSYMTAB,soskipit.*/
@@ -1172,41 +1221,623 @@ static int linker_append_elf_syms(struct bpf_linker *linker, struct src_obj *objreturn-EINVAL;}-if(sym->st_shndx&&sym->st_shndx<SHN_LORESERVE){-src_sec=&obj->secs[sym->st_shndx];-if(src_sec->skipped)+err=linker_append_elf_sym(linker,obj,sym,sym_name,i);+if(err)+returnerr;+}++return0;+}++staticElf64_Sym*get_sym_by_idx(structbpf_linker*linker,size_tsym_idx)+{+structdst_sec*symtab=&linker->secs[linker->symtab_sec_idx];+Elf64_Sym*syms=symtab->raw_data;++return&syms[sym_idx];+}++staticstructglob_sym*find_glob_sym(structbpf_linker*linker,constchar*sym_name)+{+structglob_sym*glob_sym;+constchar*name;+inti;++for(i=0;i<linker->glob_sym_cnt;i++){+glob_sym=&linker->glob_syms[i];+name=strset__data(linker->strtab_strs)+glob_sym->name_off;++if(strcmp(name,sym_name)==0)+returnglob_sym;+}++returnNULL;+}++staticstructglob_sym*add_glob_sym(structbpf_linker*linker)+{+structglob_sym*syms,*sym;++syms=libbpf_reallocarray(linker->glob_syms,linker->glob_sym_cnt+1,+sizeof(*linker->glob_syms));+if(!syms)+returnNULL;++sym=&syms[linker->glob_sym_cnt];+memset(sym,0,sizeof(*sym));+sym->var_idx=-1;++linker->glob_syms=syms;+linker->glob_sym_cnt++;++returnsym;+}++staticboolglob_sym_btf_matches(constchar*sym_name,boolexact,+conststructbtf*btf1,__u32id1,+conststructbtf*btf2,__u32id2)+{+conststructbtf_type*t1,*t2;+boolis_static1,is_static2;+constchar*n1,*n2;+inti,n;++recur:+n1=n2=NULL;+t1=skip_mods_and_typedefs(btf1,id1,&id1);+t2=skip_mods_and_typedefs(btf2,id2,&id2);++/* check if only one side is FWD, otherwise handle with common logic */+if(!exact&&btf_is_fwd(t1)!=btf_is_fwd(t2)){+n1=btf__str_by_offset(btf1,t1->name_off);+n2=btf__str_by_offset(btf2,t2->name_off);+if(strcmp(n1,n2)!=0){+pr_warn("global '%s': incompatible forward declaration names '%s' and '%s'\n",+sym_name,n1,n2);+returnfalse;+}+/* validate if FWD kind matches concrete kind */+if(btf_is_fwd(t1)){+if(btf_kflag(t1)&&btf_is_union(t2))+returntrue;+if(!btf_kflag(t1)&&btf_is_struct(t2))+returntrue;+pr_warn("global '%s': incompatible %s forward declaration and concrete kind %s\n",+sym_name,btf_kflag(t1)?"union":"struct",btf_kind_str(t2));+}else{+if(btf_kflag(t2)&&btf_is_union(t1))+returntrue;+if(!btf_kflag(t2)&&btf_is_struct(t1))+returntrue;+pr_warn("global '%s': incompatible %s forward declaration and concrete kind %s\n",+sym_name,btf_kflag(t2)?"union":"struct",btf_kind_str(t1));+}+returnfalse;+}++if(btf_kind(t1)!=btf_kind(t2)){+pr_warn("global '%s': incompatible BTF kinds %s and %s\n",+sym_name,btf_kind_str(t1),btf_kind_str(t2));+returnfalse;+}++switch(btf_kind(t1)){+caseBTF_KIND_STRUCT:+caseBTF_KIND_UNION:+caseBTF_KIND_ENUM:+caseBTF_KIND_FWD:+caseBTF_KIND_FUNC:+caseBTF_KIND_VAR:+n1=btf__str_by_offset(btf1,t1->name_off);+n2=btf__str_by_offset(btf2,t2->name_off);+if(strcmp(n1,n2)!=0){+pr_warn("global '%s': incompatible %s names '%s' and '%s'\n",+sym_name,btf_kind_str(t1),n1,n2);+returnfalse;+}+break;+default:+break;+}++switch(btf_kind(t1)){+caseBTF_KIND_UNKN:/* void */+caseBTF_KIND_FWD:+returntrue;+caseBTF_KIND_INT:+caseBTF_KIND_FLOAT:+caseBTF_KIND_ENUM:+/* ignore encoding for int and enum values for enum */+if(t1->size!=t2->size){+pr_warn("global '%s': incompatible %s '%s' size %u and %u\n",+sym_name,btf_kind_str(t1),n1,t1->size,t2->size);+returnfalse;+}+returntrue;+caseBTF_KIND_PTR:+/* just validate overall shape of the referenced type, so no+*contentscomparisonforstruct/union,andallowdfwdvs+*struct/union+*/+exact=false;+id1=t1->type;+id2=t2->type;+gotorecur;+caseBTF_KIND_ARRAY:+/* ignore index type and array size */+id1=btf_array(t1)->type;+id2=btf_array(t2)->type;+gotorecur;+caseBTF_KIND_FUNC:+/* extern and global linkages are compatible */+is_static1=btf_func_linkage(t1)==BTF_FUNC_STATIC;+is_static2=btf_func_linkage(t2)==BTF_FUNC_STATIC;+if(is_static1!=is_static2){+pr_warn("global '%s': incompatible func '%s' linkage\n",sym_name,n1);+returnfalse;+}++id1=t1->type;+id2=t2->type;+gotorecur;+caseBTF_KIND_VAR:+/* extern and global linkages are compatible */+is_static1=btf_var(t1)->linkage==BTF_VAR_STATIC;+is_static2=btf_var(t2)->linkage==BTF_VAR_STATIC;+if(is_static1!=is_static2){+pr_warn("global '%s': incompatible var '%s' linkage\n",sym_name,n1);+returnfalse;+}++id1=t1->type;+id2=t2->type;+gotorecur;+caseBTF_KIND_STRUCT:+caseBTF_KIND_UNION:{+conststructbtf_member*m1,*m2;++if(!exact)+returntrue;++if(btf_vlen(t1)!=btf_vlen(t2)){+pr_warn("global '%s': incompatible number of %s fields %u and %u\n",+sym_name,btf_kind_str(t1),btf_vlen(t1),btf_vlen(t2));+returnfalse;+}++n=btf_vlen(t1);+m1=btf_members(t1);+m2=btf_members(t2);+for(i=0;i<n;i++,m1++,m2++){+n1=btf__str_by_offset(btf1,m1->name_off);+n2=btf__str_by_offset(btf2,m2->name_off);+if(strcmp(n1,n2)!=0){+pr_warn("global '%s': incompatible field #%d names '%s' and '%s'\n",+sym_name,i,n1,n2);+returnfalse;+}+if(m1->offset!=m2->offset){+pr_warn("global '%s': incompatible field #%d ('%s') offsets\n",+sym_name,i,n1);+returnfalse;+}+if(!glob_sym_btf_matches(sym_name,exact,btf1,m1->type,btf2,m2->type))+returnfalse;+}++returntrue;+}+caseBTF_KIND_FUNC_PROTO:{+conststructbtf_param*m1,*m2;++if(btf_vlen(t1)!=btf_vlen(t2)){+pr_warn("global '%s': incompatible number of %s params %u and %u\n",+sym_name,btf_kind_str(t1),btf_vlen(t1),btf_vlen(t2));+returnfalse;+}++n=btf_vlen(t1);+m1=btf_params(t1);+m2=btf_params(t2);+for(i=0;i<n;i++,m1++,m2++){+/* ignore func arg names */+if(!glob_sym_btf_matches(sym_name,exact,btf1,m1->type,btf2,m2->type))+returnfalse;+}++/* now check return type as well */+id1=t1->type;+id2=t2->type;+gotorecur;+}++caseBTF_KIND_TYPEDEF:+caseBTF_KIND_VOLATILE:+caseBTF_KIND_CONST:+caseBTF_KIND_RESTRICT:+caseBTF_KIND_DATASEC:+default:+pr_warn("global '%s': unsupported BTF kind %s\n",+sym_name,btf_kind_str(t1));+returnfalse;+}+}++staticboolglob_syms_match(constchar*sym_name,+structbpf_linker*linker,structglob_sym*glob_sym,+structsrc_obj*obj,Elf64_Sym*sym,size_tsym_idx,intbtf_id)+{+conststructbtf_type*src_t;++/* if we are dealing with externs, BTF types describing both global+*andexternVARs/FUNCsshouldbecompletelypresentinallfiles+*/+if(!glob_sym->btf_id||!btf_id){+pr_warn("BTF info is missing for global symbol '%s'\n",sym_name);+returnfalse;+}++src_t=btf__type_by_id(obj->btf,btf_id);+if(!btf_is_var(src_t)&&!btf_is_func(src_t)){+pr_warn("only extern variables and functions are supported, but got '%s' for '%s'\n",+btf_kind_str(src_t),sym_name);+returnfalse;+}++if(!glob_sym_btf_matches(sym_name,true/*exact*/,+linker->btf,glob_sym->btf_id,obj->btf,btf_id))+returnfalse;++returntrue;+}++staticboolbtf_is_non_static(conststructbtf_type*t)+{+return(btf_is_var(t)&&btf_var(t)->linkage!=BTF_VAR_STATIC)+||(btf_is_func(t)&&btf_func_linkage(t)!=BTF_FUNC_STATIC);+}++staticintfind_glob_sym_btf(structsrc_obj*obj,Elf64_Sym*sym,constchar*sym_name,+int*out_btf_sec_id,int*out_btf_id)+{+inti,j,n=btf__get_nr_types(obj->btf),m,btf_id=0;+conststructbtf_type*t;+conststructbtf_var_secinfo*vi;+constchar*name;++for(i=1;i<=n;i++){+t=btf__type_by_id(obj->btf,i);++/* some global and extern FUNCs and VARs might not be associated with any+*DATASEC,sotrytodetecttheminthesamepass+*/+if(btf_is_non_static(t)){+name=btf__str_by_offset(obj->btf,t->name_off);+if(strcmp(name,sym_name)!=0)continue;-dst_sec=&linker->secs[src_sec->dst_id];-/* allow only one STT_SECTION symbol per section */-if(ELF64_ST_TYPE(sym->st_info)==STT_SECTION&&dst_sec->sec_sym_idx){-obj->sym_map[i]=dst_sec->sec_sym_idx;+/* remember and still try to find DATASEC */+btf_id=i;+continue;+}++if(!btf_is_datasec(t))+continue;++vi=btf_var_secinfos(t);+for(j=0,m=btf_vlen(t);j<m;j++,vi++){+t=btf__type_by_id(obj->btf,vi->type);+name=btf__str_by_offset(obj->btf,t->name_off);++if(strcmp(name,sym_name)!=0)+continue;+if(btf_is_var(t)&&btf_var(t)->linkage==BTF_VAR_STATIC)continue;+if(btf_is_func(t)&&btf_func_linkage(t)==BTF_FUNC_STATIC)+continue;++if(btf_id&&btf_id!=vi->type){+pr_warn("global/extern '%s' BTF is ambiguous: both types #%d and #%u match\n",+sym_name,btf_id,vi->type);+return-EINVAL;}++*out_btf_sec_id=i;+*out_btf_id=vi->type;++return0;}+}-name_off=strset__add_str(linker->strtab_strs,sym_name);-if(name_off<0)-returnname_off;+/* free-floating extern or global FUNC */+if(btf_id){+*out_btf_sec_id=0;+*out_btf_id=btf_id;+return0;+}-dst_sym=add_new_sym(linker,&dst_sym_idx);-if(!dst_sym)-return-ENOMEM;+pr_warn("failed to find BTF info for global/extern symbol '%s'\n",sym_name);+return-ENOENT;+}-dst_sym->st_name=name_off;-dst_sym->st_info=sym->st_info;-dst_sym->st_other=sym->st_other;-dst_sym->st_shndx=src_sec?dst_sec->sec_idx:sym->st_shndx;-dst_sym->st_value=(src_sec?src_sec->dst_off:0)+sym->st_value;-dst_sym->st_size=sym->st_size;+staticstructsrc_sec*find_src_sec_by_name(structsrc_obj*obj,constchar*sec_name)+{+structsrc_sec*sec;+inti;-obj->sym_map[i]=dst_sym_idx;+for(i=1;i<obj->sec_cnt;i++){+sec=&obj->secs[i];-if(ELF64_ST_TYPE(sym->st_info)==STT_SECTION&&dst_sym){-dst_sec->sec_sym_idx=dst_sym_idx;-dst_sym->st_value=0;+if(strcmp(sec->sec_name,sec_name)==0)+returnsec;+}++returnNULL;+}++staticintcomplete_extern_btf_info(structbtf*dst_btf,intdst_id,+structbtf*src_btf,intsrc_id)+{+structbtf_type*dst_t=btf_type_by_id(dst_btf,dst_id);+structbtf_type*src_t=btf_type_by_id(src_btf,src_id);+structbtf_param*src_p,*dst_p;+constchar*s;+inti,n,off;++/* We already made sure that source and destination types (FUNC or+*VAR)matchintermsoftypesandargumentnames.+*/+if(btf_is_var(dst_t)){+btf_var(dst_t)->linkage=BTF_VAR_GLOBAL_ALLOCATED;+return0;+}++dst_t->info=btf_type_info(BTF_KIND_FUNC,BTF_FUNC_GLOBAL,0);++/* now onto FUNC_PROTO types */+src_t=btf_type_by_id(src_btf,src_t->type);+dst_t=btf_type_by_id(dst_btf,dst_t->type);++/* Fill in all the argument names, which for extern FUNCs are missing.+*We'llendupwithtwocopiesofFUNCs/VARsforexterns,butthat+*willbetakencareofbyBTFdedupattheveryend.+*ItmightbethatBTFtypesforexterninonefilehasless/moreBTF+*information(e.g.,FWDinsteadoffullSTRUCT/UNIONinformation),+*butthatshouldbe(inmostcases,subjecttoBTFdeduprules)+*handledandresolvedbyBTFdedupalgorithmaswell,sowewon't+*worryaboutit.Ouronlyjobistomakesurethatargumentnames+*arepopulatedonbothsides,otherwiseBTFdedupwillpedantically+*considerthemdifferent.+*/+src_p=btf_params(src_t);+dst_p=btf_params(dst_t);+for(i=0,n=btf_vlen(dst_t);i<n;i++,src_p++,dst_p++){+if(!src_p->name_off)+continue;++/* src_btf has more complete info, so add name to dst_btf */+s=btf__str_by_offset(src_btf,src_p->name_off);+off=btf__add_str(dst_btf,s);+if(off<0)+returnoff;+dst_p->name_off=off;+}+return0;+}++staticvoidsym_update_bind(Elf64_Sym*sym,intsym_bind)+{+sym->st_info=ELF64_ST_INFO(sym_bind,ELF64_ST_TYPE(sym->st_info));+}++staticvoidsym_update_type(Elf64_Sym*sym,intsym_type)+{+sym->st_info=ELF64_ST_INFO(ELF64_ST_BIND(sym->st_info),sym_type);+}++staticvoidsym_update_visibility(Elf64_Sym*sym,intsym_vis)+{+/* libelf doesn't provide setters for ST_VISIBILITY,+*butitisstoredinthelower2bitsofst_other+*/+sym->st_other&=0x03;+sym->st_other|=sym_vis;+}++staticintlinker_append_elf_sym(structbpf_linker*linker,structsrc_obj*obj,+Elf64_Sym*sym,constchar*sym_name,intsrc_sym_idx)+{+structsrc_sec*src_sec=NULL;+structdst_sec*dst_sec=NULL;+structglob_sym*glob_sym=NULL;+intname_off,sym_type,sym_bind,sym_vis,err;+intbtf_sec_id=0,btf_id=0;+size_tdst_sym_idx;+Elf64_Sym*dst_sym;+boolsym_is_extern;++sym_type=ELF64_ST_TYPE(sym->st_info);+sym_bind=ELF64_ST_BIND(sym->st_info);+sym_vis=ELF64_ST_VISIBILITY(sym->st_other);+sym_is_extern=sym->st_shndx==SHN_UNDEF;++if(sym_is_extern){+if(!obj->btf){+pr_warn("externs without BTF info are not supported\n");+return-ENOTSUP;+}+}elseif(sym->st_shndx<SHN_LORESERVE){+src_sec=&obj->secs[sym->st_shndx];+if(src_sec->skipped)+return0;+dst_sec=&linker->secs[src_sec->dst_id];++/* allow only one STT_SECTION symbol per section */+if(sym_type==STT_SECTION&&dst_sec->sec_sym_idx){+obj->sym_map[src_sym_idx]=dst_sec->sec_sym_idx;+return0;+}+}++if(sym_bind==STB_LOCAL)+gotoadd_sym;++/* find matching BTF info */+err=find_glob_sym_btf(obj,sym,sym_name,&btf_sec_id,&btf_id);+if(err)+returnerr;++if(sym_is_extern&&btf_sec_id){+constchar*sec_name=NULL;+conststructbtf_type*t;++t=btf__type_by_id(obj->btf,btf_sec_id);+sec_name=btf__str_by_offset(obj->btf,t->name_off);++/* Clang puts unannotated extern vars into+*'.extern'BTFDATASEC.Treatthemthesame+*asunannotatedexternfuncs(whichare+*currentlynotputintoanyDATASECs).+*Thosedon'thaveassociatedsrc_sec/dst_sec.+*/+if(strcmp(sec_name,BTF_EXTERN_SEC)!=0){+src_sec=find_src_sec_by_name(obj,sec_name);+if(!src_sec){+pr_warn("failed to find matching ELF sec '%s'\n",sec_name);+return-ENOENT;+}+dst_sec=&linker->secs[src_sec->dst_id];+}+}++glob_sym=find_glob_sym(linker,sym_name);+if(glob_sym){+/* Preventively resolve to existing symbol. This is+*neededforfurtherrelocationsymbolremappingin+*thenextstepoflinking.+*/+obj->sym_map[src_sym_idx]=glob_sym->sym_idx;++/* If both symbols are non-externs, at least one of+*themhastobeSTB_WEAK,otherwisetheyarein+*aconflictwitheachother.+*/+if(!sym_is_extern&&!glob_sym->is_extern+&&!glob_sym->is_weak&&sym_bind!=STB_WEAK){+pr_warn("conflicting non-weak symbol #%d (%s) definition in '%s'\n",+src_sym_idx,sym_name,obj->filename);+return-EINVAL;}+if(!glob_syms_match(sym_name,linker,glob_sym,obj,sym,src_sym_idx,btf_id))+return-EINVAL;++dst_sym=get_sym_by_idx(linker,glob_sym->sym_idx);++/* If new symbol is strong, then force dst_sym to be strong as+*well;thiswayamixofweakandnon-weakextern+*definitionswillendupbeingstrong.+*/+if(sym_bind==STB_GLOBAL){+/* We still need to preserve type (NOTYPE or+*OBJECT/FUNC,dependingonwhetherthesymbolis+*externornot)+*/+sym_update_bind(dst_sym,STB_GLOBAL);+glob_sym->is_weak=false;+}++/* Non-default visibility is "contaminating", with stricter+*visibilityoverwritingmorepermissiveones,evenifmore+*permissivevisibilitycomesfromjustanexterndefinition+*/+if(sym_vis>ELF64_ST_VISIBILITY(dst_sym->st_other))+sym_update_visibility(dst_sym,sym_vis);++/* If the new symbol is extern, then regardless if+*existingsymbolisexternorresolvedglobal,just+*keeptheexistingoneuntouched.+*/+if(sym_is_extern)+return0;++/* If existing symbol is a strong resolved symbol, bail out,+*becausewelostresolutionbattlehavenothingto+*contribute.Wealreadycheckedaboverthatthereisno+*strong-strongconflict.Wealsoalreadytightenedbinding+*andvisibility,sonothingelsetocontributeatthatpoint.+*/+if(!glob_sym->is_extern&&sym_bind==STB_WEAK)+return0;++/* At this point, new symbol is strong non-extern,+*sooverwriteglob_symwithnewsymbolinformation.+*Preservebindingandvisibility.+*/+sym_update_type(dst_sym,sym_type);+dst_sym->st_shndx=dst_sec->sec_idx;+dst_sym->st_value=src_sec->dst_off+sym->st_value;+dst_sym->st_size=sym->st_size;++/* see comment below about dst_sec->id vs dst_sec->sec_idx */+glob_sym->sec_id=dst_sec->id;+glob_sym->is_extern=false;+/* never relax strong to weak binding */+if(sym_bind==STB_GLOBAL)+glob_sym->is_weak=false;++if(complete_extern_btf_info(linker->btf,glob_sym->btf_id,+obj->btf,btf_id))+return-EINVAL;++/* request updating VAR's/FUNC's underlying BTF type when appending BTF type */+glob_sym->underlying_btf_id=0;++obj->sym_map[src_sym_idx]=glob_sym->sym_idx;+return0;+}++add_sym:+name_off=strset__add_str(linker->strtab_strs,sym_name);+if(name_off<0)+returnname_off;++dst_sym=add_new_sym(linker,&dst_sym_idx);+if(!dst_sym)+return-ENOMEM;++dst_sym->st_name=name_off;+dst_sym->st_info=sym->st_info;+dst_sym->st_other=sym->st_other;+dst_sym->st_shndx=dst_sec?dst_sec->sec_idx:sym->st_shndx;+dst_sym->st_value=(src_sec?src_sec->dst_off:0)+sym->st_value;+dst_sym->st_size=sym->st_size;++obj->sym_map[src_sym_idx]=dst_sym_idx;++if(sym_type==STT_SECTION&&dst_sym){+dst_sec->sec_sym_idx=dst_sym_idx;+dst_sym->st_value=0;+}++if(sym_bind!=STB_LOCAL){+glob_sym=add_glob_sym(linker);+if(!glob_sym)+return-ENOMEM;++glob_sym->sym_idx=dst_sym_idx;+/* we use dst_sec->id (and not dst_sec->sec_idx), because+*ephemeralsections(.kconfig,.ksyms,etc)don'thave+*sec_idx(astheydon'thavecorrespondingELFsection),but+*stillhaveid..externdoesn'thaveevenephemeralsection+*associatedwithit,sodst_sec->id==dst_sec->sec_idx==0.+*/+glob_sym->sec_id=dst_sec?dst_sec->id:0;+glob_sym->name_off=name_off;+/* we will fill btf_id in during BTF merging step */+glob_sym->btf_id=0;+glob_sym->is_extern=sym_is_extern;+glob_sym->is_weak=sym_bind==STB_WEAK;}return0;
@@ -1378,12 +1994,32 @@ static int linker_fixup_btf(struct src_obj *obj)t->size=sec->shdr->sh_size;}else{/* BTF can have some sections that are not represented-*inELF,e.g.,.kconfigand.ksyms,whichareused-*forspecialexternvariables.Herewe'll-*pre-create"section shells"forthemtobeableto-*keeptrackofextraper-sectionmetadatalater-*(e.g.,BTFvariables).+*inELF,e.g.,.kconfig,.ksyms,.extern,whichareused+*forspecialexternvariables.+*+*Forallbutonesuchspecial(ephemeral)+*sections,wepre-create"section shells"tobeable+*tokeeptrackofextraper-sectionmetadatalater+*(e.g.,thoseBTFexternvariables).+*+*.externisevenmorespecial,though,becauseit+*containsexternvariablesthatneedtoberesolved+*bystaticlinker,notlibbpfandkernel.Whensuch+*externsareresolved,wearegoingtoremovethem+*from.externBTFsectionandmightendupnot+*needingitatall.Eachresolvedexternshouldhave+*matchingnon-externVAR/FUNCinothersections.+*+*Wedosupportleavingsomeoftheexterns+*unresolved,though,tosupportcasesofbuilding+*libraries,whichwilllaterbelinkedagainstfinal+*BPFapplications.Soifatfinalizationwestill+*seeunresolvedexterns,we'llcreate.extern+*sectiononourown.*/+if(strcmp(sec_name,BTF_EXTERN_SEC)==0)+continue;+sec=add_src_sec(obj,sec_name);if(!sec)return-ENOMEM;
@@ -1454,12 +2091,40 @@ static int linker_append_btf(struct bpf_linker *linker, struct src_obj *obj)return-ENOMEM;for(i=1;i<=n;i++){+structglob_sym*glob_sym=NULL;+t=btf__type_by_id(obj->btf,i);/* DATASECs are handled specially below */if(btf_kind(t)==BTF_KIND_DATASEC)continue;+if(btf_is_non_static(t)){+/* there should be glob_sym already */+name=btf__str_by_offset(obj->btf,t->name_off);+glob_sym=find_glob_sym(linker,name);++/* VARs without corresponding glob_sym are those that+*belongtoskipped/deduplicatedsections(i.e.,+*licenseandversion),sojustskipthem+*/+if(!glob_sym)+continue;++if(glob_sym->underlying_btf_id==0)+glob_sym->underlying_btf_id=-t->type;++/* globals from previous object files that match our+*VAR/FUNCalreadyhaveacorrespondingassociated+*BTFtype,sojustmakesuretouseit+*/+if(glob_sym->btf_id){+/* reuse existing BTF type for global var/func */+obj->btf_type_map[i]=glob_sym->btf_id;+continue;+}+}+id=btf__add_type(linker->btf,obj->btf,t);if(id<0){pr_warn("failed to append BTF type #%d from file '%s'\n",i,obj->filename);
@@ -1467,6 +2132,12 @@ static int linker_append_btf(struct bpf_linker *linker, struct src_obj *obj)}obj->btf_type_map[i]=id;++/* record just appended BTF type for var/func */+if(glob_sym){+glob_sym->btf_id=id;+glob_sym->underlying_btf_id=-t->type;+}}/* remap all the types except DATASECs */
@@ -1478,6 +2149,22 @@ static int linker_append_btf(struct bpf_linker *linker, struct src_obj *obj)return-EINVAL;}+/* Rewrite VAR/FUNC underlying types (i.e., FUNC's FUNC_PROTO and VAR's+*actualtype),ifnecessary+*/+for(i=0;i<linker->glob_sym_cnt;i++){+structglob_sym*glob_sym=&linker->glob_syms[i];+structbtf_type*glob_t;++if(glob_sym->underlying_btf_id>=0)+continue;++glob_sym->underlying_btf_id=obj->btf_type_map[-glob_sym->underlying_btf_id];++glob_t=btf_type_by_id(linker->btf,glob_sym->btf_id);+glob_t->type=glob_sym->underlying_btf_id;+}+/* append DATASEC info */for(i=1;i<obj->sec_cnt;i++){structsrc_sec*src_sec;
@@ -1505,6 +2192,42 @@ static int linker_append_btf(struct bpf_linker *linker, struct src_obj *obj)n=btf_vlen(t);for(j=0;j<n;j++,src_var++){void*sec_vars=dst_sec->sec_vars;+intnew_id=obj->btf_type_map[src_var->type];+structglob_sym*glob_sym=NULL;++t=btf_type_by_id(linker->btf,new_id);+if(btf_is_non_static(t)){+name=btf__str_by_offset(linker->btf,t->name_off);+glob_sym=find_glob_sym(linker,name);+if(glob_sym->sec_id!=dst_sec->id){+pr_warn("global '%s': section mismatch %d vs %d\n",+name,glob_sym->sec_id,dst_sec->id);+return-EINVAL;+}+}++/* If there is already a member (VAR or FUNC) mapped+*tothesametype,don'taddaduplicateentry.+*Thiswillhappenwhenmultipleobjectfilesdefine+*thesameexternVARs/FUNCs.+*/+if(glob_sym&&glob_sym->var_idx>=0){+__s64sz;++dst_var=&dst_sec->sec_vars[glob_sym->var_idx];+/* Because underlying BTF type might have+*changed,somightitssizehavechanged,so+*re-calculateandupdateitinsec_var.+*/+sz=btf__resolve_size(linker->btf,glob_sym->underlying_btf_id);+if(sz<0){+pr_warn("global '%s': failed to resolve size of underlying type: %d\n",+name,(int)sz);+return-EINVAL;+}+dst_var->size=sz;+continue;+}sec_vars=libbpf_reallocarray(sec_vars,dst_sec->sec_var_cnt+1,
It should never fail, but if it does, it's better to know about this rather
than end up with nonsensical type IDs.
Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
---
tools/lib/bpf/linker.c | 9 +++++++++
1 file changed, 9 insertions(+)
@@ -1423,6 +1423,15 @@ static int linker_fixup_btf(struct src_obj *obj)staticintremap_type_id(__u32*type_id,void*ctx){int*id_map=ctx;+intnew_id=id_map[*type_id];++if(*type_id==0)+return0;++if(new_id==0){+pr_warn("failed to find new ID mapping for original BTF type ID %u\n",*type_id);+return-EINVAL;+}*type_id=id_map[*type_id];
Make skip_mods_and_typedefs(), btf_kind_str(), and btf_func_linkage() helpers
available outside of libbpf.c, to be used by static linker code.
Also do few cleanups (error code fixes, comment clean up, etc) that don't
deserve their own commit.
Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
---
tools/lib/bpf/libbpf.c | 13 +++----------
tools/lib/bpf/libbpf_internal.h | 7 +++++++
tools/lib/bpf/linker.c | 14 ++++++--------
3 files changed, 16 insertions(+), 18 deletions(-)
@@ -69,8 +69,6 @@#define __printf(a, b) __attribute__((format(printf, a, b)))staticstructbpf_map*bpf_object__add_map(structbpf_object*obj);-staticconststructbtf_type*-skip_mods_and_typedefs(conststructbtf*btf,__u32id,__u32*res_id);staticboolprog_is_subprog(conststructbpf_object*obj,conststructbpf_program*prog);staticint__base_pr(enumlibbpf_print_levellevel,constchar*format,
@@ -1221,7 +1219,7 @@ static int linker_append_elf_relos(struct bpf_linker *linker, struct src_obj *obreturnerr;}}elseif(!secs_match(dst_sec,src_sec)){-pr_warn("Secs %s are not compatible\n",src_sec->sec_name);+pr_warn("sections %s are not compatible\n",src_sec->sec_name);return-1;}
Factor out logic for sanity checking SHT_SYMTAB and SHT_REL sections into
separate sections. They are already quite extensive and are suffering from too
deep indentation. Subsequent changes will extend SYMTAB sanity checking
further, so it's better to factor each into a separate function.
No functional changes are intended.
Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
---
tools/lib/bpf/linker.c | 233 ++++++++++++++++++++++-------------------
1 file changed, 127 insertions(+), 106 deletions(-)
@@ -663,8 +665,8 @@ static bool is_pow_of_2(size_t x)staticintlinker_sanity_check_elf(structsrc_obj*obj){-structsrc_sec*sec,*link_sec;-inti,j,n;+structsrc_sec*sec;+inti,err;if(!obj->symtab_sec_idx){pr_warn("ELF is missing SYMTAB section in %s\n",obj->filename);
@@ -692,43 +694,11 @@ static int linker_sanity_check_elf(struct src_obj *obj)return-EINVAL;switch(sec->shdr->sh_type){-caseSHT_SYMTAB:{-Elf64_Sym*sym;--if(sec->shdr->sh_entsize!=sizeof(Elf64_Sym))-return-EINVAL;-if(sec->shdr->sh_size%sec->shdr->sh_entsize!=0)-return-EINVAL;--if(!sec->shdr->sh_link||sec->shdr->sh_link>=obj->sec_cnt){-pr_warn("ELF SYMTAB section #%zu points to missing STRTAB section #%zu in %s\n",-sec->sec_idx,(size_t)sec->shdr->sh_link,obj->filename);-return-EINVAL;-}-link_sec=&obj->secs[sec->shdr->sh_link];-if(link_sec->shdr->sh_type!=SHT_STRTAB){-pr_warn("ELF SYMTAB section #%zu points to invalid STRTAB section #%zu in %s\n",-sec->sec_idx,(size_t)sec->shdr->sh_link,obj->filename);-return-EINVAL;-}--n=sec->shdr->sh_size/sec->shdr->sh_entsize;-sym=sec->data->d_buf;-for(j=0;j<n;j++,sym++){-if(sym->st_shndx-&&sym->st_shndx<SHN_LORESERVE-&&sym->st_shndx>=obj->sec_cnt){-pr_warn("ELF sym #%d in section #%zu points to missing section #%zu in %s\n",-j,sec->sec_idx,(size_t)sym->st_shndx,obj->filename);-return-EINVAL;-}-if(ELF64_ST_TYPE(sym->st_info)==STT_SECTION){-if(sym->st_value!=0)-return-EINVAL;-}-}+caseSHT_SYMTAB:+err=linker_sanity_check_elf_symtab(obj,sec);+if(err)+returnerr;break;-}caseSHT_STRTAB:break;caseSHT_PROGBITS:
@@ -739,87 +709,138 @@ static int linker_sanity_check_elf(struct src_obj *obj)break;caseSHT_NOBITS:break;-caseSHT_REL:{-Elf64_Rel*relo;-structsrc_sec*sym_sec;+caseSHT_REL:+err=linker_sanity_check_elf_relos(obj,sec);+if(err)+returnerr;+break;+caseSHT_LLVM_ADDRSIG:+break;+default:+pr_warn("ELF section #%zu (%s) has unrecognized type %zu in %s\n",+sec->sec_idx,sec->sec_name,(size_t)sec->shdr->sh_type,obj->filename);+return-EINVAL;+}+}-if(sec->shdr->sh_entsize!=sizeof(Elf64_Rel))-return-EINVAL;-if(sec->shdr->sh_size%sec->shdr->sh_entsize!=0)-return-EINVAL;+return0;+}-/* SHT_REL's sh_link should point to SYMTAB */-if(sec->shdr->sh_link!=obj->symtab_sec_idx){-pr_warn("ELF relo section #%zu points to invalid SYMTAB section #%zu in %s\n",-sec->sec_idx,(size_t)sec->shdr->sh_link,obj->filename);-return-EINVAL;-}+staticintlinker_sanity_check_elf_symtab(structsrc_obj*obj,structsrc_sec*sec)+{+structsrc_sec*link_sec;+Elf64_Sym*sym;+inti,n;-/* SHT_REL's sh_info points to relocated section */-if(!sec->shdr->sh_info||sec->shdr->sh_info>=obj->sec_cnt){-pr_warn("ELF relo section #%zu points to missing section #%zu in %s\n",-sec->sec_idx,(size_t)sec->shdr->sh_info,obj->filename);-return-EINVAL;-}-link_sec=&obj->secs[sec->shdr->sh_info];+if(sec->shdr->sh_entsize!=sizeof(Elf64_Sym))+return-EINVAL;+if(sec->shdr->sh_size%sec->shdr->sh_entsize!=0)+return-EINVAL;++if(!sec->shdr->sh_link||sec->shdr->sh_link>=obj->sec_cnt){+pr_warn("ELF SYMTAB section #%zu points to missing STRTAB section #%zu in %s\n",+sec->sec_idx,(size_t)sec->shdr->sh_link,obj->filename);+return-EINVAL;+}+link_sec=&obj->secs[sec->shdr->sh_link];+if(link_sec->shdr->sh_type!=SHT_STRTAB){+pr_warn("ELF SYMTAB section #%zu points to invalid STRTAB section #%zu in %s\n",+sec->sec_idx,(size_t)sec->shdr->sh_link,obj->filename);+return-EINVAL;+}-/* .rel<secname> -> <secname> pattern is followed */-if(strncmp(sec->sec_name,".rel",sizeof(".rel")-1)!=0-||strcmp(sec->sec_name+sizeof(".rel")-1,link_sec->sec_name)!=0){-pr_warn("ELF relo section #%zu name has invalid name in %s\n",-sec->sec_idx,obj->filename);+n=sec->shdr->sh_size/sec->shdr->sh_entsize;+sym=sec->data->d_buf;+for(i=0;i<n;i++,sym++){+if(sym->st_shndx+&&sym->st_shndx<SHN_LORESERVE+&&sym->st_shndx>=obj->sec_cnt){+pr_warn("ELF sym #%d in section #%zu points to missing section #%zu in %s\n",+i,sec->sec_idx,(size_t)sym->st_shndx,obj->filename);+return-EINVAL;+}+if(ELF64_ST_TYPE(sym->st_info)==STT_SECTION){+if(sym->st_value!=0)return-EINVAL;-}+continue;+}+}-/* don't further validate relocations for ignored sections */-if(link_sec->skipped)-break;+return0;+}-/* relocatable section is data or instructions */-if(link_sec->shdr->sh_type!=SHT_PROGBITS-&&link_sec->shdr->sh_type!=SHT_NOBITS){-pr_warn("ELF relo section #%zu points to invalid section #%zu in %s\n",-sec->sec_idx,(size_t)sec->shdr->sh_info,obj->filename);-return-EINVAL;-}+staticintlinker_sanity_check_elf_relos(structsrc_obj*obj,structsrc_sec*sec)+{+structsrc_sec*link_sec,*sym_sec;+Elf64_Rel*relo;+inti,n;-/* check sanity of each relocation */-n=sec->shdr->sh_size/sec->shdr->sh_entsize;-relo=sec->data->d_buf;-sym_sec=&obj->secs[obj->symtab_sec_idx];-for(j=0;j<n;j++,relo++){-size_tsym_idx=ELF64_R_SYM(relo->r_info);-size_tsym_type=ELF64_R_TYPE(relo->r_info);--if(sym_type!=R_BPF_64_64&&sym_type!=R_BPF_64_32){-pr_warn("ELF relo #%d in section #%zu has unexpected type %zu in %s\n",-j,sec->sec_idx,sym_type,obj->filename);-return-EINVAL;-}+if(sec->shdr->sh_entsize!=sizeof(Elf64_Rel))+return-EINVAL;+if(sec->shdr->sh_size%sec->shdr->sh_entsize!=0)+return-EINVAL;-if(!sym_idx||sym_idx*sizeof(Elf64_Sym)>=sym_sec->shdr->sh_size){-pr_warn("ELF relo #%d in section #%zu points to invalid symbol #%zu in %s\n",-j,sec->sec_idx,sym_idx,obj->filename);-return-EINVAL;-}+/* SHT_REL's sh_link should point to SYMTAB */+if(sec->shdr->sh_link!=obj->symtab_sec_idx){+pr_warn("ELF relo section #%zu points to invalid SYMTAB section #%zu in %s\n",+sec->sec_idx,(size_t)sec->shdr->sh_link,obj->filename);+return-EINVAL;+}-if(link_sec->shdr->sh_flags&SHF_EXECINSTR){-if(relo->r_offset%sizeof(structbpf_insn)!=0){-pr_warn("ELF relo #%d in section #%zu points to missing symbol #%zu in %s\n",-j,sec->sec_idx,sym_idx,obj->filename);-return-EINVAL;-}-}-}-break;+/* SHT_REL's sh_info points to relocated section */+if(!sec->shdr->sh_info||sec->shdr->sh_info>=obj->sec_cnt){+pr_warn("ELF relo section #%zu points to missing section #%zu in %s\n",+sec->sec_idx,(size_t)sec->shdr->sh_info,obj->filename);+return-EINVAL;+}+link_sec=&obj->secs[sec->shdr->sh_info];++/* .rel<secname> -> <secname> pattern is followed */+if(strncmp(sec->sec_name,".rel",sizeof(".rel")-1)!=0+||strcmp(sec->sec_name+sizeof(".rel")-1,link_sec->sec_name)!=0){+pr_warn("ELF relo section #%zu name has invalid name in %s\n",+sec->sec_idx,obj->filename);+return-EINVAL;+}++/* don't further validate relocations for ignored sections */+if(link_sec->skipped)+return0;++/* relocatable section is data or instructions */+if(link_sec->shdr->sh_type!=SHT_PROGBITS&&link_sec->shdr->sh_type!=SHT_NOBITS){+pr_warn("ELF relo section #%zu points to invalid section #%zu in %s\n",+sec->sec_idx,(size_t)sec->shdr->sh_info,obj->filename);+return-EINVAL;+}++/* check sanity of each relocation */+n=sec->shdr->sh_size/sec->shdr->sh_entsize;+relo=sec->data->d_buf;+sym_sec=&obj->secs[obj->symtab_sec_idx];+for(i=0;i<n;i++,relo++){+size_tsym_idx=ELF64_R_SYM(relo->r_info);+size_tsym_type=ELF64_R_TYPE(relo->r_info);++if(sym_type!=R_BPF_64_64&&sym_type!=R_BPF_64_32){+pr_warn("ELF relo #%d in section #%zu has unexpected type %zu in %s\n",+i,sec->sec_idx,sym_type,obj->filename);+return-EINVAL;}-caseSHT_LLVM_ADDRSIG:-break;-default:-pr_warn("ELF section #%zu (%s) has unrecognized type %zu in %s\n",-sec->sec_idx,sec->sec_name,(size_t)sec->shdr->sh_type,obj->filename);++if(!sym_idx||sym_idx*sizeof(Elf64_Sym)>=sym_sec->shdr->sh_size){+pr_warn("ELF relo #%d in section #%zu points to invalid symbol #%zu in %s\n",+i,sec->sec_idx,sym_idx,obj->filename);return-EINVAL;}++if(link_sec->shdr->sh_flags&SHF_EXECINSTR){+if(relo->r_offset%sizeof(structbpf_insn)!=0){+pr_warn("ELF relo #%d in section #%zu points to missing symbol #%zu in %s\n",+i,sec->sec_idx,sym_idx,obj->filename);+return-EINVAL;+}+}}return0;
Skip generating individual BPF skeletons for files that are supposed to be
linked together to form the final BPF object file. Very often such files are
"incomplete" BPF object files, which will fail libbpf bpf_object__open() step,
if used individually, thus failing BPF skeleton generation. This is by design,
so skip individual BPF skeletons and only validate them as part of their
linked final BPF object file and skeleton.
Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
---
tools/testing/selftests/bpf/Makefile | 4 +++-
1 file changed, 3 insertions(+), 1 deletion(-)
@@ -312,6 +312,8 @@ LINKED_SKELS := test_static_linked.skel.htest_static_linked.skel.h-deps:=test_static_linked1.otest_static_linked2.o+LINKED_BPF_SRCS:=$(patsubst%.o,%.c,$(foreachskel,$(LINKED_SKELS),$($(skel)-deps)))+# Set up extra TRUNNER_XXX "temporary" variables in the environment (relies on# $eval()) and pass control to DEFINE_TEST_RUNNER_RULES.# Parameters:
Add selftest validating various aspects of statically linking functions:
- no conflicts and correct resolution for name-conflicting static funcs;
- correct resolution of extern functions;
- correct handling of weak functions, both resolution itself and libbpf's
handling of unused weak function that "lost" (it leaves gaps in code with
no ELF symbols);
- correct handling of hidden visibility to turn global function into
"static" for the purpose of BPF verification.
Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
---
tools/testing/selftests/bpf/Makefile | 3 +-
.../selftests/bpf/prog_tests/linked_funcs.c | 42 +++++++++++
.../selftests/bpf/progs/linked_funcs1.c | 73 +++++++++++++++++++
.../selftests/bpf/progs/linked_funcs2.c | 73 +++++++++++++++++++
4 files changed, 190 insertions(+), 1 deletion(-)
create mode 100644 tools/testing/selftests/bpf/prog_tests/linked_funcs.c
create mode 100644 tools/testing/selftests/bpf/progs/linked_funcs1.c
create mode 100644 tools/testing/selftests/bpf/progs/linked_funcs2.c
@@ -0,0 +1,73 @@+// SPDX-License-Identifier: GPL-2.0+/* Copyright (c) 2021 Facebook */++#include"vmlinux.h"+#include<bpf/bpf_helpers.h>+#include<bpf/bpf_tracing.h>++/* weak and shared between two files */+constvolatileintmy_tid__weak=0;+constvolatilelongsyscall_id__weak=0;++intoutput_val1=0;+intoutput_ctx1=0;+intoutput_weak1=0;++/* same "subprog" name in all files, but it's ok because they all are static */+static__noinlineintsubprog(intx)+{+/* but different formula */+returnx*1;+}++/* Global functions can't be void */+intset_output_val1(intx)+{+output_val1=x+subprog(x);+returnx;+}++/* This function can't be verified as global, as it assumes raw_tp/sys_enter+*contextandaccessessyscallid(secondargument).Sowemarkitas+*__hidden,sothatlibbpfwillmarkitasstaticinthefinalobjectfile,+*rightbeforeverifyingitinthekernel.+*+*Butwedon'tmarkitas__hiddenhere,ratheratexternsite.__hiddenis+*"contaminating"visibility,soitwillgetpropagatedfromeitherexternor+*actualdefinition(includingfromthelosing__weakdefinition).+*/+voidset_output_ctx1(__u64*ctx)+{+output_ctx1=ctx[1];/* long id, same as in BPF_PROG below */+}++/* this weak instance should win because it's the first one */+__weakintset_output_weak(intx)+{+output_weak1=x;+returnx;+}++externintset_output_val2(intx);++/* here we'll force set_output_ctx2() to be __hidden in the final obj file */+__hiddenexternvoidset_output_ctx2(__u64*ctx);++SEC("raw_tp/sys_enter")+intBPF_PROG(handler1,structpt_regs*regs,longid)+{+if(my_tid!=(u32)bpf_get_current_pid_tgid()||id!=syscall_id)+return0;++set_output_val2(1000);+set_output_ctx2(ctx);/* ctx definition is hidden in BPF_PROG macro */++/* keep input value the same across both files to avoid dependency on+*handlercallorder;differentiatebyoutput_weak1vsoutput_weak2.+*/+set_output_weak(42);++return0;+}++charLICENSE[]SEC("license")="GPL";
@@ -0,0 +1,73 @@+// SPDX-License-Identifier: GPL-2.0+/* Copyright (c) 2021 Facebook */++#include"vmlinux.h"+#include<bpf/bpf_helpers.h>+#include<bpf/bpf_tracing.h>++/* weak and shared between both files */+constvolatileintmy_tid__weak=0;+constvolatilelongsyscall_id__weak=0;++intoutput_val2=0;+intoutput_ctx2=0;+intoutput_weak2=0;/* should stay zero */++/* same "subprog" name in all files, but it's ok because they all are static */+static__noinlineintsubprog(intx)+{+/* but different formula */+returnx*2;+}++/* Global functions can't be void */+intset_output_val2(intx)+{+output_val2=2*x+2*subprog(x);+return2*x;+}++/* This function can't be verified as global, as it assumes raw_tp/sys_enter+*contextandaccessessyscallid(secondargument).Sowemarkitas+*__hidden,sothatlibbpfwillmarkitasstaticinthefinalobjectfile,+*rightbeforeverifyingitinthekernel.+*+*Butwedon'tmarkitas__hiddenhere,ratheratexternsite.__hiddenis+*"contaminating"visibility,soitwillgetpropagatedfromeitherexternor+*actualdefinition(includingfromthelosing__weakdefinition).+*/+voidset_output_ctx2(__u64*ctx)+{+output_ctx2=ctx[1];/* long id, same as in BPF_PROG below */+}++/* this weak instance should lose, because it will be processed second */+__weakintset_output_weak(intx)+{+output_weak2=x;+return2*x;+}++externintset_output_val1(intx);++/* here we'll force set_output_ctx1() to be __hidden in the final obj file */+__hiddenexternvoidset_output_ctx1(__u64*ctx);++SEC("raw_tp/sys_enter")+intBPF_PROG(handler2,structpt_regs*regs,longid)+{+if(my_tid!=(u32)bpf_get_current_pid_tgid()||id!=syscall_id)+return0;++set_output_val1(2000);+set_output_ctx1(ctx);/* ctx definition is hidden in BPF_PROG macro */++/* keep input value the same across both files to avoid dependency on+*handlercallorder;differentiatebyoutput_weak1vsoutput_weak2.+*/+set_output_weak(42);++return0;+}++charLICENSE[]SEC("license")="GPL";
While -Og is designed to work well with debugger, it's still inferior to -O0
in terms of debuggability experience. It will cause some variables to still be
inlined, it will also prevent single-stepping some statements and otherwise
interfere with debugging experience. So switch to -O0 which turns off any
optimization and provides the best debugging experience.
Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
---
tools/testing/selftests/bpf/Makefile | 8 ++++----
1 file changed, 4 insertions(+), 4 deletions(-)
Add extra logic to handle map externs (only BTF-defined maps are supported for
linking). Re-use the map parsing logic used during bpf_object__open(). Map
externs are currently restricted to always match complete map definition. So
all the specified attributes will be compared (down to pining, map_flags,
numa_node, etc). In the future this restriction might be relaxed with no
backwards compatibility issues. If any attribute is mismatched between extern
and actual map definition, linker will report an error, pointing out which one
mismatches.
The original intent was to allow for extern to specify attributes that matters
(to user) to enforce. E.g., if you specify just key information and omit
value, then any value fits. Similarly, it should have been possible to enforce
map_flags, pinning, and any other possible map attribute. Unfortunately, that
means that multiple externs can be only partially overlapping with each other,
which means linker would need to combine their type definitions to end up with
the most restrictive and fullest map definition. This requires an extra amount
of BTF manipulation which at this time was deemed unnecessary and would
require further extending generic BTF writer APIs. So that is left for future
follow ups, if there will be demand for that. But the idea seems intresting
and useful, so I want to document it here.
Weak definitions are also supported, but are pretty strict as well, just
like externs: all weak map definitions have to match exactly. In the follow up
patches this most probably will be relaxed, with __weak map definitions being
able to differ between each other (with non-weak definition always winning, of
course).
Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
---
tools/lib/bpf/linker.c | 132 +++++++++++++++++++++++++++++++++++++++++
1 file changed, 132 insertions(+)
@@ -1463,6 +1463,134 @@ static bool glob_sym_btf_matches(const char *sym_name, bool exact,}}+staticboolmap_defs_match(constchar*sym_name,+conststructbtf*main_btf,+conststructbtf_map_def*main_def,+conststructbtf_map_def*main_inner_def,+conststructbtf*extra_btf,+conststructbtf_map_def*extra_def,+conststructbtf_map_def*extra_inner_def)+{+constchar*reason;++if(main_def->map_type!=extra_def->map_type){+reason="type";+gotomismatch;+}++/* check key type/size match */+if(main_def->key_size!=extra_def->key_size){+reason="key_size";+gotomismatch;+}+if(!!main_def->key_type_id!=!!extra_def->key_type_id){+reason="key type";+gotomismatch;+}+if((main_def->parts&MAP_DEF_KEY_TYPE)+&&!glob_sym_btf_matches(sym_name,true/*exact*/,+main_btf,main_def->key_type_id,+extra_btf,extra_def->key_type_id)){+reason="key type";+gotomismatch;+}++/* validate value type/size match */+if(main_def->value_size!=extra_def->value_size){+reason="value_size";+gotomismatch;+}+if(!!main_def->value_type_id!=!!extra_def->value_type_id){+reason="value type";+gotomismatch;+}+if((main_def->parts&MAP_DEF_VALUE_TYPE)+&&!glob_sym_btf_matches(sym_name,true/*exact*/,+main_btf,main_def->value_type_id,+extra_btf,extra_def->value_type_id)){+reason="key type";+gotomismatch;+}++if(main_def->max_entries!=extra_def->max_entries){+reason="max_entries";+gotomismatch;+}+if(main_def->map_flags!=extra_def->map_flags){+reason="map_flags";+gotomismatch;+}+if(main_def->numa_node!=extra_def->numa_node){+reason="numa_node";+gotomismatch;+}+if(main_def->pinning!=extra_def->pinning){+reason="pinning";+gotomismatch;+}++if((main_def->parts&MAP_DEF_INNER_MAP)!=(extra_def->parts&MAP_DEF_INNER_MAP)){+reason="inner map";+gotomismatch;+}++if(main_def->parts&MAP_DEF_INNER_MAP){+charinner_map_name[128];++snprintf(inner_map_name,sizeof(inner_map_name),"%s.inner",sym_name);++returnmap_defs_match(inner_map_name,+main_btf,main_inner_def,NULL,+extra_btf,extra_inner_def,NULL);+}++returntrue;++mismatch:+pr_warn("global '%s': map %s mismatch\n",sym_name,reason);+returnfalse;+}++staticboolglob_map_defs_match(constchar*sym_name,+structbpf_linker*linker,structglob_sym*glob_sym,+structsrc_obj*obj,Elf64_Sym*sym,intbtf_id)+{+structbtf_map_defdst_def={},dst_inner_def={};+structbtf_map_defsrc_def={},src_inner_def={};+conststructbtf_type*t;+interr;++t=btf__type_by_id(obj->btf,btf_id);+if(!btf_is_var(t)){+pr_warn("global '%s': invalid map definition type [%d]\n",sym_name,btf_id);+returnfalse;+}+t=skip_mods_and_typedefs(obj->btf,t->type,NULL);++err=parse_btf_map_def(sym_name,obj->btf,t,true/*strict*/,&src_def,&src_inner_def);+if(err){+pr_warn("global '%s': invalid map definition\n",sym_name);+returnfalse;+}++/* re-parse existing map definition */+t=btf__type_by_id(linker->btf,glob_sym->btf_id);+t=skip_mods_and_typedefs(linker->btf,t->type,NULL);+err=parse_btf_map_def(sym_name,linker->btf,t,true/*strict*/,&dst_def,&dst_inner_def);+if(err){+/* this should not happen, because we already validated it */+pr_warn("global '%s': invalid dst map definition\n",sym_name);+returnfalse;+}++/* Currently extern map definition has to be complete and match+*concretemapdefinitionexactly.Thisrestrictionmightbelifted+*inthefuture.+*/+returnmap_defs_match(sym_name,linker->btf,&dst_def,&dst_inner_def,+obj->btf,&src_def,&src_inner_def);+}+staticboolglob_syms_match(constchar*sym_name,structbpf_linker*linker,structglob_sym*glob_sym,structsrc_obj*obj,Elf64_Sym*sym,size_tsym_idx,intbtf_id)
@@ -0,0 +1,43 @@+// SPDX-License-Identifier: GPL-2.0+/* Copyright (c) 2021 Facebook */++#include<test_progs.h>+#include<sys/syscall.h>+#include"linked_vars.skel.h"++voidtest_linked_vars(void)+{+interr;+structlinked_vars*skel;++skel=linked_vars__open();+if(!ASSERT_OK_PTR(skel,"skel_open"))+return;++skel->bss->input_bss1=1000;+skel->bss->input_bss2=2000;+skel->bss->input_bss_weak=3000;++err=linked_vars__load(skel);+if(!ASSERT_OK(err,"skel_load"))+gotocleanup;++err=linked_vars__attach(skel);+if(!ASSERT_OK(err,"skel_attach"))+gotocleanup;++/* trigger */+syscall(SYS_getpgid);++ASSERT_EQ(skel->bss->output_bss1,1000+2000+3000,"output_bss1");+ASSERT_EQ(skel->bss->output_bss2,1000+2000+3000,"output_bss2");+/* 10 comes from "winner" input_data_weak in first obj file */+ASSERT_EQ(skel->bss->output_data1,1+2+10,"output_bss1");+ASSERT_EQ(skel->bss->output_data2,1+2+10,"output_bss2");+/* 10 comes from "winner" input_rodata_weak in first obj file */+ASSERT_EQ(skel->bss->output_rodata1,11+22+100,"output_weak1");+ASSERT_EQ(skel->bss->output_rodata2,11+22+100,"output_weak2");++cleanup:+linked_vars__destroy(skel);+}
@@ -0,0 +1,54 @@+// SPDX-License-Identifier: GPL-2.0+/* Copyright (c) 2021 Facebook */++#include"vmlinux.h"+#include<bpf/bpf_helpers.h>+#include<bpf/bpf_tracing.h>++externintLINUX_KERNEL_VERSION__kconfig;+/* this weak extern will be strict due to the other file's strong extern */+externboolCONFIG_BPF_SYSCALL__kconfig__weak;+externconstvoidbpf_link_fops__ksym__weak;++intinput_bss1=0;+intinput_data1=1;+constvolatileintinput_rodata1=11;++intinput_bss_weak__weak=0;+/* these two definitions should win */+intinput_data_weak__weak=10;+constvolatileintinput_rodata_weak__weak=100;++externintinput_bss2;+externintinput_data2;+externconstintinput_rodata2;++intoutput_bss1=0;+intoutput_data1=0;+intoutput_rodata1=0;++longoutput_sink1=0;++static__noinlineintget_bss_res(void)+{+/* just make sure all the relocations work against .text as well */+returninput_bss1+input_bss2+input_bss_weak;+}++SEC("raw_tp/sys_enter")+intBPF_PROG(handler1)+{+output_bss1=get_bss_res();+output_data1=input_data1+input_data2+input_data_weak;+output_rodata1=input_rodata1+input_rodata2+input_rodata_weak;++/* make sure we actually use above special externs, otherwise compiler+*willoptimizethemout+*/+output_sink1=LINUX_KERNEL_VERSION++CONFIG_BPF_SYSCALL++(long)&bpf_link_fops;+return0;+}++charLICENSE[]SEC("license")="GPL";
@@ -0,0 +1,55 @@+// SPDX-License-Identifier: GPL-2.0+/* Copyright (c) 2021 Facebook */++#include"vmlinux.h"+#include<bpf/bpf_helpers.h>+#include<bpf/bpf_tracing.h>++externintLINUX_KERNEL_VERSION__kconfig;+/* when an extern is defined as both strong and weak, resulting symbol will be strong */+externboolCONFIG_BPF_SYSCALL__kconfig;+externconstvoid__start_BTF__ksym;++intinput_bss2=0;+intinput_data2=2;+constvolatileintinput_rodata2=22;++intinput_bss_weak__weak=0;+/* these two weak variables should lose */+intinput_data_weak__weak=20;+constvolatileintinput_rodata_weak__weak=200;++externintinput_bss1;+externintinput_data1;+externconstintinput_rodata1;++intoutput_bss2=0;+intoutput_data2=0;+intoutput_rodata2=0;++intoutput_sink2=0;++static__noinlineintget_data_res(void)+{+/* just make sure all the relocations work against .text as well */+returninput_data1+input_data2+input_data_weak;+}++SEC("raw_tp/sys_enter")+intBPF_PROG(handler2)+{+output_bss2=input_bss1+input_bss2+input_bss_weak;+output_data2=get_data_res();+output_rodata2=input_rodata1+input_rodata2+input_rodata_weak;++/* make sure we actually use above special externs, otherwise compiler+*willoptimizethemout+*/+output_sink2=LINUX_KERNEL_VERSION++CONFIG_BPF_SYSCALL++(long)&__start_BTF;++return0;+}++charLICENSE[]SEC("license")="GPL";
Add selftest validating various aspects of statically linking BTF-defined map
definitions. Legacy map definitions do not support extern resolution between
object files. Some of the aspects validated:
- correct resolution of extern maps against concrete map definitions;
- extern maps can currently only specify map type and key/value size and/or
type information;
- weak concrete map definitions are resolved properly.
Static map definitions are not yet supported by libbpf, so they are not
explicitly tested, though manual testing showes that BPF linker handles them
properly.
Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
---
tools/testing/selftests/bpf/Makefile | 4 +-
.../selftests/bpf/prog_tests/linked_maps.c | 30 +++++++
.../selftests/bpf/progs/linked_maps1.c | 82 +++++++++++++++++++
.../selftests/bpf/progs/linked_maps2.c | 76 +++++++++++++++++
4 files changed, 191 insertions(+), 1 deletion(-)
create mode 100644 tools/testing/selftests/bpf/prog_tests/linked_maps.c
create mode 100644 tools/testing/selftests/bpf/progs/linked_maps1.c
create mode 100644 tools/testing/selftests/bpf/progs/linked_maps2.c
@@ -0,0 +1,82 @@+// SPDX-License-Identifier: GPL-2.0+/* Copyright (c) 2021 Facebook */++#include"vmlinux.h"+#include<bpf/bpf_helpers.h>+#include<bpf/bpf_tracing.h>++structmy_key{longx;};+structmy_value{longx;};++struct{+__uint(type,BPF_MAP_TYPE_HASH);+__type(key,structmy_key);+__type(value,structmy_value);+__uint(max_entries,16);+}map1SEC(".maps");++/* Matches map2 definition in linked_maps2.c. Order of the attributes doesn't+*matter.+*/+typedefstruct{+__uint(max_entries,8);+__type(key,int);+__type(value,int);+__uint(type,BPF_MAP_TYPE_ARRAY);+}map2_t;++externmap2_tmap2SEC(".maps");++/* This should be the winning map definition, but we have no way of verifying,+*sowejustmakesurethatitlinksandworkswithouterrors+*/+struct{+__uint(type,BPF_MAP_TYPE_ARRAY);+__type(key,int);+__type(value,int);+__uint(max_entries,16);+}map_weak__weakSEC(".maps");++intoutput_first1=0;+intoutput_second1=0;+intoutput_weak1=0;++SEC("raw_tp/sys_enter")+intBPF_PROG(handler_enter1)+{+/* update values with key = 1 */+intkey=1,val=1;+structmy_keykey_struct={.x=1};+structmy_valueval_struct={.x=1000};++bpf_map_update_elem(&map1,&key_struct,&val_struct,0);+bpf_map_update_elem(&map2,&key,&val,0);+bpf_map_update_elem(&map_weak,&key,&val,0);++return0;+}++SEC("raw_tp/sys_exit")+intBPF_PROG(handler_exit1)+{+/* lookup values with key = 2, set in another file */+intkey=2,*val;+structmy_keykey_struct={.x=2};+structmy_value*value_struct;++value_struct=bpf_map_lookup_elem(&map1,&key_struct);+if(value_struct)+output_first1=value_struct->x;++val=bpf_map_lookup_elem(&map2,&key);+if(val)+output_second1=*val;++val=bpf_map_lookup_elem(&map_weak,&key);+if(val)+output_weak1=*val;++return0;+}++charLICENSE[]SEC("license")="GPL";
@@ -0,0 +1,76 @@+// SPDX-License-Identifier: GPL-2.0+/* Copyright (c) 2021 Facebook */++#include"vmlinux.h"+#include<bpf/bpf_helpers.h>+#include<bpf/bpf_tracing.h>++/* modifiers and typedefs are ignored when comparing key/value types */+typedefstructmy_key{longx;}key_type;+typedefstructmy_value{longx;}value_type;++externstruct{+__uint(max_entries,16);+__type(key,key_type);+__type(value,value_type);+__uint(type,BPF_MAP_TYPE_HASH);+}map1SEC(".maps");++struct{+__uint(type,BPF_MAP_TYPE_ARRAY);+__type(key,int);+__type(value,int);+__uint(max_entries,8);+}map2SEC(".maps");++/* this definition will lose, but it has to exactly match the winner */+struct{+__uint(type,BPF_MAP_TYPE_ARRAY);+__type(key,int);+__type(value,int);+__uint(max_entries,16);+}map_weak__weakSEC(".maps");++intoutput_first2=0;+intoutput_second2=0;+intoutput_weak2=0;++SEC("raw_tp/sys_enter")+intBPF_PROG(handler_enter2)+{+/* update values with key = 2 */+intkey=2,val=2;+key_typekey_struct={.x=2};+value_typeval_struct={.x=2000};++bpf_map_update_elem(&map1,&key_struct,&val_struct,0);+bpf_map_update_elem(&map2,&key,&val,0);+bpf_map_update_elem(&map_weak,&key,&val,0);++return0;+}++SEC("raw_tp/sys_exit")+intBPF_PROG(handler_exit2)+{+/* lookup values with key = 1, set in another file */+intkey=1,*val;+key_typekey_struct={.x=1};+value_type*value_struct;++value_struct=bpf_map_lookup_elem(&map1,&key_struct);+if(value_struct)+output_first2=value_struct->x;++val=bpf_map_lookup_elem(&map2,&key);+if(val)+output_second2=*val;++val=bpf_map_lookup_elem(&map_weak,&key);+if(val)+output_weak2=*val;++return0;+}++charLICENSE[]SEC("license")="GPL";
From: Yonghong Song <hidden> Date: 2021-04-22 03:02:29
On 4/16/21 1:23 PM, Andrii Nakryiko wrote:
Add dumping of "extern" linkage for BTF VAR kind. Also shorten
"global-allocated" to "global" to be in line with FUNC's "global".
Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
From: Yonghong Song <hidden> Date: 2021-04-22 03:09:51
On 4/16/21 1:23 PM, Andrii Nakryiko wrote:
Dump succinct information for each member of DATASEC: its kinds and name. This
is extremely helpful to see at a quick glance what is inside each DATASEC of
This is indeed very useful and user friendly!
a given BTF. Without this, one has to jump around BTF data to just find out
the name of a VAR or FUNC. DATASEC's var_secinfo member is special in that
regard because it doesn't itself contain the name of the member, delegating
that to the referenced VAR and FUNC kinds. Other kinds, like
STRUCT/UNION/FUNC/ENUM, encode member names directly and thus are clearly
identifiable in BTF dump.
The new output looks like this:
[35] DATASEC '.bss' size=0 vlen=6
type_id=8 offset=0 size=4 (VAR 'input_bss1')
type_id=13 offset=0 size=4 (VAR 'input_bss_weak')
type_id=16 offset=0 size=4 (VAR 'output_bss1')
type_id=17 offset=0 size=4 (VAR 'output_data1')
type_id=18 offset=0 size=4 (VAR 'output_rodata1')
type_id=20 offset=0 size=8 (VAR 'output_sink1')
[36] DATASEC '.data' size=0 vlen=2
type_id=9 offset=0 size=4 (VAR 'input_data1')
type_id=14 offset=0 size=4 (VAR 'input_data_weak')
[37] DATASEC '.kconfig' size=0 vlen=2
type_id=25 offset=0 size=4 (VAR 'LINUX_KERNEL_VERSION')
type_id=28 offset=0 size=1 (VAR 'CONFIG_BPF_SYSCALL')
[38] DATASEC '.ksyms' size=0 vlen=1
type_id=30 offset=0 size=1 (VAR 'bpf_link_fops')
[39] DATASEC '.rodata' size=0 vlen=2
type_id=12 offset=0 size=4 (VAR 'input_rodata1')
type_id=15 offset=0 size=4 (VAR 'input_rodata_weak')
[40] DATASEC 'license' size=0 vlen=1
type_id=24 offset=0 size=4 (VAR 'LICENSE')
Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
From: Yonghong Song <hidden> Date: 2021-04-22 03:48:20
On 4/16/21 1:23 PM, Andrii Nakryiko wrote:
When used on externs SEC() macro will trigger compilation warning about
inapplicable `__attribute__((used))`. That's expected for extern declarations,
so suppress it with the corresponding _Pragma.
Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
Ack with some comments below.
Acked-by: Yonghong Song <redacted>
The 'used' attribute is mostly useful for static variable/functions
since otherwise if not really used, the compiler could delete them
freely. The 'used' attribute does not really have an impact on
global variables regardless whether they are used or not in a particular
compilation unit. We could drop 'used' here and selftests should still
work. The only worry is that if people define something like
static int _version SEC("version") = 1;
static char _license[] SEC("license") = "GPL";
Removing 'used' may cause failure.
Since we don't want to remove 'used', then adding _Pragma to silence
the warning is a right thing to do here.
/* Avoid 'linux/stddef.h' definition of '__always_inline'. */
#undef __always_inline
On Wed, Apr 21, 2021 at 8:48 PM Yonghong Song [off-list ref] wrote:
On 4/16/21 1:23 PM, Andrii Nakryiko wrote:
quoted
When used on externs SEC() macro will trigger compilation warning about
inapplicable `__attribute__((used))`. That's expected for extern declarations,
so suppress it with the corresponding _Pragma.
Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
Ack with some comments below.
Acked-by: Yonghong Song <redacted>
The 'used' attribute is mostly useful for static variable/functions
since otherwise if not really used, the compiler could delete them
freely. The 'used' attribute does not really have an impact on
global variables regardless whether they are used or not in a particular
compilation unit. We could drop 'used' here and selftests should still
work. The only worry is that if people define something like
static int _version SEC("version") = 1;
static char _license[] SEC("license") = "GPL";
Removing 'used' may cause failure.
Since we don't want to remove 'used', then adding _Pragma to silence
the warning is a right thing to do here.
Right, SEC() has become a universal macro used for functions,
variables, and externs. I didn't want to introduce multiple variants,
but also we can't break existing use cases. So pragma, luckily,
allowed to support all cases.
quoted
/* Avoid 'linux/stddef.h' definition of '__always_inline'. */
#undef __always_inline
From: Yonghong Song <hidden> Date: 2021-04-22 05:43:34
On 4/16/21 1:23 PM, Andrii Nakryiko wrote:
quoted hunk
Define __hidden helper macro in bpf_helpers.h, which is a short-hand for
__attribute__((visibility("hidden"))). Add libbpf support to mark BPF
subprograms marked with __hidden as static in BTF information to enforce BPF
verifier's static function validation algorithm, which takes more information
(caller's context) into account during a subprogram validation.
Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
---
tools/lib/bpf/bpf_helpers.h | 8 ++++++
tools/lib/bpf/btf.c | 5 ----
tools/lib/bpf/libbpf.c | 45 ++++++++++++++++++++++++++++++++-
tools/lib/bpf/libbpf_internal.h | 6 +++++
4 files changed, 58 insertions(+), 6 deletions(-)
To prevent potential external __hidden macro definition conflict, how
about
#ifdef __hidden
#undef __hidden
#define __hidden __attribute__((visibility("hidden")))
#endif
quoted hunk
+
/* When utilizing vmlinux.h with BPF CO-RE, user BPF programs can't include
* any system-level headers (such as stddef.h, linux/version.h, etc), and
* commonly-used macros like NULL and KERNEL_VERSION aren't available through
@@ -698,6 +700,15 @@ bpf_object__add_programs(struct bpf_object *obj, Elf_Data *sec_data,if(err)returnerr;+/* if function is a global/weak symbol, but has hidden+*visibility(oranynon-defaultone),markitsBTFFUNCas+*statictoenablemorepermissiveBPFverificationmodewith+*moreoutsidecontextavailabletoBPFverifier+*/+if(GELF_ST_BIND(sym.st_info)!=STB_LOCAL+&&GELF_ST_VISIBILITY(sym.st_other)!=STV_DEFAULT)
Maybe we should check GELF_ST_VISIBILITY(sym.st_other) == STV_HIDDEN
instead?
@@ -2618,7 +2629,7 @@ static int bpf_object__sanitize_and_load_btf(struct bpf_object *obj) { struct btf *kern_btf = obj->btf; bool btf_mandatory, sanitize;- int err = 0;+ int i, err = 0; if (!obj->btf) return 0;
@@ -2632,6 +2643,38 @@ static int bpf_object__sanitize_and_load_btf(struct bpf_object *obj) return 0; }+ /* Even though some subprogs are global/weak, user might prefer more+ * permissive BPF verification process that BPF verifier performs for+ * static functions, taking into account more context from the caller+ * functions. In such case, they need to mark such subprogs with+ * __attribute__((visibility("hidden"))) and libbpf will adjust+ * corresponding FUNC BTF type to be marked as static and trigger more+ * involved BPF verification process.+ */+ for (i = 0; i < obj->nr_programs; i++) {+ struct bpf_program *prog = &obj->programs[i];+ struct btf_type *t;+ const char *name;+ int j, n;++ if (!prog->mark_btf_static || !prog_is_subprog(obj, prog))+ continue;++ n = btf__get_nr_types(obj->btf);+ for (j = 1; j <= n; j++) {+ t = btf_type_by_id(obj->btf, j);+ if (!btf_is_func(t) || btf_func_linkage(t) != BTF_FUNC_GLOBAL)+ continue;++ name = btf__str_by_offset(obj->btf, t->name_off);+ if (strcmp(name, prog->name) != 0)+ continue;++ t->info = btf_type_info(BTF_KIND_FUNC, BTF_FUNC_STATIC, 0);+ break;+ }+ }+ sanitize = btf_needs_sanitization(obj);
From: Yonghong Song <hidden> Date: 2021-04-22 06:26:20
On 4/16/21 1:23 PM, Andrii Nakryiko wrote:
Currently libbpf is very strict about parsing BPF program isnstruction
isnstruction => instruction
sections. No gaps are allowed between sequential BPF programs within a given
ELF section. Libbpf enforced that by keeping track of the next section offset
that should start a new BPF (sub)program and cross-checks that by searching for
a corresponding STT_FUNC ELF symbol.
But this is too restrictive once we allow to have weak BPF programs and link
together two or more BPF object files. In such case, some weak BPF programs
might be "overriden" by either non-weak BPF program with the same name and
overriden => overridden
signature, or even by another weak BPF program that just happened to be linked
first. That, in turn, leaves BPF instructions of the "lost" BPF (sub)program
intact, but there is no corresponding ELF symbol, because no one is going to
be referencing it.
Libbpf already correctly handles such cases in the sense that it won't append
such dead code to actual BPF programs loaded into kernel. So the only change
that needs to be done is to relax the logic of parsing BPF instruction
sections. Instead of assuming next BPF (sub)program section offset, iterate
available STT_FUNC ELF symbols to discover all available BPF subprograms and
programs.
Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
Ack with a minor suggestion below.
Acked-by: Yonghong Song <redacted>
From: Yonghong Song <hidden> Date: 2021-04-22 15:33:37
On 4/16/21 1:23 PM, Andrii Nakryiko wrote:
quoted hunk
Refactor BTF-defined maps parsing logic to allow it to be nicely reused by BPF
static linker. Further, at least for BPF static linker, it's important to know
which attributes of a BPF map were defined explicitly, so provide a bit set
for each known portion of BTF map definition. This allows BPF static linker to
do a simple check when dealing with extern map declarations.
The same capabilities allow to distinguish attributes explicitly set to zero
(e.g., __uint(max_entries, 0)) vs the case of not specifying it at all (no
max_entries attribute at all). Libbpf is currently not utilizing that, but it
could be useful for backwards compatibility reasons later.
Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
---
tools/lib/bpf/libbpf.c | 256 ++++++++++++++++++--------------
tools/lib/bpf/libbpf_internal.h | 32 ++++
2 files changed, 177 insertions(+), 111 deletions(-)
}
else if (strcmp(name, "values") == 0) {
+ char inner_map_name[128];
int err;
if (is_inner) {
pr_warn("map '%s': multi-level inner maps not supported.\n",
- map->name);
+ map_name);
return -ENOTSUP;
}
if (i != vlen - 1) {
pr_warn("map '%s': '%s' member should be last.\n",
- map->name, name);
+ map_name, name);
return -EINVAL;
}
- if (!bpf_map_type__is_map_in_map(map->def.type)) {
+ if (!bpf_map_type__is_map_in_map(map_def->map_type)) {
pr_warn("map '%s': should be map-in-map.\n",
- map->name);
+ map_name);
return -ENOTSUP;
}
- if (map->def.value_size && map->def.value_size != 4) {
+ if (map_def->value_size && map_def->value_size != 4) {
pr_warn("map '%s': conflicting value size %u != 4.\n",
- map->name, map->def.value_size);
+ map_name, map_def->value_size);
return -EINVAL;
}
- map->def.value_size = 4;
- t = btf__type_by_id(obj->btf, m->type);
+ map_def->value_size = 4;
+ t = btf__type_by_id(btf, m->type);
if (!t) {
pr_warn("map '%s': map-in-map inner type [%d] not found.\n",
- map->name, m->type);
+ map_name, m->type);
return -EINVAL;
}
if (!btf_is_array(t) || btf_array(t)->nelems) {
pr_warn("map '%s': map-in-map inner spec is not a zero-sized array.\n",
- map->name);
+ map_name);
return -EINVAL;
}
- t = skip_mods_and_typedefs(obj->btf, btf_array(t)->type,
- NULL);
+ t = skip_mods_and_typedefs(btf, btf_array(t)->type, NULL);
if (!btf_is_ptr(t)) {
pr_warn("map '%s': map-in-map inner def is of unexpected kind %s.\n",
- map->name, btf_kind_str(t));
+ map_name, btf_kind_str(t));
return -EINVAL;
}
- t = skip_mods_and_typedefs(obj->btf, t->type, NULL);
+ t = skip_mods_and_typedefs(btf, t->type, NULL);
if (!btf_is_struct(t)) {
pr_warn("map '%s': map-in-map inner def is of unexpected kind %s.\n",
- map->name, btf_kind_str(t));
+ map_name, btf_kind_str(t));
return -EINVAL;
}
- map->inner_map = calloc(1, sizeof(*map->inner_map));
- if (!map->inner_map)
- return -ENOMEM;
- map->inner_map->fd = -1;
- map->inner_map->sec_idx = obj->efile.btf_maps_shndx;
The refactoring didn't set map->inner_map->sec_idx. I guess since
inner_map is only used internally by libbpf, so it does not
have a user visible sec_idx and hence useless? It is worthwhile to
mention in the commit message for this difference, I think.
From: Yonghong Song <hidden> Date: 2021-04-22 16:06:35
On 4/16/21 1:23 PM, Andrii Nakryiko wrote:
Factor out logic for sanity checking SHT_SYMTAB and SHT_REL sections into
separate sections. They are already quite extensive and are suffering from too
deep indentation. Subsequent changes will extend SYMTAB sanity checking
further, so it's better to factor each into a separate function.
No functional changes are intended.
Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
Ack with a minor suggestion below.
Acked-by: Yonghong Song <redacted>
@@ -663,8 +665,8 @@ static bool is_pow_of_2(size_t x)staticintlinker_sanity_check_elf(structsrc_obj*obj){-structsrc_sec*sec,*link_sec;-inti,j,n;+structsrc_sec*sec;+inti,err;if(!obj->symtab_sec_idx){pr_warn("ELF is missing SYMTAB section in %s\n",obj->filename);
@@ -692,43 +694,11 @@ static int linker_sanity_check_elf(struct src_obj *obj)return-EINVAL;switch(sec->shdr->sh_type){-caseSHT_SYMTAB:{-Elf64_Sym*sym;--if(sec->shdr->sh_entsize!=sizeof(Elf64_Sym))-return-EINVAL;-if(sec->shdr->sh_size%sec->shdr->sh_entsize!=0)-return-EINVAL;--if(!sec->shdr->sh_link||sec->shdr->sh_link>=obj->sec_cnt){-pr_warn("ELF SYMTAB section #%zu points to missing STRTAB section #%zu in %s\n",-sec->sec_idx,(size_t)sec->shdr->sh_link,obj->filename);-return-EINVAL;-}-link_sec=&obj->secs[sec->shdr->sh_link];-if(link_sec->shdr->sh_type!=SHT_STRTAB){-pr_warn("ELF SYMTAB section #%zu points to invalid STRTAB section #%zu in %s\n",-sec->sec_idx,(size_t)sec->shdr->sh_link,obj->filename);-return-EINVAL;-}--n=sec->shdr->sh_size/sec->shdr->sh_entsize;-sym=sec->data->d_buf;-for(j=0;j<n;j++,sym++){-if(sym->st_shndx-&&sym->st_shndx<SHN_LORESERVE-&&sym->st_shndx>=obj->sec_cnt){-pr_warn("ELF sym #%d in section #%zu points to missing section #%zu in %s\n",-j,sec->sec_idx,(size_t)sym->st_shndx,obj->filename);-return-EINVAL;-}-if(ELF64_ST_TYPE(sym->st_info)==STT_SECTION){-if(sym->st_value!=0)-return-EINVAL;-}-}+caseSHT_SYMTAB:+err=linker_sanity_check_elf_symtab(obj,sec);+if(err)+returnerr;break;-}caseSHT_STRTAB:break;caseSHT_PROGBITS:
@@ -739,87 +709,138 @@ static int linker_sanity_check_elf(struct src_obj *obj)break;caseSHT_NOBITS:break;-caseSHT_REL:{-Elf64_Rel*relo;-structsrc_sec*sym_sec;+caseSHT_REL:+err=linker_sanity_check_elf_relos(obj,sec);+if(err)+returnerr;+break;+caseSHT_LLVM_ADDRSIG:+break;+default:+pr_warn("ELF section #%zu (%s) has unrecognized type %zu in %s\n",+sec->sec_idx,sec->sec_name,(size_t)sec->shdr->sh_type,obj->filename);+return-EINVAL;+}+}-if(sec->shdr->sh_entsize!=sizeof(Elf64_Rel))-return-EINVAL;-if(sec->shdr->sh_size%sec->shdr->sh_entsize!=0)-return-EINVAL;+return0;+}-/* SHT_REL's sh_link should point to SYMTAB */-if(sec->shdr->sh_link!=obj->symtab_sec_idx){-pr_warn("ELF relo section #%zu points to invalid SYMTAB section #%zu in %s\n",-sec->sec_idx,(size_t)sec->shdr->sh_link,obj->filename);-return-EINVAL;-}+staticintlinker_sanity_check_elf_symtab(structsrc_obj*obj,structsrc_sec*sec)+{+structsrc_sec*link_sec;+Elf64_Sym*sym;+inti,n;-/* SHT_REL's sh_info points to relocated section */-if(!sec->shdr->sh_info||sec->shdr->sh_info>=obj->sec_cnt){-pr_warn("ELF relo section #%zu points to missing section #%zu in %s\n",-sec->sec_idx,(size_t)sec->shdr->sh_info,obj->filename);-return-EINVAL;-}-link_sec=&obj->secs[sec->shdr->sh_info];+if(sec->shdr->sh_entsize!=sizeof(Elf64_Sym))+return-EINVAL;+if(sec->shdr->sh_size%sec->shdr->sh_entsize!=0)+return-EINVAL;++if(!sec->shdr->sh_link||sec->shdr->sh_link>=obj->sec_cnt){+pr_warn("ELF SYMTAB section #%zu points to missing STRTAB section #%zu in %s\n",+sec->sec_idx,(size_t)sec->shdr->sh_link,obj->filename);+return-EINVAL;+}+link_sec=&obj->secs[sec->shdr->sh_link];+if(link_sec->shdr->sh_type!=SHT_STRTAB){+pr_warn("ELF SYMTAB section #%zu points to invalid STRTAB section #%zu in %s\n",+sec->sec_idx,(size_t)sec->shdr->sh_link,obj->filename);+return-EINVAL;+}-/* .rel<secname> -> <secname> pattern is followed */-if(strncmp(sec->sec_name,".rel",sizeof(".rel")-1)!=0-||strcmp(sec->sec_name+sizeof(".rel")-1,link_sec->sec_name)!=0){-pr_warn("ELF relo section #%zu name has invalid name in %s\n",-sec->sec_idx,obj->filename);+n=sec->shdr->sh_size/sec->shdr->sh_entsize;+sym=sec->data->d_buf;+for(i=0;i<n;i++,sym++){+if(sym->st_shndx+&&sym->st_shndx<SHN_LORESERVE+&&sym->st_shndx>=obj->sec_cnt){+pr_warn("ELF sym #%d in section #%zu points to missing section #%zu in %s\n",+i,sec->sec_idx,(size_t)sym->st_shndx,obj->filename);+return-EINVAL;+}+if(ELF64_ST_TYPE(sym->st_info)==STT_SECTION){+if(sym->st_value!=0)return-EINVAL;-}+continue;
I think this "continue" is not necessary.
+ }
+ }
- /* don't further validate relocations for ignored sections */
- if (link_sec->skipped)
- break;
+ return 0;
+}
- /* relocatable section is data or instructions */
- if (link_sec->shdr->sh_type != SHT_PROGBITS
- && link_sec->shdr->sh_type != SHT_NOBITS) {
- pr_warn("ELF relo section #%zu points to invalid section #%zu in %s\n",
- sec->sec_idx, (size_t)sec->shdr->sh_info, obj->filename);
- return -EINVAL;
- }
From: Yonghong Song <hidden> Date: 2021-04-22 16:19:31
On 4/16/21 1:23 PM, Andrii Nakryiko wrote:
Make skip_mods_and_typedefs(), btf_kind_str(), and btf_func_linkage() helpers
available outside of libbpf.c, to be used by static linker code.
Also do few cleanups (error code fixes, comment clean up, etc) that don't
deserve their own commit.
In general, a separate commit will be good. In this case, we already
have quite some commits in the patch set. So it should be okay
to add some minor cleanups here.
From: Yonghong Song <hidden> Date: 2021-04-22 16:35:24
On 4/16/21 1:23 PM, Andrii Nakryiko wrote:
quoted hunk
Add logic to validate extern symbols, plus some other minor extra checks, like
ELF symbol #0 validation, general symbol visibility and binding validations.
Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
---
tools/lib/bpf/linker.c | 43 +++++++++++++++++++++++++++++++++---------
1 file changed, 34 insertions(+), 9 deletions(-)
@@ -750,14 +750,39 @@ static int linker_sanity_check_elf_symtab(struct src_obj *obj, struct src_sec *sn=sec->shdr->sh_size/sec->shdr->sh_entsize;sym=sec->data->d_buf;for(i=0;i<n;i++,sym++){-if(sym->st_shndx-&&sym->st_shndx<SHN_LORESERVE-&&sym->st_shndx>=obj->sec_cnt){+intsym_type=ELF64_ST_TYPE(sym->st_info);+intsym_bind=ELF64_ST_BIND(sym->st_info);++if(i==0){+if(sym->st_name!=0||sym->st_info!=0+||sym->st_other!=0||sym->st_shndx!=0+||sym->st_value!=0||sym->st_size!=0){+pr_warn("ELF sym #0 is invalid in %s\n",obj->filename);+return-EINVAL;+}+continue;+}
In ELF file, the first entry of symbol table and section table (index 0)
is invalid/undefined.
Symbol table '.symtab' contains 9 entries:
Num: Value Size Type Bind Vis Ndx Name
0: 0000000000000000 0 NOTYPE LOCAL DEFAULT UND
Section Headers:
[Nr] Name Type Address Off Size
ES Flg Lk Inf Al
[ 0] NULL 0000000000000000 000000 000000
00 0 0 0
Instead of validating them, I think we can skip traversal of the index =
0 entry for symbol table and section header table. What do you think?
quoted hunk
+ if (sym_bind != STB_LOCAL && sym_bind != STB_GLOBAL && sym_bind != STB_WEAK) {
+ pr_warn("ELF sym #%d is section #%zu has unsupported symbol binding %d\n",
+ i, sec->sec_idx, sym_bind);
+ return -EINVAL;
+ }
+ if (sym->st_shndx == 0) {
+ if (sym_type != STT_NOTYPE || sym_bind == STB_LOCAL
+ || sym->st_value != 0 || sym->st_size != 0) {
+ pr_warn("ELF sym #%d is invalid extern symbol in %s\n",
+ i, obj->filename);
+
+ return -EINVAL;
+ }
+ continue;
+ }
+ if (sym->st_shndx < SHN_LORESERVE && sym->st_shndx >= obj->sec_cnt) {
pr_warn("ELF sym #%d in section #%zu points to missing section #%zu in %s\n",
i, sec->sec_idx, (size_t)sym->st_shndx, obj->filename);
return -EINVAL;
}
- if (ELF64_ST_TYPE(sym->st_info) == STT_SECTION) {
+ if (sym_type == STT_SECTION) {
if (sym->st_value != 0)
return -EINVAL;
continue;
@@ -1135,16 +1160,16 @@ static int linker_append_elf_syms(struct bpf_linker *linker, struct src_obj *obj size_t dst_sym_idx; int name_off;- /* we already have all-zero initial symbol */- if (sym->st_name == 0 && sym->st_info == 0 &&- sym->st_other == 0 && sym->st_shndx == SHN_UNDEF &&- sym->st_value == 0 && sym->st_size ==0)+ /* We already validated all-zero symbol #0 and we already+ * appended it preventively to the final SYMTAB, so skip it.+ */+ if (i == 0) continue; sym_name = elf_strptr(obj->elf, str_sec_idx, sym->st_name); if (!sym_name) { pr_warn("can't fetch symbol name for symbol #%d in '%s'\n", i, obj->filename);- return -1;+ return -EINVAL; } if (sym->st_shndx && sym->st_shndx < SHN_LORESERVE) {
From: Yonghong Song <hidden> Date: 2021-04-22 16:50:45
On 4/16/21 1:23 PM, Andrii Nakryiko wrote:
It should never fail, but if it does, it's better to know about this rather
than end up with nonsensical type IDs.
So this is defensive programming. Maybe do another round of
audit of the callers and if you didn't find any issue, you
do not need to check not-happening condition here?
@@ -1423,6 +1423,15 @@ static int linker_fixup_btf(struct src_obj *obj)staticintremap_type_id(__u32*type_id,void*ctx){int*id_map=ctx;+intnew_id=id_map[*type_id];++if(*type_id==0)+return0;++if(new_id==0){+pr_warn("failed to find new ID mapping for original BTF type ID %u\n",*type_id);+return-EINVAL;+}*type_id=id_map[*type_id];
On Wed, Apr 21, 2021 at 10:43 PM Yonghong Song [off-list ref] wrote:
On 4/16/21 1:23 PM, Andrii Nakryiko wrote:
quoted
Define __hidden helper macro in bpf_helpers.h, which is a short-hand for
__attribute__((visibility("hidden"))). Add libbpf support to mark BPF
subprograms marked with __hidden as static in BTF information to enforce BPF
verifier's static function validation algorithm, which takes more information
(caller's context) into account during a subprogram validation.
Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
---
tools/lib/bpf/bpf_helpers.h | 8 ++++++
tools/lib/bpf/btf.c | 5 ----
tools/lib/bpf/libbpf.c | 45 ++++++++++++++++++++++++++++++++-
tools/lib/bpf/libbpf_internal.h | 6 +++++
4 files changed, 58 insertions(+), 6 deletions(-)
To prevent potential external __hidden macro definition conflict, how
about
#ifdef __hidden
#undef __hidden
#define __hidden __attribute__((visibility("hidden")))
#endif
We do force #undef only with __always_inline because of the bad
definition in linux/stddef.h And we check #ifndef for __weak, because
__weak is defined in kernel headers. This is not really the case for
__hidden, the only definition is in
tools/lib/traceevent/event-parse-local.h, which I don't think we
should worry about in BPF context. So I wanted to keep it simple and
fix only if that really causes some real conflicts.
And keep in mind that in BPF code bpf_helpers.h is usually included as
one of the first few headers anyways.
quoted
+
/* When utilizing vmlinux.h with BPF CO-RE, user BPF programs can't include
* any system-level headers (such as stddef.h, linux/version.h, etc), and
* commonly-used macros like NULL and KERNEL_VERSION aren't available through
[...]
quoted
@@ -698,6 +700,15 @@ bpf_object__add_programs(struct bpf_object *obj, Elf_Data *sec_data, if (err) return err;+ /* if function is a global/weak symbol, but has hidden+ * visibility (or any non-default one), mark its BTF FUNC as+ * static to enable more permissive BPF verification mode with+ * more outside context available to BPF verifier+ */+ if (GELF_ST_BIND(sym.st_info) != STB_LOCAL+ && GELF_ST_VISIBILITY(sym.st_other) != STV_DEFAULT)
Maybe we should check GELF_ST_VISIBILITY(sym.st_other) == STV_HIDDEN
instead?
It felt like only STV_DEFAULT should be "exported", semantically
speaking. Everything else would be treated as if it was static, except
that C rules require that function has to be global. Do you think
there is some danger to do it this way?
Currently static linker doesn't do anything special for STV_INTERNAL
and STV_PROTECTED, so we could just disable those. Do you prefer that?
I just felt that there is no risk of regression if we do this for
non-STV_DEFAULT generically.
On Wed, Apr 21, 2021 at 11:26 PM Yonghong Song [off-list ref] wrote:
On 4/16/21 1:23 PM, Andrii Nakryiko wrote:
quoted
Currently libbpf is very strict about parsing BPF program isnstruction
isnstruction => instruction
will fix
quoted
sections. No gaps are allowed between sequential BPF programs within a given
ELF section. Libbpf enforced that by keeping track of the next section offset
that should start a new BPF (sub)program and cross-checks that by searching for
a corresponding STT_FUNC ELF symbol.
But this is too restrictive once we allow to have weak BPF programs and link
together two or more BPF object files. In such case, some weak BPF programs
might be "overriden" by either non-weak BPF program with the same name and
overriden => overridden
will fix
quoted
signature, or even by another weak BPF program that just happened to be linked
first. That, in turn, leaves BPF instructions of the "lost" BPF (sub)program
intact, but there is no corresponding ELF symbol, because no one is going to
be referencing it.
Libbpf already correctly handles such cases in the sense that it won't append
such dead code to actual BPF programs loaded into kernel. So the only change
that needs to be done is to relax the logic of parsing BPF instruction
sections. Instead of assuming next BPF (sub)program section offset, iterate
available STT_FUNC ELF symbols to discover all available BPF subprograms and
programs.
Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
Ack with a minor suggestion below.
Acked-by: Yonghong Song <redacted>
On Thu, Apr 22, 2021 at 9:06 AM Yonghong Song [off-list ref] wrote:
On 4/16/21 1:23 PM, Andrii Nakryiko wrote:
quoted
Factor out logic for sanity checking SHT_SYMTAB and SHT_REL sections into
separate sections. They are already quite extensive and are suffering from too
deep indentation. Subsequent changes will extend SYMTAB sanity checking
further, so it's better to factor each into a separate function.
No functional changes are intended.
Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
Ack with a minor suggestion below.
Acked-by: Yonghong Song <redacted>
- /* .rel<secname> -> <secname> pattern is followed */
- if (strncmp(sec->sec_name, ".rel", sizeof(".rel") - 1) != 0
- || strcmp(sec->sec_name + sizeof(".rel") - 1, link_sec->sec_name) != 0) {
- pr_warn("ELF relo section #%zu name has invalid name in %s\n",
- sec->sec_idx, obj->filename);
+ n = sec->shdr->sh_size / sec->shdr->sh_entsize;
+ sym = sec->data->d_buf;
+ for (i = 0; i < n; i++, sym++) {
+ if (sym->st_shndx
+ && sym->st_shndx < SHN_LORESERVE
+ && sym->st_shndx >= obj->sec_cnt) {
+ pr_warn("ELF sym #%d in section #%zu points to missing section #%zu in %s\n",
+ i, sec->sec_idx, (size_t)sym->st_shndx, obj->filename);
+ return -EINVAL;
+ }
+ if (ELF64_ST_TYPE(sym->st_info) == STT_SECTION) {
+ if (sym->st_value != 0)
return -EINVAL;
- }
+ continue;
I think this "continue" is not necessary.
yes, but I wanted to make it clear that there should be no more logic
handling STT_SECTION afterwards, if/when we extend the logic of this
loop. We have similar patterns with goto to keep logic more modular
and easily extensible:
if (something) {
err = -EINVAL;
goto err_out;
}
err_out:
return err;
So let's keep it as is.
quoted
+ }
+ }
- /* don't further validate relocations for ignored sections */
- if (link_sec->skipped)
- break;
+ return 0;
+}
- /* relocatable section is data or instructions */
- if (link_sec->shdr->sh_type != SHT_PROGBITS
- && link_sec->shdr->sh_type != SHT_NOBITS) {
- pr_warn("ELF relo section #%zu points to invalid section #%zu in %s\n",
- sec->sec_idx, (size_t)sec->shdr->sh_info, obj->filename);
- return -EINVAL;
- }
On Thu, Apr 22, 2021 at 9:19 AM Yonghong Song [off-list ref] wrote:
On 4/16/21 1:23 PM, Andrii Nakryiko wrote:
quoted
Make skip_mods_and_typedefs(), btf_kind_str(), and btf_func_linkage() helpers
available outside of libbpf.c, to be used by static linker code.
Also do few cleanups (error code fixes, comment clean up, etc) that don't
deserve their own commit.
In general, a separate commit will be good. In this case, we already
have quite some commits in the patch set. So it should be okay
to add some minor cleanups here.
Yeah, that and it always feels like those minor clean ups that you
only ever discover as you work on some bigger feature would be a waste
of everyone's time, if submitted as individual patches.
On Thu, Apr 22, 2021 at 9:35 AM Yonghong Song [off-list ref] wrote:
On 4/16/21 1:23 PM, Andrii Nakryiko wrote:
quoted
Add logic to validate extern symbols, plus some other minor extra checks, like
ELF symbol #0 validation, general symbol visibility and binding validations.
Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
---
tools/lib/bpf/linker.c | 43 +++++++++++++++++++++++++++++++++---------
1 file changed, 34 insertions(+), 9 deletions(-)
@@ -750,14 +750,39 @@ static int linker_sanity_check_elf_symtab(struct src_obj *obj, struct src_sec *sn=sec->shdr->sh_size/sec->shdr->sh_entsize;sym=sec->data->d_buf;for(i=0;i<n;i++,sym++){-if(sym->st_shndx-&&sym->st_shndx<SHN_LORESERVE-&&sym->st_shndx>=obj->sec_cnt){+intsym_type=ELF64_ST_TYPE(sym->st_info);+intsym_bind=ELF64_ST_BIND(sym->st_info);++if(i==0){+if(sym->st_name!=0||sym->st_info!=0+||sym->st_other!=0||sym->st_shndx!=0+||sym->st_value!=0||sym->st_size!=0){+pr_warn("ELF sym #0 is invalid in %s\n",obj->filename);+return-EINVAL;+}+continue;+}
In ELF file, the first entry of symbol table and section table (index 0)
is invalid/undefined.
Symbol table '.symtab' contains 9 entries:
Num: Value Size Type Bind Vis Ndx Name
0: 0000000000000000 0 NOTYPE LOCAL DEFAULT UND
Section Headers:
[Nr] Name Type Address Off Size
ES Flg Lk Inf Al
[ 0] NULL 0000000000000000 000000 000000
00 0 0 0
Instead of validating them, I think we can skip traversal of the index =
0 entry for symbol table and section header table. What do you think?
In valid ELF yes. But then entire sanity check logic is not needed if
we just assume correct ELF. But I don't want to make that potentially
dangerous assumption :) Here I'm validating that ELF is sane with
minimal efforts. I do skip symbol #0 later because I validated that
it's all-zero one, as expected by ELF standard.
quoted
+ if (sym_bind != STB_LOCAL && sym_bind != STB_GLOBAL && sym_bind != STB_WEAK) {
+ pr_warn("ELF sym #%d is section #%zu has unsupported symbol binding %d\n",
+ i, sec->sec_idx, sym_bind);
+ return -EINVAL;
+ }
+ if (sym->st_shndx == 0) {
+ if (sym_type != STT_NOTYPE || sym_bind == STB_LOCAL
+ || sym->st_value != 0 || sym->st_size != 0) {
+ pr_warn("ELF sym #%d is invalid extern symbol in %s\n",
+ i, obj->filename);
+
+ return -EINVAL;
+ }
+ continue;
+ }
+ if (sym->st_shndx < SHN_LORESERVE && sym->st_shndx >= obj->sec_cnt) {
pr_warn("ELF sym #%d in section #%zu points to missing section #%zu in %s\n",
i, sec->sec_idx, (size_t)sym->st_shndx, obj->filename);
return -EINVAL;
}
- if (ELF64_ST_TYPE(sym->st_info) == STT_SECTION) {
+ if (sym_type == STT_SECTION) {
if (sym->st_value != 0)
return -EINVAL;
continue;
@@ -1135,16 +1160,16 @@ static int linker_append_elf_syms(struct bpf_linker *linker, struct src_obj *obj size_t dst_sym_idx; int name_off;- /* we already have all-zero initial symbol */- if (sym->st_name == 0 && sym->st_info == 0 &&- sym->st_other == 0 && sym->st_shndx == SHN_UNDEF &&- sym->st_value == 0 && sym->st_size ==0)+ /* We already validated all-zero symbol #0 and we already+ * appended it preventively to the final SYMTAB, so skip it.+ */+ if (i == 0) continue; sym_name = elf_strptr(obj->elf, str_sec_idx, sym->st_name); if (!sym_name) { pr_warn("can't fetch symbol name for symbol #%d in '%s'\n", i, obj->filename);- return -1;+ return -EINVAL; } if (sym->st_shndx && sym->st_shndx < SHN_LORESERVE) {
On Thu, Apr 22, 2021 at 9:50 AM Yonghong Song [off-list ref] wrote:
On 4/16/21 1:23 PM, Andrii Nakryiko wrote:
quoted
It should never fail, but if it does, it's better to know about this rather
than end up with nonsensical type IDs.
So this is defensive programming. Maybe do another round of
audit of the callers and if you didn't find any issue, you
do not need to check not-happening condition here?
It's far from obvious that this will never happen, because we do a
decently complicated BTF processing (we skip some types altogether
believing that they are not used, for example) and it will only get
more complicated with time. Just as there are "verifier bug" checks in
kernel, this prevents things from going wild if non-trivial bugs will
inevitably happen.
@@ -1423,6 +1423,15 @@ static int linker_fixup_btf(struct src_obj *obj)staticintremap_type_id(__u32*type_id,void*ctx){int*id_map=ctx;+intnew_id=id_map[*type_id];++if(*type_id==0)+return0;++if(new_id==0){+pr_warn("failed to find new ID mapping for original BTF type ID %u\n",*type_id);+return-EINVAL;+}*type_id=id_map[*type_id];
On Thu, Apr 22, 2021 at 8:33 AM Yonghong Song [off-list ref] wrote:
On 4/16/21 1:23 PM, Andrii Nakryiko wrote:
quoted
Refactor BTF-defined maps parsing logic to allow it to be nicely reused by BPF
static linker. Further, at least for BPF static linker, it's important to know
which attributes of a BPF map were defined explicitly, so provide a bit set
for each known portion of BTF map definition. This allows BPF static linker to
do a simple check when dealing with extern map declarations.
The same capabilities allow to distinguish attributes explicitly set to zero
(e.g., __uint(max_entries, 0)) vs the case of not specifying it at all (no
max_entries attribute at all). Libbpf is currently not utilizing that, but it
could be useful for backwards compatibility reasons later.
Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
---
tools/lib/bpf/libbpf.c | 256 ++++++++++++++++++--------------
tools/lib/bpf/libbpf_internal.h | 32 ++++
2 files changed, 177 insertions(+), 111 deletions(-)
}
else if (strcmp(name, "values") == 0) {
+ char inner_map_name[128];
int err;
if (is_inner) {
pr_warn("map '%s': multi-level inner maps not supported.\n",
- map->name);
+ map_name);
return -ENOTSUP;
}
if (i != vlen - 1) {
pr_warn("map '%s': '%s' member should be last.\n",
- map->name, name);
+ map_name, name);
return -EINVAL;
}
- if (!bpf_map_type__is_map_in_map(map->def.type)) {
+ if (!bpf_map_type__is_map_in_map(map_def->map_type)) {
pr_warn("map '%s': should be map-in-map.\n",
- map->name);
+ map_name);
return -ENOTSUP;
}
- if (map->def.value_size && map->def.value_size != 4) {
+ if (map_def->value_size && map_def->value_size != 4) {
pr_warn("map '%s': conflicting value size %u != 4.\n",
- map->name, map->def.value_size);
+ map_name, map_def->value_size);
return -EINVAL;
}
- map->def.value_size = 4;
- t = btf__type_by_id(obj->btf, m->type);
+ map_def->value_size = 4;
+ t = btf__type_by_id(btf, m->type);
if (!t) {
pr_warn("map '%s': map-in-map inner type [%d] not found.\n",
- map->name, m->type);
+ map_name, m->type);
return -EINVAL;
}
if (!btf_is_array(t) || btf_array(t)->nelems) {
pr_warn("map '%s': map-in-map inner spec is not a zero-sized array.\n",
- map->name);
+ map_name);
return -EINVAL;
}
- t = skip_mods_and_typedefs(obj->btf, btf_array(t)->type,
- NULL);
+ t = skip_mods_and_typedefs(btf, btf_array(t)->type, NULL);
if (!btf_is_ptr(t)) {
pr_warn("map '%s': map-in-map inner def is of unexpected kind %s.\n",
- map->name, btf_kind_str(t));
+ map_name, btf_kind_str(t));
return -EINVAL;
}
- t = skip_mods_and_typedefs(obj->btf, t->type, NULL);
+ t = skip_mods_and_typedefs(btf, t->type, NULL);
if (!btf_is_struct(t)) {
pr_warn("map '%s': map-in-map inner def is of unexpected kind %s.\n",
- map->name, btf_kind_str(t));
+ map_name, btf_kind_str(t));
return -EINVAL;
}
- map->inner_map = calloc(1, sizeof(*map->inner_map));
- if (!map->inner_map)
- return -ENOMEM;
- map->inner_map->fd = -1;
- map->inner_map->sec_idx = obj->efile.btf_maps_shndx;
The refactoring didn't set map->inner_map->sec_idx. I guess since
inner_map is only used internally by libbpf, so it does not
have a user visible sec_idx and hence useless? It is worthwhile to
mention in the commit message for this difference, I think.
I'll add it back here, but it was never really necessary. More for the
completeness sake. sec_idx is used only to match relo to a map, and
this inner_map is never matched and never referenced by a relocation.
From: Yonghong Song <hidden> Date: 2021-04-22 21:27:27
On 4/16/21 1:23 PM, Andrii Nakryiko wrote:
Add BPF static linker logic to resolve extern variables and functions across
multiple linked together BPF object files.
For that, linker maintains a separate list of struct glob_sym structures,
which keeps track of few pieces of metadata (is it extern or resolved global,
is it a weak symbol, which ELF section it belongs to, etc) and ties together
BTF type info and ELF symbol information and keeps them in sync.
With adding support for extern variables/funcs, it's now possible for some
sections to contain both extern and non-extern definitions. This means that
some sections may start out as ephemeral (if only externs are present and thus
there is not corresponding ELF section), but will be "upgraded" to actual ELF
section as symbols are resolved or new non-extern definitions are appended.
Additional care is taken to not duplicate extern entries in sections like
.kconfig and .ksyms.
Given libbpf requires BTF type to always be present for .kconfig/.ksym
externs, linker extends this requirement to all the externs, even those that
are supposed to be resolved during static linking and which won't be visible
to libbpf. With BTF information always present, static linker will check not
just ELF symbol matches, but entire BTF type signature match as well. That
logic is stricter that BPF CO-RE checks. It probably should be re-used by
.ksym resolution logic in libbpf as well, but that's left for follow up
patches.
To make it unnecessary to rewrite ELF symbols and minimize BTF type
rewriting/removal, ELF symbols that correspond to externs initially will be
updated in place once they are resolved. Similarly for BTF type info, VAR/FUNC
and var_secinfo's (sec_vars in struct bpf_linker) are staying stable, but
types they point to might get replaced when extern is resolved. This might
leave some left-over types (even though we try to minimize this for common
cases of having extern funcs with not argument names vs concrete function with
names properly specified). That can be addresses later with a generic BTF
garbage collection. That's left for a follow up as well.
Given BTF type appending phase is separate from ELF symbol
appending/resolution, special struct glob_sym->underlying_btf_id variable is
used to communicate resolution and rewrite decisions. 0 means
underlying_btf_id needs to be appended (it's not yet in final linker->btf), <0
values are used for temporary storage of source BTF type ID (not yet
rewritten), so -glob_sym->underlying_btf_id is BTF type id in obj-btf. But by
the end of linker_append_btf() phase, that underlying_btf_id will be remapped
and will always be > 0. This is the uglies part of the whole process, but
keeps the other parts much simpler due to stability of sec_var and VAR/FUNC
types, as well as ELF symbol, so please keep that in mind while reviewing.
This is indeed complicated. I has some comments below. Please check
whether my understanding is correct or not.
quoted hunk
BTF-defined maps require some extra custom logic and is addressed separate in
the next patch, so that to keep this one smaller and easier to review.
Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
---
tools/lib/bpf/linker.c | 844 ++++++++++++++++++++++++++++++++++++++---
1 file changed, 785 insertions(+), 59 deletions(-)
@@ -22,6 +22,8 @@#include"libbpf_internal.h"#include"strset.h"+#define BTF_EXTERN_SEC ".extern"+structsrc_sec{constchar*sec_name;/* positional (not necessarily ELF) index in an array of sections */
@@ -74,11 +76,36 @@ struct btf_ext_sec_data {void*recs;};+structglob_sym{+/* ELF symbol index */+intsym_idx;+/* associated section id for .ksyms, .kconfig, etc, but not .extern */+intsec_id;+/* extern name offset in STRTAB */+intname_off;+/* optional associated BTF type ID */+intbtf_id;+/* BTF type ID to which VAR/FUNC type is pointing to; used for+*rewritingtypeswhenexternVAR/FUNCisresolvedtoaconcrete+*definition+*/+intunderlying_btf_id;+/* sec_var index in the corresponding dst_sec, if exists */+intvar_idx;++/* extern or resolved/global symbol */+boolis_extern;+/* weak or strong symbol, never goes back from strong to weak */+boolis_weak;+};+structdst_sec{char*sec_name;/* positional (not necessarily ELF) index in an array of sections */intid;+boolephemeral;+/* ELF info */size_tsec_idx;Elf_Scn*scn;
+
+static bool glob_sym_btf_matches(const char *sym_name, bool exact,
+ const struct btf *btf1, __u32 id1,
+ const struct btf *btf2, __u32 id2)
+{
+ const struct btf_type *t1, *t2;
+ bool is_static1, is_static2;
+ const char *n1, *n2;
+ int i, n;
+
+recur:
+ n1 = n2 = NULL;
+ t1 = skip_mods_and_typedefs(btf1, id1, &id1);
+ t2 = skip_mods_and_typedefs(btf2, id2, &id2);
+
+ /* check if only one side is FWD, otherwise handle with common logic */
+ if (!exact && btf_is_fwd(t1) != btf_is_fwd(t2)) {
+ n1 = btf__str_by_offset(btf1, t1->name_off);
+ n2 = btf__str_by_offset(btf2, t2->name_off);
+ if (strcmp(n1, n2) != 0) {
+ pr_warn("global '%s': incompatible forward declaration names '%s' and '%s'\n",
+ sym_name, n1, n2);
+ return false;
+ }
+ /* validate if FWD kind matches concrete kind */
+ if (btf_is_fwd(t1)) {
+ if (btf_kflag(t1) && btf_is_union(t2))
+ return true;
+ if (!btf_kflag(t1) && btf_is_struct(t2))
+ return true;
+ pr_warn("global '%s': incompatible %s forward declaration and concrete kind %s\n",
+ sym_name, btf_kflag(t1) ? "union" : "struct", btf_kind_str(t2));
+ } else {
+ if (btf_kflag(t2) && btf_is_union(t1))
+ return true;
+ if (!btf_kflag(t2) && btf_is_struct(t1))
+ return true;
+ pr_warn("global '%s': incompatible %s forward declaration and concrete kind %s\n",
+ sym_name, btf_kflag(t2) ? "union" : "struct", btf_kind_str(t1));
+ }
+ return false;
+ }
+
+ if (btf_kind(t1) != btf_kind(t2)) {
+ pr_warn("global '%s': incompatible BTF kinds %s and %s\n",
+ sym_name, btf_kind_str(t1), btf_kind_str(t2));
+ return false;
+ }
+
+ switch (btf_kind(t1)) {
+ case BTF_KIND_STRUCT:
+ case BTF_KIND_UNION:
+ case BTF_KIND_ENUM:
+ case BTF_KIND_FWD:
+ case BTF_KIND_FUNC:
+ case BTF_KIND_VAR:
+ n1 = btf__str_by_offset(btf1, t1->name_off);
+ n2 = btf__str_by_offset(btf2, t2->name_off);
+ if (strcmp(n1, n2) != 0) {
+ pr_warn("global '%s': incompatible %s names '%s' and '%s'\n",
+ sym_name, btf_kind_str(t1), n1, n2);
+ return false;
+ }
+ break;
+ default:
+ break;
+ }
+
+ switch (btf_kind(t1)) {
+ case BTF_KIND_UNKN: /* void */
+ case BTF_KIND_FWD:
+ return true;
+ case BTF_KIND_INT:
+ case BTF_KIND_FLOAT:
+ case BTF_KIND_ENUM:
+ /* ignore encoding for int and enum values for enum */
+ if (t1->size != t2->size) {
+ pr_warn("global '%s': incompatible %s '%s' size %u and %u\n",
+ sym_name, btf_kind_str(t1), n1, t1->size, t2->size);
+ return false;
+ }
+ return true;
+ case BTF_KIND_PTR:
+ /* just validate overall shape of the referenced type, so no
+ * contents comparison for struct/union, and allowd fwd vs
+ * struct/union
+ */
+ exact = false;
+ id1 = t1->type;
+ id2 = t2->type;
+ goto recur;
+ case BTF_KIND_ARRAY:
+ /* ignore index type and array size */
+ id1 = btf_array(t1)->type;
+ id2 = btf_array(t2)->type;
+ goto recur;
+ case BTF_KIND_FUNC:
+ /* extern and global linkages are compatible */
+ is_static1 = btf_func_linkage(t1) == BTF_FUNC_STATIC;
+ is_static2 = btf_func_linkage(t2) == BTF_FUNC_STATIC;
+ if (is_static1 != is_static2) {
+ pr_warn("global '%s': incompatible func '%s' linkage\n", sym_name, n1);
+ return false;
+ }
+
+ id1 = t1->type;
+ id2 = t2->type;
+ goto recur;
+ case BTF_KIND_VAR:
+ /* extern and global linkages are compatible */
+ is_static1 = btf_var(t1)->linkage == BTF_VAR_STATIC;
+ is_static2 = btf_var(t2)->linkage == BTF_VAR_STATIC;
+ if (is_static1 != is_static2) {
+ pr_warn("global '%s': incompatible var '%s' linkage\n", sym_name, n1);
+ return false;
+ }
+
+ id1 = t1->type;
+ id2 = t2->type;
+ goto recur;
+ case BTF_KIND_STRUCT:
+ case BTF_KIND_UNION: {
+ const struct btf_member *m1, *m2;
+
+ if (!exact)
+ return true;
+
+ if (btf_vlen(t1) != btf_vlen(t2)) {
+ pr_warn("global '%s': incompatible number of %s fields %u and %u\n",
+ sym_name, btf_kind_str(t1), btf_vlen(t1), btf_vlen(t2));
+ return false;
+ }
+
+ n = btf_vlen(t1);
+ m1 = btf_members(t1);
+ m2 = btf_members(t2);
+ for (i = 0; i < n; i++, m1++, m2++) {
+ n1 = btf__str_by_offset(btf1, m1->name_off);
+ n2 = btf__str_by_offset(btf2, m2->name_off);
+ if (strcmp(n1, n2) != 0) {
+ pr_warn("global '%s': incompatible field #%d names '%s' and '%s'\n",
+ sym_name, i, n1, n2);
+ return false;
+ }
+ if (m1->offset != m2->offset) {
+ pr_warn("global '%s': incompatible field #%d ('%s') offsets\n",
+ sym_name, i, n1);
+ return false;
+ }
+ if (!glob_sym_btf_matches(sym_name, exact, btf1, m1->type, btf2, m2->type))
+ return false;
+ }
+
+ return true;
+ }
+ case BTF_KIND_FUNC_PROTO: {
+ const struct btf_param *m1, *m2;
+
+ if (btf_vlen(t1) != btf_vlen(t2)) {
+ pr_warn("global '%s': incompatible number of %s params %u and %u\n",
+ sym_name, btf_kind_str(t1), btf_vlen(t1), btf_vlen(t2));
+ return false;
+ }
+
+ n = btf_vlen(t1);
+ m1 = btf_params(t1);
+ m2 = btf_params(t2);
+ for (i = 0; i < n; i++, m1++, m2++) {
+ /* ignore func arg names */
+ if (!glob_sym_btf_matches(sym_name, exact, btf1, m1->type, btf2, m2->type))
+ return false;
+ }
+
+ /* now check return type as well */
+ id1 = t1->type;
+ id2 = t2->type;
+ goto recur;
+ }
+
+ case BTF_KIND_TYPEDEF:
+ case BTF_KIND_VOLATILE:
+ case BTF_KIND_CONST:
+ case BTF_KIND_RESTRICT:
We already did skip_mods_and_typedefs() before. Unless something serious
wrong, we should not hit the above four types. So I think we can skip
them here.
+ case BTF_KIND_DATASEC:
+ default:
+ pr_warn("global '%s': unsupported BTF kind %s\n",
+ sym_name, btf_kind_str(t1));
+ return false;
+ }
+}
+
+static bool glob_syms_match(const char *sym_name,
+ struct bpf_linker *linker, struct glob_sym *glob_sym,
+ struct src_obj *obj, Elf64_Sym *sym, size_t sym_idx, int btf_id)
+{
+ const struct btf_type *src_t;
+
+ /* if we are dealing with externs, BTF types describing both global
+ * and extern VARs/FUNCs should be completely present in all files
+ */
+ if (!glob_sym->btf_id || !btf_id) {
+ pr_warn("BTF info is missing for global symbol '%s'\n", sym_name);
+ return false;
+ }
+
+ src_t = btf__type_by_id(obj->btf, btf_id);
+ if (!btf_is_var(src_t) && !btf_is_func(src_t)) {
+ pr_warn("only extern variables and functions are supported, but got '%s' for '%s'\n",
+ btf_kind_str(src_t), sym_name);
+ return false;
+ }
+
+ if (!glob_sym_btf_matches(sym_name, true /*exact*/,
+ linker->btf, glob_sym->btf_id, obj->btf, btf_id))
+ return false;
+
+ return true;
+}
+
[...]
+
+static void sym_update_visibility(Elf64_Sym *sym, int sym_vis)
+{
+ /* libelf doesn't provide setters for ST_VISIBILITY,
+ * but it is stored in the lower 2 bits of st_other
+ */
+ sym->st_other &= 0x03;
+ sym->st_other |= sym_vis;
+}
+
+static int linker_append_elf_sym(struct bpf_linker *linker, struct src_obj *obj,
+ Elf64_Sym *sym, const char *sym_name, int src_sym_idx)
+{
+ struct src_sec *src_sec = NULL;
+ struct dst_sec *dst_sec = NULL;
+ struct glob_sym *glob_sym = NULL;
+ int name_off, sym_type, sym_bind, sym_vis, err;
+ int btf_sec_id = 0, btf_id = 0;
+ size_t dst_sym_idx;
+ Elf64_Sym *dst_sym;
+ bool sym_is_extern;
+
+ sym_type = ELF64_ST_TYPE(sym->st_info);
+ sym_bind = ELF64_ST_BIND(sym->st_info);
+ sym_vis = ELF64_ST_VISIBILITY(sym->st_other);
+ sym_is_extern = sym->st_shndx == SHN_UNDEF;
+
+ if (sym_is_extern) {
+ if (!obj->btf) {
+ pr_warn("externs without BTF info are not supported\n");
+ return -ENOTSUP;
+ }
+ } else if (sym->st_shndx < SHN_LORESERVE) {
So what happens if sym->st_shndx >= SHN_LORESERVE. Maybe return failures
here? In general, bpf program shouldn't hit sym->st_shndx >= SHN_LORESERVE.
+ src_sec = &obj->secs[sym->st_shndx];
+ if (src_sec->skipped)
+ return 0;
+ dst_sec = &linker->secs[src_sec->dst_id];
+
+ /* allow only one STT_SECTION symbol per section */
+ if (sym_type == STT_SECTION && dst_sec->sec_sym_idx) {
+ obj->sym_map[src_sym_idx] = dst_sec->sec_sym_idx;
+ return 0;
+ }
+ }
+
+ if (sym_bind == STB_LOCAL)
+ goto add_sym;
+
+ /* find matching BTF info */
+ err = find_glob_sym_btf(obj, sym, sym_name, &btf_sec_id, &btf_id);
+ if (err)
+ return err;
+
+ if (sym_is_extern && btf_sec_id) {
+ const char *sec_name = NULL;
+ const struct btf_type *t;
+
+ t = btf__type_by_id(obj->btf, btf_sec_id);
+ sec_name = btf__str_by_offset(obj->btf, t->name_off);
+
+ /* Clang puts unannotated extern vars into
+ * '.extern' BTF DATASEC. Treat them the same
+ * as unannotated extern funcs (which are
+ * currently not put into any DATASECs).
+ * Those don't have associated src_sec/dst_sec.
+ */
+ if (strcmp(sec_name, BTF_EXTERN_SEC) != 0) {
+ src_sec = find_src_sec_by_name(obj, sec_name);
+ if (!src_sec) {
+ pr_warn("failed to find matching ELF sec '%s'\n", sec_name);
+ return -ENOENT;
+ }
+ dst_sec = &linker->secs[src_sec->dst_id];
+ }
+ }
+
+ glob_sym = find_glob_sym(linker, sym_name);
+ if (glob_sym) {
+ /* Preventively resolve to existing symbol. This is
+ * needed for further relocation symbol remapping in
+ * the next step of linking.
+ */
+ obj->sym_map[src_sym_idx] = glob_sym->sym_idx;
+
+ /* If both symbols are non-externs, at least one of
+ * them has to be STB_WEAK, otherwise they are in
+ * a conflict with each other.
+ */
+ if (!sym_is_extern && !glob_sym->is_extern
+ && !glob_sym->is_weak && sym_bind != STB_WEAK) {
+ pr_warn("conflicting non-weak symbol #%d (%s) definition in '%s'\n",
+ src_sym_idx, sym_name, obj->filename);
+ return -EINVAL;
}
+ if (!glob_syms_match(sym_name, linker, glob_sym, obj, sym, src_sym_idx, btf_id))
+ return -EINVAL;
+
+ dst_sym = get_sym_by_idx(linker, glob_sym->sym_idx);
+
+ /* If new symbol is strong, then force dst_sym to be strong as
+ * well; this way a mix of weak and non-weak extern
+ * definitions will end up being strong.
+ */
+ if (sym_bind == STB_GLOBAL) {
+ /* We still need to preserve type (NOTYPE or
+ * OBJECT/FUNC, depending on whether the symbol is
+ * extern or not)
+ */
+ sym_update_bind(dst_sym, STB_GLOBAL);
+ glob_sym->is_weak = false;
+ }
+
+ /* Non-default visibility is "contaminating", with stricter
+ * visibility overwriting more permissive ones, even if more
+ * permissive visibility comes from just an extern definition
+ */
+ if (sym_vis > ELF64_ST_VISIBILITY(dst_sym->st_other))
+ sym_update_visibility(dst_sym, sym_vis);
For visibility, maybe we can just handle DEFAULT and HIDDEN, and others
are not supported? DEFAULT + DEFAULT/HIDDEN => DEFAULT, HIDDEN + HIDDEN
=> HIDDEN?
+
+ /* If the new symbol is extern, then regardless if
+ * existing symbol is extern or resolved global, just
+ * keep the existing one untouched.
+ */
+ if (sym_is_extern)
+ return 0;
+
+ /* If existing symbol is a strong resolved symbol, bail out,
+ * because we lost resolution battle have nothing to
+ * contribute. We already checked abover that there is no
+ * strong-strong conflict. We also already tightened binding
+ * and visibility, so nothing else to contribute at that point.
+ */
+ if (!glob_sym->is_extern && sym_bind == STB_WEAK)
+ return 0;
+
+ /* At this point, new symbol is strong non-extern,
+ * so overwrite glob_sym with new symbol information.
+ * Preserve binding and visibility.
+ */
+ sym_update_type(dst_sym, sym_type);
+ dst_sym->st_shndx = dst_sec->sec_idx;
+ dst_sym->st_value = src_sec->dst_off + sym->st_value;
+ dst_sym->st_size = sym->st_size;
+
+ /* see comment below about dst_sec->id vs dst_sec->sec_idx */
+ glob_sym->sec_id = dst_sec->id;
+ glob_sym->is_extern = false;
+ /* never relax strong to weak binding */
+ if (sym_bind == STB_GLOBAL)
+ glob_sym->is_weak = false;
In the above, we already set glob_sym->is_weak to false if STB_GLOBAL.
quoted hunk
+
+ if (complete_extern_btf_info(linker->btf, glob_sym->btf_id,
+ obj->btf, btf_id))
+ return -EINVAL;
+
+ /* request updating VAR's/FUNC's underlying BTF type when appending BTF type */
+ glob_sym->underlying_btf_id = 0;
+
+ obj->sym_map[src_sym_idx] = glob_sym->sym_idx;
+ return 0;
+ }
+
+add_sym:
+ name_off = strset__add_str(linker->strtab_strs, sym_name);
+ if (name_off < 0)
+ return name_off;
+
+ dst_sym = add_new_sym(linker, &dst_sym_idx);
+ if (!dst_sym)
+ return -ENOMEM;
+
+ dst_sym->st_name = name_off;
+ dst_sym->st_info = sym->st_info;
+ dst_sym->st_other = sym->st_other;
+ dst_sym->st_shndx = dst_sec ? dst_sec->sec_idx : sym->st_shndx;
+ dst_sym->st_value = (src_sec ? src_sec->dst_off : 0) + sym->st_value;
+ dst_sym->st_size = sym->st_size;
+
+ obj->sym_map[src_sym_idx] = dst_sym_idx;
+
+ if (sym_type == STT_SECTION && dst_sym) {
+ dst_sec->sec_sym_idx = dst_sym_idx;
+ dst_sym->st_value = 0;
+ }
+
+ if (sym_bind != STB_LOCAL) {
+ glob_sym = add_glob_sym(linker);
+ if (!glob_sym)
+ return -ENOMEM;
+
+ glob_sym->sym_idx = dst_sym_idx;
+ /* we use dst_sec->id (and not dst_sec->sec_idx), because
+ * ephemeral sections (.kconfig, .ksyms, etc) don't have
+ * sec_idx (as they don't have corresponding ELF section), but
+ * still have id. .extern doesn't have even ephemeral section
+ * associated with it, so dst_sec->id == dst_sec->sec_idx == 0.
+ */
+ glob_sym->sec_id = dst_sec ? dst_sec->id : 0;
+ glob_sym->name_off = name_off;
+ /* we will fill btf_id in during BTF merging step */
+ glob_sym->btf_id = 0;
+ glob_sym->is_extern = sym_is_extern;
+ glob_sym->is_weak = sym_bind == STB_WEAK;
}
return 0;
@@ -1442,6 +2078,7 @@ static int linker_append_btf(struct bpf_linker *linker, struct src_obj *obj) { const struct btf_type *t; int i, j, n, start_id, id;+ const char *name; if (!obj->btf) return 0;
@@ -1454,12 +2091,40 @@ static int linker_append_btf(struct bpf_linker *linker, struct src_obj *obj) return -ENOMEM; for (i = 1; i <= n; i++) {+ struct glob_sym *glob_sym = NULL;+ t = btf__type_by_id(obj->btf, i); /* DATASECs are handled specially below */ if (btf_kind(t) == BTF_KIND_DATASEC) continue;+ if (btf_is_non_static(t)) {+ /* there should be glob_sym already */+ name = btf__str_by_offset(obj->btf, t->name_off);+ glob_sym = find_glob_sym(linker, name);++ /* VARs without corresponding glob_sym are those that+ * belong to skipped/deduplicated sections (i.e.,+ * license and version), so just skip them+ */+ if (!glob_sym)+ continue;++ if (glob_sym->underlying_btf_id == 0)+ glob_sym->underlying_btf_id = -t->type;
Is this needed? If glob_sym->btf_id is not NULL, then
glob_sym->underlying_btf_id has been set by the previous object.
If it is NULL, it will set probably after this
if (btf_is_non_static(t)) { ...}, is this right?
quoted hunk
+
+ /* globals from previous object files that match our
+ * VAR/FUNC already have a corresponding associated
+ * BTF type, so just make sure to use it
+ */
+ if (glob_sym->btf_id) {
+ /* reuse existing BTF type for global var/func */
+ obj->btf_type_map[i] = glob_sym->btf_id;
+ continue;
+ }
+ }
+
id = btf__add_type(linker->btf, obj->btf, t);
if (id < 0) {
pr_warn("failed to append BTF type #%d from file '%s'\n", i, obj->filename);
@@ -1467,6 +2132,12 @@ static int linker_append_btf(struct bpf_linker *linker, struct src_obj *obj) } obj->btf_type_map[i] = id;++ /* record just appended BTF type for var/func */+ if (glob_sym) {+ glob_sym->btf_id = id;+ glob_sym->underlying_btf_id = -t->type;+ } } /* remap all the types except DATASECs */
@@ -1478,6 +2149,22 @@ static int linker_append_btf(struct bpf_linker *linker, struct src_obj *obj) return -EINVAL; }+ /* Rewrite VAR/FUNC underlying types (i.e., FUNC's FUNC_PROTO and VAR's+ * actual type), if necessary+ */+ for (i = 0; i < linker->glob_sym_cnt; i++) {+ struct glob_sym *glob_sym = &linker->glob_syms[i];+ struct btf_type *glob_t;++ if (glob_sym->underlying_btf_id >= 0)+ continue;++ glob_sym->underlying_btf_id = obj->btf_type_map[-glob_sym->underlying_btf_id];
After this point, any new *extern* variables will hit the below in the
previous code:
> + if (glob_sym->btf_id) {
> + /* reuse existing BTF type for global var/func */
> + obj->btf_type_map[i] = glob_sym->btf_id;
> + continue;
> + }
quoted hunk
+
+ glob_t = btf_type_by_id(linker->btf, glob_sym->btf_id);
+ glob_t->type = glob_sym->underlying_btf_id;
+ }
+
/* append DATASEC info */
for (i = 1; i < obj->sec_cnt; i++) {
struct src_sec *src_sec;
On Thu, Apr 22, 2021 at 2:27 PM Yonghong Song [off-list ref] wrote:
On 4/16/21 1:23 PM, Andrii Nakryiko wrote:
quoted
Add BPF static linker logic to resolve extern variables and functions across
multiple linked together BPF object files.
For that, linker maintains a separate list of struct glob_sym structures,
which keeps track of few pieces of metadata (is it extern or resolved global,
is it a weak symbol, which ELF section it belongs to, etc) and ties together
BTF type info and ELF symbol information and keeps them in sync.
With adding support for extern variables/funcs, it's now possible for some
sections to contain both extern and non-extern definitions. This means that
some sections may start out as ephemeral (if only externs are present and thus
there is not corresponding ELF section), but will be "upgraded" to actual ELF
section as symbols are resolved or new non-extern definitions are appended.
Additional care is taken to not duplicate extern entries in sections like
.kconfig and .ksyms.
Given libbpf requires BTF type to always be present for .kconfig/.ksym
externs, linker extends this requirement to all the externs, even those that
are supposed to be resolved during static linking and which won't be visible
to libbpf. With BTF information always present, static linker will check not
just ELF symbol matches, but entire BTF type signature match as well. That
logic is stricter that BPF CO-RE checks. It probably should be re-used by
.ksym resolution logic in libbpf as well, but that's left for follow up
patches.
To make it unnecessary to rewrite ELF symbols and minimize BTF type
rewriting/removal, ELF symbols that correspond to externs initially will be
updated in place once they are resolved. Similarly for BTF type info, VAR/FUNC
and var_secinfo's (sec_vars in struct bpf_linker) are staying stable, but
types they point to might get replaced when extern is resolved. This might
leave some left-over types (even though we try to minimize this for common
cases of having extern funcs with not argument names vs concrete function with
names properly specified). That can be addresses later with a generic BTF
garbage collection. That's left for a follow up as well.
Given BTF type appending phase is separate from ELF symbol
appending/resolution, special struct glob_sym->underlying_btf_id variable is
used to communicate resolution and rewrite decisions. 0 means
underlying_btf_id needs to be appended (it's not yet in final linker->btf), <0
values are used for temporary storage of source BTF type ID (not yet
rewritten), so -glob_sym->underlying_btf_id is BTF type id in obj-btf. But by
the end of linker_append_btf() phase, that underlying_btf_id will be remapped
and will always be > 0. This is the uglies part of the whole process, but
keeps the other parts much simpler due to stability of sec_var and VAR/FUNC
types, as well as ELF symbol, so please keep that in mind while reviewing.
This is indeed complicated. I has some comments below. Please check
whether my understanding is correct or not.
quoted
BTF-defined maps require some extra custom logic and is addressed separate in
the next patch, so that to keep this one smaller and easier to review.
Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
---
tools/lib/bpf/linker.c | 844 ++++++++++++++++++++++++++++++++++++++---
1 file changed, 785 insertions(+), 59 deletions(-)
@@ -22,6 +22,8 @@#include"libbpf_internal.h"#include"strset.h"+#define BTF_EXTERN_SEC ".extern"+structsrc_sec{constchar*sec_name;/* positional (not necessarily ELF) index in an array of sections */
@@ -74,11 +76,36 @@ struct btf_ext_sec_data {void*recs;};+structglob_sym{+/* ELF symbol index */+intsym_idx;+/* associated section id for .ksyms, .kconfig, etc, but not .extern */+intsec_id;+/* extern name offset in STRTAB */+intname_off;+/* optional associated BTF type ID */+intbtf_id;+/* BTF type ID to which VAR/FUNC type is pointing to; used for+*rewritingtypeswhenexternVAR/FUNCisresolvedtoaconcrete+*definition+*/+intunderlying_btf_id;+/* sec_var index in the corresponding dst_sec, if exists */+intvar_idx;++/* extern or resolved/global symbol */+boolis_extern;+/* weak or strong symbol, never goes back from strong to weak */+boolis_weak;+};+structdst_sec{char*sec_name;/* positional (not necessarily ELF) index in an array of sections */intid;+boolephemeral;+/* ELF info */size_tsec_idx;Elf_Scn*scn;
+
+static bool glob_sym_btf_matches(const char *sym_name, bool exact,
+ const struct btf *btf1, __u32 id1,
+ const struct btf *btf2, __u32 id2)
+{
+ const struct btf_type *t1, *t2;
+ bool is_static1, is_static2;
+ const char *n1, *n2;
+ int i, n;
+
+recur:
+ n1 = n2 = NULL;
+ t1 = skip_mods_and_typedefs(btf1, id1, &id1);
+ t2 = skip_mods_and_typedefs(btf2, id2, &id2);
+
+ /* check if only one side is FWD, otherwise handle with common logic */
+ if (!exact && btf_is_fwd(t1) != btf_is_fwd(t2)) {
+ n1 = btf__str_by_offset(btf1, t1->name_off);
+ n2 = btf__str_by_offset(btf2, t2->name_off);
+ if (strcmp(n1, n2) != 0) {
+ pr_warn("global '%s': incompatible forward declaration names '%s' and '%s'\n",
+ sym_name, n1, n2);
+ return false;
+ }
+ /* validate if FWD kind matches concrete kind */
+ if (btf_is_fwd(t1)) {
+ if (btf_kflag(t1) && btf_is_union(t2))
+ return true;
+ if (!btf_kflag(t1) && btf_is_struct(t2))
+ return true;
+ pr_warn("global '%s': incompatible %s forward declaration and concrete kind %s\n",
+ sym_name, btf_kflag(t1) ? "union" : "struct", btf_kind_str(t2));
+ } else {
+ if (btf_kflag(t2) && btf_is_union(t1))
+ return true;
+ if (!btf_kflag(t2) && btf_is_struct(t1))
+ return true;
+ pr_warn("global '%s': incompatible %s forward declaration and concrete kind %s\n",
+ sym_name, btf_kflag(t2) ? "union" : "struct", btf_kind_str(t1));
+ }
+ return false;
+ }
+
+ if (btf_kind(t1) != btf_kind(t2)) {
+ pr_warn("global '%s': incompatible BTF kinds %s and %s\n",
+ sym_name, btf_kind_str(t1), btf_kind_str(t2));
+ return false;
+ }
+
+ switch (btf_kind(t1)) {
+ case BTF_KIND_STRUCT:
+ case BTF_KIND_UNION:
+ case BTF_KIND_ENUM:
+ case BTF_KIND_FWD:
+ case BTF_KIND_FUNC:
+ case BTF_KIND_VAR:
+ n1 = btf__str_by_offset(btf1, t1->name_off);
+ n2 = btf__str_by_offset(btf2, t2->name_off);
+ if (strcmp(n1, n2) != 0) {
+ pr_warn("global '%s': incompatible %s names '%s' and '%s'\n",
+ sym_name, btf_kind_str(t1), n1, n2);
+ return false;
+ }
+ break;
+ default:
+ break;
+ }
+
+ switch (btf_kind(t1)) {
+ case BTF_KIND_UNKN: /* void */
+ case BTF_KIND_FWD:
+ return true;
+ case BTF_KIND_INT:
+ case BTF_KIND_FLOAT:
+ case BTF_KIND_ENUM:
+ /* ignore encoding for int and enum values for enum */
+ if (t1->size != t2->size) {
+ pr_warn("global '%s': incompatible %s '%s' size %u and %u\n",
+ sym_name, btf_kind_str(t1), n1, t1->size, t2->size);
+ return false;
+ }
+ return true;
+ case BTF_KIND_PTR:
+ /* just validate overall shape of the referenced type, so no
+ * contents comparison for struct/union, and allowd fwd vs
+ * struct/union
+ */
+ exact = false;
+ id1 = t1->type;
+ id2 = t2->type;
+ goto recur;
+ case BTF_KIND_ARRAY:
+ /* ignore index type and array size */
+ id1 = btf_array(t1)->type;
+ id2 = btf_array(t2)->type;
+ goto recur;
+ case BTF_KIND_FUNC:
+ /* extern and global linkages are compatible */
+ is_static1 = btf_func_linkage(t1) == BTF_FUNC_STATIC;
+ is_static2 = btf_func_linkage(t2) == BTF_FUNC_STATIC;
+ if (is_static1 != is_static2) {
+ pr_warn("global '%s': incompatible func '%s' linkage\n", sym_name, n1);
+ return false;
+ }
+
+ id1 = t1->type;
+ id2 = t2->type;
+ goto recur;
+ case BTF_KIND_VAR:
+ /* extern and global linkages are compatible */
+ is_static1 = btf_var(t1)->linkage == BTF_VAR_STATIC;
+ is_static2 = btf_var(t2)->linkage == BTF_VAR_STATIC;
+ if (is_static1 != is_static2) {
+ pr_warn("global '%s': incompatible var '%s' linkage\n", sym_name, n1);
+ return false;
+ }
+
+ id1 = t1->type;
+ id2 = t2->type;
+ goto recur;
+ case BTF_KIND_STRUCT:
+ case BTF_KIND_UNION: {
+ const struct btf_member *m1, *m2;
+
+ if (!exact)
+ return true;
+
+ if (btf_vlen(t1) != btf_vlen(t2)) {
+ pr_warn("global '%s': incompatible number of %s fields %u and %u\n",
+ sym_name, btf_kind_str(t1), btf_vlen(t1), btf_vlen(t2));
+ return false;
+ }
+
+ n = btf_vlen(t1);
+ m1 = btf_members(t1);
+ m2 = btf_members(t2);
+ for (i = 0; i < n; i++, m1++, m2++) {
+ n1 = btf__str_by_offset(btf1, m1->name_off);
+ n2 = btf__str_by_offset(btf2, m2->name_off);
+ if (strcmp(n1, n2) != 0) {
+ pr_warn("global '%s': incompatible field #%d names '%s' and '%s'\n",
+ sym_name, i, n1, n2);
+ return false;
+ }
+ if (m1->offset != m2->offset) {
+ pr_warn("global '%s': incompatible field #%d ('%s') offsets\n",
+ sym_name, i, n1);
+ return false;
+ }
+ if (!glob_sym_btf_matches(sym_name, exact, btf1, m1->type, btf2, m2->type))
+ return false;
+ }
+
+ return true;
+ }
+ case BTF_KIND_FUNC_PROTO: {
+ const struct btf_param *m1, *m2;
+
+ if (btf_vlen(t1) != btf_vlen(t2)) {
+ pr_warn("global '%s': incompatible number of %s params %u and %u\n",
+ sym_name, btf_kind_str(t1), btf_vlen(t1), btf_vlen(t2));
+ return false;
+ }
+
+ n = btf_vlen(t1);
+ m1 = btf_params(t1);
+ m2 = btf_params(t2);
+ for (i = 0; i < n; i++, m1++, m2++) {
+ /* ignore func arg names */
+ if (!glob_sym_btf_matches(sym_name, exact, btf1, m1->type, btf2, m2->type))
+ return false;
+ }
+
+ /* now check return type as well */
+ id1 = t1->type;
+ id2 = t2->type;
+ goto recur;
+ }
+
+ case BTF_KIND_TYPEDEF:
+ case BTF_KIND_VOLATILE:
+ case BTF_KIND_CONST:
+ case BTF_KIND_RESTRICT:
We already did skip_mods_and_typedefs() before. Unless something serious
wrong, we should not hit the above four types. So I think we can skip
them here.
This is the way of documenting explicitly that I'm aware of those
kinds and they shouldn't be encountered. Otherwise one might wonder if
we just forgot to handle them.
quoted
+ case BTF_KIND_DATASEC:
+ default:
+ pr_warn("global '%s': unsupported BTF kind %s\n",
+ sym_name, btf_kind_str(t1));
+ return false;
+ }
+}
+
+static bool glob_syms_match(const char *sym_name,
+ struct bpf_linker *linker, struct glob_sym *glob_sym,
+ struct src_obj *obj, Elf64_Sym *sym, size_t sym_idx, int btf_id)
+{
+ const struct btf_type *src_t;
+
+ /* if we are dealing with externs, BTF types describing both global
+ * and extern VARs/FUNCs should be completely present in all files
+ */
+ if (!glob_sym->btf_id || !btf_id) {
+ pr_warn("BTF info is missing for global symbol '%s'\n", sym_name);
+ return false;
+ }
+
+ src_t = btf__type_by_id(obj->btf, btf_id);
+ if (!btf_is_var(src_t) && !btf_is_func(src_t)) {
+ pr_warn("only extern variables and functions are supported, but got '%s' for '%s'\n",
+ btf_kind_str(src_t), sym_name);
+ return false;
+ }
+
+ if (!glob_sym_btf_matches(sym_name, true /*exact*/,
+ linker->btf, glob_sym->btf_id, obj->btf, btf_id))
+ return false;
+
+ return true;
+}
+
[...]
quoted
+
+static void sym_update_visibility(Elf64_Sym *sym, int sym_vis)
+{
+ /* libelf doesn't provide setters for ST_VISIBILITY,
+ * but it is stored in the lower 2 bits of st_other
+ */
+ sym->st_other &= 0x03;
+ sym->st_other |= sym_vis;
+}
+
+static int linker_append_elf_sym(struct bpf_linker *linker, struct src_obj *obj,
+ Elf64_Sym *sym, const char *sym_name, int src_sym_idx)
+{
+ struct src_sec *src_sec = NULL;
+ struct dst_sec *dst_sec = NULL;
+ struct glob_sym *glob_sym = NULL;
+ int name_off, sym_type, sym_bind, sym_vis, err;
+ int btf_sec_id = 0, btf_id = 0;
+ size_t dst_sym_idx;
+ Elf64_Sym *dst_sym;
+ bool sym_is_extern;
+
+ sym_type = ELF64_ST_TYPE(sym->st_info);
+ sym_bind = ELF64_ST_BIND(sym->st_info);
+ sym_vis = ELF64_ST_VISIBILITY(sym->st_other);
+ sym_is_extern = sym->st_shndx == SHN_UNDEF;
+
+ if (sym_is_extern) {
+ if (!obj->btf) {
+ pr_warn("externs without BTF info are not supported\n");
+ return -ENOTSUP;
+ }
+ } else if (sym->st_shndx < SHN_LORESERVE) {
So what happens if sym->st_shndx >= SHN_LORESERVE. Maybe return failures
here? In general, bpf program shouldn't hit sym->st_shndx >= SHN_LORESERVE.
There is at least SHN_ABS (0xfff1), which is an informational STT_FILE
symbol. libbpf doesn't error out on such special symbols, and linker
will just pass-through them and append to the final object file.
quoted
+ src_sec = &obj->secs[sym->st_shndx];
+ if (src_sec->skipped)
+ return 0;
+ dst_sec = &linker->secs[src_sec->dst_id];
+
+ /* allow only one STT_SECTION symbol per section */
+ if (sym_type == STT_SECTION && dst_sec->sec_sym_idx) {
+ obj->sym_map[src_sym_idx] = dst_sec->sec_sym_idx;
+ return 0;
+ }
+ }
+
+ if (sym_bind == STB_LOCAL)
+ goto add_sym;
+
+ /* find matching BTF info */
+ err = find_glob_sym_btf(obj, sym, sym_name, &btf_sec_id, &btf_id);
+ if (err)
+ return err;
+
+ if (sym_is_extern && btf_sec_id) {
+ const char *sec_name = NULL;
+ const struct btf_type *t;
+
+ t = btf__type_by_id(obj->btf, btf_sec_id);
+ sec_name = btf__str_by_offset(obj->btf, t->name_off);
+
+ /* Clang puts unannotated extern vars into
+ * '.extern' BTF DATASEC. Treat them the same
+ * as unannotated extern funcs (which are
+ * currently not put into any DATASECs).
+ * Those don't have associated src_sec/dst_sec.
+ */
+ if (strcmp(sec_name, BTF_EXTERN_SEC) != 0) {
+ src_sec = find_src_sec_by_name(obj, sec_name);
+ if (!src_sec) {
+ pr_warn("failed to find matching ELF sec '%s'\n", sec_name);
+ return -ENOENT;
+ }
+ dst_sec = &linker->secs[src_sec->dst_id];
+ }
+ }
+
+ glob_sym = find_glob_sym(linker, sym_name);
+ if (glob_sym) {
+ /* Preventively resolve to existing symbol. This is
+ * needed for further relocation symbol remapping in
+ * the next step of linking.
+ */
+ obj->sym_map[src_sym_idx] = glob_sym->sym_idx;
+
+ /* If both symbols are non-externs, at least one of
+ * them has to be STB_WEAK, otherwise they are in
+ * a conflict with each other.
+ */
+ if (!sym_is_extern && !glob_sym->is_extern
+ && !glob_sym->is_weak && sym_bind != STB_WEAK) {
+ pr_warn("conflicting non-weak symbol #%d (%s) definition in '%s'\n",
+ src_sym_idx, sym_name, obj->filename);
+ return -EINVAL;
}
+ if (!glob_syms_match(sym_name, linker, glob_sym, obj, sym, src_sym_idx, btf_id))
+ return -EINVAL;
+
+ dst_sym = get_sym_by_idx(linker, glob_sym->sym_idx);
+
+ /* If new symbol is strong, then force dst_sym to be strong as
+ * well; this way a mix of weak and non-weak extern
+ * definitions will end up being strong.
+ */
+ if (sym_bind == STB_GLOBAL) {
+ /* We still need to preserve type (NOTYPE or
+ * OBJECT/FUNC, depending on whether the symbol is
+ * extern or not)
+ */
+ sym_update_bind(dst_sym, STB_GLOBAL);
+ glob_sym->is_weak = false;
+ }
+
+ /* Non-default visibility is "contaminating", with stricter
+ * visibility overwriting more permissive ones, even if more
+ * permissive visibility comes from just an extern definition
+ */
+ if (sym_vis > ELF64_ST_VISIBILITY(dst_sym->st_other))
+ sym_update_visibility(dst_sym, sym_vis);
For visibility, maybe we can just handle DEFAULT and HIDDEN, and others
are not supported? DEFAULT + DEFAULT/HIDDEN => DEFAULT, HIDDEN + HIDDEN
=> HIDDEN?
Sure, we can restrict this to STV_DEFAULT and STV_HIDDEN for now.
quoted
+
+ /* If the new symbol is extern, then regardless if
+ * existing symbol is extern or resolved global, just
+ * keep the existing one untouched.
+ */
+ if (sym_is_extern)
+ return 0;
+
+ /* If existing symbol is a strong resolved symbol, bail out,
+ * because we lost resolution battle have nothing to
+ * contribute. We already checked abover that there is no
+ * strong-strong conflict. We also already tightened binding
+ * and visibility, so nothing else to contribute at that point.
+ */
+ if (!glob_sym->is_extern && sym_bind == STB_WEAK)
+ return 0;
+
+ /* At this point, new symbol is strong non-extern,
+ * so overwrite glob_sym with new symbol information.
+ * Preserve binding and visibility.
+ */
+ sym_update_type(dst_sym, sym_type);
+ dst_sym->st_shndx = dst_sec->sec_idx;
+ dst_sym->st_value = src_sec->dst_off + sym->st_value;
+ dst_sym->st_size = sym->st_size;
+
+ /* see comment below about dst_sec->id vs dst_sec->sec_idx */
+ glob_sym->sec_id = dst_sec->id;
+ glob_sym->is_extern = false;
+ /* never relax strong to weak binding */
+ if (sym_bind == STB_GLOBAL)
+ glob_sym->is_weak = false;
In the above, we already set glob_sym->is_weak to false if STB_GLOBAL.
yep, you are right, this is unnecessary, I'll remove
quoted
+
+ if (complete_extern_btf_info(linker->btf, glob_sym->btf_id,
+ obj->btf, btf_id))
+ return -EINVAL;
+
+ /* request updating VAR's/FUNC's underlying BTF type when appending BTF type */
+ glob_sym->underlying_btf_id = 0;
+
+ obj->sym_map[src_sym_idx] = glob_sym->sym_idx;
+ return 0;
+ }
+
+add_sym:
+ name_off = strset__add_str(linker->strtab_strs, sym_name);
+ if (name_off < 0)
+ return name_off;
+
+ dst_sym = add_new_sym(linker, &dst_sym_idx);
+ if (!dst_sym)
+ return -ENOMEM;
+
+ dst_sym->st_name = name_off;
+ dst_sym->st_info = sym->st_info;
+ dst_sym->st_other = sym->st_other;
+ dst_sym->st_shndx = dst_sec ? dst_sec->sec_idx : sym->st_shndx;
+ dst_sym->st_value = (src_sec ? src_sec->dst_off : 0) + sym->st_value;
+ dst_sym->st_size = sym->st_size;
+
+ obj->sym_map[src_sym_idx] = dst_sym_idx;
+
+ if (sym_type == STT_SECTION && dst_sym) {
+ dst_sec->sec_sym_idx = dst_sym_idx;
+ dst_sym->st_value = 0;
+ }
+
+ if (sym_bind != STB_LOCAL) {
+ glob_sym = add_glob_sym(linker);
+ if (!glob_sym)
+ return -ENOMEM;
+
+ glob_sym->sym_idx = dst_sym_idx;
+ /* we use dst_sec->id (and not dst_sec->sec_idx), because
+ * ephemeral sections (.kconfig, .ksyms, etc) don't have
+ * sec_idx (as they don't have corresponding ELF section), but
+ * still have id. .extern doesn't have even ephemeral section
+ * associated with it, so dst_sec->id == dst_sec->sec_idx == 0.
+ */
+ glob_sym->sec_id = dst_sec ? dst_sec->id : 0;
+ glob_sym->name_off = name_off;
+ /* we will fill btf_id in during BTF merging step */
+ glob_sym->btf_id = 0;
+ glob_sym->is_extern = sym_is_extern;
+ glob_sym->is_weak = sym_bind == STB_WEAK;
}
return 0;
@@ -1442,6 +2078,7 @@ static int linker_append_btf(struct bpf_linker *linker, struct src_obj *obj) { const struct btf_type *t; int i, j, n, start_id, id;+ const char *name; if (!obj->btf) return 0;
@@ -1454,12 +2091,40 @@ static int linker_append_btf(struct bpf_linker *linker, struct src_obj *obj) return -ENOMEM; for (i = 1; i <= n; i++) {+ struct glob_sym *glob_sym = NULL;+ t = btf__type_by_id(obj->btf, i); /* DATASECs are handled specially below */ if (btf_kind(t) == BTF_KIND_DATASEC) continue;+ if (btf_is_non_static(t)) {+ /* there should be glob_sym already */+ name = btf__str_by_offset(obj->btf, t->name_off);+ glob_sym = find_glob_sym(linker, name);++ /* VARs without corresponding glob_sym are those that+ * belong to skipped/deduplicated sections (i.e.,+ * license and version), so just skip them+ */+ if (!glob_sym)+ continue;++ if (glob_sym->underlying_btf_id == 0)+ glob_sym->underlying_btf_id = -t->type;
Is this needed? If glob_sym->btf_id is not NULL, then
glob_sym->underlying_btf_id has been set by the previous object.
If it is NULL, it will set probably after this
if (btf_is_non_static(t)) { ...}, is this right?
I think it's still needed. Here's the scenario.
1. Obj file A contains extern symbol X. We create corresponding
glob_sym (with is_extern=true), and store btf_id to point to
BTF_KIND_VAR, and btf_underlying_id to point to the type that
BTF_KIND_VAR points to.
2. Obj file B contains non-extern symbol X. At this point
linker_append_elf_sym() will update glob_sym to is_extern = false, it
will keep btf_id to re-use already appended BTF_KIND_VAR, but it will
zero-out underlying_btf_id, because for externs type could be
incomplete (e.g. for functions it won't contain function argument
names, for maps it could differ even more drastically later). So then
we get here, we see that glob_sym->underlying_btf_id is zero, so needs
updating. We store it as -Y, because Y is BTF type ID in obj->btf, not
in linker->btf (yet). Then the if (glob_sym->btf_id) below sees that
glob_sym->btf_id is already set, so we just keep using already
appended BTF_KIND_VAR (we already set its linkage to
BTF_VAR_GLOBAL_ALLOCATED in complete_extern_btf_info(), called from
linker_append_elf_sym(). So we'll skip appending another BTF_KIND_VAR.
But we do want to point existing BTF_KIND_VAR to a new type that
corresponds to ID -Y.
quoted
+
+ /* globals from previous object files that match our
+ * VAR/FUNC already have a corresponding associated
+ * BTF type, so just make sure to use it
+ */
+ if (glob_sym->btf_id) {
+ /* reuse existing BTF type for global var/func */
+ obj->btf_type_map[i] = glob_sym->btf_id;
+ continue;
+ }
+ }
+
id = btf__add_type(linker->btf, obj->btf, t);
if (id < 0) {
pr_warn("failed to append BTF type #%d from file '%s'\n", i, obj->filename);
@@ -1467,6 +2132,12 @@ static int linker_append_btf(struct bpf_linker *linker, struct src_obj *obj) } obj->btf_type_map[i] = id;++ /* record just appended BTF type for var/func */+ if (glob_sym) {+ glob_sym->btf_id = id;+ glob_sym->underlying_btf_id = -t->type;+ } } /* remap all the types except DATASECs */
@@ -1478,6 +2149,22 @@ static int linker_append_btf(struct bpf_linker *linker, struct src_obj *obj) return -EINVAL; }+ /* Rewrite VAR/FUNC underlying types (i.e., FUNC's FUNC_PROTO and VAR's+ * actual type), if necessary+ */+ for (i = 0; i < linker->glob_sym_cnt; i++) {+ struct glob_sym *glob_sym = &linker->glob_syms[i];+ struct btf_type *glob_t;++ if (glob_sym->underlying_btf_id >= 0)+ continue;++ glob_sym->underlying_btf_id = obj->btf_type_map[-glob_sym->underlying_btf_id];
After this point, any new *extern* variables will hit the below in the
previous code:
Right, but we want to hit this for existing glob_syms that went from
extern to non-extern or from weak to non-weak. See
/* request updating VAR's/FUNC's underlying BTF type when
appending BTF type */
glob_sym->underlying_btf_id = 0;
in linker_append_elf_sym().
And we'll use that even more extensively when extending __weak and
extern map definitions later.
> + if (glob_sym->btf_id) {
> + /* reuse existing BTF type for global var/func */
> + obj->btf_type_map[i] = glob_sym->btf_id;
> + continue;
> + }
quoted
+
+ glob_t = btf_type_by_id(linker->btf, glob_sym->btf_id);
+ glob_t->type = glob_sym->underlying_btf_id;
+ }
+
/* append DATASEC info */
for (i = 1; i < obj->sec_cnt; i++) {
struct src_sec *src_sec;
From: Yonghong Song <hidden> Date: 2021-04-22 22:56:24
On 4/16/21 1:23 PM, Andrii Nakryiko wrote:
Add extra logic to handle map externs (only BTF-defined maps are supported for
linking). Re-use the map parsing logic used during bpf_object__open(). Map
externs are currently restricted to always match complete map definition. So
all the specified attributes will be compared (down to pining, map_flags,
numa_node, etc). In the future this restriction might be relaxed with no
backwards compatibility issues. If any attribute is mismatched between extern
and actual map definition, linker will report an error, pointing out which one
mismatches.
The original intent was to allow for extern to specify attributes that matters
(to user) to enforce. E.g., if you specify just key information and omit
value, then any value fits. Similarly, it should have been possible to enforce
map_flags, pinning, and any other possible map attribute. Unfortunately, that
means that multiple externs can be only partially overlapping with each other,
which means linker would need to combine their type definitions to end up with
the most restrictive and fullest map definition. This requires an extra amount
of BTF manipulation which at this time was deemed unnecessary and would
require further extending generic BTF writer APIs. So that is left for future
follow ups, if there will be demand for that. But the idea seems intresting
and useful, so I want to document it here.
Weak definitions are also supported, but are pretty strict as well, just
like externs: all weak map definitions have to match exactly. In the follow up
patches this most probably will be relaxed, with __weak map definitions being
able to differ between each other (with non-weak definition always winning, of
course).
Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
I think strict enforcement of extern/global map definitions is good.
If library want people will use its maps, it may put the map definition
into one of its headers and application can include and have
exact the same definition.
Acked-by: Yonghong Song <redacted>
From: Yonghong Song <hidden> Date: 2021-04-22 23:00:41
On 4/22/21 11:09 AM, Andrii Nakryiko wrote:
On Wed, Apr 21, 2021 at 10:43 PM Yonghong Song [off-list ref] wrote:
quoted
On 4/16/21 1:23 PM, Andrii Nakryiko wrote:
quoted
Define __hidden helper macro in bpf_helpers.h, which is a short-hand for
__attribute__((visibility("hidden"))). Add libbpf support to mark BPF
subprograms marked with __hidden as static in BTF information to enforce BPF
verifier's static function validation algorithm, which takes more information
(caller's context) into account during a subprogram validation.
Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
---
tools/lib/bpf/bpf_helpers.h | 8 ++++++
tools/lib/bpf/btf.c | 5 ----
tools/lib/bpf/libbpf.c | 45 ++++++++++++++++++++++++++++++++-
tools/lib/bpf/libbpf_internal.h | 6 +++++
4 files changed, 58 insertions(+), 6 deletions(-)
To prevent potential external __hidden macro definition conflict, how
about
#ifdef __hidden
#undef __hidden
#define __hidden __attribute__((visibility("hidden")))
#endif
We do force #undef only with __always_inline because of the bad
definition in linux/stddef.h And we check #ifndef for __weak, because
__weak is defined in kernel headers. This is not really the case for
__hidden, the only definition is in
tools/lib/traceevent/event-parse-local.h, which I don't think we
should worry about in BPF context. So I wanted to keep it simple and
fix only if that really causes some real conflicts.
And keep in mind that in BPF code bpf_helpers.h is usually included as
one of the first few headers anyways.
That is fine. Conflict of __hidden is a low risk and we can deal with it
later if needed.
quoted
quoted
+
/* When utilizing vmlinux.h with BPF CO-RE, user BPF programs can't include
* any system-level headers (such as stddef.h, linux/version.h, etc), and
* commonly-used macros like NULL and KERNEL_VERSION aren't available through
[...]
quoted
quoted
@@ -698,6 +700,15 @@ bpf_object__add_programs(struct bpf_object *obj, Elf_Data *sec_data, if (err) return err;+ /* if function is a global/weak symbol, but has hidden+ * visibility (or any non-default one), mark its BTF FUNC as+ * static to enable more permissive BPF verification mode with+ * more outside context available to BPF verifier+ */+ if (GELF_ST_BIND(sym.st_info) != STB_LOCAL+ && GELF_ST_VISIBILITY(sym.st_other) != STV_DEFAULT)
Maybe we should check GELF_ST_VISIBILITY(sym.st_other) == STV_HIDDEN
instead?
It felt like only STV_DEFAULT should be "exported", semantically
speaking. Everything else would be treated as if it was static, except
that C rules require that function has to be global. Do you think
there is some danger to do it this way?
Currently static linker doesn't do anything special for STV_INTERNAL
and STV_PROTECTED, so we could just disable those. Do you prefer that?
Yes, let us just deal with STV_DEFAULT and STV_HIDDEN. We already
specialized STV_HIDDEN, so we should not treat STV_INTERNAL/PROTECTED
as what they mean in ELF standard, so let us disable them for now.
I just felt that there is no risk of regression if we do this for
non-STV_DEFAULT generically.
From: Yonghong Song <hidden> Date: 2021-04-22 23:13:48
On 4/22/21 11:20 AM, Andrii Nakryiko wrote:
On Thu, Apr 22, 2021 at 9:35 AM Yonghong Song [off-list ref] wrote:
quoted
On 4/16/21 1:23 PM, Andrii Nakryiko wrote:
quoted
Add logic to validate extern symbols, plus some other minor extra checks, like
ELF symbol #0 validation, general symbol visibility and binding validations.
Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
---
tools/lib/bpf/linker.c | 43 +++++++++++++++++++++++++++++++++---------
1 file changed, 34 insertions(+), 9 deletions(-)
@@ -750,14 +750,39 @@ static int linker_sanity_check_elf_symtab(struct src_obj *obj, struct src_sec *sn=sec->shdr->sh_size/sec->shdr->sh_entsize;sym=sec->data->d_buf;for(i=0;i<n;i++,sym++){-if(sym->st_shndx-&&sym->st_shndx<SHN_LORESERVE-&&sym->st_shndx>=obj->sec_cnt){+intsym_type=ELF64_ST_TYPE(sym->st_info);+intsym_bind=ELF64_ST_BIND(sym->st_info);++if(i==0){+if(sym->st_name!=0||sym->st_info!=0+||sym->st_other!=0||sym->st_shndx!=0+||sym->st_value!=0||sym->st_size!=0){+pr_warn("ELF sym #0 is invalid in %s\n",obj->filename);+return-EINVAL;+}+continue;+}
In ELF file, the first entry of symbol table and section table (index 0)
is invalid/undefined.
Symbol table '.symtab' contains 9 entries:
Num: Value Size Type Bind Vis Ndx Name
0: 0000000000000000 0 NOTYPE LOCAL DEFAULT UND
Section Headers:
[Nr] Name Type Address Off Size
ES Flg Lk Inf Al
[ 0] NULL 0000000000000000 000000 000000
00 0 0 0
Instead of validating them, I think we can skip traversal of the index =
0 entry for symbol table and section header table. What do you think?
In valid ELF yes. But then entire sanity check logic is not needed if
we just assume correct ELF. But I don't want to make that potentially
dangerous assumption :) Here I'm validating that ELF is sane with
minimal efforts. I do skip symbol #0 later because I validated that
it's all-zero one, as expected by ELF standard.
I just feel we can ignore entry 0 regardless of its contents. But I
guess this is still useful since libbpf linker generates its own
ELF file and it is worthwhile to do an in-depth check for that.
So I am okay with this.
quoted
quoted
+ if (sym_bind != STB_LOCAL && sym_bind != STB_GLOBAL && sym_bind != STB_WEAK) {
+ pr_warn("ELF sym #%d is section #%zu has unsupported symbol binding %d\n",
+ i, sec->sec_idx, sym_bind);
+ return -EINVAL;
+ }
+ if (sym->st_shndx == 0) {
+ if (sym_type != STT_NOTYPE || sym_bind == STB_LOCAL
+ || sym->st_value != 0 || sym->st_size != 0) {
+ pr_warn("ELF sym #%d is invalid extern symbol in %s\n",
+ i, obj->filename);
+
+ return -EINVAL;
+ }
+ continue;
+ }
On Thu, Apr 22, 2021 at 4:00 PM Yonghong Song [off-list ref] wrote:
On 4/22/21 11:09 AM, Andrii Nakryiko wrote:
quoted
On Wed, Apr 21, 2021 at 10:43 PM Yonghong Song [off-list ref] wrote:
quoted
On 4/16/21 1:23 PM, Andrii Nakryiko wrote:
quoted
Define __hidden helper macro in bpf_helpers.h, which is a short-hand for
__attribute__((visibility("hidden"))). Add libbpf support to mark BPF
subprograms marked with __hidden as static in BTF information to enforce BPF
verifier's static function validation algorithm, which takes more information
(caller's context) into account during a subprogram validation.
Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
---
tools/lib/bpf/bpf_helpers.h | 8 ++++++
tools/lib/bpf/btf.c | 5 ----
tools/lib/bpf/libbpf.c | 45 ++++++++++++++++++++++++++++++++-
tools/lib/bpf/libbpf_internal.h | 6 +++++
4 files changed, 58 insertions(+), 6 deletions(-)
To prevent potential external __hidden macro definition conflict, how
about
#ifdef __hidden
#undef __hidden
#define __hidden __attribute__((visibility("hidden")))
#endif
We do force #undef only with __always_inline because of the bad
definition in linux/stddef.h And we check #ifndef for __weak, because
__weak is defined in kernel headers. This is not really the case for
__hidden, the only definition is in
tools/lib/traceevent/event-parse-local.h, which I don't think we
should worry about in BPF context. So I wanted to keep it simple and
fix only if that really causes some real conflicts.
And keep in mind that in BPF code bpf_helpers.h is usually included as
one of the first few headers anyways.
That is fine. Conflict of __hidden is a low risk and we can deal with it
later if needed.
quoted
quoted
quoted
+
/* When utilizing vmlinux.h with BPF CO-RE, user BPF programs can't include
* any system-level headers (such as stddef.h, linux/version.h, etc), and
* commonly-used macros like NULL and KERNEL_VERSION aren't available through
[...]
quoted
quoted
@@ -698,6 +700,15 @@ bpf_object__add_programs(struct bpf_object *obj, Elf_Data *sec_data, if (err) return err;+ /* if function is a global/weak symbol, but has hidden+ * visibility (or any non-default one), mark its BTF FUNC as+ * static to enable more permissive BPF verification mode with+ * more outside context available to BPF verifier+ */+ if (GELF_ST_BIND(sym.st_info) != STB_LOCAL+ && GELF_ST_VISIBILITY(sym.st_other) != STV_DEFAULT)
Maybe we should check GELF_ST_VISIBILITY(sym.st_other) == STV_HIDDEN
instead?
It felt like only STV_DEFAULT should be "exported", semantically
speaking. Everything else would be treated as if it was static, except
that C rules require that function has to be global. Do you think
there is some danger to do it this way?
Currently static linker doesn't do anything special for STV_INTERNAL
and STV_PROTECTED, so we could just disable those. Do you prefer that?
Yes, let us just deal with STV_DEFAULT and STV_HIDDEN. We already
specialized STV_HIDDEN, so we should not treat STV_INTERNAL/PROTECTED
as what they mean in ELF standard, so let us disable them for now.
Yep, will do
quoted
I just felt that there is no risk of regression if we do this for
non-STV_DEFAULT generically.
On Thu, Apr 22, 2021 at 3:56 PM Yonghong Song [off-list ref] wrote:
On 4/16/21 1:23 PM, Andrii Nakryiko wrote:
quoted
Add extra logic to handle map externs (only BTF-defined maps are supported for
linking). Re-use the map parsing logic used during bpf_object__open(). Map
externs are currently restricted to always match complete map definition. So
all the specified attributes will be compared (down to pining, map_flags,
numa_node, etc). In the future this restriction might be relaxed with no
backwards compatibility issues. If any attribute is mismatched between extern
and actual map definition, linker will report an error, pointing out which one
mismatches.
The original intent was to allow for extern to specify attributes that matters
(to user) to enforce. E.g., if you specify just key information and omit
value, then any value fits. Similarly, it should have been possible to enforce
map_flags, pinning, and any other possible map attribute. Unfortunately, that
means that multiple externs can be only partially overlapping with each other,
which means linker would need to combine their type definitions to end up with
the most restrictive and fullest map definition. This requires an extra amount
of BTF manipulation which at this time was deemed unnecessary and would
require further extending generic BTF writer APIs. So that is left for future
follow ups, if there will be demand for that. But the idea seems intresting
and useful, so I want to document it here.
Weak definitions are also supported, but are pretty strict as well, just
like externs: all weak map definitions have to match exactly. In the follow up
patches this most probably will be relaxed, with __weak map definitions being
able to differ between each other (with non-weak definition always winning, of
course).
Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
I think strict enforcement of extern/global map definitions is good.
If library want people will use its maps, it may put the map definition
into one of its headers and application can include and have
exact the same definition.
In a lot of cases yes. But imagine I, as BPF library creator, started
out with just a typical hashmap definition, and then decided to add
pinning and maybe map_flags BPF_F_NO_PREALLOC. Why would that change
necessitate extern definition? But as you said, library provider can
(and should) provide extern definition that will be kept 100% in sync,
so this is not something that I urgently want to change.
From: Yonghong Song <hidden> Date: 2021-04-22 23:58:26
On 4/22/21 3:12 PM, Andrii Nakryiko wrote:
On Thu, Apr 22, 2021 at 2:27 PM Yonghong Song [off-list ref] wrote:
quoted
On 4/16/21 1:23 PM, Andrii Nakryiko wrote:
quoted
Add BPF static linker logic to resolve extern variables and functions across
multiple linked together BPF object files.
For that, linker maintains a separate list of struct glob_sym structures,
which keeps track of few pieces of metadata (is it extern or resolved global,
is it a weak symbol, which ELF section it belongs to, etc) and ties together
BTF type info and ELF symbol information and keeps them in sync.
With adding support for extern variables/funcs, it's now possible for some
sections to contain both extern and non-extern definitions. This means that
some sections may start out as ephemeral (if only externs are present and thus
there is not corresponding ELF section), but will be "upgraded" to actual ELF
section as symbols are resolved or new non-extern definitions are appended.
Additional care is taken to not duplicate extern entries in sections like
.kconfig and .ksyms.
Given libbpf requires BTF type to always be present for .kconfig/.ksym
externs, linker extends this requirement to all the externs, even those that
are supposed to be resolved during static linking and which won't be visible
to libbpf. With BTF information always present, static linker will check not
just ELF symbol matches, but entire BTF type signature match as well. That
logic is stricter that BPF CO-RE checks. It probably should be re-used by
.ksym resolution logic in libbpf as well, but that's left for follow up
patches.
To make it unnecessary to rewrite ELF symbols and minimize BTF type
rewriting/removal, ELF symbols that correspond to externs initially will be
updated in place once they are resolved. Similarly for BTF type info, VAR/FUNC
and var_secinfo's (sec_vars in struct bpf_linker) are staying stable, but
types they point to might get replaced when extern is resolved. This might
leave some left-over types (even though we try to minimize this for common
cases of having extern funcs with not argument names vs concrete function with
names properly specified). That can be addresses later with a generic BTF
garbage collection. That's left for a follow up as well.
Given BTF type appending phase is separate from ELF symbol
appending/resolution, special struct glob_sym->underlying_btf_id variable is
used to communicate resolution and rewrite decisions. 0 means
underlying_btf_id needs to be appended (it's not yet in final linker->btf), <0
values are used for temporary storage of source BTF type ID (not yet
rewritten), so -glob_sym->underlying_btf_id is BTF type id in obj-btf. But by
the end of linker_append_btf() phase, that underlying_btf_id will be remapped
and will always be > 0. This is the uglies part of the whole process, but
keeps the other parts much simpler due to stability of sec_var and VAR/FUNC
types, as well as ELF symbol, so please keep that in mind while reviewing.
This is indeed complicated. I has some comments below. Please check
whether my understanding is correct or not.
quoted
BTF-defined maps require some extra custom logic and is addressed separate in
the next patch, so that to keep this one smaller and easier to review.
Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
---
tools/lib/bpf/linker.c | 844 ++++++++++++++++++++++++++++++++++++++---
1 file changed, 785 insertions(+), 59 deletions(-)
@@ -22,6 +22,8 @@#include"libbpf_internal.h"#include"strset.h"+#define BTF_EXTERN_SEC ".extern"+structsrc_sec{constchar*sec_name;/* positional (not necessarily ELF) index in an array of sections */
@@ -74,11 +76,36 @@ struct btf_ext_sec_data {void*recs;};+structglob_sym{+/* ELF symbol index */+intsym_idx;+/* associated section id for .ksyms, .kconfig, etc, but not .extern */+intsec_id;+/* extern name offset in STRTAB */+intname_off;+/* optional associated BTF type ID */+intbtf_id;+/* BTF type ID to which VAR/FUNC type is pointing to; used for+*rewritingtypeswhenexternVAR/FUNCisresolvedtoaconcrete+*definition+*/+intunderlying_btf_id;+/* sec_var index in the corresponding dst_sec, if exists */+intvar_idx;++/* extern or resolved/global symbol */+boolis_extern;+/* weak or strong symbol, never goes back from strong to weak */+boolis_weak;+};+structdst_sec{char*sec_name;/* positional (not necessarily ELF) index in an array of sections */intid;+boolephemeral;+/* ELF info */size_tsec_idx;Elf_Scn*scn;
+
+ case BTF_KIND_TYPEDEF:
+ case BTF_KIND_VOLATILE:
+ case BTF_KIND_CONST:
+ case BTF_KIND_RESTRICT:
We already did skip_mods_and_typedefs() before. Unless something serious
wrong, we should not hit the above four types. So I think we can skip
them here.
This is the way of documenting explicitly that I'm aware of those
kinds and they shouldn't be encountered. Otherwise one might wonder if
we just forgot to handle them.
maybe add a short comment like "shouldn't happen due to
skip_mods_and_typedefs(), added here for completeness"?
quoted
quoted
+ case BTF_KIND_DATASEC:
+ default:
+ pr_warn("global '%s': unsupported BTF kind %s\n",
+ sym_name, btf_kind_str(t1));
+ return false;
+ }
+}
+
+static bool glob_syms_match(const char *sym_name,
+ struct bpf_linker *linker, struct glob_sym *glob_sym,
+ struct src_obj *obj, Elf64_Sym *sym, size_t sym_idx, int btf_id)
+{
+ const struct btf_type *src_t;
+
+ /* if we are dealing with externs, BTF types describing both global
+ * and extern VARs/FUNCs should be completely present in all files
+ */
+ if (!glob_sym->btf_id || !btf_id) {
+ pr_warn("BTF info is missing for global symbol '%s'\n", sym_name);
+ return false;
+ }
+
+ src_t = btf__type_by_id(obj->btf, btf_id);
+ if (!btf_is_var(src_t) && !btf_is_func(src_t)) {
+ pr_warn("only extern variables and functions are supported, but got '%s' for '%s'\n",
+ btf_kind_str(src_t), sym_name);
+ return false;
+ }
+
+ if (!glob_sym_btf_matches(sym_name, true /*exact*/,
+ linker->btf, glob_sym->btf_id, obj->btf, btf_id))
+ return false;
+
+ return true;
+}
+
[...]
quoted
+
+static void sym_update_visibility(Elf64_Sym *sym, int sym_vis)
+{
+ /* libelf doesn't provide setters for ST_VISIBILITY,
+ * but it is stored in the lower 2 bits of st_other
+ */
+ sym->st_other &= 0x03;
+ sym->st_other |= sym_vis;
+}
+
+static int linker_append_elf_sym(struct bpf_linker *linker, struct src_obj *obj,
+ Elf64_Sym *sym, const char *sym_name, int src_sym_idx)
+{
+ struct src_sec *src_sec = NULL;
+ struct dst_sec *dst_sec = NULL;
+ struct glob_sym *glob_sym = NULL;
+ int name_off, sym_type, sym_bind, sym_vis, err;
+ int btf_sec_id = 0, btf_id = 0;
+ size_t dst_sym_idx;
+ Elf64_Sym *dst_sym;
+ bool sym_is_extern;
+
+ sym_type = ELF64_ST_TYPE(sym->st_info);
+ sym_bind = ELF64_ST_BIND(sym->st_info);
+ sym_vis = ELF64_ST_VISIBILITY(sym->st_other);
+ sym_is_extern = sym->st_shndx == SHN_UNDEF;
+
+ if (sym_is_extern) {
+ if (!obj->btf) {
+ pr_warn("externs without BTF info are not supported\n");
+ return -ENOTSUP;
+ }
+ } else if (sym->st_shndx < SHN_LORESERVE) {
So what happens if sym->st_shndx >= SHN_LORESERVE. Maybe return failures
here? In general, bpf program shouldn't hit sym->st_shndx >= SHN_LORESERVE.
There is at least SHN_ABS (0xfff1), which is an informational STT_FILE
symbol. libbpf doesn't error out on such special symbols, and linker
will just pass-through them and append to the final object file.
Okay, I see. Never paid attention to it.
quoted
quoted
+ src_sec = &obj->secs[sym->st_shndx];
+ if (src_sec->skipped)
+ return 0;
+ dst_sec = &linker->secs[src_sec->dst_id];
+
+ /* allow only one STT_SECTION symbol per section */
+ if (sym_type == STT_SECTION && dst_sec->sec_sym_idx) {
+ obj->sym_map[src_sym_idx] = dst_sec->sec_sym_idx;
+ return 0;
+ }
+ }
+
+ if (sym_bind == STB_LOCAL)
+ goto add_sym;
+
+ /* find matching BTF info */
+ err = find_glob_sym_btf(obj, sym, sym_name, &btf_sec_id, &btf_id);
+ if (err)
+ return err;
+
+ if (sym_is_extern && btf_sec_id) {
+ const char *sec_name = NULL;
+ const struct btf_type *t;
+
+ t = btf__type_by_id(obj->btf, btf_sec_id);
+ sec_name = btf__str_by_offset(obj->btf, t->name_off);
+
+ /* Clang puts unannotated extern vars into
+ * '.extern' BTF DATASEC. Treat them the same
+ * as unannotated extern funcs (which are
+ * currently not put into any DATASECs).
+ * Those don't have associated src_sec/dst_sec.
+ */
+ if (strcmp(sec_name, BTF_EXTERN_SEC) != 0) {
+ src_sec = find_src_sec_by_name(obj, sec_name);
+ if (!src_sec) {
+ pr_warn("failed to find matching ELF sec '%s'\n", sec_name);
+ return -ENOENT;
+ }
+ dst_sec = &linker->secs[src_sec->dst_id];
+ }
+ }
+
+ glob_sym = find_glob_sym(linker, sym_name);
+ if (glob_sym) {
+ /* Preventively resolve to existing symbol. This is
+ * needed for further relocation symbol remapping in
+ * the next step of linking.
+ */
+ obj->sym_map[src_sym_idx] = glob_sym->sym_idx;
+
+ /* If both symbols are non-externs, at least one of
+ * them has to be STB_WEAK, otherwise they are in
+ * a conflict with each other.
+ */
+ if (!sym_is_extern && !glob_sym->is_extern
+ && !glob_sym->is_weak && sym_bind != STB_WEAK) {
+ pr_warn("conflicting non-weak symbol #%d (%s) definition in '%s'\n",
+ src_sym_idx, sym_name, obj->filename);
+ return -EINVAL;
}
+ if (!glob_syms_match(sym_name, linker, glob_sym, obj, sym, src_sym_idx, btf_id))
+ return -EINVAL;
+
+ dst_sym = get_sym_by_idx(linker, glob_sym->sym_idx);
+
+ /* If new symbol is strong, then force dst_sym to be strong as
+ * well; this way a mix of weak and non-weak extern
+ * definitions will end up being strong.
+ */
+ if (sym_bind == STB_GLOBAL) {
+ /* We still need to preserve type (NOTYPE or
+ * OBJECT/FUNC, depending on whether the symbol is
+ * extern or not)
+ */
+ sym_update_bind(dst_sym, STB_GLOBAL);
+ glob_sym->is_weak = false;
+ }
+
+ /* Non-default visibility is "contaminating", with stricter
+ * visibility overwriting more permissive ones, even if more
+ * permissive visibility comes from just an extern definition
+ */
+ if (sym_vis > ELF64_ST_VISIBILITY(dst_sym->st_other))
+ sym_update_visibility(dst_sym, sym_vis);
For visibility, maybe we can just handle DEFAULT and HIDDEN, and others
are not supported? DEFAULT + DEFAULT/HIDDEN => DEFAULT, HIDDEN + HIDDEN
=> HIDDEN?
Sure, we can restrict this to STV_DEFAULT and STV_HIDDEN for now.
quoted
quoted
+
+ /* If the new symbol is extern, then regardless if
+ * existing symbol is extern or resolved global, just
+ * keep the existing one untouched.
+ */
+ if (sym_is_extern)
+ return 0;
+
+ /* If existing symbol is a strong resolved symbol, bail out,
+ * because we lost resolution battle have nothing to
+ * contribute. We already checked abover that there is no
+ * strong-strong conflict. We also already tightened binding
+ * and visibility, so nothing else to contribute at that point.
+ */
+ if (!glob_sym->is_extern && sym_bind == STB_WEAK)
+ return 0;
+
+ /* At this point, new symbol is strong non-extern,
+ * so overwrite glob_sym with new symbol information.
+ * Preserve binding and visibility.
+ */
+ sym_update_type(dst_sym, sym_type);
+ dst_sym->st_shndx = dst_sec->sec_idx;
+ dst_sym->st_value = src_sec->dst_off + sym->st_value;
+ dst_sym->st_size = sym->st_size;
+
+ /* see comment below about dst_sec->id vs dst_sec->sec_idx */
+ glob_sym->sec_id = dst_sec->id;
+ glob_sym->is_extern = false;
+ /* never relax strong to weak binding */
+ if (sym_bind == STB_GLOBAL)
+ glob_sym->is_weak = false;
In the above, we already set glob_sym->is_weak to false if STB_GLOBAL.
yep, you are right, this is unnecessary, I'll remove
quoted
quoted
+
+ if (complete_extern_btf_info(linker->btf, glob_sym->btf_id,
+ obj->btf, btf_id))
+ return -EINVAL;
+
+ /* request updating VAR's/FUNC's underlying BTF type when appending BTF type */
+ glob_sym->underlying_btf_id = 0;
+
+ obj->sym_map[src_sym_idx] = glob_sym->sym_idx;
+ return 0;
+ }
+
+add_sym:
+ name_off = strset__add_str(linker->strtab_strs, sym_name);
+ if (name_off < 0)
+ return name_off;
+
+ dst_sym = add_new_sym(linker, &dst_sym_idx);
+ if (!dst_sym)
+ return -ENOMEM;
+
+ dst_sym->st_name = name_off;
+ dst_sym->st_info = sym->st_info;
+ dst_sym->st_other = sym->st_other;
+ dst_sym->st_shndx = dst_sec ? dst_sec->sec_idx : sym->st_shndx;
+ dst_sym->st_value = (src_sec ? src_sec->dst_off : 0) + sym->st_value;
+ dst_sym->st_size = sym->st_size;
+
+ obj->sym_map[src_sym_idx] = dst_sym_idx;
+
+ if (sym_type == STT_SECTION && dst_sym) {
+ dst_sec->sec_sym_idx = dst_sym_idx;
+ dst_sym->st_value = 0;
+ }
+
+ if (sym_bind != STB_LOCAL) {
+ glob_sym = add_glob_sym(linker);
+ if (!glob_sym)
+ return -ENOMEM;
+
+ glob_sym->sym_idx = dst_sym_idx;
+ /* we use dst_sec->id (and not dst_sec->sec_idx), because
+ * ephemeral sections (.kconfig, .ksyms, etc) don't have
+ * sec_idx (as they don't have corresponding ELF section), but
+ * still have id. .extern doesn't have even ephemeral section
+ * associated with it, so dst_sec->id == dst_sec->sec_idx == 0.
+ */
+ glob_sym->sec_id = dst_sec ? dst_sec->id : 0;
+ glob_sym->name_off = name_off;
+ /* we will fill btf_id in during BTF merging step */
+ glob_sym->btf_id = 0;
+ glob_sym->is_extern = sym_is_extern;
+ glob_sym->is_weak = sym_bind == STB_WEAK;
}
return 0;
@@ -1442,6 +2078,7 @@ static int linker_append_btf(struct bpf_linker *linker, struct src_obj *obj) { const struct btf_type *t; int i, j, n, start_id, id;+ const char *name; if (!obj->btf) return 0;
@@ -1454,12 +2091,40 @@ static int linker_append_btf(struct bpf_linker *linker, struct src_obj *obj) return -ENOMEM; for (i = 1; i <= n; i++) {+ struct glob_sym *glob_sym = NULL;+ t = btf__type_by_id(obj->btf, i); /* DATASECs are handled specially below */ if (btf_kind(t) == BTF_KIND_DATASEC) continue;+ if (btf_is_non_static(t)) {+ /* there should be glob_sym already */+ name = btf__str_by_offset(obj->btf, t->name_off);+ glob_sym = find_glob_sym(linker, name);++ /* VARs without corresponding glob_sym are those that+ * belong to skipped/deduplicated sections (i.e.,+ * license and version), so just skip them+ */+ if (!glob_sym)+ continue;++ if (glob_sym->underlying_btf_id == 0)+ glob_sym->underlying_btf_id = -t->type;
Is this needed? If glob_sym->btf_id is not NULL, then
glob_sym->underlying_btf_id has been set by the previous object.
If it is NULL, it will set probably after this
if (btf_is_non_static(t)) { ...}, is this right?
I think it's still needed. Here's the scenario.
1. Obj file A contains extern symbol X. We create corresponding
glob_sym (with is_extern=true), and store btf_id to point to
BTF_KIND_VAR, and btf_underlying_id to point to the type that
BTF_KIND_VAR points to.
2. Obj file B contains non-extern symbol X. At this point
linker_append_elf_sym() will update glob_sym to is_extern = false, it
will keep btf_id to re-use already appended BTF_KIND_VAR, but it will
zero-out underlying_btf_id, because for externs type could be
incomplete (e.g. for functions it won't contain function argument
names, for maps it could differ even more drastically later). So then
we get here, we see that glob_sym->underlying_btf_id is zero, so needs
updating. We store it as -Y, because Y is BTF type ID in obj->btf, not
in linker->btf (yet). Then the if (glob_sym->btf_id) below sees that
glob_sym->btf_id is already set, so we just keep using already
appended BTF_KIND_VAR (we already set its linkage to
BTF_VAR_GLOBAL_ALLOCATED in complete_extern_btf_info(), called from
linker_append_elf_sym(). So we'll skip appending another BTF_KIND_VAR.
But we do want to point existing BTF_KIND_VAR to a new type that
corresponds to ID -Y.
Thanks for explanation, I missed the code in
/* request updating VAR's/FUNC's underlying BTF type when
appending BTF type */
glob_sym->underlying_btf_id = 0;
in linker_append_elf_sym().
Maybe add a comment above the code with something like
underlying_btf_id may have been reset to 0 due to the presence of
a strong global variable.
?
quoted
quoted
+
+ /* globals from previous object files that match our
+ * VAR/FUNC already have a corresponding associated
+ * BTF type, so just make sure to use it
+ */
+ if (glob_sym->btf_id) {
+ /* reuse existing BTF type for global var/func */
+ obj->btf_type_map[i] = glob_sym->btf_id;
+ continue;
+ }
+ }
+
id = btf__add_type(linker->btf, obj->btf, t);
if (id < 0) {
pr_warn("failed to append BTF type #%d from file '%s'\n", i, obj->filename);
@@ -1467,6 +2132,12 @@ static int linker_append_btf(struct bpf_linker *linker, struct src_obj *obj) } obj->btf_type_map[i] = id;++ /* record just appended BTF type for var/func */+ if (glob_sym) {+ glob_sym->btf_id = id;+ glob_sym->underlying_btf_id = -t->type;+ } } /* remap all the types except DATASECs */
@@ -1478,6 +2149,22 @@ static int linker_append_btf(struct bpf_linker *linker, struct src_obj *obj) return -EINVAL; }+ /* Rewrite VAR/FUNC underlying types (i.e., FUNC's FUNC_PROTO and VAR's+ * actual type), if necessary+ */+ for (i = 0; i < linker->glob_sym_cnt; i++) {+ struct glob_sym *glob_sym = &linker->glob_syms[i];+ struct btf_type *glob_t;++ if (glob_sym->underlying_btf_id >= 0)+ continue;++ glob_sym->underlying_btf_id = obj->btf_type_map[-glob_sym->underlying_btf_id];
After this point, any new *extern* variables will hit the below in the
previous code:
Right, but we want to hit this for existing glob_syms that went from
extern to non-extern or from weak to non-weak. See
/* request updating VAR's/FUNC's underlying BTF type when
appending BTF type */
glob_sym->underlying_btf_id = 0;
in linker_append_elf_sym().
And we'll use that even more extensively when extending __weak and
extern map definitions later.
quoted
> + if (glob_sym->btf_id) {
> + /* reuse existing BTF type for global var/func */
> + obj->btf_type_map[i] = glob_sym->btf_id;
> + continue;
> + }
quoted
+
+ glob_t = btf_type_by_id(linker->btf, glob_sym->btf_id);
+ glob_t->type = glob_sym->underlying_btf_id;
+ }
+
/* append DATASEC info */
for (i = 1; i < obj->sec_cnt; i++) {
struct src_sec *src_sec;
From: Yonghong Song <hidden> Date: 2021-04-23 00:05:36
On 4/16/21 1:24 PM, Andrii Nakryiko wrote:
While -Og is designed to work well with debugger, it's still inferior to -O0
in terms of debuggability experience. It will cause some variables to still be
inlined, it will also prevent single-stepping some statements and otherwise
interfere with debugging experience. So switch to -O0 which turns off any
optimization and provides the best debugging experience.
Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
From: Yonghong Song <hidden> Date: 2021-04-23 00:14:35
On 4/16/21 1:24 PM, Andrii Nakryiko wrote:
Skip generating individual BPF skeletons for files that are supposed to be
linked together to form the final BPF object file. Very often such files are
"incomplete" BPF object files, which will fail libbpf bpf_object__open() step,
if used individually, thus failing BPF skeleton generation. This is by design,
so skip individual BPF skeletons and only validate them as part of their
linked final BPF object file and skeleton.
Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
From: Yonghong Song <hidden> Date: 2021-04-23 00:50:44
On 4/16/21 1:24 PM, Andrii Nakryiko wrote:
Add selftest validating various aspects of statically linking functions:
- no conflicts and correct resolution for name-conflicting static funcs;
- correct resolution of extern functions;
- correct handling of weak functions, both resolution itself and libbpf's
handling of unused weak function that "lost" (it leaves gaps in code with
no ELF symbols);
- correct handling of hidden visibility to turn global function into
"static" for the purpose of BPF verification.
Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
Ack with a small nit below.
Acked-by: Yonghong Song <redacted>
@@ -0,0 +1,73 @@+// SPDX-License-Identifier: GPL-2.0+/* Copyright (c) 2021 Facebook */++#include"vmlinux.h"+#include<bpf/bpf_helpers.h>+#include<bpf/bpf_tracing.h>++/* weak and shared between two files */+constvolatileintmy_tid__weak=0;+constvolatilelongsyscall_id__weak=0;
Since the new compiler (llvm13) is recommended for this patch set.
We can simplify the above two definition with
int my_tid __weak;
long syscall_id __weak;
The same for the other file.
But I am also okay with the current form
to *satisfy* llvm10 some people may still use.
+
+int output_val1 = 0;
+int output_ctx1 = 0;
+int output_weak1 = 0;
+
+/* same "subprog" name in all files, but it's ok because they all are static */
+static __noinline int subprog(int x)
+{
+ /* but different formula */
+ return x * 1;
+}
+
+/* Global functions can't be void */
+int set_output_val1(int x)
+{
+ output_val1 = x + subprog(x);
+ return x;
+}
+
+/* This function can't be verified as global, as it assumes raw_tp/sys_enter
+ * context and accesses syscall id (second argument). So we mark it as
+ * __hidden, so that libbpf will mark it as static in the final object file,
+ * right before verifying it in the kernel.
+ *
+ * But we don't mark it as __hidden here, rather at extern site. __hidden is
+ * "contaminating" visibility, so it will get propagated from either extern or
+ * actual definition (including from the losing __weak definition).
+ */
+void set_output_ctx1(__u64 *ctx)
+{
+ output_ctx1 = ctx[1]; /* long id, same as in BPF_PROG below */
+}
+
+/* this weak instance should win because it's the first one */
+__weak int set_output_weak(int x)
+{
+ output_weak1 = x;
+ return x;
+}
+
+extern int set_output_val2(int x);
+
+/* here we'll force set_output_ctx2() to be __hidden in the final obj file */
+__hidden extern void set_output_ctx2(__u64 *ctx);
+
From: Yonghong Song <hidden> Date: 2021-04-23 01:02:20
On 4/16/21 1:24 PM, Andrii Nakryiko wrote:
Add selftest validating various aspects of statically linking global
variables:
- correct resolution of extern variables across .bss, .data, and .rodata
sections;
- correct handling of weak definitions;
- correct de-duplication of repeating special externs (.kconfig, .ksyms).
Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
Ack with a minor nit below.
Acked-by: Yonghong Song <redacted>
@@ -0,0 +1,43 @@+// SPDX-License-Identifier: GPL-2.0+/* Copyright (c) 2021 Facebook */++#include<test_progs.h>+#include<sys/syscall.h>+#include"linked_vars.skel.h"++voidtest_linked_vars(void)+{+interr;+structlinked_vars*skel;++skel=linked_vars__open();+if(!ASSERT_OK_PTR(skel,"skel_open"))+return;++skel->bss->input_bss1=1000;+skel->bss->input_bss2=2000;+skel->bss->input_bss_weak=3000;++err=linked_vars__load(skel);+if(!ASSERT_OK(err,"skel_load"))+gotocleanup;++err=linked_vars__attach(skel);+if(!ASSERT_OK(err,"skel_attach"))+gotocleanup;++/* trigger */+syscall(SYS_getpgid);++ASSERT_EQ(skel->bss->output_bss1,1000+2000+3000,"output_bss1");+ASSERT_EQ(skel->bss->output_bss2,1000+2000+3000,"output_bss2");+/* 10 comes from "winner" input_data_weak in first obj file */+ASSERT_EQ(skel->bss->output_data1,1+2+10,"output_bss1");+ASSERT_EQ(skel->bss->output_data2,1+2+10,"output_bss2");+/* 10 comes from "winner" input_rodata_weak in first obj file */
@@ -0,0 +1,54 @@+// SPDX-License-Identifier: GPL-2.0+/* Copyright (c) 2021 Facebook */++#include"vmlinux.h"+#include<bpf/bpf_helpers.h>+#include<bpf/bpf_tracing.h>++externintLINUX_KERNEL_VERSION__kconfig;+/* this weak extern will be strict due to the other file's strong extern */+externboolCONFIG_BPF_SYSCALL__kconfig__weak;+externconstvoidbpf_link_fops__ksym__weak;++intinput_bss1=0;+intinput_data1=1;+constvolatileintinput_rodata1=11;++intinput_bss_weak__weak=0;+/* these two definitions should win */+intinput_data_weak__weak=10;+constvolatileintinput_rodata_weak__weak=100;+
From: Yonghong Song <hidden> Date: 2021-04-23 01:20:27
On 4/16/21 1:24 PM, Andrii Nakryiko wrote:
Add selftest validating various aspects of statically linking BTF-defined map
definitions. Legacy map definitions do not support extern resolution between
object files. Some of the aspects validated:
- correct resolution of extern maps against concrete map definitions;
- extern maps can currently only specify map type and key/value size and/or
type information;
- weak concrete map definitions are resolved properly.
Static map definitions are not yet supported by libbpf, so they are not
explicitly tested, though manual testing showes that BPF linker handles them
properly.
Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
Ack with a nit below.
Acked-by: Yonghong Song <redacted>
@@ -0,0 +1,82 @@+// SPDX-License-Identifier: GPL-2.0+/* Copyright (c) 2021 Facebook */++#include"vmlinux.h"+#include<bpf/bpf_helpers.h>+#include<bpf/bpf_tracing.h>++structmy_key{longx;};+structmy_value{longx;};++struct{+__uint(type,BPF_MAP_TYPE_HASH);+__type(key,structmy_key);+__type(value,structmy_value);+__uint(max_entries,16);+}map1SEC(".maps");++/* Matches map2 definition in linked_maps2.c. Order of the attributes doesn't+*matter.+*/+typedefstruct{+__uint(max_entries,8);+__type(key,int);+__type(value,int);+__uint(type,BPF_MAP_TYPE_ARRAY);+}map2_t;++externmap2_tmap2SEC(".maps");++/* This should be the winning map definition, but we have no way of verifying,+*sowejustmakesurethatitlinksandworkswithouterrors+*/
If in debug output, you output something like
map_weak in obj linked_maps1.o wins
you can open debug mode for this test, capture debug output and check
the above substring. But this is really fragile and not really
recommended. So as long as functionality works, we should be fine.
On Thu, Apr 22, 2021 at 5:51 PM Yonghong Song [off-list ref] wrote:
quoted
+
+/* weak and shared between two files */
+const volatile int my_tid __weak = 0;
+const volatile long syscall_id __weak = 0;
Since the new compiler (llvm13) is recommended for this patch set.
We can simplify the above two definition with
int my_tid __weak;
long syscall_id __weak;
The same for the other file.
But I am also okay with the current form
to *satisfy* llvm10 some people may still use.
The test won't work with anything, but the latest llvm trunk,
so " = 0" is useless.
Let's remove it.
Especially from the tests that rely on the latest llvm.
No one can backport the latest llvm BPF backend to llvm10 front-end.
From: Yonghong Song <hidden> Date: 2021-04-23 02:36:59
On 4/22/21 4:57 PM, Yonghong Song wrote:
On 4/22/21 3:12 PM, Andrii Nakryiko wrote:
quoted
On Thu, Apr 22, 2021 at 2:27 PM Yonghong Song [off-list ref] wrote:
quoted
On 4/16/21 1:23 PM, Andrii Nakryiko wrote:
quoted
Add BPF static linker logic to resolve extern variables and
functions across
multiple linked together BPF object files.
For that, linker maintains a separate list of struct glob_sym
structures,
which keeps track of few pieces of metadata (is it extern or
resolved global,
is it a weak symbol, which ELF section it belongs to, etc) and ties
together
BTF type info and ELF symbol information and keeps them in sync.
With adding support for extern variables/funcs, it's now possible
for some
sections to contain both extern and non-extern definitions. This
means that
some sections may start out as ephemeral (if only externs are
present and thus
there is not corresponding ELF section), but will be "upgraded" to
actual ELF
section as symbols are resolved or new non-extern definitions are
appended.
Additional care is taken to not duplicate extern entries in sections
like
.kconfig and .ksyms.
Given libbpf requires BTF type to always be present for .kconfig/.ksym
externs, linker extends this requirement to all the externs, even
those that
are supposed to be resolved during static linking and which won't be
visible
to libbpf. With BTF information always present, static linker will
check not
just ELF symbol matches, but entire BTF type signature match as
well. That
logic is stricter that BPF CO-RE checks. It probably should be
re-used by
.ksym resolution logic in libbpf as well, but that's left for follow up
patches.
To make it unnecessary to rewrite ELF symbols and minimize BTF type
rewriting/removal, ELF symbols that correspond to externs initially
will be
updated in place once they are resolved. Similarly for BTF type
info, VAR/FUNC
and var_secinfo's (sec_vars in struct bpf_linker) are staying
stable, but
types they point to might get replaced when extern is resolved. This
might
leave some left-over types (even though we try to minimize this for
common
cases of having extern funcs with not argument names vs concrete
function with
names properly specified). That can be addresses later with a
generic BTF
garbage collection. That's left for a follow up as well.
Given BTF type appending phase is separate from ELF symbol
appending/resolution, special struct glob_sym->underlying_btf_id
variable is
used to communicate resolution and rewrite decisions. 0 means
underlying_btf_id needs to be appended (it's not yet in final
linker->btf), <0
values are used for temporary storage of source BTF type ID (not yet
rewritten), so -glob_sym->underlying_btf_id is BTF type id in
obj-btf. But by
the end of linker_append_btf() phase, that underlying_btf_id will be
remapped
and will always be > 0. This is the uglies part of the whole
process, but
keeps the other parts much simpler due to stability of sec_var and
VAR/FUNC
types, as well as ELF symbol, so please keep that in mind while
reviewing.
This is indeed complicated. I has some comments below. Please check
whether my understanding is correct or not.
quoted
BTF-defined maps require some extra custom logic and is addressed
separate in
the next patch, so that to keep this one smaller and easier to review.
Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
---
tools/lib/bpf/linker.c | 844
++++++++++++++++++++++++++++++++++++++---
1 file changed, 785 insertions(+), 59 deletions(-)
[...]
quoted
quoted
quoted
+ src_sec = &obj->secs[sym->st_shndx];
+ if (src_sec->skipped)
+ return 0;
+ dst_sec = &linker->secs[src_sec->dst_id];
+
+ /* allow only one STT_SECTION symbol per section */
+ if (sym_type == STT_SECTION && dst_sec->sec_sym_idx) {
+ obj->sym_map[src_sym_idx] = dst_sec->sec_sym_idx;
+ return 0;
+ }
+ }
+
+ if (sym_bind == STB_LOCAL)
+ goto add_sym;
+
+ /* find matching BTF info */
+ err = find_glob_sym_btf(obj, sym, sym_name, &btf_sec_id,
&btf_id);
+ if (err)
+ return err;
+
+ if (sym_is_extern && btf_sec_id) {
+ const char *sec_name = NULL;
+ const struct btf_type *t;
+
+ t = btf__type_by_id(obj->btf, btf_sec_id);
+ sec_name = btf__str_by_offset(obj->btf, t->name_off);
+
+ /* Clang puts unannotated extern vars into
+ * '.extern' BTF DATASEC. Treat them the same
+ * as unannotated extern funcs (which are
+ * currently not put into any DATASECs).
+ * Those don't have associated src_sec/dst_sec.
+ */
+ if (strcmp(sec_name, BTF_EXTERN_SEC) != 0) {
+ src_sec = find_src_sec_by_name(obj, sec_name);
+ if (!src_sec) {
+ pr_warn("failed to find matching ELF
sec '%s'\n", sec_name);
+ return -ENOENT;
+ }
+ dst_sec = &linker->secs[src_sec->dst_id];
+ }
+ }
+
+ glob_sym = find_glob_sym(linker, sym_name);
+ if (glob_sym) {
+ /* Preventively resolve to existing symbol. This is
+ * needed for further relocation symbol remapping in
+ * the next step of linking.
+ */
+ obj->sym_map[src_sym_idx] = glob_sym->sym_idx;
+
+ /* If both symbols are non-externs, at least one of
+ * them has to be STB_WEAK, otherwise they are in
+ * a conflict with each other.
+ */
+ if (!sym_is_extern && !glob_sym->is_extern
+ && !glob_sym->is_weak && sym_bind != STB_WEAK) {
+ pr_warn("conflicting non-weak symbol #%d (%s)
definition in '%s'\n",
+ src_sym_idx, sym_name, obj->filename);
+ return -EINVAL;
}
+ if (!glob_syms_match(sym_name, linker, glob_sym, obj,
sym, src_sym_idx, btf_id))
+ return -EINVAL;
+
+ dst_sym = get_sym_by_idx(linker, glob_sym->sym_idx);
+
+ /* If new symbol is strong, then force dst_sym to be
strong as
+ * well; this way a mix of weak and non-weak extern
+ * definitions will end up being strong.
+ */
+ if (sym_bind == STB_GLOBAL) {
+ /* We still need to preserve type (NOTYPE or
+ * OBJECT/FUNC, depending on whether the
symbol is
+ * extern or not)
+ */
+ sym_update_bind(dst_sym, STB_GLOBAL);
+ glob_sym->is_weak = false;
+ }
+
+ /* Non-default visibility is "contaminating", with
stricter
+ * visibility overwriting more permissive ones, even
if more
+ * permissive visibility comes from just an extern
definition
+ */
+ if (sym_vis > ELF64_ST_VISIBILITY(dst_sym->st_other))
+ sym_update_visibility(dst_sym, sym_vis);
For visibility, maybe we can just handle DEFAULT and HIDDEN, and others
are not supported? DEFAULT + DEFAULT/HIDDEN => DEFAULT, HIDDEN + HIDDEN
=> HIDDEN?
Looking at your selftest. Your current approach, DEFAULT + DEFAULT ->
DEFAULT, HIDDEN + HIDDEN/DEFAULT -> HIDDEN should work fine. This is
also align with ELF principal to accommodate the least permissive
visibility.
quoted
quoted
Sure, we can restrict this to STV_DEFAULT and STV_HIDDEN for now. >>
On Thu, Apr 22, 2021 at 11:24 AM Andrii Nakryiko
[off-list ref] wrote:
On Thu, Apr 22, 2021 at 9:50 AM Yonghong Song [off-list ref] wrote:
quoted
On 4/16/21 1:23 PM, Andrii Nakryiko wrote:
quoted
It should never fail, but if it does, it's better to know about this rather
than end up with nonsensical type IDs.
So this is defensive programming. Maybe do another round of
audit of the callers and if you didn't find any issue, you
do not need to check not-happening condition here?
It's far from obvious that this will never happen, because we do a
decently complicated BTF processing (we skip some types altogether
believing that they are not used, for example) and it will only get
more complicated with time. Just as there are "verifier bug" checks in
kernel, this prevents things from going wild if non-trivial bugs will
inevitably happen.
I agree with Yonghong. This doesn't look right.
The callback will be called for all non-void types, right?
so *type_id == 0 shouldn't never happen.
If it does there is a bug somewhere that should be investigated
instead of ignored.
The
if (new_id == 0) pr_warn
bit makes sense.
My reading that it will abort the whole linking process and
this linker bug will be reported back to us.
So it's good.
On Thu, Apr 22, 2021 at 7:54 PM Alexei Starovoitov
[off-list ref] wrote:
On Thu, Apr 22, 2021 at 11:24 AM Andrii Nakryiko
[off-list ref] wrote:
quoted
On Thu, Apr 22, 2021 at 9:50 AM Yonghong Song [off-list ref] wrote:
quoted
On 4/16/21 1:23 PM, Andrii Nakryiko wrote:
quoted
It should never fail, but if it does, it's better to know about this rather
than end up with nonsensical type IDs.
So this is defensive programming. Maybe do another round of
audit of the callers and if you didn't find any issue, you
do not need to check not-happening condition here?
It's far from obvious that this will never happen, because we do a
decently complicated BTF processing (we skip some types altogether
believing that they are not used, for example) and it will only get
more complicated with time. Just as there are "verifier bug" checks in
kernel, this prevents things from going wild if non-trivial bugs will
inevitably happen.
I agree with Yonghong. This doesn't look right.
I read it as Yonghong was asking about the entire patch. You seem to
be concerned with one particular check, right?
The callback will be called for all non-void types, right?
so *type_id == 0 shouldn't never happen.
If it does there is a bug somewhere that should be investigated
instead of ignored.
See btf_type_visit_type_ids() and btf_ext_visit_type_ids(), they call
callback for every field that contains type ID, even if it points to
VOID. So this can happen and is expected.
The
if (new_id == 0) pr_warn
bit makes sense.
Right, and this is the point of this patch. id_map[] will have zeroes
for any unmapped type, so I just need to make sure I'm not false
erroring on id_map[0] (== 0, which is valid, but never used).
My reading that it will abort the whole linking process and
this linker bug will be reported back to us.
So it's good.
On Thu, Apr 22, 2021 at 4:58 PM Yonghong Song [off-list ref] wrote:
On 4/22/21 3:12 PM, Andrii Nakryiko wrote:
quoted
On Thu, Apr 22, 2021 at 2:27 PM Yonghong Song [off-list ref] wrote:
quoted
On 4/16/21 1:23 PM, Andrii Nakryiko wrote:
quoted
Add BPF static linker logic to resolve extern variables and functions across
multiple linked together BPF object files.
For that, linker maintains a separate list of struct glob_sym structures,
which keeps track of few pieces of metadata (is it extern or resolved global,
is it a weak symbol, which ELF section it belongs to, etc) and ties together
BTF type info and ELF symbol information and keeps them in sync.
With adding support for extern variables/funcs, it's now possible for some
sections to contain both extern and non-extern definitions. This means that
some sections may start out as ephemeral (if only externs are present and thus
there is not corresponding ELF section), but will be "upgraded" to actual ELF
section as symbols are resolved or new non-extern definitions are appended.
Additional care is taken to not duplicate extern entries in sections like
.kconfig and .ksyms.
Given libbpf requires BTF type to always be present for .kconfig/.ksym
externs, linker extends this requirement to all the externs, even those that
are supposed to be resolved during static linking and which won't be visible
to libbpf. With BTF information always present, static linker will check not
just ELF symbol matches, but entire BTF type signature match as well. That
logic is stricter that BPF CO-RE checks. It probably should be re-used by
.ksym resolution logic in libbpf as well, but that's left for follow up
patches.
To make it unnecessary to rewrite ELF symbols and minimize BTF type
rewriting/removal, ELF symbols that correspond to externs initially will be
updated in place once they are resolved. Similarly for BTF type info, VAR/FUNC
and var_secinfo's (sec_vars in struct bpf_linker) are staying stable, but
types they point to might get replaced when extern is resolved. This might
leave some left-over types (even though we try to minimize this for common
cases of having extern funcs with not argument names vs concrete function with
names properly specified). That can be addresses later with a generic BTF
garbage collection. That's left for a follow up as well.
Given BTF type appending phase is separate from ELF symbol
appending/resolution, special struct glob_sym->underlying_btf_id variable is
used to communicate resolution and rewrite decisions. 0 means
underlying_btf_id needs to be appended (it's not yet in final linker->btf), <0
values are used for temporary storage of source BTF type ID (not yet
rewritten), so -glob_sym->underlying_btf_id is BTF type id in obj-btf. But by
the end of linker_append_btf() phase, that underlying_btf_id will be remapped
and will always be > 0. This is the uglies part of the whole process, but
keeps the other parts much simpler due to stability of sec_var and VAR/FUNC
types, as well as ELF symbol, so please keep that in mind while reviewing.
This is indeed complicated. I has some comments below. Please check
whether my understanding is correct or not.
quoted
BTF-defined maps require some extra custom logic and is addressed separate in
the next patch, so that to keep this one smaller and easier to review.
Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
---
tools/lib/bpf/linker.c | 844 ++++++++++++++++++++++++++++++++++++++---
1 file changed, 785 insertions(+), 59 deletions(-)
@@ -22,6 +22,8 @@#include"libbpf_internal.h"#include"strset.h"+#define BTF_EXTERN_SEC ".extern"+structsrc_sec{constchar*sec_name;/* positional (not necessarily ELF) index in an array of sections */
@@ -74,11 +76,36 @@ struct btf_ext_sec_data {void*recs;};+structglob_sym{+/* ELF symbol index */+intsym_idx;+/* associated section id for .ksyms, .kconfig, etc, but not .extern */+intsec_id;+/* extern name offset in STRTAB */+intname_off;+/* optional associated BTF type ID */+intbtf_id;+/* BTF type ID to which VAR/FUNC type is pointing to; used for+*rewritingtypeswhenexternVAR/FUNCisresolvedtoaconcrete+*definition+*/+intunderlying_btf_id;+/* sec_var index in the corresponding dst_sec, if exists */+intvar_idx;++/* extern or resolved/global symbol */+boolis_extern;+/* weak or strong symbol, never goes back from strong to weak */+boolis_weak;+};+structdst_sec{char*sec_name;/* positional (not necessarily ELF) index in an array of sections */intid;+boolephemeral;+/* ELF info */size_tsec_idx;Elf_Scn*scn;
+
+ case BTF_KIND_TYPEDEF:
+ case BTF_KIND_VOLATILE:
+ case BTF_KIND_CONST:
+ case BTF_KIND_RESTRICT:
We already did skip_mods_and_typedefs() before. Unless something serious
wrong, we should not hit the above four types. So I think we can skip
them here.
This is the way of documenting explicitly that I'm aware of those
kinds and they shouldn't be encountered. Otherwise one might wonder if
we just forgot to handle them.
maybe add a short comment like "shouldn't happen due to
skip_mods_and_typedefs(), added here for completeness"?
sure, I'll leave the comment
quoted
quoted
quoted
+ case BTF_KIND_DATASEC:
+ default:
+ pr_warn("global '%s': unsupported BTF kind %s\n",
+ sym_name, btf_kind_str(t1));
+ return false;
+ }
+}
+
+static bool glob_syms_match(const char *sym_name,
+ struct bpf_linker *linker, struct glob_sym *glob_sym,
+ struct src_obj *obj, Elf64_Sym *sym, size_t sym_idx, int btf_id)
+{
+ const struct btf_type *src_t;
+
+ /* if we are dealing with externs, BTF types describing both global
+ * and extern VARs/FUNCs should be completely present in all files
+ */
+ if (!glob_sym->btf_id || !btf_id) {
+ pr_warn("BTF info is missing for global symbol '%s'\n", sym_name);
+ return false;
+ }
+
+ src_t = btf__type_by_id(obj->btf, btf_id);
+ if (!btf_is_var(src_t) && !btf_is_func(src_t)) {
+ pr_warn("only extern variables and functions are supported, but got '%s' for '%s'\n",
+ btf_kind_str(src_t), sym_name);
+ return false;
+ }
+
+ if (!glob_sym_btf_matches(sym_name, true /*exact*/,
+ linker->btf, glob_sym->btf_id, obj->btf, btf_id))
+ return false;
+
+ return true;
+}
+
[...]
quoted
+
+static void sym_update_visibility(Elf64_Sym *sym, int sym_vis)
+{
+ /* libelf doesn't provide setters for ST_VISIBILITY,
+ * but it is stored in the lower 2 bits of st_other
+ */
+ sym->st_other &= 0x03;
+ sym->st_other |= sym_vis;
+}
+
+static int linker_append_elf_sym(struct bpf_linker *linker, struct src_obj *obj,
+ Elf64_Sym *sym, const char *sym_name, int src_sym_idx)
+{
+ struct src_sec *src_sec = NULL;
+ struct dst_sec *dst_sec = NULL;
+ struct glob_sym *glob_sym = NULL;
+ int name_off, sym_type, sym_bind, sym_vis, err;
+ int btf_sec_id = 0, btf_id = 0;
+ size_t dst_sym_idx;
+ Elf64_Sym *dst_sym;
+ bool sym_is_extern;
+
+ sym_type = ELF64_ST_TYPE(sym->st_info);
+ sym_bind = ELF64_ST_BIND(sym->st_info);
+ sym_vis = ELF64_ST_VISIBILITY(sym->st_other);
+ sym_is_extern = sym->st_shndx == SHN_UNDEF;
+
+ if (sym_is_extern) {
+ if (!obj->btf) {
+ pr_warn("externs without BTF info are not supported\n");
+ return -ENOTSUP;
+ }
+ } else if (sym->st_shndx < SHN_LORESERVE) {
So what happens if sym->st_shndx >= SHN_LORESERVE. Maybe return failures
here? In general, bpf program shouldn't hit sym->st_shndx >= SHN_LORESERVE.
There is at least SHN_ABS (0xfff1), which is an informational STT_FILE
symbol. libbpf doesn't error out on such special symbols, and linker
will just pass-through them and append to the final object file.
Okay, I see. Never paid attention to it.
quoted
quoted
quoted
+ src_sec = &obj->secs[sym->st_shndx];
+ if (src_sec->skipped)
+ return 0;
+ dst_sec = &linker->secs[src_sec->dst_id];
+
+ /* allow only one STT_SECTION symbol per section */
+ if (sym_type == STT_SECTION && dst_sec->sec_sym_idx) {
+ obj->sym_map[src_sym_idx] = dst_sec->sec_sym_idx;
+ return 0;
+ }
+ }
+
+ if (sym_bind == STB_LOCAL)
+ goto add_sym;
+
+ /* find matching BTF info */
+ err = find_glob_sym_btf(obj, sym, sym_name, &btf_sec_id, &btf_id);
+ if (err)
+ return err;
+
+ if (sym_is_extern && btf_sec_id) {
+ const char *sec_name = NULL;
+ const struct btf_type *t;
+
+ t = btf__type_by_id(obj->btf, btf_sec_id);
+ sec_name = btf__str_by_offset(obj->btf, t->name_off);
+
+ /* Clang puts unannotated extern vars into
+ * '.extern' BTF DATASEC. Treat them the same
+ * as unannotated extern funcs (which are
+ * currently not put into any DATASECs).
+ * Those don't have associated src_sec/dst_sec.
+ */
+ if (strcmp(sec_name, BTF_EXTERN_SEC) != 0) {
+ src_sec = find_src_sec_by_name(obj, sec_name);
+ if (!src_sec) {
+ pr_warn("failed to find matching ELF sec '%s'\n", sec_name);
+ return -ENOENT;
+ }
+ dst_sec = &linker->secs[src_sec->dst_id];
+ }
+ }
+
+ glob_sym = find_glob_sym(linker, sym_name);
+ if (glob_sym) {
+ /* Preventively resolve to existing symbol. This is
+ * needed for further relocation symbol remapping in
+ * the next step of linking.
+ */
+ obj->sym_map[src_sym_idx] = glob_sym->sym_idx;
+
+ /* If both symbols are non-externs, at least one of
+ * them has to be STB_WEAK, otherwise they are in
+ * a conflict with each other.
+ */
+ if (!sym_is_extern && !glob_sym->is_extern
+ && !glob_sym->is_weak && sym_bind != STB_WEAK) {
+ pr_warn("conflicting non-weak symbol #%d (%s) definition in '%s'\n",
+ src_sym_idx, sym_name, obj->filename);
+ return -EINVAL;
}
+ if (!glob_syms_match(sym_name, linker, glob_sym, obj, sym, src_sym_idx, btf_id))
+ return -EINVAL;
+
+ dst_sym = get_sym_by_idx(linker, glob_sym->sym_idx);
+
+ /* If new symbol is strong, then force dst_sym to be strong as
+ * well; this way a mix of weak and non-weak extern
+ * definitions will end up being strong.
+ */
+ if (sym_bind == STB_GLOBAL) {
+ /* We still need to preserve type (NOTYPE or
+ * OBJECT/FUNC, depending on whether the symbol is
+ * extern or not)
+ */
+ sym_update_bind(dst_sym, STB_GLOBAL);
+ glob_sym->is_weak = false;
+ }
+
+ /* Non-default visibility is "contaminating", with stricter
+ * visibility overwriting more permissive ones, even if more
+ * permissive visibility comes from just an extern definition
+ */
+ if (sym_vis > ELF64_ST_VISIBILITY(dst_sym->st_other))
+ sym_update_visibility(dst_sym, sym_vis);
For visibility, maybe we can just handle DEFAULT and HIDDEN, and others
are not supported? DEFAULT + DEFAULT/HIDDEN => DEFAULT, HIDDEN + HIDDEN
=> HIDDEN?
Sure, we can restrict this to STV_DEFAULT and STV_HIDDEN for now.
quoted
quoted
+
+ /* If the new symbol is extern, then regardless if
+ * existing symbol is extern or resolved global, just
+ * keep the existing one untouched.
+ */
+ if (sym_is_extern)
+ return 0;
+
+ /* If existing symbol is a strong resolved symbol, bail out,
+ * because we lost resolution battle have nothing to
+ * contribute. We already checked abover that there is no
+ * strong-strong conflict. We also already tightened binding
+ * and visibility, so nothing else to contribute at that point.
+ */
+ if (!glob_sym->is_extern && sym_bind == STB_WEAK)
+ return 0;
+
+ /* At this point, new symbol is strong non-extern,
+ * so overwrite glob_sym with new symbol information.
+ * Preserve binding and visibility.
+ */
+ sym_update_type(dst_sym, sym_type);
+ dst_sym->st_shndx = dst_sec->sec_idx;
+ dst_sym->st_value = src_sec->dst_off + sym->st_value;
+ dst_sym->st_size = sym->st_size;
+
+ /* see comment below about dst_sec->id vs dst_sec->sec_idx */
+ glob_sym->sec_id = dst_sec->id;
+ glob_sym->is_extern = false;
+ /* never relax strong to weak binding */
+ if (sym_bind == STB_GLOBAL)
+ glob_sym->is_weak = false;
In the above, we already set glob_sym->is_weak to false if STB_GLOBAL.
yep, you are right, this is unnecessary, I'll remove
quoted
quoted
+
+ if (complete_extern_btf_info(linker->btf, glob_sym->btf_id,
+ obj->btf, btf_id))
+ return -EINVAL;
+
+ /* request updating VAR's/FUNC's underlying BTF type when appending BTF type */
+ glob_sym->underlying_btf_id = 0;
+
+ obj->sym_map[src_sym_idx] = glob_sym->sym_idx;
+ return 0;
+ }
+
+add_sym:
+ name_off = strset__add_str(linker->strtab_strs, sym_name);
+ if (name_off < 0)
+ return name_off;
+
+ dst_sym = add_new_sym(linker, &dst_sym_idx);
+ if (!dst_sym)
+ return -ENOMEM;
+
+ dst_sym->st_name = name_off;
+ dst_sym->st_info = sym->st_info;
+ dst_sym->st_other = sym->st_other;
+ dst_sym->st_shndx = dst_sec ? dst_sec->sec_idx : sym->st_shndx;
+ dst_sym->st_value = (src_sec ? src_sec->dst_off : 0) + sym->st_value;
+ dst_sym->st_size = sym->st_size;
+
+ obj->sym_map[src_sym_idx] = dst_sym_idx;
+
+ if (sym_type == STT_SECTION && dst_sym) {
+ dst_sec->sec_sym_idx = dst_sym_idx;
+ dst_sym->st_value = 0;
+ }
+
+ if (sym_bind != STB_LOCAL) {
+ glob_sym = add_glob_sym(linker);
+ if (!glob_sym)
+ return -ENOMEM;
+
+ glob_sym->sym_idx = dst_sym_idx;
+ /* we use dst_sec->id (and not dst_sec->sec_idx), because
+ * ephemeral sections (.kconfig, .ksyms, etc) don't have
+ * sec_idx (as they don't have corresponding ELF section), but
+ * still have id. .extern doesn't have even ephemeral section
+ * associated with it, so dst_sec->id == dst_sec->sec_idx == 0.
+ */
+ glob_sym->sec_id = dst_sec ? dst_sec->id : 0;
+ glob_sym->name_off = name_off;
+ /* we will fill btf_id in during BTF merging step */
+ glob_sym->btf_id = 0;
+ glob_sym->is_extern = sym_is_extern;
+ glob_sym->is_weak = sym_bind == STB_WEAK;
}
return 0;
@@ -1442,6 +2078,7 @@ static int linker_append_btf(struct bpf_linker *linker, struct src_obj *obj) { const struct btf_type *t; int i, j, n, start_id, id;+ const char *name; if (!obj->btf) return 0;
@@ -1454,12 +2091,40 @@ static int linker_append_btf(struct bpf_linker *linker, struct src_obj *obj) return -ENOMEM; for (i = 1; i <= n; i++) {+ struct glob_sym *glob_sym = NULL;+ t = btf__type_by_id(obj->btf, i); /* DATASECs are handled specially below */ if (btf_kind(t) == BTF_KIND_DATASEC) continue;+ if (btf_is_non_static(t)) {+ /* there should be glob_sym already */+ name = btf__str_by_offset(obj->btf, t->name_off);+ glob_sym = find_glob_sym(linker, name);++ /* VARs without corresponding glob_sym are those that+ * belong to skipped/deduplicated sections (i.e.,+ * license and version), so just skip them+ */+ if (!glob_sym)+ continue;++ if (glob_sym->underlying_btf_id == 0)+ glob_sym->underlying_btf_id = -t->type;
Is this needed? If glob_sym->btf_id is not NULL, then
glob_sym->underlying_btf_id has been set by the previous object.
If it is NULL, it will set probably after this
if (btf_is_non_static(t)) { ...}, is this right?
I think it's still needed. Here's the scenario.
1. Obj file A contains extern symbol X. We create corresponding
glob_sym (with is_extern=true), and store btf_id to point to
BTF_KIND_VAR, and btf_underlying_id to point to the type that
BTF_KIND_VAR points to.
2. Obj file B contains non-extern symbol X. At this point
linker_append_elf_sym() will update glob_sym to is_extern = false, it
will keep btf_id to re-use already appended BTF_KIND_VAR, but it will
zero-out underlying_btf_id, because for externs type could be
incomplete (e.g. for functions it won't contain function argument
names, for maps it could differ even more drastically later). So then
we get here, we see that glob_sym->underlying_btf_id is zero, so needs
updating. We store it as -Y, because Y is BTF type ID in obj->btf, not
in linker->btf (yet). Then the if (glob_sym->btf_id) below sees that
glob_sym->btf_id is already set, so we just keep using already
appended BTF_KIND_VAR (we already set its linkage to
BTF_VAR_GLOBAL_ALLOCATED in complete_extern_btf_info(), called from
linker_append_elf_sym(). So we'll skip appending another BTF_KIND_VAR.
But we do want to point existing BTF_KIND_VAR to a new type that
corresponds to ID -Y.
Thanks for explanation, I missed the code in
/* request updating VAR's/FUNC's underlying BTF type when
appending BTF type */
glob_sym->underlying_btf_id = 0;
in linker_append_elf_sym().
Maybe add a comment above the code with something like
underlying_btf_id may have been reset to 0 due to the presence of
a strong global variable.
?
ok, sure, I'll add the comment
quoted
quoted
quoted
+
+ /* globals from previous object files that match our
+ * VAR/FUNC already have a corresponding associated
+ * BTF type, so just make sure to use it
+ */
+ if (glob_sym->btf_id) {
+ /* reuse existing BTF type for global var/func */
+ obj->btf_type_map[i] = glob_sym->btf_id;
+ continue;
+ }
+ }
+
id = btf__add_type(linker->btf, obj->btf, t);
if (id < 0) {
pr_warn("failed to append BTF type #%d from file '%s'\n", i, obj->filename);
@@ -1467,6 +2132,12 @@ static int linker_append_btf(struct bpf_linker *linker, struct src_obj *obj) } obj->btf_type_map[i] = id;++ /* record just appended BTF type for var/func */+ if (glob_sym) {+ glob_sym->btf_id = id;+ glob_sym->underlying_btf_id = -t->type;+ } } /* remap all the types except DATASECs */
@@ -1478,6 +2149,22 @@ static int linker_append_btf(struct bpf_linker *linker, struct src_obj *obj) return -EINVAL; }+ /* Rewrite VAR/FUNC underlying types (i.e., FUNC's FUNC_PROTO and VAR's+ * actual type), if necessary+ */+ for (i = 0; i < linker->glob_sym_cnt; i++) {+ struct glob_sym *glob_sym = &linker->glob_syms[i];+ struct btf_type *glob_t;++ if (glob_sym->underlying_btf_id >= 0)+ continue;++ glob_sym->underlying_btf_id = obj->btf_type_map[-glob_sym->underlying_btf_id];
After this point, any new *extern* variables will hit the below in the
previous code:
Right, but we want to hit this for existing glob_syms that went from
extern to non-extern or from weak to non-weak. See
/* request updating VAR's/FUNC's underlying BTF type when
appending BTF type */
glob_sym->underlying_btf_id = 0;
in linker_append_elf_sym().
And we'll use that even more extensively when extending __weak and
extern map definitions later.
quoted
> + if (glob_sym->btf_id) {
> + /* reuse existing BTF type for global var/func */
> + obj->btf_type_map[i] = glob_sym->btf_id;
> + continue;
> + }
quoted
+
+ glob_t = btf_type_by_id(linker->btf, glob_sym->btf_id);
+ glob_t->type = glob_sym->underlying_btf_id;
+ }
+
/* append DATASEC info */
for (i = 1; i < obj->sec_cnt; i++) {
struct src_sec *src_sec;
On Thu, Apr 22, 2021 at 7:36 PM Yonghong Song [off-list ref] wrote:
On 4/22/21 4:57 PM, Yonghong Song wrote:
quoted
On 4/22/21 3:12 PM, Andrii Nakryiko wrote:
quoted
On Thu, Apr 22, 2021 at 2:27 PM Yonghong Song [off-list ref] wrote:
quoted
On 4/16/21 1:23 PM, Andrii Nakryiko wrote:
quoted
Add BPF static linker logic to resolve extern variables and
functions across
multiple linked together BPF object files.
For that, linker maintains a separate list of struct glob_sym
structures,
which keeps track of few pieces of metadata (is it extern or
resolved global,
is it a weak symbol, which ELF section it belongs to, etc) and ties
together
BTF type info and ELF symbol information and keeps them in sync.
With adding support for extern variables/funcs, it's now possible
for some
sections to contain both extern and non-extern definitions. This
means that
some sections may start out as ephemeral (if only externs are
present and thus
there is not corresponding ELF section), but will be "upgraded" to
actual ELF
section as symbols are resolved or new non-extern definitions are
appended.
Additional care is taken to not duplicate extern entries in sections
like
.kconfig and .ksyms.
Given libbpf requires BTF type to always be present for .kconfig/.ksym
externs, linker extends this requirement to all the externs, even
those that
are supposed to be resolved during static linking and which won't be
visible
to libbpf. With BTF information always present, static linker will
check not
just ELF symbol matches, but entire BTF type signature match as
well. That
logic is stricter that BPF CO-RE checks. It probably should be
re-used by
.ksym resolution logic in libbpf as well, but that's left for follow up
patches.
To make it unnecessary to rewrite ELF symbols and minimize BTF type
rewriting/removal, ELF symbols that correspond to externs initially
will be
updated in place once they are resolved. Similarly for BTF type
info, VAR/FUNC
and var_secinfo's (sec_vars in struct bpf_linker) are staying
stable, but
types they point to might get replaced when extern is resolved. This
might
leave some left-over types (even though we try to minimize this for
common
cases of having extern funcs with not argument names vs concrete
function with
names properly specified). That can be addresses later with a
generic BTF
garbage collection. That's left for a follow up as well.
Given BTF type appending phase is separate from ELF symbol
appending/resolution, special struct glob_sym->underlying_btf_id
variable is
used to communicate resolution and rewrite decisions. 0 means
underlying_btf_id needs to be appended (it's not yet in final
linker->btf), <0
values are used for temporary storage of source BTF type ID (not yet
rewritten), so -glob_sym->underlying_btf_id is BTF type id in
obj-btf. But by
the end of linker_append_btf() phase, that underlying_btf_id will be
remapped
and will always be > 0. This is the uglies part of the whole
process, but
keeps the other parts much simpler due to stability of sec_var and
VAR/FUNC
types, as well as ELF symbol, so please keep that in mind while
reviewing.
This is indeed complicated. I has some comments below. Please check
whether my understanding is correct or not.
quoted
BTF-defined maps require some extra custom logic and is addressed
separate in
the next patch, so that to keep this one smaller and easier to review.
Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
---
tools/lib/bpf/linker.c | 844
++++++++++++++++++++++++++++++++++++++---
1 file changed, 785 insertions(+), 59 deletions(-)
[...]
quoted
quoted
quoted
quoted
+ src_sec = &obj->secs[sym->st_shndx];
+ if (src_sec->skipped)
+ return 0;
+ dst_sec = &linker->secs[src_sec->dst_id];
+
+ /* allow only one STT_SECTION symbol per section */
+ if (sym_type == STT_SECTION && dst_sec->sec_sym_idx) {
+ obj->sym_map[src_sym_idx] = dst_sec->sec_sym_idx;
+ return 0;
+ }
+ }
+
+ if (sym_bind == STB_LOCAL)
+ goto add_sym;
+
+ /* find matching BTF info */
+ err = find_glob_sym_btf(obj, sym, sym_name, &btf_sec_id,
&btf_id);
+ if (err)
+ return err;
+
+ if (sym_is_extern && btf_sec_id) {
+ const char *sec_name = NULL;
+ const struct btf_type *t;
+
+ t = btf__type_by_id(obj->btf, btf_sec_id);
+ sec_name = btf__str_by_offset(obj->btf, t->name_off);
+
+ /* Clang puts unannotated extern vars into
+ * '.extern' BTF DATASEC. Treat them the same
+ * as unannotated extern funcs (which are
+ * currently not put into any DATASECs).
+ * Those don't have associated src_sec/dst_sec.
+ */
+ if (strcmp(sec_name, BTF_EXTERN_SEC) != 0) {
+ src_sec = find_src_sec_by_name(obj, sec_name);
+ if (!src_sec) {
+ pr_warn("failed to find matching ELF
sec '%s'\n", sec_name);
+ return -ENOENT;
+ }
+ dst_sec = &linker->secs[src_sec->dst_id];
+ }
+ }
+
+ glob_sym = find_glob_sym(linker, sym_name);
+ if (glob_sym) {
+ /* Preventively resolve to existing symbol. This is
+ * needed for further relocation symbol remapping in
+ * the next step of linking.
+ */
+ obj->sym_map[src_sym_idx] = glob_sym->sym_idx;
+
+ /* If both symbols are non-externs, at least one of
+ * them has to be STB_WEAK, otherwise they are in
+ * a conflict with each other.
+ */
+ if (!sym_is_extern && !glob_sym->is_extern
+ && !glob_sym->is_weak && sym_bind != STB_WEAK) {
+ pr_warn("conflicting non-weak symbol #%d (%s)
definition in '%s'\n",
+ src_sym_idx, sym_name, obj->filename);
+ return -EINVAL;
}
+ if (!glob_syms_match(sym_name, linker, glob_sym, obj,
sym, src_sym_idx, btf_id))
+ return -EINVAL;
+
+ dst_sym = get_sym_by_idx(linker, glob_sym->sym_idx);
+
+ /* If new symbol is strong, then force dst_sym to be
strong as
+ * well; this way a mix of weak and non-weak extern
+ * definitions will end up being strong.
+ */
+ if (sym_bind == STB_GLOBAL) {
+ /* We still need to preserve type (NOTYPE or
+ * OBJECT/FUNC, depending on whether the
symbol is
+ * extern or not)
+ */
+ sym_update_bind(dst_sym, STB_GLOBAL);
+ glob_sym->is_weak = false;
+ }
+
+ /* Non-default visibility is "contaminating", with
stricter
+ * visibility overwriting more permissive ones, even
if more
+ * permissive visibility comes from just an extern
definition
+ */
+ if (sym_vis > ELF64_ST_VISIBILITY(dst_sym->st_other))
+ sym_update_visibility(dst_sym, sym_vis);
For visibility, maybe we can just handle DEFAULT and HIDDEN, and others
are not supported? DEFAULT + DEFAULT/HIDDEN => DEFAULT, HIDDEN + HIDDEN
=> HIDDEN?
Looking at your selftest. Your current approach, DEFAULT + DEFAULT ->
DEFAULT, HIDDEN + HIDDEN/DEFAULT -> HIDDEN should work fine. This is
also align with ELF principal to accommodate the least permissive
visibility.
Yes, and also PROTECTED + HIDDEN/DEFAULT -> PROTECTED. But we don't
special handle PROTECTED right now. So I think it makes sense to error
out if anyone tries to use PROTECTED, which is why I'll restrict to
HIDDEN and DEFAULT for now.
quoted
quoted
quoted
Sure, we can restrict this to STV_DEFAULT and STV_HIDDEN for now. >>
On Thu, Apr 22, 2021 at 7:35 PM Alexei Starovoitov
[off-list ref] wrote:
On Thu, Apr 22, 2021 at 5:51 PM Yonghong Song [off-list ref] wrote:
quoted
quoted
+
+/* weak and shared between two files */
+const volatile int my_tid __weak = 0;
+const volatile long syscall_id __weak = 0;
Since the new compiler (llvm13) is recommended for this patch set.
We can simplify the above two definition with
int my_tid __weak;
long syscall_id __weak;
The same for the other file.
But I am also okay with the current form
to *satisfy* llvm10 some people may still use.
The test won't work with anything, but the latest llvm trunk,
so " = 0" is useless.
Let's remove it.
Especially from the tests that rely on the latest llvm.
No one can backport the latest llvm BPF backend to llvm10 front-end.
On Thu, Apr 22, 2021 at 9:25 PM Andrii Nakryiko
[off-list ref] wrote:
On Thu, Apr 22, 2021 at 7:54 PM Alexei Starovoitov
[off-list ref] wrote:
quoted
On Thu, Apr 22, 2021 at 11:24 AM Andrii Nakryiko
[off-list ref] wrote:
quoted
On Thu, Apr 22, 2021 at 9:50 AM Yonghong Song [off-list ref] wrote:
quoted
On 4/16/21 1:23 PM, Andrii Nakryiko wrote:
quoted
It should never fail, but if it does, it's better to know about this rather
than end up with nonsensical type IDs.
So this is defensive programming. Maybe do another round of
audit of the callers and if you didn't find any issue, you
do not need to check not-happening condition here?
It's far from obvious that this will never happen, because we do a
decently complicated BTF processing (we skip some types altogether
believing that they are not used, for example) and it will only get
more complicated with time. Just as there are "verifier bug" checks in
kernel, this prevents things from going wild if non-trivial bugs will
inevitably happen.
I agree with Yonghong. This doesn't look right.
I read it as Yonghong was asking about the entire patch. You seem to
be concerned with one particular check, right?
quoted
The callback will be called for all non-void types, right?
so *type_id == 0 shouldn't never happen.
If it does there is a bug somewhere that should be investigated
instead of ignored.
See btf_type_visit_type_ids() and btf_ext_visit_type_ids(), they call
callback for every field that contains type ID, even if it points to
VOID. So this can happen and is expected.
I see. So something like 'extern cosnt void foo __ksym' would
point to void type?
But then why is it not a part of the id_map[] and has
to be handled explicitly?
quoted
The
if (new_id == 0) pr_warn
bit makes sense.
Right, and this is the point of this patch. id_map[] will have zeroes
for any unmapped type, so I just need to make sure I'm not false
erroring on id_map[0] (== 0, which is valid, but never used).
Right, id_map[0] should be 0.
I'm still missing something in this combination of 'if's.
May be do it as:
if (new_id == 0 && *type_id != 0) { pr_warn
?
That was the idea?
On Thu, Apr 22, 2021 at 10:11 PM Alexei Starovoitov
[off-list ref] wrote:
On Thu, Apr 22, 2021 at 9:25 PM Andrii Nakryiko
[off-list ref] wrote:
quoted
On Thu, Apr 22, 2021 at 7:54 PM Alexei Starovoitov
[off-list ref] wrote:
quoted
On Thu, Apr 22, 2021 at 11:24 AM Andrii Nakryiko
[off-list ref] wrote:
quoted
On Thu, Apr 22, 2021 at 9:50 AM Yonghong Song [off-list ref] wrote:
quoted
On 4/16/21 1:23 PM, Andrii Nakryiko wrote:
quoted
It should never fail, but if it does, it's better to know about this rather
than end up with nonsensical type IDs.
So this is defensive programming. Maybe do another round of
audit of the callers and if you didn't find any issue, you
do not need to check not-happening condition here?
It's far from obvious that this will never happen, because we do a
decently complicated BTF processing (we skip some types altogether
believing that they are not used, for example) and it will only get
more complicated with time. Just as there are "verifier bug" checks in
kernel, this prevents things from going wild if non-trivial bugs will
inevitably happen.
I agree with Yonghong. This doesn't look right.
I read it as Yonghong was asking about the entire patch. You seem to
be concerned with one particular check, right?
quoted
The callback will be called for all non-void types, right?
so *type_id == 0 shouldn't never happen.
If it does there is a bug somewhere that should be investigated
instead of ignored.
See btf_type_visit_type_ids() and btf_ext_visit_type_ids(), they call
callback for every field that contains type ID, even if it points to
VOID. So this can happen and is expected.
I see. So something like 'extern cosnt void foo __ksym' would
point to void type?
But then why is it not a part of the id_map[] and has
to be handled explicitly?
const void foo will be VAR -> CONST -> VOID. But any `void *` anywhere
will be PTR -> VOID. Any void bla(int x) would have return type VOID
(0), and so on. There are a lot of cases when we use VOID as type_id.
VOID always is handled specially, because it stays zero despite any
transformation: during BTF concatenation, BTF dedup, BTF generation,
etc.
quoted
quoted
The
if (new_id == 0) pr_warn
bit makes sense.
Right, and this is the point of this patch. id_map[] will have zeroes
for any unmapped type, so I just need to make sure I'm not false
erroring on id_map[0] (== 0, which is valid, but never used).
Right, id_map[0] should be 0.
I'm still missing something in this combination of 'if's.
May be do it as:
if (new_id == 0 && *type_id != 0) { pr_warn
?
That was the idea?
That's the idea, there is just no need to do VOID -> VOID
transformation, but I'll rewrite it to a combined if if it makes it
easier to follow. Here's full source of remap_type_id with few
comments to added:
static int remap_type_id(__u32 *type_id, void *ctx)
{
int *id_map = ctx;
int new_id = id_map[*type_id];
/* Here VOID stays VOID, that's all */
if (*type_id == 0)
return 0;
/* This means whatever type we are trying to remap didn't get a new ID
assigned in linker->btf and that's an error */
if (new_id == 0) {
pr_warn("failed to find new ID mapping for original
BTF type ID %u\n", *type_id);
return -EINVAL;
}
*type_id = id_map[*type_id];
return 0;
}
On Fri, Apr 23, 2021 at 9:09 AM Andrii Nakryiko
[off-list ref] wrote:
On Thu, Apr 22, 2021 at 10:11 PM Alexei Starovoitov
[off-list ref] wrote:
quoted
On Thu, Apr 22, 2021 at 9:25 PM Andrii Nakryiko
[off-list ref] wrote:
quoted
On Thu, Apr 22, 2021 at 7:54 PM Alexei Starovoitov
[off-list ref] wrote:
quoted
On Thu, Apr 22, 2021 at 11:24 AM Andrii Nakryiko
[off-list ref] wrote:
quoted
On Thu, Apr 22, 2021 at 9:50 AM Yonghong Song [off-list ref] wrote:
quoted
On 4/16/21 1:23 PM, Andrii Nakryiko wrote:
quoted
It should never fail, but if it does, it's better to know about this rather
than end up with nonsensical type IDs.
So this is defensive programming. Maybe do another round of
audit of the callers and if you didn't find any issue, you
do not need to check not-happening condition here?
It's far from obvious that this will never happen, because we do a
decently complicated BTF processing (we skip some types altogether
believing that they are not used, for example) and it will only get
more complicated with time. Just as there are "verifier bug" checks in
kernel, this prevents things from going wild if non-trivial bugs will
inevitably happen.
I agree with Yonghong. This doesn't look right.
I read it as Yonghong was asking about the entire patch. You seem to
be concerned with one particular check, right?
quoted
The callback will be called for all non-void types, right?
so *type_id == 0 shouldn't never happen.
If it does there is a bug somewhere that should be investigated
instead of ignored.
See btf_type_visit_type_ids() and btf_ext_visit_type_ids(), they call
callback for every field that contains type ID, even if it points to
VOID. So this can happen and is expected.
I see. So something like 'extern cosnt void foo __ksym' would
point to void type?
But then why is it not a part of the id_map[] and has
to be handled explicitly?
const void foo will be VAR -> CONST -> VOID. But any `void *` anywhere
will be PTR -> VOID. Any void bla(int x) would have return type VOID
(0), and so on. There are a lot of cases when we use VOID as type_id.
VOID always is handled specially, because it stays zero despite any
transformation: during BTF concatenation, BTF dedup, BTF generation,
etc.
quoted
quoted
quoted
The
if (new_id == 0) pr_warn
bit makes sense.
Right, and this is the point of this patch. id_map[] will have zeroes
for any unmapped type, so I just need to make sure I'm not false
erroring on id_map[0] (== 0, which is valid, but never used).
Right, id_map[0] should be 0.
I'm still missing something in this combination of 'if's.
May be do it as:
if (new_id == 0 && *type_id != 0) { pr_warn
?
That was the idea?
That's the idea, there is just no need to do VOID -> VOID
transformation, but I'll rewrite it to a combined if if it makes it
easier to follow. Here's full source of remap_type_id with few
comments to added:
static int remap_type_id(__u32 *type_id, void *ctx)
{
int *id_map = ctx;
int new_id = id_map[*type_id];
/* Here VOID stays VOID, that's all */
if (*type_id == 0)
return 0;
Does it mean that id_map[0] is a garbage value?
and all other code that might be doing id_map[idx] might be reading
garbage if it doesn't have a check for idx == 0 ?
/* This means whatever type we are trying to remap didn't get a new ID
assigned in linker->btf and that's an error */
if (new_id == 0) {
pr_warn("failed to find new ID mapping for original
BTF type ID %u\n", *type_id);
return -EINVAL;
}
*type_id = id_map[*type_id];
return 0;
}
On Fri, Apr 23, 2021 at 9:18 AM Alexei Starovoitov
[off-list ref] wrote:
On Fri, Apr 23, 2021 at 9:09 AM Andrii Nakryiko
[off-list ref] wrote:
quoted
On Thu, Apr 22, 2021 at 10:11 PM Alexei Starovoitov
[off-list ref] wrote:
quoted
On Thu, Apr 22, 2021 at 9:25 PM Andrii Nakryiko
[off-list ref] wrote:
quoted
On Thu, Apr 22, 2021 at 7:54 PM Alexei Starovoitov
[off-list ref] wrote:
quoted
On Thu, Apr 22, 2021 at 11:24 AM Andrii Nakryiko
[off-list ref] wrote:
quoted
On Thu, Apr 22, 2021 at 9:50 AM Yonghong Song [off-list ref] wrote:
quoted
On 4/16/21 1:23 PM, Andrii Nakryiko wrote:
quoted
It should never fail, but if it does, it's better to know about this rather
than end up with nonsensical type IDs.
So this is defensive programming. Maybe do another round of
audit of the callers and if you didn't find any issue, you
do not need to check not-happening condition here?
It's far from obvious that this will never happen, because we do a
decently complicated BTF processing (we skip some types altogether
believing that they are not used, for example) and it will only get
more complicated with time. Just as there are "verifier bug" checks in
kernel, this prevents things from going wild if non-trivial bugs will
inevitably happen.
I agree with Yonghong. This doesn't look right.
I read it as Yonghong was asking about the entire patch. You seem to
be concerned with one particular check, right?
quoted
The callback will be called for all non-void types, right?
so *type_id == 0 shouldn't never happen.
If it does there is a bug somewhere that should be investigated
instead of ignored.
See btf_type_visit_type_ids() and btf_ext_visit_type_ids(), they call
callback for every field that contains type ID, even if it points to
VOID. So this can happen and is expected.
I see. So something like 'extern cosnt void foo __ksym' would
point to void type?
But then why is it not a part of the id_map[] and has
to be handled explicitly?
const void foo will be VAR -> CONST -> VOID. But any `void *` anywhere
will be PTR -> VOID. Any void bla(int x) would have return type VOID
(0), and so on. There are a lot of cases when we use VOID as type_id.
VOID always is handled specially, because it stays zero despite any
transformation: during BTF concatenation, BTF dedup, BTF generation,
etc.
quoted
quoted
quoted
The
if (new_id == 0) pr_warn
bit makes sense.
Right, and this is the point of this patch. id_map[] will have zeroes
for any unmapped type, so I just need to make sure I'm not false
erroring on id_map[0] (== 0, which is valid, but never used).
Right, id_map[0] should be 0.
I'm still missing something in this combination of 'if's.
May be do it as:
if (new_id == 0 && *type_id != 0) { pr_warn
?
That was the idea?
That's the idea, there is just no need to do VOID -> VOID
transformation, but I'll rewrite it to a combined if if it makes it
easier to follow. Here's full source of remap_type_id with few
comments to added:
static int remap_type_id(__u32 *type_id, void *ctx)
{
int *id_map = ctx;
int new_id = id_map[*type_id];
/* Here VOID stays VOID, that's all */
if (*type_id == 0)
return 0;
Does it mean that id_map[0] is a garbage value?
and all other code that might be doing id_map[idx] might be reading
garbage if it doesn't have a check for idx == 0 ?
No, id_map[0] == 0 by construction (id_map is obj->btf_type_map and is
calloc()'ed) and can be used as id_map[idx].
quoted
/* This means whatever type we are trying to remap didn't get a new ID
assigned in linker->btf and that's an error */
if (new_id == 0) {
pr_warn("failed to find new ID mapping for original
BTF type ID %u\n", *type_id);
return -EINVAL;
}
*type_id = id_map[*type_id];
return 0;
}
On Fri, Apr 23, 2021 at 9:31 AM Andrii Nakryiko
[off-list ref] wrote:
quoted
quoted
static int remap_type_id(__u32 *type_id, void *ctx)
{
int *id_map = ctx;
int new_id = id_map[*type_id];
/* Here VOID stays VOID, that's all */
if (*type_id == 0)
return 0;
Does it mean that id_map[0] is a garbage value?
and all other code that might be doing id_map[idx] might be reading
garbage if it doesn't have a check for idx == 0 ?
No, id_map[0] == 0 by construction (id_map is obj->btf_type_map and is
calloc()'ed) and can be used as id_map[idx].
Ok. Then why are you insisting on this micro optimization to return 0
directly?
That's the confusing part for me.
If it was:
"if (new_id == 0 && *type_id != 0) { pr_warn"
Then it would be clear what error condition is about.
But 'return 0' messing things up in my mind,
because it's far from obvious that first check is really a combination
with the 2nd check and by itself it's a micro optimization to avoid
reading id_map[0].
On Fri, Apr 23, 2021 at 9:34 AM Alexei Starovoitov
[off-list ref] wrote:
On Fri, Apr 23, 2021 at 9:31 AM Andrii Nakryiko
[off-list ref] wrote:
quoted
quoted
quoted
static int remap_type_id(__u32 *type_id, void *ctx)
{
int *id_map = ctx;
int new_id = id_map[*type_id];
/* Here VOID stays VOID, that's all */
if (*type_id == 0)
return 0;
Does it mean that id_map[0] is a garbage value?
and all other code that might be doing id_map[idx] might be reading
garbage if it doesn't have a check for idx == 0 ?
No, id_map[0] == 0 by construction (id_map is obj->btf_type_map and is
calloc()'ed) and can be used as id_map[idx].
Ok. Then why are you insisting on this micro optimization to return 0
directly?
That's the confusing part for me.
I'm not insisting:
> but I'll rewrite it to a combined if if it makes it easier to follow
So I'm confused why you are confused.
If it was:
"if (new_id == 0 && *type_id != 0) { pr_warn"
Then it would be clear what error condition is about.
But 'return 0' messing things up in my mind,
because it's far from obvious that first check is really a combination
with the 2nd check and by itself it's a micro optimization to avoid
reading id_map[0].
I didn't try to micro optimize, that's how I naturally think about the
problem. I'll rewrite the if, don't know why we are spending emails on
this still.
On Thu, Apr 22, 2021 at 5:50 PM Yonghong Song [off-list ref] wrote:
On 4/16/21 1:24 PM, Andrii Nakryiko wrote:
quoted
Add selftest validating various aspects of statically linking functions:
- no conflicts and correct resolution for name-conflicting static funcs;
- correct resolution of extern functions;
- correct handling of weak functions, both resolution itself and libbpf's
handling of unused weak function that "lost" (it leaves gaps in code with
no ELF symbols);
- correct handling of hidden visibility to turn global function into
"static" for the purpose of BPF verification.
Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
Ack with a small nit below.
Acked-by: Yonghong Song <redacted>
@@ -0,0 +1,73 @@+// SPDX-License-Identifier: GPL-2.0+/* Copyright (c) 2021 Facebook */++#include"vmlinux.h"+#include<bpf/bpf_helpers.h>+#include<bpf/bpf_tracing.h>++/* weak and shared between two files */+constvolatileintmy_tid__weak=0;+constvolatilelongsyscall_id__weak=0;
Since the new compiler (llvm13) is recommended for this patch set.
We can simplify the above two definition with
int my_tid __weak;
long syscall_id __weak;
The same for the other file.
This is not about old vs new compilers. I wanted to use .rodata
variables, but I'll switch to .bss, no problem.
But I am also okay with the current form
to *satisfy* llvm10 some people may still use.
quoted
+
+int output_val1 = 0;
+int output_ctx1 = 0;
+int output_weak1 = 0;
+
+/* same "subprog" name in all files, but it's ok because they all are static */
+static __noinline int subprog(int x)
+{
+ /* but different formula */
+ return x * 1;
+}
+
+/* Global functions can't be void */
+int set_output_val1(int x)
+{
+ output_val1 = x + subprog(x);
+ return x;
+}
+
+/* This function can't be verified as global, as it assumes raw_tp/sys_enter
+ * context and accesses syscall id (second argument). So we mark it as
+ * __hidden, so that libbpf will mark it as static in the final object file,
+ * right before verifying it in the kernel.
+ *
+ * But we don't mark it as __hidden here, rather at extern site. __hidden is
+ * "contaminating" visibility, so it will get propagated from either extern or
+ * actual definition (including from the losing __weak definition).
+ */
+void set_output_ctx1(__u64 *ctx)
+{
+ output_ctx1 = ctx[1]; /* long id, same as in BPF_PROG below */
+}
+
+/* this weak instance should win because it's the first one */
+__weak int set_output_weak(int x)
+{
+ output_weak1 = x;
+ return x;
+}
+
+extern int set_output_val2(int x);
+
+/* here we'll force set_output_ctx2() to be __hidden in the final obj file */
+__hidden extern void set_output_ctx2(__u64 *ctx);
+
From: Yonghong Song <hidden> Date: 2021-04-23 17:35:24
On 4/23/21 10:18 AM, Andrii Nakryiko wrote:
On Thu, Apr 22, 2021 at 5:50 PM Yonghong Song [off-list ref] wrote:
quoted
On 4/16/21 1:24 PM, Andrii Nakryiko wrote:
quoted
Add selftest validating various aspects of statically linking functions:
- no conflicts and correct resolution for name-conflicting static funcs;
- correct resolution of extern functions;
- correct handling of weak functions, both resolution itself and libbpf's
handling of unused weak function that "lost" (it leaves gaps in code with
no ELF symbols);
- correct handling of hidden visibility to turn global function into
"static" for the purpose of BPF verification.
Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
Ack with a small nit below.
Acked-by: Yonghong Song <redacted>
@@ -0,0 +1,73 @@+// SPDX-License-Identifier: GPL-2.0+/* Copyright (c) 2021 Facebook */++#include"vmlinux.h"+#include<bpf/bpf_helpers.h>+#include<bpf/bpf_tracing.h>++/* weak and shared between two files */+constvolatileintmy_tid__weak=0;+constvolatilelongsyscall_id__weak=0;
Since the new compiler (llvm13) is recommended for this patch set.
We can simplify the above two definition with
int my_tid __weak;
long syscall_id __weak;
The same for the other file.
This is not about old vs new compilers. I wanted to use .rodata
variables, but I'll switch to .bss, no problem.
I see. You can actually hone one "const volatile ing my_tid __weak = 0"
and another "long syscall_id __weak". This way, you will be able to
test both .rodata and .bss section.
quoted
But I am also okay with the current form
to *satisfy* llvm10 some people may still use.
quoted
+
+int output_val1 = 0;
+int output_ctx1 = 0;
+int output_weak1 = 0;
+
+/* same "subprog" name in all files, but it's ok because they all are static */
+static __noinline int subprog(int x)
+{
+ /* but different formula */
+ return x * 1;
+}
+
+/* Global functions can't be void */
+int set_output_val1(int x)
+{
+ output_val1 = x + subprog(x);
+ return x;
+}
+
+/* This function can't be verified as global, as it assumes raw_tp/sys_enter
+ * context and accesses syscall id (second argument). So we mark it as
+ * __hidden, so that libbpf will mark it as static in the final object file,
+ * right before verifying it in the kernel.
+ *
+ * But we don't mark it as __hidden here, rather at extern site. __hidden is
+ * "contaminating" visibility, so it will get propagated from either extern or
+ * actual definition (including from the losing __weak definition).
+ */
+void set_output_ctx1(__u64 *ctx)
+{
+ output_ctx1 = ctx[1]; /* long id, same as in BPF_PROG below */
+}
+
+/* this weak instance should win because it's the first one */
+__weak int set_output_weak(int x)
+{
+ output_weak1 = x;
+ return x;
+}
+
+extern int set_output_val2(int x);
+
+/* here we'll force set_output_ctx2() to be __hidden in the final obj file */
+__hidden extern void set_output_ctx2(__u64 *ctx);
+
On Fri, Apr 23, 2021 at 10:35 AM Yonghong Song [off-list ref] wrote:
On 4/23/21 10:18 AM, Andrii Nakryiko wrote:
quoted
On Thu, Apr 22, 2021 at 5:50 PM Yonghong Song [off-list ref] wrote:
quoted
On 4/16/21 1:24 PM, Andrii Nakryiko wrote:
quoted
Add selftest validating various aspects of statically linking functions:
- no conflicts and correct resolution for name-conflicting static funcs;
- correct resolution of extern functions;
- correct handling of weak functions, both resolution itself and libbpf's
handling of unused weak function that "lost" (it leaves gaps in code with
no ELF symbols);
- correct handling of hidden visibility to turn global function into
"static" for the purpose of BPF verification.
Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
Ack with a small nit below.
Acked-by: Yonghong Song <redacted>
@@ -0,0 +1,73 @@+// SPDX-License-Identifier: GPL-2.0+/* Copyright (c) 2021 Facebook */++#include"vmlinux.h"+#include<bpf/bpf_helpers.h>+#include<bpf/bpf_tracing.h>++/* weak and shared between two files */+constvolatileintmy_tid__weak=0;+constvolatilelongsyscall_id__weak=0;
Since the new compiler (llvm13) is recommended for this patch set.
We can simplify the above two definition with
int my_tid __weak;
long syscall_id __weak;
The same for the other file.
This is not about old vs new compilers. I wanted to use .rodata
variables, but I'll switch to .bss, no problem.
I see. You can actually hone one "const volatile ing my_tid __weak = 0"
and another "long syscall_id __weak". This way, you will be able to
test both .rodata and .bss section.
I wonder if you meant to have one my_tid __weak in .bss and another
my_tid __weak in .rodata. Or just my_tid in .bss and syscall_id in
.rodata?
If the former (mixing ELF sections across definitions of the same
symbol), then it's disallowed right now. libbpf will error out on
mismatched sections. I tested this with normal compilation, it does
work and the final section is the section of the winner.
But I think that's quite confusing, actually, so I'm going to leave it
disallowed for now. E.g., if one file expects a read-write variable
and another expects that same variable to be read-only, and the winner
ends up being read-only one, then the file expecting read-write will
essentially have incorrect code (and will be rejected by BPF verifier,
if anything attempts to write). So I think it's better to reject it at
the linking time.
But I'll do one (my_tid) as .bss, and another (syscall_id) as .rodata.
quoted
quoted
But I am also okay with the current form
to *satisfy* llvm10 some people may still use.
quoted
+
+int output_val1 = 0;
+int output_ctx1 = 0;
+int output_weak1 = 0;
+
+/* same "subprog" name in all files, but it's ok because they all are static */
+static __noinline int subprog(int x)
+{
+ /* but different formula */
+ return x * 1;
+}
+
+/* Global functions can't be void */
+int set_output_val1(int x)
+{
+ output_val1 = x + subprog(x);
+ return x;
+}
+
+/* This function can't be verified as global, as it assumes raw_tp/sys_enter
+ * context and accesses syscall id (second argument). So we mark it as
+ * __hidden, so that libbpf will mark it as static in the final object file,
+ * right before verifying it in the kernel.
+ *
+ * But we don't mark it as __hidden here, rather at extern site. __hidden is
+ * "contaminating" visibility, so it will get propagated from either extern or
+ * actual definition (including from the losing __weak definition).
+ */
+void set_output_ctx1(__u64 *ctx)
+{
+ output_ctx1 = ctx[1]; /* long id, same as in BPF_PROG below */
+}
+
+/* this weak instance should win because it's the first one */
+__weak int set_output_weak(int x)
+{
+ output_weak1 = x;
+ return x;
+}
+
+extern int set_output_val2(int x);
+
+/* here we'll force set_output_ctx2() to be __hidden in the final obj file */
+__hidden extern void set_output_ctx2(__u64 *ctx);
+
From: Yonghong Song <hidden> Date: 2021-04-23 17:59:00
On 4/23/21 10:55 AM, Andrii Nakryiko wrote:
On Fri, Apr 23, 2021 at 10:35 AM Yonghong Song [off-list ref] wrote:
quoted
On 4/23/21 10:18 AM, Andrii Nakryiko wrote:
quoted
On Thu, Apr 22, 2021 at 5:50 PM Yonghong Song [off-list ref] wrote:
quoted
On 4/16/21 1:24 PM, Andrii Nakryiko wrote:
quoted
Add selftest validating various aspects of statically linking functions:
- no conflicts and correct resolution for name-conflicting static funcs;
- correct resolution of extern functions;
- correct handling of weak functions, both resolution itself and libbpf's
handling of unused weak function that "lost" (it leaves gaps in code with
no ELF symbols);
- correct handling of hidden visibility to turn global function into
"static" for the purpose of BPF verification.
Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
Ack with a small nit below.
Acked-by: Yonghong Song <redacted>
@@ -0,0 +1,73 @@+// SPDX-License-Identifier: GPL-2.0+/* Copyright (c) 2021 Facebook */++#include"vmlinux.h"+#include<bpf/bpf_helpers.h>+#include<bpf/bpf_tracing.h>++/* weak and shared between two files */+constvolatileintmy_tid__weak=0;+constvolatilelongsyscall_id__weak=0;
Since the new compiler (llvm13) is recommended for this patch set.
We can simplify the above two definition with
int my_tid __weak;
long syscall_id __weak;
The same for the other file.
This is not about old vs new compilers. I wanted to use .rodata
variables, but I'll switch to .bss, no problem.
I see. You can actually hone one "const volatile ing my_tid __weak = 0"
and another "long syscall_id __weak". This way, you will be able to
test both .rodata and .bss section.
I wonder if you meant to have one my_tid __weak in .bss and another
my_tid __weak in .rodata. Or just my_tid in .bss and syscall_id in
.rodata?
If the former (mixing ELF sections across definitions of the same
symbol), then it's disallowed right now. libbpf will error out on
mismatched sections. I tested this with normal compilation, it does
work and the final section is the section of the winner.
But I think that's quite confusing, actually, so I'm going to leave it
disallowed for now. E.g., if one file expects a read-write variable
and another expects that same variable to be read-only, and the winner
ends up being read-only one, then the file expecting read-write will
essentially have incorrect code (and will be rejected by BPF verifier,
if anything attempts to write). So I think it's better to reject it at
the linking time.
But I'll do one (my_tid) as .bss, and another (syscall_id) as .rodata.
I mean this one. Permitting the same variable in both .bss and .rodata
sections is never a good practice.
quoted
quoted
quoted
But I am also okay with the current form
to *satisfy* llvm10 some people may still use.
quoted
+
+int output_val1 = 0;
+int output_ctx1 = 0;
+int output_weak1 = 0;
+
+/* same "subprog" name in all files, but it's ok because they all are static */
+static __noinline int subprog(int x)
+{
+ /* but different formula */
+ return x * 1;
+}
+
+/* Global functions can't be void */
+int set_output_val1(int x)
+{
+ output_val1 = x + subprog(x);
+ return x;
+}
+
+/* This function can't be verified as global, as it assumes raw_tp/sys_enter
+ * context and accesses syscall id (second argument). So we mark it as
+ * __hidden, so that libbpf will mark it as static in the final object file,
+ * right before verifying it in the kernel.
+ *
+ * But we don't mark it as __hidden here, rather at extern site. __hidden is
+ * "contaminating" visibility, so it will get propagated from either extern or
+ * actual definition (including from the losing __weak definition).
+ */
+void set_output_ctx1(__u64 *ctx)
+{
+ output_ctx1 = ctx[1]; /* long id, same as in BPF_PROG below */
+}
+
+/* this weak instance should win because it's the first one */
+__weak int set_output_weak(int x)
+{
+ output_weak1 = x;
+ return x;
+}
+
+extern int set_output_val2(int x);
+
+/* here we'll force set_output_ctx2() to be __hidden in the final obj file */
+__hidden extern void set_output_ctx2(__u64 *ctx);
+
On Fri, Apr 23, 2021 at 10:58 AM Yonghong Song [off-list ref] wrote:
On 4/23/21 10:55 AM, Andrii Nakryiko wrote:
quoted
On Fri, Apr 23, 2021 at 10:35 AM Yonghong Song [off-list ref] wrote:
quoted
On 4/23/21 10:18 AM, Andrii Nakryiko wrote:
quoted
On Thu, Apr 22, 2021 at 5:50 PM Yonghong Song [off-list ref] wrote:
quoted
On 4/16/21 1:24 PM, Andrii Nakryiko wrote:
quoted
Add selftest validating various aspects of statically linking functions:
- no conflicts and correct resolution for name-conflicting static funcs;
- correct resolution of extern functions;
- correct handling of weak functions, both resolution itself and libbpf's
handling of unused weak function that "lost" (it leaves gaps in code with
no ELF symbols);
- correct handling of hidden visibility to turn global function into
"static" for the purpose of BPF verification.
Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
Ack with a small nit below.
Acked-by: Yonghong Song <redacted>
@@ -0,0 +1,73 @@+// SPDX-License-Identifier: GPL-2.0+/* Copyright (c) 2021 Facebook */++#include"vmlinux.h"+#include<bpf/bpf_helpers.h>+#include<bpf/bpf_tracing.h>++/* weak and shared between two files */+constvolatileintmy_tid__weak=0;+constvolatilelongsyscall_id__weak=0;
Since the new compiler (llvm13) is recommended for this patch set.
We can simplify the above two definition with
int my_tid __weak;
long syscall_id __weak;
The same for the other file.
This is not about old vs new compilers. I wanted to use .rodata
variables, but I'll switch to .bss, no problem.
I see. You can actually hone one "const volatile ing my_tid __weak = 0"
and another "long syscall_id __weak". This way, you will be able to
test both .rodata and .bss section.
I wonder if you meant to have one my_tid __weak in .bss and another
my_tid __weak in .rodata. Or just my_tid in .bss and syscall_id in
.rodata?
If the former (mixing ELF sections across definitions of the same
symbol), then it's disallowed right now. libbpf will error out on
mismatched sections. I tested this with normal compilation, it does
work and the final section is the section of the winner.
But I think that's quite confusing, actually, so I'm going to leave it
disallowed for now. E.g., if one file expects a read-write variable
and another expects that same variable to be read-only, and the winner
ends up being read-only one, then the file expecting read-write will
essentially have incorrect code (and will be rejected by BPF verifier,
if anything attempts to write). So I think it's better to reject it at
the linking time.
But I'll do one (my_tid) as .bss, and another (syscall_id) as .rodata.
I mean this one. Permitting the same variable in both .bss and .rodata
sections is never a good practice.
Ok, cool, that's what we do right now. I wonder why it is allowed by
user-space linkers, it seems dangerous.
quoted
quoted
quoted
quoted
But I am also okay with the current form
to *satisfy* llvm10 some people may still use.
quoted
+
+int output_val1 = 0;
+int output_ctx1 = 0;
+int output_weak1 = 0;
+
+/* same "subprog" name in all files, but it's ok because they all are static */
+static __noinline int subprog(int x)
+{
+ /* but different formula */
+ return x * 1;
+}
+
+/* Global functions can't be void */
+int set_output_val1(int x)
+{
+ output_val1 = x + subprog(x);
+ return x;
+}
+
+/* This function can't be verified as global, as it assumes raw_tp/sys_enter
+ * context and accesses syscall id (second argument). So we mark it as
+ * __hidden, so that libbpf will mark it as static in the final object file,
+ * right before verifying it in the kernel.
+ *
+ * But we don't mark it as __hidden here, rather at extern site. __hidden is
+ * "contaminating" visibility, so it will get propagated from either extern or
+ * actual definition (including from the losing __weak definition).
+ */
+void set_output_ctx1(__u64 *ctx)
+{
+ output_ctx1 = ctx[1]; /* long id, same as in BPF_PROG below */
+}
+
+/* this weak instance should win because it's the first one */
+__weak int set_output_weak(int x)
+{
+ output_weak1 = x;
+ return x;
+}
+
+extern int set_output_val2(int x);
+
+/* here we'll force set_output_ctx2() to be __hidden in the final obj file */
+__hidden extern void set_output_ctx2(__u64 *ctx);
+