From: Eric Leblond <hidden> Date: 2016-10-16 22:00:37
Hello,
Here's a patchset on the libbpf library that can be found in
tools/lib/bpf.
Patch 0 to patch 4 add a new function to be able to set the BPF
program type. Till then program type such as network filter can't
be loaded by the library:
* tools lib bpf: add error functions
* uapi linux bpf: add max value to enum
* tools: Sync tools/include/uapi/linux/bpf.h with the
* tools lib bpf: export function to set type
Patch 5 is adding functions that were missing to handle maps in
userspace.
* tools lib bpf: add missing functions
Patch 7 fixes a bug in the parsing of BPF ELF file.
* tools lib bpf: fix maps resolution
Patch 8 update 'make install' to install the header on the system.
* tools lib bpf: install header file
Patchset statistics:
include/uapi/linux/bpf.h | 1 +
tools/include/uapi/linux/bpf.h | 56 ++++++++++++++++++++++++++++++++++++++++++++++++++++++--
tools/lib/bpf/Makefile | 11 +++++++++--
tools/lib/bpf/bpf.c | 35 ++++++++++++++++++++++++++++++++++-
tools/lib/bpf/bpf.h | 2 --
tools/lib/bpf/libbpf.c | 83 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++--------------------------
tools/lib/bpf/libbpf.h | 12 +++++++++++-
7 files changed, 166 insertions(+), 34 deletions(-)
Best regards,
From: Eric Leblond <hidden> Date: 2016-10-16 21:59:01
It will be used to detect userspace trying to set invalid value.
Signed-off-by: Eric Leblond <redacted>
---
include/uapi/linux/bpf.h | 1 +
1 file changed, 1 insertion(+)
From: Eric Leblond <hidden> Date: 2016-10-16 21:59:07
It is not correct to assimilate the elf data of the maps section
to an array of map definition. In fact the sizes differ. The
offset provided in the symbol section has to be used instead.
This patch fixes a bug causing a elf with two maps not to load
correctly.
Signed-off-by: Eric Leblond <redacted>
---
tools/lib/bpf/libbpf.c | 50 +++++++++++++++++++++++++++++++++++---------------
1 file changed, 35 insertions(+), 15 deletions(-)
@@ -529,13 +530,6 @@ bpf_object__init_maps(struct bpf_object *obj, void *data,pr_debug("maps in %s: %zd bytes\n",obj->path,size);-obj->maps=calloc(nr_maps,sizeof(obj->maps[0]));-if(!obj->maps){-pr_warning("alloc maps for object failed\n");-return-ENOMEM;-}-obj->nr_maps=nr_maps;-for(i=0;i<nr_maps;i++){structbpf_map_def*def=&obj->maps[i].def;
@@ -547,23 +541,42 @@ bpf_object__init_maps(struct bpf_object *obj, void *data,obj->maps[i].fd=-1;/* Save map definition into obj->maps */-*def=((structbpf_map_def*)data)[i];+*def=*(structbpf_map_def*)(data+obj->maps[i].offset);}return0;}staticint-bpf_object__init_maps_name(structbpf_object*obj)+bpf_object__init_maps_symbol(structbpf_object*obj){inti;+intnr_maps=0;Elf_Data*symbols=obj->efile.symbols;+size_tmap_idx=0;if(!symbols||obj->efile.maps_shndx<0)return-EINVAL;+/* get the number of maps */+for(i=0;i<symbols->d_size/sizeof(GElf_Sym);i++){+GElf_Symsym;++if(!gelf_getsym(symbols,i,&sym))+continue;+if(sym.st_shndx!=obj->efile.maps_shndx)+continue;+nr_maps++;+}++obj->maps=calloc(nr_maps,sizeof(obj->maps[0]));+if(!obj->maps){+pr_warning("alloc maps for object failed\n");+return-ENOMEM;+}+obj->nr_maps=nr_maps;+for(i=0;i<symbols->d_size/sizeof(GElf_Sym);i++){GElf_Symsym;-size_tmap_idx;constchar*map_name;if(!gelf_getsym(symbols,i,&sym))
@@ -574,12 +587,12 @@ bpf_object__init_maps_name(struct bpf_object *obj)map_name=elf_strptr(obj->efile.elf,obj->efile.strtabidx,sym.st_name);-map_idx=sym.st_value/sizeof(structbpf_map_def);if(map_idx>=obj->nr_maps){pr_warning("index of map \"%s\" is buggy: %zu > %zu\n",map_name,map_idx,obj->nr_maps);continue;}+obj->maps[map_idx].offset=sym.st_value;obj->maps[map_idx].name=strdup(map_name);if(!obj->maps[map_idx].name){pr_warning("failed to alloc map name\n");
@@ -587,6 +600,7 @@ bpf_object__init_maps_name(struct bpf_object *obj)}pr_debug("map %zu is \"%s\"\n",map_idx,obj->maps[map_idx].name);+map_idx++;}return0;}
@@ -647,8 +661,6 @@ static int bpf_object__elf_collect(struct bpf_object *obj)data->d_buf,data->d_size);elseif(strcmp(name,"maps")==0){-err=bpf_object__init_maps(obj,data->d_buf,-data->d_size);obj->efile.maps_shndx=idx;}elseif(sh.sh_type==SHT_SYMTAB){if(obj->efile.symbols){
@@ -698,8 +710,16 @@ static int bpf_object__elf_collect(struct bpf_object *obj)pr_warning("Corrupted ELF file: index of strtab invalid\n");returnLIBBPF_ERRNO__FORMAT;}-if(obj->efile.maps_shndx>=0)-err=bpf_object__init_maps_name(obj);+if(obj->efile.maps_shndx>=0){+Elf_Data*data;+err=bpf_object__init_maps_symbol(obj);+if(err)+gotoout;++scn=elf_getscn(elf,obj->efile.maps_shndx);+data=elf_getdata(scn,0);+err=bpf_object__init_maps(obj,data->d_buf,data->d_size);+}out:returnerr;}
From: Eric Leblond <hidden> Date: 2016-10-16 22:00:34
Makefile was not installing the header file of the library and a
manual copy was needed to have a usable library on the system.
Signed-off-by: Eric Leblond <redacted>
---
tools/lib/bpf/Makefile | 11 +++++++++--
1 file changed, 9 insertions(+), 2 deletions(-)
From: Eric Leblond <hidden> Date: 2016-10-16 22:00:39
Some functions were missing in the library to be able to use it
in the case where the userspace is handling the maps in kernel.
The patch also renames functions to have a homogeneous naming
convention.
Signed-off-by: Eric Leblond <redacted>
---
tools/lib/bpf/bpf.c | 35 ++++++++++++++++++++++++++++++++++-
tools/lib/bpf/bpf.h | 2 --
tools/lib/bpf/libbpf.h | 5 +++++
3 files changed, 39 insertions(+), 3 deletions(-)
From: Eric Leblond <hidden> Date: 2016-10-16 22:01:32
The include of err.h is not explicitely needed in exported
functions and it was causing include conflict with some existing
code due to redefining some macros.
To fix this, let's have error handling functions provided by the
library. Furthermore this will allow user to have an homogeneous
API.
Signed-off-by: Eric Leblond <redacted>
---
tools/lib/bpf/libbpf.c | 11 +++++++++++
tools/lib/bpf/libbpf.h | 4 +++-
2 files changed, 14 insertions(+), 1 deletion(-)
From: Eric Leblond <hidden> Date: 2016-10-16 22:02:04
Current API was not allowing the user to set a type like socket
filter. To avoid a setter function for each type, the patch simply
exports a set function that takes the type in parameter.
Signed-off-by: Eric Leblond <redacted>
---
tools/lib/bpf/libbpf.c | 19 +++++++++----------
tools/lib/bpf/libbpf.h | 3 +++
2 files changed, 12 insertions(+), 10 deletions(-)
@@ -1336,26 +1336,25 @@ int bpf_program__nth_fd(struct bpf_program *prog, int n)returnfd;}-staticvoidbpf_program__set_type(structbpf_program*prog,-enumbpf_prog_typetype)+intbpf_program__set_type(structbpf_program*prog,unsignedinttype){+if(!prog)+return-EINVAL;+if(type>=__MAX_BPF_PROG_TYPE)+return-EINVAL;+prog->type=type;+return0;}intbpf_program__set_tracepoint(structbpf_program*prog){-if(!prog)-return-EINVAL;-bpf_program__set_type(prog,BPF_PROG_TYPE_TRACEPOINT);-return0;+returnbpf_program__set_type(prog,BPF_PROG_TYPE_TRACEPOINT);}intbpf_program__set_kprobe(structbpf_program*prog){-if(!prog)-return-EINVAL;-bpf_program__set_type(prog,BPF_PROG_TYPE_KPROBE);-return0;+returnbpf_program__set_type(prog,BPF_PROG_TYPE_KPROBE);}staticboolbpf_program__is_type(structbpf_program*prog,
The include of err.h is not explicitely needed in exported
functions and it was causing include conflict with some existing
code due to redefining some macros.
To fix this, let's have error handling functions provided by the
library. Furthermore this will allow user to have an homogeneous
API.
Signed-off-by: Eric Leblond <redacted>
---
tools/lib/bpf/libbpf.c | 11 +++++++++++
tools/lib/bpf/libbpf.h | 4 +++-
2 files changed, 14 insertions(+), 1 deletion(-)
Current API was not allowing the user to set a type like socket
filter. To avoid a setter function for each type, the patch simply
exports a set function that takes the type in parameter.
Signed-off-by: Eric Leblond <redacted>
---
tools/lib/bpf/libbpf.c | 19 +++++++++----------
tools/lib/bpf/libbpf.h | 3 +++
2 files changed, 12 insertions(+), 10 deletions(-)
@@ -1336,26 +1336,25 @@ int bpf_program__nth_fd(struct bpf_program *prog, int n)returnfd;}-staticvoidbpf_program__set_type(structbpf_program*prog,-enumbpf_prog_typetype)+intbpf_program__set_type(structbpf_program*prog,unsignedinttype){+if(!prog)+return-EINVAL;+if(type>=__MAX_BPF_PROG_TYPE)+return-EINVAL;+prog->type=type;+return0;}intbpf_program__set_tracepoint(structbpf_program*prog){-if(!prog)-return-EINVAL;-bpf_program__set_type(prog,BPF_PROG_TYPE_TRACEPOINT);-return0;+returnbpf_program__set_type(prog,BPF_PROG_TYPE_TRACEPOINT);}intbpf_program__set_kprobe(structbpf_program*prog){-if(!prog)-return-EINVAL;-bpf_program__set_type(prog,BPF_PROG_TYPE_KPROBE);-return0;+returnbpf_program__set_type(prog,BPF_PROG_TYPE_KPROBE);}staticboolbpf_program__is_type(structbpf_program*prog,
@@ -173,6 +173,9 @@ int bpf_program__set_kprobe(struct bpf_program *prog);boolbpf_program__is_tracepoint(structbpf_program*prog);boolbpf_program__is_kprobe(structbpf_program*prog);+intbpf_program__set_type(structbpf_program*prog,+unsignedinttype);+
Although you don't include uapi/linux/bpf.h in this patch, logically
you add this dependency.
Please continously add bpf_program__set_socket_filter() and
bpf_program__is_socket_filter() like what we do for tracepoint.
This way libbpf.h is indenpendent from kernel header.
We can use macro in both .h and .c.
Thank you.
Some functions were missing in the library to be able to use it
in the case where the userspace is handling the maps in kernel.
The patch also renames functions to have a homogeneous naming
convention.
Signed-off-by: Eric Leblond <redacted>
---
tools/lib/bpf/bpf.c | 35 ++++++++++++++++++++++++++++++++++-
tools/lib/bpf/bpf.h | 2 --
tools/lib/bpf/libbpf.h | 5 +++++
3 files changed, 39 insertions(+), 3 deletions(-)
Please don't use '__' style API here. It is easily be confused with
bpf_map__*() in libbpf.h. They are APIs at different level.
bpf_map__*() are APIs for 'struct bpf_map's, they are object introduced
by libbpf, defined in libbpf.h. bpf_map_*() APIs operate on fd, they are
objects defined by kernel. bpf_map_*() APIs are declared in bpf.h.
In libbpf, bpf.h directly operates on kernel objects (fd), APIs in it
are named bpf_map_*(); libbpf.h operates on 'struct bpf_map' object,
APIs in it are named using bpf_map__*(). libbpf.h and bpf.h are independent
with each other.
It is not correct to assimilate the elf data of the maps section
to an array of map definition. In fact the sizes differ. The
offset provided in the symbol section has to be used instead.
This patch fixes a bug causing a elf with two maps not to load
correctly.
Could you please give an example so we can understand why
section 'maps' is not an array?
Thank you.
From: Joe Stringer <hidden> Date: 2016-10-18 22:57:40
On 16 October 2016 at 14:18, Eric Leblond [off-list ref] wrote:
The include of err.h is not explicitely needed in exported
functions and it was causing include conflict with some existing
code due to redefining some macros.
To fix this, let's have error handling functions provided by the
library. Furthermore this will allow user to have an homogeneous
API.
Signed-off-by: Eric Leblond <redacted>
Does it need to return the error like this or should we just fix up
the bpf_object__open() API to return errors in a simpler form?
There's already libbpf_set_print(...) for outputting errors, is it
reasonable to just change the library to return NULLs in error cases
instead?
On 16 October 2016 at 14:18, Eric Leblond [off-list ref] wrote:
quoted
The include of err.h is not explicitely needed in exported
functions and it was causing include conflict with some existing
code due to redefining some macros.
To fix this, let's have error handling functions provided by the
library. Furthermore this will allow user to have an homogeneous
API.
Signed-off-by: Eric Leblond <redacted>
Does it need to return the error like this or should we just fix up
the bpf_object__open() API to return errors in a simpler form?
There's already libbpf_set_print(...) for outputting errors, is it
reasonable to just change the library to return NULLs in error cases
instead?
Returning error code to caller so caller knows what happen.
Other subsystems in perf also do this.
Perf hides libbpf's error output (make it silent unless -v),
so it needs a way for receiving libbpf's error code.
I think this patch is good, decouple libbpf.h and kernel headers.
Thank you.
Hi Eric,
Are you still working in this patch set?
Now I know why maps section is not a simple array
from a patch set from Joe Stringer:
https://www.mail-archive.com/netdev@vger.kernel.org/msg135088.html
So I think this patch is really useful.
Are you going to resend the whole patch set? If not, let me collect
this patch 7/8 into my local code base and send to Arnaldo
with my other patches.
Thank you.
On 2016/10/17 5:18, Eric Leblond wrote:
quoted hunk
It is not correct to assimilate the elf data of the maps section
to an array of map definition. In fact the sizes differ. The
offset provided in the symbol section has to be used instead.
This patch fixes a bug causing a elf with two maps not to load
correctly.
Signed-off-by: Eric Leblond <redacted>
---
tools/lib/bpf/libbpf.c | 50 +++++++++++++++++++++++++++++++++++---------------
1 file changed, 35 insertions(+), 15 deletions(-)
@@ -529,13 +530,6 @@ bpf_object__init_maps(struct bpf_object *obj, void *data,pr_debug("maps in %s: %zd bytes\n",obj->path,size);-obj->maps=calloc(nr_maps,sizeof(obj->maps[0]));-if(!obj->maps){-pr_warning("alloc maps for object failed\n");-return-ENOMEM;-}-obj->nr_maps=nr_maps;-for(i=0;i<nr_maps;i++){structbpf_map_def*def=&obj->maps[i].def;
@@ -547,23 +541,42 @@ bpf_object__init_maps(struct bpf_object *obj, void *data,obj->maps[i].fd=-1;/* Save map definition into obj->maps */-*def=((structbpf_map_def*)data)[i];+*def=*(structbpf_map_def*)(data+obj->maps[i].offset);}return0;}staticint-bpf_object__init_maps_name(structbpf_object*obj)+bpf_object__init_maps_symbol(structbpf_object*obj){inti;+intnr_maps=0;Elf_Data*symbols=obj->efile.symbols;+size_tmap_idx=0;if(!symbols||obj->efile.maps_shndx<0)return-EINVAL;+/* get the number of maps */+for(i=0;i<symbols->d_size/sizeof(GElf_Sym);i++){+GElf_Symsym;++if(!gelf_getsym(symbols,i,&sym))+continue;+if(sym.st_shndx!=obj->efile.maps_shndx)+continue;+nr_maps++;+}++obj->maps=calloc(nr_maps,sizeof(obj->maps[0]));+if(!obj->maps){+pr_warning("alloc maps for object failed\n");+return-ENOMEM;+}+obj->nr_maps=nr_maps;+for(i=0;i<symbols->d_size/sizeof(GElf_Sym);i++){GElf_Symsym;-size_tmap_idx;constchar*map_name;if(!gelf_getsym(symbols,i,&sym))
@@ -574,12 +587,12 @@ bpf_object__init_maps_name(struct bpf_object *obj)map_name=elf_strptr(obj->efile.elf,obj->efile.strtabidx,sym.st_name);-map_idx=sym.st_value/sizeof(structbpf_map_def);if(map_idx>=obj->nr_maps){pr_warning("index of map \"%s\" is buggy: %zu > %zu\n",map_name,map_idx,obj->nr_maps);continue;}+obj->maps[map_idx].offset=sym.st_value;obj->maps[map_idx].name=strdup(map_name);if(!obj->maps[map_idx].name){pr_warning("failed to alloc map name\n");
@@ -587,6 +600,7 @@ bpf_object__init_maps_name(struct bpf_object *obj)}pr_debug("map %zu is \"%s\"\n",map_idx,obj->maps[map_idx].name);+map_idx++;}return0;}
@@ -647,8 +661,6 @@ static int bpf_object__elf_collect(struct bpf_object *obj)data->d_buf,data->d_size);elseif(strcmp(name,"maps")==0){-err=bpf_object__init_maps(obj,data->d_buf,-data->d_size);obj->efile.maps_shndx=idx;}elseif(sh.sh_type==SHT_SYMTAB){if(obj->efile.symbols){
@@ -698,8 +710,16 @@ static int bpf_object__elf_collect(struct bpf_object *obj)pr_warning("Corrupted ELF file: index of strtab invalid\n");returnLIBBPF_ERRNO__FORMAT;}-if(obj->efile.maps_shndx>=0)-err=bpf_object__init_maps_name(obj);+if(obj->efile.maps_shndx>=0){+Elf_Data*data;+err=bpf_object__init_maps_symbol(obj);+if(err)+gotoout;++scn=elf_getscn(elf,obj->efile.maps_shndx);+data=elf_getdata(scn,0);+err=bpf_object__init_maps(obj,data->d_buf,data->d_size);+}out:returnerr;}
From: Eric Leblond <hidden> Date: 2016-11-07 18:41:33
Hi,
On Tue, 2016-11-08 at 02:23 +0800, Wangnan (F) wrote:
Hi Eric,
Are you still working in this patch set?
Sorry to lag on this, I've been taken by a series of other projects. I
did not yet reworked it yet but I was planning to do a bit on it this
week.
Now I know why maps section is not a simple array
from a patch set from Joe Stringer:
https://www.mail-archive.com/netdev@vger.kernel.org/msg135088.html
So I think this patch is really useful.
Are you going to resend the whole patch set? If not, let me collect
this patch 7/8 into my local code base and send to Arnaldo
with my other patches.
If ok with you, I propose that you collect patch 7/8 it you have no
news from me on Friday. If an issue for you, just collect it now and I
will synchronize with updated code when resending my patchset.
BR,
--
Eric Leblond [off-list ref]
Blog: https://home.regit.org/
Hi Eric,
During testing this patch I find a segfault, please see inline comment.
In addition, since both the BPF map array and map names should be done
after symbol table is collected, merging bpf_object__init_maps and
bpf_object__init_maps_name would be a good practice, making code
simpler.
So I prepare a new patch. Please have a look at:
http://lkml.kernel.org/g/20161108215734.28905-1-wangnan0@huawei.com
New version ensure not crashing in any case user provides a corrupted
maps section, including array of bpf maps, maps with different definition
structures and very short map definition.
Thank you.
On 2016/10/16 14:18, Eric Leblond wrote:
quoted hunk
It is not correct to assimilate the elf data of the maps section
to an array of map definition. In fact the sizes differ. The
offset provided in the symbol section has to be used instead.
This patch fixes a bug causing a elf with two maps not to load
correctly.
Signed-off-by: Eric Leblond <redacted>
---
tools/lib/bpf/libbpf.c | 50 +++++++++++++++++++++++++++++++++++---------------
1 file changed, 35 insertions(+), 15 deletions(-)