On Wed, Oct 27, 2021 at 11:23:55PM IST, Andrii Nakryiko wrote:

On Wed, Oct 27, 2021 at 1:53 AM Jiri Olsa [off-list ref] wrote:

quoted

On Tue, Oct 26, 2021 at 09:12:31PM -0700, Andrii Nakryiko wrote:

quoted

On Tue, Oct 26, 2021 at 5:03 AM Jiri Olsa [off-list ref] wrote:

quoted

On Mon, Oct 25, 2021 at 09:54:48PM -0700, Andrii Nakryiko wrote:

quoted

On Sat, Oct 23, 2021 at 5:05 AM Jiri Olsa [off-list ref] wrote:

quoted

hi,
I'm trying to enable BTF for kernel module in fedora,
and I'm getting big increase on modules sizes on s390x arch.

Size of modules in total - kernel dir under /lib/modules/VER/
from kernel-core and kernel-module packages:

               current   new
      aarch64      60M   76M
      ppc64le      53M   66M
      s390x        21M   41M
      x86_64       64M   79M

The reason for higher increase on s390x was that dedup algorithm
did not detect some of the big kernel structs like 'struct module',
so they are duplicated in the kernel module BTF data. The s390x
has many small modules that increased significantly in size because
of that even after compression.

First issues was that the '--btf_gen_floats' option is not passed
to pahole for kernel module BTF generation.

The other problem is more tricky and is the reason why this patchset
is RFC ;-)

The s390x compiler generates multiple definitions of the same struct
and dedup algorithm does not seem to handle this at the moment.

I put the debuginfo and btf dump of the s390x pnet.ko module in here:
  http://people.redhat.com/~jolsa/kmodbtf/

Please let me know if you'd like to see other info/files.

Hard to tell what's going on without vmlinux itself. Can you upload a
corresponding kernel image with BTF in it?

sure, uploaded

vmlinux.btfdump:

[174] FLOAT 'float' size=4
[175] FLOAT 'double' size=8

VS

pnet.btfdump:

[89318] INT 'float' size=4 bits_offset=0 nr_bits=32 encoding=(none)
[89319] INT 'double' size=8 bits_offset=0 nr_bits=64 encoding=(none)

ugh, that's with no fix applied, sry

I applied the first patch and uploaded new files

now when I compare the 'module' struct from vmlinux:

        [885] STRUCT 'module' size=1280 vlen=70

and same one from pnet.ko:

        [89323] STRUCT 'module' size=1280 vlen=70

they seem to completely match, all the fields
and yet it still appears in the kmod's BTF

Ok, now struct module is identical down to the types referenced from
the fields, which means it should have been deduplicated completely.
This will require a more time-consuming debugging, though, so I'll put
it on my TODO list for now. If you get to this earlier, see where the
equivalence check fails in btf_dedup (sprinkle debug outputs around to
see what's going on).

Hello Andrii,

I think I'm seeing something similar when working on the conntrack patches [0],
I was looking to match whether the type in a PTR_TO_BTF_ID register is same as
struct nf_conn, but it seems that there can be two BTF IDs for the same struct
type.

When doing bpftool dump, I see:

 ; bpftool btf dump file /sys/kernel/btf/vmlinux format raw | grep nf_conn
 ...
[89224] STRUCT 'nf_conn' size=256 vlen=15
 ...

 ; bpftool btf dump file /sys/kernel/btf/nf_conntrack format raw | grep nf_conn
 ...
[103077] STRUCT 'nf_conn' size=256 vlen=15
[104988] STRUCT 'nf_conn' size=256 vlen=15
[106490] STRUCT 'nf_conn' size=256 vlen=15
[108187] STRUCT 'nf_conn' size=256 vlen=15
 ...

Inside the kernel, when trying to match both, register PTR_TO_BTF_ID refers to
the nf_conntrack BTF ID, while the BTF_ID_LIST resolves to the one in vmlinux,
this ends up making the job of matching the two struct types a bit difficult
(for now, I am thinking of going with btf_struct_ids_match). My original plan
was to compare the result of btf_types_by_id.

[0]: https://github.com/kkdwivedi/linux/commits/conntrack-bpf

quoted

thanks,
jirka

quoted

quoted

jirka

quoted

quoted

I found code in dedup that seems to handle such situation for arrays,
and added 'some' fix for structs. With that change I can no longer
see vmlinux's structs in kernel module BTF data, but I have no idea
if that breaks anything else.

thoughts? thanks,
jirka


---
Jiri Olsa (2):
      kbuild: Unify options for BTF generation for vmlinux and modules
      bpf: Add support to detect and dedup instances of same structs

 Makefile                  |  3 +++
 scripts/Makefile.modfinal |  2 +-
 scripts/link-vmlinux.sh   | 11 +----------
 scripts/pahole-flags.sh   | 20 ++++++++++++++++++++
 tools/lib/bpf/btf.c       | 12 ++++++++++--
 5 files changed, 35 insertions(+), 13 deletions(-)
 create mode 100755 scripts/pahole-flags.sh

--
Kartikeya