Re: [PATCH dwarves 0/3] add option to merge more dwarf cu's into

On 3/25/21 6:10 AM, Arnaldo Carvalho de Melo wrote:

Em Wed, Mar 24, 2021 at 11:53:16PM -0700, Yonghong Song escreveu:

quoted

For vmlinux built with clang thin-lto or lto for latest bpf-next,
there exist cross cu debuginfo type references. For example,
       compile unit 1:
          tag 10:  type A
       compile unit 2:
          ...
            refer to type A (tag 10 in compile unit 1)
I only checked a few but have seen type A may be a simple type
like "unsigned char" or a complex type like an array of base types.
I am using latest llvm trunk and bpf-next. I suspect llvm12 or
linus tree >= 5.12 rc2 should be able to exhibit the issue as well.
Both thin-lto and lto have the same issues.

Current pahole cannot handle this. It will report types cannot
be found error. Bill Wendling has attempted to fix the issue
with [1] by permitting all tags/types are hashed to the same
hash table and then process cu's one by one. This does not
really work. The reason is that each cu resolves types locally
so for the above example we may have
   compile unit 1:
     type A : type_id = 10
   compile unit 2:
     refer to type A : type A will be resolved as type id = 10
But id 10 refers to compile unit 1, we will get either out
of bound type id or incorrect one.

This patch set is a continuation of Bill's work. We still
increase the hashtable size and traverse all cu's before
recoding and finalization. But instead of creating one-to-one
mapping between debuginfo cu and pahole cu, we just create
one pahole cu, which should solve the above incorrect type
id issue.

Patches #1 and #2 are refactoring the existing code
and Patch #3 added an option "merge_cus" to permit
merging all debuginfo cu's into one pahole cu.
For vmlinux built, it can be detected that if LTO or Thin-LTO
is enabled, "merge_cus" can be added into pahole
command line.

   [1] https://www.spinics.net/lists/dwarves/msg00999.html

Thanks for working on this, I'll look at it today.

Thanks! In case that you want to test with the kernel, I attached a 
patch on top of bpf-next to use --merge_cus when building kernel and 
modules.

- Arnaldo