Re: FAILED unresolved symbol vfs_truncate on arm64 with LLVM

On Tue, Feb 09, 2021 at 12:09:31PM -0800, Nick Desaulniers wrote:

On Tue, Feb 9, 2021 at 11:06 AM Jiri Olsa [off-list ref] wrote:

quoted

On Tue, Feb 09, 2021 at 05:13:42PM +0100, Jiri Olsa wrote:

quoted

On Tue, Feb 09, 2021 at 04:09:36PM +0100, Jiri Olsa wrote:

SNIP

quoted

quoted

quoted

quoted

quoted

                DW_AT_prototyped        (true)
                DW_AT_type      (0x01cfdfe4 "long int")
                DW_AT_external  (true)

Ok, the problem appears to be not in DWARF, but in mcount_loc data.
vfs_truncate's address is not recorded as ftrace-attachable, and thus
pahole ignores it. I don't know why this happens and it's quite
strange, given vfs_truncate is just a normal global function.

right, I can't see it in mcount adresses.. but it begins with instructions
that appears to be nops, which would suggest it's traceable

  ffff80001031f430 <vfs_truncate>:
  ffff80001031f430: 5f 24 03 d5   hint    #34
  ffff80001031f434: 1f 20 03 d5   nop
  ffff80001031f438: 1f 20 03 d5   nop
  ffff80001031f43c: 3f 23 03 d5   hint    #25

quoted

quoted

I'd like to understand this issue before we try to fix it, but there
is at least one improvement we can make: pahole should check ftrace
addresses only for static functions, not the global ones (global ones
should be always attachable, unless they are special, e.g., notrace
and stuff). We can easily check that by looking at the corresponding
symbol. But I'd like to verify that vfs_truncate is ftrace-attachable

I'm still trying to build the kernel.. however ;-)

I finally reproduced.. however arm's not using mcount_loc
but some other special section.. so it's new mess for me

so ftrace data actualy has vfs_truncate address but with extra 4 bytes:

        ffff80001031f434

real vfs_truncate address:

        ffff80001031f430 g     F .text  0000000000000168 vfs_truncate

vfs_truncate disasm:

        ffff80001031f430 <vfs_truncate>:
        ffff80001031f430: 5f 24 03 d5   hint    #34
        ffff80001031f434: 1f 20 03 d5   nop
        ffff80001031f438: 1f 20 03 d5   nop
        ffff80001031f43c: 3f 23 03 d5   hint    #25

thats why we don't match it in pahole.. I checked few other functions
and some have the same problem and some match the function boundary

those that match don't have that first hint instrucion, like:

        ffff800010321e40 <do_faccessat>:
        ffff800010321e40: 1f 20 03 d5   nop
        ffff800010321e44: 1f 20 03 d5   nop
        ffff800010321e48: 3f 23 03 d5   hint    #25

any hints about hint instructions? ;-)

aarch64 makes *some* newer instructions reuse the "hint" ie "nop"
encoding space to make software backwards compatible on older hardware
that doesn't support such instructions.  Is this BTI, perhaps? (The
function is perhaps the destination of an indirect call?)

I see, I think we can't take ftrace addresses as start of the function
because there could be extra instruction(s) related to the call before
it like here

we need to check ftrace address be within the function/symbol,
not exact start

jirka