Hi Yonghong.

On 9/13/21 9:40 AM, Jose E. Marchesi wrote:

quoted

quoted

cc Jose E. Marchesi

Hi, Jose, just let you know that the BTF format for BTF_KIND_TAG is
changed since v1 as the new format can simplify kernel/libbpf
implementation. Thanks!

Noted.  Thanks for the update.

Hi, Jose,

This is just another update on btf_tag development.
Now, btf_tag is divided into btf_decl_tag and btf_type_tag
for tagging declarations and types as clang compiler prefers
not to mix them with each other. All compiler works in llvm
has done and you can check upstream llvm-project "main" branch
for implementation.

The patch set below (under review)
   https://lore.kernel.org/bpf/20211209173537.1525283-1-yhs@fb.com/ (local)
actually tried to use btf_type_tag for linux kernel __user
annotation so bpf verifier can use it.

Noted, thanks for the heads up :)
We have not yet started to implement this.

Another question from Omar (open source drgn maintainer)

https://developers.facebook.com/blog/post/2021/12/09/drgn-how-linux-kernel-team-meta-debugs-kernel-scale/
mentioned that btf_tag information will also help drgn since it
can then especially distinguish between __percpu pointer from
other pointers. Currently drgn is using dwarf, clang compiled
kernel puts btf_tag information in dwarf. Based on our earlier
discussion, gcc intends to generate btf tags for BTF only. Maybe
we could discuss to also generate for dwarf? Do we need a flag?

It seems to me that there are three different orthogonal issues/topics
here, even if somehow related.  Each would require a separated
discussion, probably on different contexts:

[Please let me know if I am wrong on any detail in the summary below.
 In part I am writing it down as a recap to help myself :)]

1) The need for BTF to convey free-text tags on certain elements, such
   as members of struct types.

   IMO there is not much to discuss about this one.  The specification
   is straightforward as is the implementation.  We will be adding it to
   GCC soon.

   Note that:
   - This is obviously BTF specific.
   - This is not strictly BPF specific, as GCC can generate BTF for any
     supported target and not just BPF.  I think you have patches for
     LLVM to the same effect.

2) The need for DWARF to convey free-text tags on certain elements, such
   as members of struct types.

   The motivation for this was originally the way the Linux kernel
   generates its BTF information, using pahole, using DWARF as a source.
   As we discussed in our last exchange on this topic, this is
   accidental, i.e. if the kernel switched to generate BTF directly from
   the compiler and the linker could merge/deduplicate BTF, there would
   be no need for using DWARF to act as the "unwilling conveyer" of this
   information.  There are additional benefits of this second approach.
   Thats why we didn't plan to add these extended DWARF DIEs to GCC.

   However, it now seems that a DWARF consumer, the drgn project, would
   also benefit from having such a support in DWARF to distinguish
   between different kind of pointers.

   So it seems to me that now we have two use-cases for adding support
   for these free-text tags to DWARF, as a proper extension to the
   format, strictly unrelated to BTF, BPF or even the kernel, since:
   - This is not kernel specific.
   - This is not directly related to BTF.
   - This is not directly related to BPF.

   Therefore I would avoid any reference to BTF in the proposal to keep
   it general, like in the name of DIE types and so on.  This would 

   Whom to involve to discuss this?  Well, I would say that at a minimum
   we need to involve GCC people, LLVM people and DWARF people, and the
   consensum of all the three parties.  Once agreed, we can all
   implement the same DIEs counting on the next version of the DWARF
   spec will catch with them.  I would really avoid rushed solutions
   based on compiler-specific extensions.

   Where to discuss this?  I don't know.  In some DWARF forum?  Or
   cross-posting gcc-patches and whatever LLVM list uses for
   development?  I am CCing Mark Wielaard who is a DWARF wizard... any
   suggestion?

3) Addition of C-family language-level constructions to specify
   free-text tags on certain language elements, such as struct fields.

   These are the attributes, or built-ins or whatever syntax.

   Note that, strictly speaking:
   - This is orthogonal to both DWARF and BTF, and any other supported
     debugging format, which may or may not be expressive enough to
     convey the free-form text tag.
   - This is not specific to BPF.

   Therefore I would avoid any reference to BTF or BPF in the attribute
   names.  Something like `__attribute__((btf_tag("arbitrary_str")))'
   makes very little sense to me; the attribute name ought to be more
   generic.

   Whom to involve to discuss this?  Definitely, the front-end chaps of
   both GCC and LLVM will have something to say about this, particularly
   the ones in charge of the C-like language front-ends like C and C++.
   A consensum among them would be ideal and would avoid
   compiler-specific hacks.

   Where to discuss this? Again, it seems that we need some neutral
   ground to discuss inter-operability issues between the different free
   software compilers...

In any case we are more than willing to help and participate in the
discussions :)

Please let me know if you have any questions.  Happy to help in
whatever way to get gcc also implementing btf tag support.

Thanks!

Yonghong

quoted

quoted

On 9/13/21 8:51 AM, Yonghong Song wrote:

quoted

LLVM14 added support for a new C attribute ([1])
    __attribute__((btf_tag("arbitrary_str")))
This attribute will be emitted to dwarf ([2]) and pahole
will convert it to BTF. Or for bpf target, this
attribute will be emitted to BTF directly ([3], [4]).
The attribute is intended to provide additional
information for
    - struct/union type or struct/union member
    - static/global variables
    - static/global function or function parameter.
This new attribute can be used to add attributes
to kernel codes, e.g., pre- or post- conditions,
allow/deny info, or any other info in which only
the kernel is interested. Such attributes will
be processed by clang frontend and emitted to
dwarf, converting to BTF by pahole. Ultimiately
the verifier can use these information for
verification purpose.
The new attribute can also be used for bpf
programs, e.g., tagging with __user attributes
for function parameters, specifying global
function preconditions, etc. Such information
may help verifier to detect user program
bugs.
After this series, pahole dwarf->btf converter
will be enhanced to support new llvm tag
for btf_tag attribute. With pahole support,
we will then try to add a few real use case,
e.g., __user/__rcu tagging, allow/deny list,
some kernel function precondition, etc,
in the kernel.
In the rest of the series, Patches 1-2 had
kernel support. Patches 3-4 added
libbpf support. Patch 5 added bpftool
support. Patches 6-10 added various selftests.
Patch 11 added documentation for the new kind.
    [1] https://reviews.llvm.org/D106614
    [2] https://reviews.llvm.org/D106621
    [3] https://reviews.llvm.org/D106622
    [4] https://reviews.llvm.org/D109560
Changelog:
    v1 -> v2:
      - BTF ELF format changed in llvm ([4] above),
        so cross-board change to use the new format.
      - Clarified in commit message that BTF_KIND_TAG
        is not emitted by bpftool btf dump format c.
      - Fix various comments from Andrii.
Yonghong Song (11):
    btf: change BTF_KIND_* macros to enums
    bpf: support for new btf kind BTF_KIND_TAG
    libbpf: rename btf_{hash,equal}_int to btf_{hash,equal}_int_tag
    libbpf: add support for BTF_KIND_TAG
    bpftool: add support for BTF_KIND_TAG
    selftests/bpf: test libbpf API function btf__add_tag()
    selftests/bpf: change NAME_NTH/IS_NAME_NTH for BTF_KIND_TAG format
    selftests/bpf: add BTF_KIND_TAG unit tests
    selftests/bpf: test BTF_KIND_TAG for deduplication
    selftests/bpf: add a test with a bpf program with btf_tag attributes
    docs/bpf: add documentation for BTF_KIND_TAG
   Documentation/bpf/btf.rst                     |  27 +-
   include/uapi/linux/btf.h                      |  52 +--
   kernel/bpf/btf.c                              | 120 +++++++
   tools/bpf/bpftool/btf.c                       |  12 +
   tools/include/uapi/linux/btf.h                |  52 +--
   tools/lib/bpf/btf.c                           |  85 ++++-
   tools/lib/bpf/btf.h                           |  15 +
   tools/lib/bpf/btf_dump.c                      |   3 +
   tools/lib/bpf/libbpf.c                        |  31 +-
   tools/lib/bpf/libbpf.map                      |   5 +
   tools/lib/bpf/libbpf_internal.h               |   2 +
   tools/testing/selftests/bpf/btf_helpers.c     |   7 +-
   tools/testing/selftests/bpf/prog_tests/btf.c  | 318 ++++++++++++++++--
   .../selftests/bpf/prog_tests/btf_tag.c        |  14 +
   .../selftests/bpf/prog_tests/btf_write.c      |  21 ++
   tools/testing/selftests/bpf/progs/tag.c       |  39 +++
   tools/testing/selftests/bpf/test_btf.h        |   3 +
   17 files changed, 736 insertions(+), 70 deletions(-)
   create mode 100644 tools/testing/selftests/bpf/prog_tests/btf_tag.c
   create mode 100644 tools/testing/selftests/bpf/progs/tag.c