Re: [PATCH v5] pgo: add clang's Profile Guided Optimization infrastructure

On Wed, Jan 20, 2021 at 6:03 PM Sedat Dilek [off-list ref] wrote:

On Mon, Jan 18, 2021 at 10:56 PM Bill Wendling [off-list ref] wrote:

quoted

On Mon, Jan 18, 2021 at 9:26 AM Sedat Dilek [off-list ref] wrote:

quoted

On Mon, Jan 18, 2021 at 1:39 PM Sedat Dilek [off-list ref] wrote:

quoted

On Mon, Jan 18, 2021 at 3:32 AM Bill Wendling [off-list ref] wrote:

quoted

On Sun, Jan 17, 2021 at 4:27 PM Sedat Dilek [off-list ref] wrote:

quoted

[ big snip ]

[More snippage.]

quoted

[ CC Fangrui ]

With the attached...

   [PATCH v3] module: Ignore _GLOBAL_OFFSET_TABLE_ when warning for
undefined symbols

...I was finally able to boot into a rebuild PGO-optimized Linux-kernel.
For details see ClangBuiltLinux issue #1250 "Unknown symbol
_GLOBAL_OFFSET_TABLE_ loading kernel modules".

Thanks for confirming that this works with the above patch.

quoted

@ Bill Nick Sami Nathan

1, Can you say something of the impact passing "LLVM_IAS=1" to make?

The integrated assembler and this option are more-or-less orthogonal
to each other. One can still use the GNU assembler with PGO. If you're
having an issue, it may be related to ClangBuiltLinux issue #1250.

quoted

2. Can you please try Nick's DWARF v5 support patchset v5 and
CONFIG_DEBUG_INFO_DWARF5=y (see attachments)?

I know Nick did several tests with PGO. He may have looked into it
already, but we can check.

Reproducible.

LLVM_IAS=1 + DWARF5 = Not bootable

I will try:

LLVM_IAS=1 + DWARF4

I was not able to boot into such a built Linux-kernel.

PGO will have no effect on debugging data. If this is an issue with
DWARF, then it's likely orthogonal to the PGO patch.

quoted

For me worked: DWARF2 and LLVM_IAS=1 *not* set.

Of course, this could be an issue with my system's LLVM/Clang.

Debian clang version
12.0.0-++20210115111113+45ef053bd709-1~exp1~20210115101809.3724

Please use the official clang 11.0.1 release
(https://releases.llvm.org/download.html), modifying the
kernel/pgo/Kconfig as I suggested above. The reason we specify clang
12 for the minimal version is because of an issue that was recently
fixed.

I downgraded to clang-11.1.0-rc1.
( See attachment. )

Doing the first build with PGO enabled plus DWARF5 and LLVM_IAS=1 works.

But again after generating vmlinux.profdata and doing the PGO-rebuild
- the resulting Linux-kernel does NOT boot in QEMU or on bare metal.
With GNU/as I can boot.

So this is independent of DWARF v4 or DWARF v5 (LLVM_IAS=1 and DWARF
v2 is not allowed).
There is something wrong (here) with passing LLVM_IAS=1 to make when
doing the PGO-rebuild.

Can someone please verify and confirm that the PGO-rebuild with
LLVM_IAS=1 boots or boots not?

I was able to build+boot with LLVM_IAS=1 on my personal laptop (no
dwarf 5, just mainline+v5).

Thanks.

- Sedat -

quoted

quoted

Can you give me a LLVM commit-id where you had success with LLVM_IAS=1
and especially CONFIG_DEBUG_INFO_DWARF5=y?
Success means I was able to boot in QEMU and/or bare metal.

The DWARF5 patch isn't in yet, so I don't want to rely upon it too much.

I agree, providing test results with patches that haven't landed yet
can cloud the interpretation of results.  It would be helpful to drop
local patch sets before trying this.

If the resulting image still isn't working for you, can you please
provide your config? Surely we'd be able to reproduce boot failures in
QEMU?  Nothing comes to mind about a change of assemblers causing an
issue; I would assume assembly cannot be instrumented by the compiler
(even though the compiler is the "driver" of the assembler).

The hash warnings are certainly curious.
IndexedInstrProfReader::getInstrProfRecord() is the only place in LLVM
sources that can emit that.
-- 
Thanks,
~Nick Desaulniers