Re: [PATCH v4] pgo: add clang's Profile Guided Optimization infrastructure
From: Bill Wendling <morbo@google.com>
Date: 2021-01-14 02:12:53
Also in:
linux-doc, lkml
On Wed, Jan 13, 2021 at 12:55 PM Nathan Chancellor [off-list ref] wrote:
Hi Bill, On Tue, Jan 12, 2021 at 10:19:58PM -0800, Bill Wendling wrote:quoted
From: Sami Tolvanen <samitolvanen@google.com> Enable the use of clang's Profile-Guided Optimization[1]. To generate a profile, the kernel is instrumented with PGO counters, a representative workload is run, and the raw profile data is collected from /sys/kernel/debug/pgo/profraw. The raw profile data must be processed by clang's "llvm-profdata" tool before it can be used during recompilation: $ cp /sys/kernel/debug/pgo/profraw vmlinux.profraw $ llvm-profdata merge --output=vmlinux.profdata vmlinux.profraw Multiple raw profiles may be merged during this step. The data can now be used by the compiler: $ make LLVM=1 KCFLAGS=-fprofile-use=vmlinux.profdata ... This initial submission is restricted to x86, as that's the platform we know works. This restriction can be lifted once other platforms have been verified to work with PGO. Note that this method of profiling the kernel is clang-native, unlike the clang support in kernel/gcov. [1] https://clang.llvm.org/docs/UsersManual.html#profile-guided-optimization Signed-off-by: Sami Tolvanen <samitolvanen@google.com> Co-developed-by: Bill Wendling <morbo@google.com> Signed-off-by: Bill Wendling <morbo@google.com> Change-Id: Ic78e69c682286d3a44c4549a0138578c98138b77Small nit: This should be removed.
Grrr....The git hook keeps adding it in there. :-(
I applied this patch on top of v5.11-rc3, built it with LLVM 12 (f1d5cbbdee5526bc86eac0a5652b115d9bc158e5 + D94470) with Microsoft's WSL 5.4 config [1] + CONFIG_PGO_CLANG=y, and ran it on WSL2. $ zgrep PGO /proc/config.gz # Profile Guided Optimization (PGO) (EXPERIMENTAL) CONFIG_ARCH_SUPPORTS_PGO_CLANG=y CONFIG_PGO_CLANG=y # end of Profile Guided Optimization (PGO) (EXPERIMENTAL) However, I see an issue with actually using the data: $ sudo -s # mount -t debugfs none /sys/kernel/debug # cp -a /sys/kernel/debug/pgo/profraw vmlinux.profraw # chown nathan:nathan vmlinux.profraw # exit $ tc-build/build/llvm/stage1/bin/llvm-profdata merge --output=vmlinux.profdata vmlinux.profraw warning: vmlinux.profraw: Invalid instrumentation profile data (bad magic) error: No profiles could be merged. Am I holding it wrong? :) Note, this is virtualized, I do not have any "real" x86 hardware that I can afford to test on right now. [1]: https://github.com/microsoft/WSL2-Linux-Kernel/raw/linux-msft-wsl-5.4.y/Microsoft/config-wsl
Could you send me the vmlinux.profraw file? (Don't CC this list.) -bw