Re: [PATCH v7 0/4] kexec: Fix kexec_file_load for llvm16 with PGO
From: Ricardo Ribalda <hidden>
Date: 2023-09-08 21:17:54
Also in:
kexec, linux-riscv, lkml, llvm, stable
Hi Song On Fri, 8 Sept 2023 at 01:08, Song Liu [off-list ref] wrote:
Hi Ricardo and folks, On Fri, May 19, 2023 at 7:48 AM Ricardo Ribalda [off-list ref] wrote:quoted
When upreving llvm I realised that kexec stopped working on my test platform. The reason seems to be that due to PGO there are multiple .text sections on the purgatory, and kexec does not supports that. Signed-off-by: Ricardo Ribalda <redacted>We are seeing WARNINGs like the following while kexec'ing a PGO and LTO enabled kernel: WARNING: CPU: 26 PID: 110894 at kernel/kexec_file.c:919 kexec_load_purgatory+0x37f/0x390 AFAICT, the warning was added by this set, and it was triggered when we have many .text sections in purgatory.ro. The kexec was actually successful. So I wonder whether we really need the WARNING here. If we disable LTO (PGO is still enabled), we don't see the WARNING any more. I also tested an older kernel (5.19 based), where we also see many .text sections with LTO. It kexec()'ed fine. (It doesn't have the WARN_ON() in kexec_purgatory_setup_sechdrs).
You have been "lucky" that the code has chosen the correct start address, you need to modify the linker script of your kernel to disable PGO. You need to backport a patch like this: https://lore.kernel.org/lkml/CAPhsuW5_qAvV0N3o+hOiAnb1=buJ1pLzqYW9D+Bwft6hxJvAeQ@mail.gmail.com/T/#md68b7f832216b0c56bbec0c9b07332e180b9ba2b (local) (assuming x86) Regards
Please help us fix this properly (as I really don't know much about kexec). Thanks in advance, Song Here is readelf -S output on purgatory.ro. With LTO: readelf -W -S purgatory.ro There are 48 section headers, starting at offset 0x4a10: Section Headers: [Nr] Name Type Address Off Size ES Flg Lk Inf Al [ 0] NULL 0000000000000000 000000 000000 00 0 0 0 [ 1] .text PROGBITS 0000000000000000 000040 0000d0 00 AX 0 0 16 [ 2] .data PROGBITS 0000000000000000 001000 001000 00 WA 0 0 4096 [ 3] .rela.text RELA 0000000000000000 003788 000228 18 I 45 1 8 [ 4] .rodata PROGBITS 0000000000000000 002000 0000e0 00 A 0 0 16 [ 5] .rela.rodata RELA 0000000000000000 0039b0 000030 18 I 45 4 8 [ 6] .bss NOBITS 0000000000000000 0020e0 001000 00 WA 0 0 4096 [ 7] .text.purgatory PROGBITS 0000000000000000 0020e0 0000df 00 AX 0 0 16 [ 8] .rela.text.purgatory RELA 0000000000000000 0039e0 000060 18 I 45 7 8 [ 9] .text.warn PROGBITS 0000000000000000 0021c0 000001 00 AX 0 0 16 [10] .kexec-purgatory PROGBITS 0000000000000000 0021d0 000120 00 WA 0 0 16 [11] .comment PROGBITS 0000000000000000 003a40 000046 01 MS 0 0 1 [12] .llvm_addrsig LOOS+0xfff4c03 0000000000000000 003a86 000005 00 E 0 0 1 [13] .text.sha256_update PROGBITS 0000000000000000 0022f0 0008eb 00 AX 0 0 16 [14] .rela.text.sha256_update RELA 0000000000000000 003a90 000060 18 I 45 13 8 [15] .text.sha224_update PROGBITS 0000000000000000 002be0 00000c 00 AX 0 0 16 [16] .rela.text.sha224_update RELA 0000000000000000 003af0 000018 18 I 45 15 8 [17] .text.sha256_final PROGBITS 0000000000000000 002bf0 0000cd 00 AX 0 0 16 [18] .rela.text.sha256_final RELA 0000000000000000 003b08 000030 18 I 45 17 8 [19] .text.sha224_final PROGBITS 0000000000000000 002cc0 0000bd 00 AX 0 0 16 [20] .rela.text.sha224_final RELA 0000000000000000 003b38 000030 18 I 45 19 8 [21] .text.sha256 PROGBITS 0000000000000000 002d80 00011d 00 AX 0 0 16 [22] .rela.text.sha256 RELA 0000000000000000 003b68 000030 18 I 45 21 8 [23] .modinfo PROGBITS 0000000000000000 002e9d 000039 00 A 0 0 1 [24] .rodata.SHA256_K PROGBITS 0000000000000000 002ee0 000100 00 A 0 0 16 [25] .rodata.__sha256_final.padding PROGBITS 0000000000000000 002fe0 000040 00 A 0 0 16 [26] .text.memcmp PROGBITS 0000000000000000 003020 00000b 00 AX 0 0 16 [27] .text.bcmp PROGBITS 0000000000000000 003030 00000b 00 AX 0 0 16 [28] .text.strcmp PROGBITS 0000000000000000 003040 000041 00 AX 0 0 16 [29] .text.strncmp PROGBITS 0000000000000000 003090 00003a 00 AX 0 0 16 [30] .text.strnlen PROGBITS 0000000000000000 0030d0 000039 00 AX 0 0 16 [31] .text.atou PROGBITS 0000000000000000 003110 000035 00 AX 0 0 16 [32] .text.simple_strtoull PROGBITS 0000000000000000 003150 0000b6 00 AX 0 0 16 [33] .text.simple_strtol PROGBITS 0000000000000000 003210 0001b6 00 AX 0 0 16 [34] .text.strlen PROGBITS 0000000000000000 0033d0 00001c 00 AX 0 0 16 [35] .text.strstr PROGBITS 0000000000000000 0033f0 00005f 00 AX 0 0 16 [36] .text.strchr PROGBITS 0000000000000000 003450 000022 00 AX 0 0 16 [37] .text.kstrtoull PROGBITS 0000000000000000 003480 000142 00 AX 0 0 16 [38] .text.boot_kstrtoul PROGBITS 0000000000000000 0035d0 00000c 00 AX 0 0 16 [39] .rela.text.boot_kstrtoul RELA 0000000000000000 003b98 000018 18 I 45 38 8 [40] .text.memset PROGBITS 0000000000000000 0035e0 00001f 00 AX 0 0 16 [41] .text.memmove PROGBITS 0000000000000000 003600 0000a6 00 AX 0 0 16 [42] .text.memcpy PROGBITS 0000000000000000 0036b0 0000a6 00 AX 0 0 16 [43] .rodata.str1.1 PROGBITS 0000000000000000 003756 000032 01 AMS 0 0 1 [44] .note.GNU-stack PROGBITS 0000000000000000 003bb0 000000 00 0 0 1 [45] .symtab SYMTAB 0000000000000000 003bb0 000948 18 47 68 8 [46] .shstrtab STRTAB 0000000000000000 0044f8 0002cd 00 0 0 1 [47] .strtab STRTAB 0000000000000000 0047c5 000248 00 0 0 1 Key to Flags: W (write), A (alloc), X (execute), M (merge), S (strings), I (info), L (link order), O (extra OS processing required), G (group), T (TLS), C (compressed), x (unknown), o (OS specific), E (exclude), l (large), p (processor specific) Without LTO: readelf -W -S purgatory.ro There are 16 section headers, starting at offset 0x4130: Section Headers: [Nr] Name Type Address Off Size ES Flg Lk Inf Al [ 0] NULL 0000000000000000 000000 000000 00 0 0 0 [ 1] .text PROGBITS 0000000000000000 000040 00131b 00 AX 0 0 16 [ 2] .rela.text RELA 0000000000000000 003290 000480 18 I 13 1 8 [ 3] .kexec-purgatory PROGBITS 0000000000000000 001360 000120 00 WA 0 0 16 [ 4] .comment PROGBITS 0000000000000000 003710 000046 01 MS 0 0 1 [ 5] .llvm_addrsig LOOS+0xfff4c03 0000000000000000 003756 000005 00 E 0 0 1 [ 6] .data PROGBITS 0000000000000000 002000 001000 00 WA 0 0 4096 [ 7] .rodata PROGBITS 0000000000000000 003000 000220 00 A 0 0 16 [ 8] .rela.rodata RELA 0000000000000000 003760 000030 18 I 13 7 8 [ 9] .bss NOBITS 0000000000000000 003220 001000 00 WA 0 0 4096 [10] .modinfo PROGBITS 0000000000000000 003220 000039 00 A 0 0 1 [11] .rodata.str1.1 PROGBITS 0000000000000000 003259 000032 01 AMS 0 0 1 [12] .note.GNU-stack PROGBITS 0000000000000000 003790 000000 00 0 0 1 [13] .symtab SYMTAB 0000000000000000 003790 0006d8 18 15 43 8 [14] .shstrtab STRTAB 0000000000000000 003e68 00009c 00 0 0 1 [15] .strtab STRTAB 0000000000000000 003f04 00022b 00 0 0 1 Key to Flags: W (write), A (alloc), X (execute), M (merge), S (strings), I (info), L (link order), O (extra OS processing required), G (group), T (TLS), C (compressed), x (unknown), o (OS specific), E (exclude), l (large), p (processor specific) [...]quoted
Ricardo Ribalda (4): kexec: Support purgatories with .text.hot sections x86/purgatory: Remove PGO flags powerpc/purgatory: Remove PGO flags riscv/purgatory: Remove PGO flags arch/powerpc/purgatory/Makefile | 5 +++++ arch/riscv/purgatory/Makefile | 5 +++++ arch/x86/purgatory/Makefile | 5 +++++ kernel/kexec_file.c | 14 +++++++++++++- 4 files changed, 28 insertions(+), 1 deletion(-) --- base-commit: 58390c8ce1bddb6c623f62e7ed36383e7fa5c02f change-id: 20230321-kexec_clang16-4510c23d129c Best regards, -- Ricardo Ribalda Delgado [off-list ref]
-- Ricardo Ribalda