Problem:
========
When a page is hard-offlined due to an uncorrectable memory error (multi
bit ECC), memory_failure() sets PG_hwpoison, unmaps it, removes it from
the buddy allocator. This information is not carried to the next kernel
that is kexeced. The new kernel kexecs and trip over that bad memory
bank _again_.
Why now:
========
Several industry trends make this increasingly important:
1) DRAM is getting more expensive
2) soldered / on-package memory (LPDDR, HBM) is becoming more common, so a
failing part can no longer simply be swapped;
3) memory is kept in service far longer (at Meta, DRAM lifetime is being
drastically extended).
4) It is more and more common to kexec instead of full reboot
5) Increase of memory per system with CXL
What was done already:
======================
In order make linux deal better with the problem above, I've done
already fixed a bunch of stuff in this area, such as:
1) Panic on unrecoverable errors, instead of "printk and carry over":
https://lore.kernel.org/all/20260630-ecc_panic-v10-0-c6ed5b62eea2@debian.org/
2) Respect poisoned memory at kexec time
https://lore.kernel.org/all/20260812-kexec_posioned-v6-0-e477887086f0@debian.org/
Now, the natural follow up is to carry the poisoned memory information
to the next kexec kernel, avoiding tripping over a known "bad page".
Proposed Solution:
==================
Carry the poisoned frames to the next kernel in a new EFI configuration
table, LINUX_EFI_POISONED_MEMORY.
The table is a bitmap with one bit per 2MB of physical memory, modeled
on LINUX_EFI_UNACCEPTED_MEMORY. The stub sizes it from the EFI memory
map and installs it empty while boot services are up -- a running kernel
cannot install a configuration table, it can only flip bits -- and a
table inherited from an earlier boot is reused as-is. That is one
fixed-size allocation, 64KB per TB of RAM, with no list to grow at
runtime and no chain to trust at parse time.
The mechanism is architecture independent, so x86 and arm64 use the same
code.
Each hard offline sets the bit for its unit. Soft-offlined pages are not
recorded: they are still functional, and were offlined predictively.
The next kernel poisons the inherited frames from mm_core_init(), right
after memblock_free_all(), the first point at which they have struct
pages. A frame that is free is taken off the buddy allocator; a frame the
kernel is already using is flagged where it is, and free_pages_prepare()
drops it if the owner ever hands it back. Either way it ends up in the
state a frame poisoned by this kernel would be in, so everything that
already understands PG_hwpoison covers it -- including the kexec segment
placement check from the series linked above, which is what keeps the
next kernel image off these frames.
Nothing is reserved in memblock. An earlier version did that from
efi_config_parse_tables(), but the page flag alone turns out to be
enough, and reserving per unit that early runs into memblock's fixed
region array before memblock_allow_resize().
Granularity is the trade-off: one bad 4KB frame costs a whole 2MB unit in
every later kernel of the chain. In exchange, a row or column fault --
roughly a quarter of the DRAM faults reported in [1], and potentially
thousands of 4KB pages scattered over gigabytes -- collapses into a bit
or two.
A bit is never cleared, which is a known limitation: it stands for a
whole unit, so an unpoison of one frame cannot tell whether the unit as a
whole is good again.
Known limitations:
==================
In order to keep this patchset digestible, I am making some trade-offs,
thus, this feature has the following limitations:
- Memory hot-added after boot is not covered: the bitmap spans the RAM
the EFI memory map describes, and a frame above it is not recorded.
Same gap the unaccepted-memory table has.
- Memory preserved across a KHO handover is not covered.
kho_preserved_memory_reserve() marks it MEMBLOCK_RSRV_NOINIT, so
memmap_init_reserved_pages() leaves those struct pages uninitialised,
and a frame there is neither free nor PageReserved when the bitmap is
applied. Its record is dropped, and the frame goes back to the
allocator once the owner releases it.
- Without a memblock reservation, early boot can allocate over a
recorded frame before the flag is applied. The frame does not reach
the allocator afterwards, but the memmap or page tables may end up
sitting on it.
- The per memory block hardware-poison counter does not include
inherited frames, because memblk_nr_poison_inc() divides by
sections_per_block and memory_dev_init() has not run yet. A block
holding an inherited frame can therefore be brought back online, and
HardwareCorrupted stays high after that memory is removed.
All of the limitations above can be fixed in follow up work. I am trying
to keep this patchset the foundation, with that work done on top.
The series is six patches:
1) add the LINUX_EFI_POISONED_MEMORY table
2) size, build and install it from both stub entry paths
3) record poisoned frames into the table from the memory_failure() path
4) add the mm helper that poisons one inherited frame
5) walk the inherited bitmap
6) apply it from mm_core_init()
This was initially discussed at
https://lore.kernel.org/all/ajut_LDQGYCShApx@gmail.com/
A special thanks to Kiryl Shutsemau, for feedbacks and suggestions.
[1] https://arxiv.org/abs/2408.15302
To: Ard Biesheuvel <ardb@kernel.org>
To: Ilias Apalodimas <ilias.apalodimas@linaro.org>
To: Miaohe Lin <linmiaohe@huawei.com>
To: Naoya Horiguchi <nao.horiguchi@gmail.com>
To: Andrew Morton <akpm@linux-foundation.org>
Cc: linux-efi@vger.kernel.org
Cc: linux-kernel@vger.kernel.org
Cc: linux-mm@kvack.org
Cc: rmikey@meta.com
To: kas@kernel.org
Cc: riel@surriel.com
To: kexec@lists.infradead.org
Signed-off-by: Breno Leitao <leitao@debian.org>
---
Changes in v3:
- Poison the inherited frames from mm_core_init() instead of reserving
them in memblock, so everything that keys off PG_hwpoison sees them,
the kexec segment placement check included (Kiryl)
- Reserve nothing in memblock: the page flag is enough, and reserving
per unit that early runs into the fixed region array (Kiryl)
- Drop the 1MB cap on the table and the unit coarsening that went with
it (Kiryl)
- Size the table from the memory types arm64 turns into RAM as well, not
just the set setup_e820() maps to E820_TYPE_RAM (Kiryl)
- Make CONFIG_EFI_POISONED_MEMORY unprompted, so it is on wherever its
dependencies allow and nobody has to decide (Kiryl, Pratyush)
- Fold the top-of-RAM helper into its only caller and make it static,
rather than adding a generic libstub API
- Fold the table build and its installation into one patch
- Spell out the known limitations in this cover letter
- Link to v2: https://patch.msgid.link/20260821-hwpoison-kho-v2-0-5743791e48e6@debian.org
Changes in v2:
- Replace the growable linked list of 4KB entries with a fixed-size
bitmap, one bit per 2MB, modeled on the unaccepted-memory table (Kiryl)
- Record hard offlines only, by hooking action_result() instead of
num_poisoned_pages_inc(), which also fires for soft offline (Kiryl)
- Allocate the table as EFI_ACPI_RECLAIM_MEMORY, so it is not System RAM
in the next kernel, and reuse an inherited table instead of installing
a second one
- Validate the geometry of an inherited table before using it
- Cap the table at 1MB, coarsening the unit instead of growing it
- Restrict to 64-bit, as the unaccepted-memory table effectively is
- Never clear a bit: an unpoison no longer un-records the unit
- Split the table definition and the stub installer into separate patches
- Link to v1: https://patch.msgid.link/20260717-hwpoison-kho-v1-0-9c5eda551998@debian.org
To: Ard Biesheuvel <ardb@kernel.org>
To: Ilias Apalodimas <ilias.apalodimas@linaro.org>
To: Miaohe Lin <linmiaohe@huawei.com>
To: Naoya Horiguchi <nao.horiguchi@gmail.com>
To: Andrew Morton <akpm@linux-foundation.org>
To: David Hildenbrand <david@kernel.org>
To: Lorenzo Stoakes <ljs@kernel.org>
To: "Liam R. Howlett" <liam@infradead.org>
To: Vlastimil Babka <vbabka@kernel.org>
To: Mike Rapoport <rppt@kernel.org>
To: Suren Baghdasaryan <surenb@google.com>
To: Michal Hocko <mhocko@suse.com>
Cc: linux-efi@vger.kernel.org
Cc: linux-kernel@vger.kernel.org
Cc: linux-mm@kvack.org
---
Breno Leitao (5):
mm/memory-failure: efi: add the LINUX_EFI_POISONED_MEMORY configuration table
mm/memory-failure: libstub: install the poisoned-memory EFI table
mm/memory-failure: efi: record hardware-poisoned frames into the poisoned-memory table
mm/memory-failure: add a helper to poison a frame at boot
mm/memory-failure: efi: replay the poisioned page in the next kernel
drivers/firmware/efi/Kconfig | 8 +
drivers/firmware/efi/Makefile | 1 +
drivers/firmware/efi/efi.c | 6 +
drivers/firmware/efi/libstub/efi-stub-helper.c | 106 +++++++++++
drivers/firmware/efi/libstub/efi-stub.c | 1 +
drivers/firmware/efi/libstub/efistub.h | 6 +
drivers/firmware/efi/libstub/x86-stub.c | 2 +
drivers/firmware/efi/poison.c | 237 +++++++++++++++++++++++++
include/linux/efi.h | 21 +++
include/linux/mm.h | 6 +
mm/memory-failure.c | 33 ++++
mm/mm_init.c | 1 +
12 files changed, 428 insertions(+)
---
base-commit: b8809969e1d7a591e0f49dd464a5d04b3cf02ab1
change-id: 20260622-hwpoison-kho-fc9db2ada8ba
Best regards,
--
Breno Leitao [off-list ref]
Hardware-poisoned page frames are tracked only in the running kernel's
data structures, so a kexec loses them and the next kernel doesn't have
this information, thus, tripping into them again.
Add an EFI configuration table to carry that information across kexec.
It is a bitmap with one bit per EFI_POISON_UNIT_SIZE (2MiB) of physical
memory, modeled on the LINUX_EFI_UNACCEPTED_MEMORY table, and it rides
the EFI system table to every kernel in the chain.
The Kconfig symbol has no prompt. There is nothing for a user to decide,
so it is on wherever it can be, and it only costs 64K per TiB of RAM on
a kernel that already has the EFI stub and MEMORY_FAILURE. It is
restricted to 64BIT because the unit arithmetic would need div_u64() on
32-bit, and there is no 32-bit EFI configuration with MEMORY_FAILURE to
test that on.
Suggested-by: Kiryl Shutsemau <kas@kernel.org>
Signed-off-by: Breno Leitao <leitao@debian.org>
---
drivers/firmware/efi/Kconfig | 8 ++++++++
drivers/firmware/efi/efi.c | 6 ++++++
include/linux/efi.h | 13 +++++++++++++
3 files changed, 27 insertions(+)
@@ -1271,6 +1274,16 @@ struct linux_efi_memreserve {#define EFI_MEMRESERVE_COUNT(size) (((size) - sizeof(struct linux_efi_memreserve)) \/sizeof_field(structlinux_efi_memreserve,entry[0]))+/* Bit N covers unit N of physical address space, counting from address 0. */+structlinux_efi_poisoned_memory{+u32version;+u32unit_size;/* bytes of phys space per bitmap bit */+u64size;/* bitmap size in bytes */+unsignedlongbitmap[];+};++#define EFI_POISON_UNIT_SIZE SZ_2M+void__initefi_arch_mem_reserve(phys_addr_taddr,u64size);/*
A EFI config table can only be installed while boot services are still
up, so the stub has to create it; the running kernel can only flip bits
in a table that already exists.
Size the bitmap from the top of usable RAM, which efi_get_ram_top()
works out by walking the UEFI memory map, since the stub has no max_pfn.
Bit N covers unit N counting from address 0, so the table tracks
max_pfn. Memory the firmware hot-adds later sits above it and is not
carried across a kexec.
One table has to serve every architecture, so efi_get_ram_top() takes
the union of the memory types they turn into RAM: what setup_e820() maps
to E820_TYPE_RAM on x86, plus the EFI_ACPI_RECLAIM_MEMORY and
EFI_PERSISTENT_MEMORY that is_usable_memory() accepts on arm64. Sizing
wide only costs bitmap bytes; sizing narrow silently drops the records
for every frame above the top.
At one bit per 2M that is 64K per TiB, and 256M at the 4PB x86
architectural maximum. The 2M granule is called "unit" here, and the
table carries it so the granule can change later without breaking the
kernels already reading it.
Allocate it as EFI_ACPI_RECLAIM_MEMORY so the next kernel does not take
it for free RAM, and install it empty.
A table installed by an earlier boot rides the system table across kexec
and is reused as-is.
x86 does not go through efi_stub_common(), so the generic stub and the
x86 stub each need the call; on x86 it has to come before exit_boot(),
which is the last point a configuration table can be installed.
Signed-off-by: Breno Leitao <leitao@debian.org>
---
drivers/firmware/efi/libstub/efi-stub-helper.c | 106 +++++++++++++++++++++++++
drivers/firmware/efi/libstub/efi-stub.c | 1 +
drivers/firmware/efi/libstub/efistub.h | 6 ++
drivers/firmware/efi/libstub/x86-stub.c | 2 +
4 files changed, 115 insertions(+)
@@ -774,3 +774,109 @@ void efi_remap_image(unsigned long image_base, unsigned alloc_size,efi_warn("Failed to remap data region non-executable\n");}}++#ifdef CONFIG_EFI_POISONED_MEMORY+/* Erring wide only costs bitmap bytes, so the type list is permissive. */+staticefi_status_tefi_get_ram_top(u64*top)+{+structefi_boot_memmap*map__free(efi_pool)=NULL;+efi_status_tstatus;+u64ram_top=0;+inti,nr_desc;++status=efi_get_memory_map(&map,false);+if(status!=EFI_SUCCESS)+returnstatus;++nr_desc=map->map_size/map->desc_size;+for(i=0;i<nr_desc;i++){+efi_memory_desc_t*d;++d=efi_memdesc_ptr((unsignedlong)map->map,map->desc_size,i);+switch(d->type){+caseEFI_LOADER_CODE:+caseEFI_LOADER_DATA:+caseEFI_BOOT_SERVICES_CODE:+caseEFI_BOOT_SERVICES_DATA:+caseEFI_CONVENTIONAL_MEMORY:+caseEFI_UNACCEPTED_MEMORY:+caseEFI_ACPI_RECLAIM_MEMORY:+caseEFI_PERSISTENT_MEMORY:+ram_top=max(ram_top,+d->phys_addr+d->num_pages*EFI_PAGE_SIZE);+break;+default:+break;+}+}+if(!ram_top)+returnEFI_NOT_FOUND;++*top=ram_top;+returnEFI_SUCCESS;+}++/* Whole words: the kernel reaches the bitmap with set_bit(). */+staticu64efi_poison_bitmap_size(u64ram_top)+{+u64bytes=DIV_ROUND_UP(DIV_ROUND_UP(ram_top,EFI_POISON_UNIT_SIZE),+BITS_PER_BYTE);++returnround_up(bytes,sizeof(unsignedlong));+}++/* ACPI reclaim memory, so the next kernel does not treat it as free RAM. */+staticstructlinux_efi_poisoned_memory*efi_poison_alloc(u64bitmap_size)+{+structlinux_efi_poisoned_memory*pm;+efi_status_tstatus;++status=efi_bs_call(allocate_pool,EFI_ACPI_RECLAIM_MEMORY,+sizeof(*pm)+bitmap_size,(void**)&pm);+if(status!=EFI_SUCCESS)+returnNULL;++pm->version=1;+pm->unit_size=EFI_POISON_UNIT_SIZE;+pm->size=bitmap_size;+memset(pm->bitmap,0,bitmap_size);++returnpm;+}++voidinstall_poisoned_memory_table(void)+{+efi_guid_tpoisoned_memory_table_guid=LINUX_EFI_POISONED_MEMORY_TABLE_GUID;+structlinux_efi_poisoned_memory*pm;+u64ram_top,bitmap_size;+efi_status_tstatus;++/* A table installed by an earlier boot rides the system table across kexec. */+pm=get_efi_config_table(poisoned_memory_table_guid);+if(pm){+if(pm->version!=1)+efi_err("Unknown version of poisoned-memory table\n");+return;+}++if(efi_get_ram_top(&ram_top)!=EFI_SUCCESS){+efi_err("Failed to size the poisoned-memory table!\n");+return;+}++bitmap_size=efi_poison_bitmap_size(ram_top);++pm=efi_poison_alloc(bitmap_size);+if(!pm){+efi_err("Failed to allocate poisoned-memory table!\n");+return;+}++status=efi_bs_call(install_configuration_table,+&poisoned_memory_table_guid,pm);+if(status!=EFI_SUCCESS){+efi_bs_call(free_pool,pm);+efi_err("Failed to install poisoned-memory config table!\n");+}+}+#endif
action_result() is where memory_failure() reports the outcome of a hard
offline, so hook it to set the frame's bit in the
LINUX_EFI_POISONED_MEMORY bitmap.
Soft-offlined pages reach num_poisoned_pages_inc() through
page_handle_poison() and are deliberately left out: they are still
functional and were offlined predictively, so recording them would turn
a prediction into a permanent loss for every kernel further down the
kexec chain.
A bit is only ever set, never cleared, given that multiple pages can set
the same bit, and it is not trivial to decide if the bit should be unset
when a page is unrecorded.
Unpoisoning a frame therefore does not hand its unit back to the next
kernel. That is a known limitation.
memory_failure() has already taken the frame out of this kernel's
allocator, so only the cross-kexec record happens here.
Suggested-by: Kiryl Shutsemau <kas@kernel.org>
Signed-off-by: Breno Leitao <leitao@debian.org>
---
drivers/firmware/efi/Makefile | 1 +
drivers/firmware/efi/poison.c | 126 ++++++++++++++++++++++++++++++++++++++++++
include/linux/efi.h | 6 ++
mm/memory-failure.c | 3 +
4 files changed, 136 insertions(+)
@@ -0,0 +1,126 @@+// SPDX-License-Identifier: GPL-2.0-only+/*+*RuntimesideoftheLINUX_EFI_POISONED_MEMORYtable:onebitper+*EFI_POISON_UNIT_SIZE,sethereasframesgobad,honoredbythenextkernel.+*+*Copyright(c)2026MetaPlatforms,Inc.andaffiliates.+*Copyright(c)2026BrenoLeitao<leitao@debian.org>+*/++#define pr_fmt(fmt) "efi: " fmt++#include<linux/bitmap.h>+#include<linux/efi.h>+#include<linux/io.h>+#include<linux/log2.h>+#include<linux/memblock.h>+#include<linux/mm.h>+#include<linux/overflow.h>++staticstructlinux_efi_poisoned_memory*efi_poison__ro_after_init;+staticu64efi_poison_nbits__ro_after_init;++staticu64__init+efi_poison_usable_size(conststructlinux_efi_poisoned_memory*pm)+{+u64nr_units=DIV_ROUND_UP(PFN_PHYS(max_pfn),pm->unit_size);+u64bytes=DIV_ROUND_UP(nr_units,BITS_PER_BYTE);++/* Whole words: the bitmap is reached an unsigned long at a time. */+returnmin(round_up(bytes,sizeof(unsignedlong)),pm->size);+}++staticbool__init+efi_poison_geometry_valid(conststructlinux_efi_poisoned_memory*pm)+{+if(!pm->size||!IS_ALIGNED(pm->size,sizeof(unsignedlong)))+returnfalse;++returnpm->unit_size>=PAGE_SIZE&&is_power_of_2(pm->unit_size);+}++staticbool__init+efi_poison_range_valid(conststructlinux_efi_poisoned_memory*pm)+{+u64nbits,span;++if(check_mul_overflow(pm->size,(u64)BITS_PER_BYTE,&nbits))+returnfalse;++return!check_mul_overflow(nbits,(u64)pm->unit_size,&span);+}++/* The table may come from an earlier kernel, so vet it before using it. */+staticbool__init+efi_poison_table_valid(conststructlinux_efi_poisoned_memory*pm)+{+if(pm->version!=1){+pr_warn("Ignoring poisoned-memory table with version %u\n",+pm->version);+returnfalse;+}++if(!efi_poison_geometry_valid(pm)||!efi_poison_range_valid(pm)){+pr_warn("Ignoring malformed poisoned-memory table\n");+returnfalse;+}++returntrue;+}++staticint__initefi_poison_init(void)+{+structlinux_efi_poisoned_memory*pm;+u64size;++if(efi.poisoned_memory==EFI_INVALID_TABLE_ADDR)+return0;++pm=memremap(efi.poisoned_memory,sizeof(*pm),MEMREMAP_WB);+if(WARN_ON_ONCE(!pm))+return0;+if(!efi_poison_table_valid(pm)){+memunmap(pm);+return0;+}+size=efi_poison_usable_size(pm);+memunmap(pm);+if(!size)+return0;++efi_poison=memremap(efi.poisoned_memory,sizeof(*pm)+size,+MEMREMAP_WB);+if(WARN_ON_ONCE(!efi_poison))+return0;++efi_poison_nbits=size*BITS_PER_BYTE;+return0;+}+early_initcall(efi_poison_init);++staticlongefi_poison_unit(unsignedlongpfn)+{+u64unit=PFN_PHYS(pfn)/efi_poison->unit_size;++if(unit>=efi_poison_nbits)+return-1;+returnunit;+}++/*+*Abitisnevercleared:itstandsforawholeEFI_POISON_UNIT_SIZE,soan+*unpoisoncannottellwhethertheunitasawholeisgoodagain.+*/+voidefi_hwpoison_record_pfn(unsignedlongpfn)+{+longunit;++if(!efi_poison)+return;++unit=efi_poison_unit(pfn);+if(unit<0)+return;++set_bit(unit,efi_poison->bitmap);+}
@@ -1286,6 +1287,8 @@ static int action_result(unsigned long pfn, enum mf_action_page_type type,if(type!=MF_MSG_ALREADY_POISONED&&type!=MF_MSG_PFN_MAP){num_poisoned_pages_inc(pfn);update_per_node_mf_stats(pfn,result);+/* Only hard offlines are carried over to the next kernel. */+efi_hwpoison_record_pfn(pfn);}pr_err("%#lx: recovery action for %s: %s\n",
A kernel that inherits a record of hardware-poisoned frames from an
earlier kernel needs a way to apply it. memory_failure() does not fit: it
takes mf_mutex, prints a line per frame, and a 2M unit is 512 of them.
Add hwpoison_boot_pfn(). A frame that is free takes the same route
memory_failure() takes for a free page: take_page_off_buddy(), the flag,
the refcount and the counter. A frame the kernel is already sitting on is
only flagged, since it cannot be taken away from whoever reserved it.
free_pages_prepare() drops such a frame if it is ever handed back, so it
does not reach the allocator either way.
The accounting cannot go through num_poisoned_pages_inc(). Its per memory
block half divides by sections_per_block, which memory_dev_init() only
sets up from driver_init(), so where this runs it is still zero and the
boot dies on a divide by zero. Bump the global counter directly; the block
counters stay short by these frames.
Signed-off-by: Breno Leitao <leitao@debian.org>
---
include/linux/mm.h | 1 +
mm/memory-failure.c | 24 ++++++++++++++++++++++++
2 files changed, 25 insertions(+)
@@ -97,6 +97,30 @@ void num_poisoned_pages_sub(unsigned long pfn, long i)memblk_nr_poison_sub(pfn,i);}+staticvoidupdate_per_node_mf_stats(unsignedlongpfn,enummf_resultresult);++bool__inithwpoison_boot_pfn(unsignedlongpfn)+{+structpage*page=pfn_to_online_page(pfn);++if(!page||PageHWPoison(page))+returnfalse;++if(is_free_buddy_page(page)){+if(!take_page_off_buddy(page))+returnfalse;+page_ref_inc(page);+}elseif(!PageReserved(page)){+returnfalse;+}++SetPageHWPoison(page);+update_per_node_mf_stats(pfn,MF_RECOVERED);+atomic_long_inc(&num_poisoned_pages);++returntrue;+}+/***MF_ATTR_RO-Createsysfsentryforeachmemoryfailurestatistics.*@_name:nameofthefileintheperNUMAsysfsdirectory.
The bitmap an earlier kernel filled in rides the EFI system table into
this one, but nothing reads it back until now.
Add efi_offline_poisoned_memory(), which vets the table header, copies
the geometry out of it, and hands every frame of every set unit to
hwpoison_boot_pfn(). The bitmap runs to megabytes on a large machine, so
it is mapped and walked a page at a time rather than in one go.
Call it through hwpoison_init_boot() from mm_core_init() right after
memblock_free_all(), the first point at which the recorded frames have
struct pages. They end up in the state a frame poisoned by this kernel
would be in, so the placement check that already understands PG_hwpoison
covers them too.
Suggested-by: Kiryl Shutsemau <kas@kernel.org>
Signed-off-by: Breno Leitao <leitao@debian.org>
---
drivers/firmware/efi/poison.c | 113 +++++++++++++++++++++++++++++++++++++++++-
include/linux/efi.h | 2 +
include/linux/mm.h | 5 ++
mm/memory-failure.c | 6 +++
mm/mm_init.c | 1 +
5 files changed, 126 insertions(+), 1 deletion(-)
@@ -9,14 +9,20 @@#define pr_fmt(fmt) "efi: " fmt-#include<linux/bitmap.h>#include<linux/efi.h>#include<linux/io.h>#include<linux/log2.h>#include<linux/memblock.h>+#include<linux/minmax.h>#include<linux/mm.h>#include<linux/overflow.h>+structefi_poison_geometry{+u64table;/* phys address of the table */+u64unit_size;+u64bitmap_size;+};+staticstructlinux_efi_poisoned_memory*efi_poison__ro_after_init;staticu64efi_poison_nbits__ro_after_init;
@@ -124,3 +130,108 @@ void efi_hwpoison_record_pfn(unsigned long pfn)set_bit(unit,efi_poison->bitmap);}++staticbool__initefi_poison_read_geometry(u64ppm,+structefi_poison_geometry*g)+{+structlinux_efi_poisoned_memory*pm;+boolvalid;++pm=early_memremap(ppm,sizeof(*pm));+if(!pm){+pr_warn("Could not map poisoned-memory table\n");+returnfalse;+}++valid=efi_poison_table_valid(pm);+if(valid){+g->table=ppm;+g->unit_size=pm->unit_size;+g->bitmap_size=efi_poison_usable_size(pm);+}else{+/* Keep the runtime side off a table this pass rejected. */+efi.poisoned_memory=EFI_INVALID_TABLE_ADDR;+}++early_memunmap(pm,sizeof(*pm));++returnvalid;+}++staticunsignedlong__init+efi_poison_offline_unit(conststructefi_poison_geometry*g,u64unit)+{+unsignedlongpfn=PHYS_PFN(unit*g->unit_size);+unsignedlongi,nr_pages=0;++for(i=0;i<g->unit_size>>PAGE_SHIFT;i++)+nr_pages+=hwpoison_boot_pfn(pfn+i);++returnnr_pages;+}++staticlong__initefi_poison_walk_chunk(conststructefi_poison_geometry*g,+u64off,unsignedlong*nr_units)+{+u64chunk=min_t(u64,PAGE_SIZE,g->bitmap_size-off);+unsignedlongbit,nbits=chunk*BITS_PER_BYTE;+unsignedlong*map,nr_pages=0;++map=early_memremap(g->table+offsetof(structlinux_efi_poisoned_memory,+bitmap)+off,chunk);+if(!map)+return-1;++for_each_set_bit(bit,map,nbits){+nr_pages+=efi_poison_offline_unit(g,off*BITS_PER_BYTE+bit);+(*nr_units)++;+}++early_memunmap(map,chunk);++returnnr_pages;+}++staticlong__initefi_poison_walk(conststructefi_poison_geometry*g,+unsignedlong*nr_units)+{+unsignedlongnr_pages=0;+u64off;++for(off=0;off<g->bitmap_size;off+=PAGE_SIZE){+longnr=efi_poison_walk_chunk(g,off,nr_units);++if(nr<0){+pr_warn("Could not map poisoned-memory bitmap\n");+return-1;+}+nr_pages+=nr;+}++returnnr_pages;+}++void__initefi_offline_poisoned_memory(void)+{+structefi_poison_geometryg;+unsignedlongnr_units=0;+longnr_pages,expected;++if(efi.poisoned_memory==EFI_INVALID_TABLE_ADDR)+return;++if(!efi_poison_read_geometry(efi.poisoned_memory,&g))+return;++nr_pages=efi_poison_walk(&g,&nr_units);+if(nr_pages<0)+return;++if(nr_pages)+pr_info("poisoned %ld page(s) inherited across kexec\n",nr_pages);++expected=nr_units*(g.unit_size>>PAGE_SHIFT);+if(nr_pages<expected)+pr_warn("%ld inherited poisoned page(s) could not be taken out of use\n",+expected-nr_pages);+}
@@ -121,6 +121,12 @@ bool __init hwpoison_boot_pfn(unsigned long pfn)returntrue;}+/* The EFI table is the only source of inherited poison today. */+void__inithwpoison_init_boot(void)+{+efi_offline_poisoned_memory();+}+/***MF_ATTR_RO-Createsysfsentryforeachmemoryfailurestatistics.*@_name:nameofthefileintheperNUMAsysfsdirectory.
On Wed, Aug 26, 2026 at 05:03:52AM -0700, Breno Leitao wrote:
quoted hunk
@@ -1271,6 +1274,16 @@ struct linux_efi_memreserve { #define EFI_MEMRESERVE_COUNT(size) (((size) - sizeof(struct linux_efi_memreserve)) \ / sizeof_field(struct linux_efi_memreserve, entry[0]))+/* Bit N covers unit N of physical address space, counting from address 0. */+struct linux_efi_poisoned_memory {+ u32 version;+ u32 unit_size; /* bytes of phys space per bitmap bit */+ u64 size; /* bitmap size in bytes */+ unsigned long bitmap[];+};
struct efi_unaccepted_memory has 'phys_base'. Do we want it here too?
It can be helpful if physical memory starts very high for some reason.
I believe it is common for risc-v.
Tenstorrent Blackhole card has 4G of RAM based at 0x400030000000.
Counting from address 0 that is a 4M bitmap for 4G of RAM. :-/
--
Kiryl Shutsemau / Kirill A. Shutemov
On Wed, Aug 26, 2026 at 05:03:53AM -0700, Breno Leitao wrote:
A EFI config table can only be installed while boot services are still
up, so the stub has to create it; the running kernel can only flip bits
in a table that already exists.
Size the bitmap from the top of usable RAM, which efi_get_ram_top()
works out by walking the UEFI memory map, since the stub has no max_pfn.
Bit N covers unit N counting from address 0, so the table tracks
max_pfn. Memory the firmware hot-adds later sits above it and is not
carried across a kexec.
One table has to serve every architecture, so efi_get_ram_top() takes
the union of the memory types they turn into RAM: what setup_e820() maps
to E820_TYPE_RAM on x86, plus the EFI_ACPI_RECLAIM_MEMORY and
EFI_PERSISTENT_MEMORY that is_usable_memory() accepts on arm64. Sizing
wide only costs bitmap bytes; sizing narrow silently drops the records
for every frame above the top.
x86 and ARM doesn't seem to agree what RAM is. On x86 it is by ->type
and on ARM it is by ->attribute (anything WB/WC/WT).
is_usable_memory() only decides nomap. Everything is_memory() lets in
lands in memblock.memory, and max_pfn is PFN_DOWN(memblock_end_of_DRAM()),
so the top of RAM there can be a type that is not on your list.
--
Kiryl Shutsemau / Kirill A. Shutemov
I don't think it works with deferred page init.
With CONFIG_DEFERRED_STRUCT_PAGE_INIT the struct pages above
first_deferred_pfn are not initialized here. memmap_init_range() breaks
out at defer_init() and the rest is done by page_alloc_init_late(), way
past this point.
But I think the shape is wrong here, not just the placement. We should
flag poisoned pages on the first add to buddy, not after the buddy is
initialized.
__free_pages_core() already consults the unaccepted table. Adding youre
case there seems logical. And all three paths -- memblock_free_pages(),
deferred_free_pages() and generic_online_page() -- gets there.
No need in special-casing for deferred page init.
It also closes the window where the page allocator is live before the
frames are taken out of it.
And patch 4 seems to be redundant in such shape.
--
Kiryl Shutsemau / Kirill A. Shutemov
Ard,
On Wed, Aug 26, 2026 at 05:03:51AM -0700, Breno Leitao wrote:
Proposed Solution:
==================
Carry the poisoned frames to the next kernel in a new EFI configuration
table, LINUX_EFI_POISONED_MEMORY.
I'm updating the series to address Kiryl's feedback, and before posting
the next version I wanted to check whether this approach is fine with you
from the EFI side.
It mirrors what unaccepted memory already does, so there's nothing new
here, but I'd like to confirm you'd be Ok with it.
Thanks,
--breno
On Fri, Aug 28, 2026 at 02:47:39PM +0100, Kiryl Shutsemau wrote:
On Wed, Aug 26, 2026 at 05:03:52AM -0700, Breno Leitao wrote:
quoted
@@ -1271,6 +1274,16 @@ struct linux_efi_memreserve { #define EFI_MEMRESERVE_COUNT(size) (((size) - sizeof(struct linux_efi_memreserve)) \ / sizeof_field(struct linux_efi_memreserve, entry[0]))+/* Bit N covers unit N of physical address space, counting from address 0. */+struct linux_efi_poisoned_memory {+ u32 version;+ u32 unit_size; /* bytes of phys space per bitmap bit */+ u64 size; /* bitmap size in bytes */+ unsigned long bitmap[];+};
struct efi_unaccepted_memory has 'phys_base'. Do we want it here too?
It can be helpful if physical memory starts very high for some reason.
I believe it is common for risc-v.
Tenstorrent Blackhole card has 4G of RAM based at 0x400030000000.
Counting from address 0 that is a 4M bitmap for 4G of RAM. :-/
Sure, although this won't be used much right now, it will prepare the
structure for future expansion. It also shouldn't add much complexity to
the process.
I will get the structure like this now
/* Bit N covers the unit at @phys_base + N * @unit_size. */
struct linux_efi_poisoned_memory {
u32 version;
u32 unit_size; /* bytes of phys space per bitmap bit */
u64 phys_base; /* address bit 0 covers */
u64 size; /* bitmap size in bytes */
unsigned long bitmap[];
};
and then offset the entities in the bitmap by phys_base.
Thanks for the feedback,
--breno
On Fri, Aug 28, 2026 at 02:59:43PM +0100, Kiryl Shutsemau wrote:
On Wed, Aug 26, 2026 at 05:03:53AM -0700, Breno Leitao wrote:
quoted
A EFI config table can only be installed while boot services are still
up, so the stub has to create it; the running kernel can only flip bits
in a table that already exists.
Size the bitmap from the top of usable RAM, which efi_get_ram_top()
works out by walking the UEFI memory map, since the stub has no max_pfn.
Bit N covers unit N counting from address 0, so the table tracks
max_pfn. Memory the firmware hot-adds later sits above it and is not
carried across a kexec.
One table has to serve every architecture, so efi_get_ram_top() takes
the union of the memory types they turn into RAM: what setup_e820() maps
to E820_TYPE_RAM on x86, plus the EFI_ACPI_RECLAIM_MEMORY and
EFI_PERSISTENT_MEMORY that is_usable_memory() accepts on arm64. Sizing
wide only costs bitmap bytes; sizing narrow silently drops the records
for every frame above the top.
x86 and ARM doesn't seem to agree what RAM is. On x86 it is by ->type
and on ARM it is by ->attribute (anything WB/WC/WT).
is_usable_memory() only decides nomap. Everything is_memory() lets in
lands in memblock.memory, and max_pfn is PFN_DOWN(memblock_end_of_DRAM()),
so the top of RAM there can be a type that is not on your list.
Thanks. I will just ignore the memory types and get the base and top
based on the memory map descriptors.
Now that I need to get the phys base (from previous patch discussion),
I will need to get the range, doing something like:
static efi_status_t efi_get_ram_range(u64 *base, u64 *top)
{
struct efi_boot_memmap *map __free(efi_pool) = NULL;
u64 ram_base = ULLONG_MAX, ram_top = 0;
efi_status_t status;
int i, nr_desc;
status = efi_get_memory_map(&map, false);
if (status != EFI_SUCCESS)
return status;
nr_desc = map->map_size / map->desc_size;
for (i = 0; i < nr_desc; i++) {
efi_memory_desc_t *d;
d = efi_memdesc_ptr((unsigned long)map->map, map->desc_size, i);
ram_base = min(ram_base, d->phys_addr);
ram_top = max(ram_top,
d->phys_addr + d->num_pages * EFI_PAGE_SIZE);
}
if (!ram_top)
return EFI_NOT_FOUND;
*base = round_down(ram_base, EFI_POISON_UNIT_SIZE);
*top = round_up(ram_top, EFI_POISON_UNIT_SIZE);
return EFI_SUCCESS;
}
is it what you had in mind?
Thanks,
--breno
On Fri, Aug 28, 2026 at 03:11:10PM +0100, Kiryl Shutsemau wrote:
On Wed, Aug 26, 2026 at 05:03:54AM -0700, Breno Leitao wrote:
quoted
+#include <linux/memblock.h>
Is it leftover? I don't see any memblock usage.
Yes, good catch, I will remove.
quoted
+#include <linux/mm.h>+#include <linux/overflow.h>++static struct linux_efi_poisoned_memory *efi_poison __ro_after_init;+static u64 efi_poison_nbits __ro_after_init;++static u64 __init+efi_poison_usable_size(const struct linux_efi_poisoned_memory *pm)+{+ u64 nr_units = DIV_ROUND_UP(PFN_PHYS(max_pfn), pm->unit_size);+ u64 bytes = DIV_ROUND_UP(nr_units, BITS_PER_BYTE);++ /* Whole words: the bitmap is reached an unsigned long at a time. */+ return min(round_up(bytes, sizeof(unsigned long)), pm->size);
Hm. Why bother to clamp to max_pfn? What's what's wrong with just using
pm->size directly?
The clamp is a min(), so it can only ever make the kernel use less of
the table than the table claims, just a silly and cheap optimization.
That said, pm->size is more than enought for now, and we optimize later.
I will drop efi_poison_usable_size() completely and replace it by
pm->size.
Thanks,
--breno