Starfive JH7110 and ESWIN EIC770X both have non cache-coherent
peripherals. On JH7110[1], GPU/VOUT/VPU/ISP are routed to the sys port,
making them not cache-coherent. On EIC770X, all peripherals are routed
to the sys port, and none is cache-coherent. To make drivers work on
such platforms, the standard solution is to use Svpbmt and map the DMA
buffer as uncacheable. However, neither SoC supports Svpbmt. Instead,
they map the system memory twice, as cached and uncached. The uncached
alias implicitly applies the uncacheable PMA. To support such platform,
a special form of Svpbmt, namely "XPbmtUC" is introduced in this patch.
It's a synthetical PTE format where a single bit (UC) is controlling
the cacheability and the bit position can be configured at runtime. It
is intended to model the physical memory aliasing with minimal effort.
On JH7110, it aligns perfectly with the HW, as the aliased UC region
happens to be offsetted by 2^34. Thus, configuring the XPbmtUC with
bit=32 (PPN is shifted by 2) is all that needs to be done.
On EIC770X, the aliased UC region is put to a awkward offset, and given
there can be 2 NUMA node (dual-die) with 2 separate memory regions and
their UC alias counterpart, we instead ask the firmware to provide a
thin-layer hypervisor to re-arrange the memory map. The XPbmtUC will be
enabled with bit=38, thus map all UC pages to 2^40 (the upper-half of
2^41), and the underlaying hypervisor will re-map the 2^40+ addresses
to the appropriate UC alias regions. (See description in PATCH 1/6)
We chose bit 38 (PPN bit 40) to make the 2-stage translation efficient.
Hypervisor can utilize Sv39x4 G-stage scheme, and map all pages as 1GB
huge page, consuming only the first-level page table (16KB total), and
several TLB entries. In practice, it's the firmware/bootloader that
configures XPbmtUC through device-tree, based on firmware capabilities,
and skip the enablement on stock firmware. This is tested on Hifive
Premier P550 with the modified OpenSBI[2]. It runs the host Linux in VS
mode, and provide the aforementioned remapping. The performance penalty
(if not running KVM in Linux) is minimal, as the CPU is never switched
to HS mode. A very slight, unavoidable, slow down is with the external
interrupt delivery. Due to the lack of AIA in EIC770X, all device irq
now needs to trap to M mode first, before forwarding to VS mode. The
overhead of running KVM in such setup is yet unknown, and may well be
noticeable, as all HS-qualified instructions will trap to M mode, and
there's also the extra cost of flushing G/VS-stage TLBs. I'm analyzing
it in parallel.
I'm aware there's an ongoing series that Samuel sent for physical
memory aliases. I haven't been following too closely, but if you're
worried about it touching to many areas, I hope my series can shed some
light on the problem. My change is very minimal and local, also fairly
easy to remove if we later decide deprecating it down the road.
[1] https://github.com/starfive-tech/JH7100_Docs/blob/main/JH7100%20Cache%20Coherence%20V1.0.pdf
[2] https://github.com/ganboing/opensbi/tree/eic77x-vspt-physalias-wip
Bo Gan (6):
riscv: Add a custom, simplified version of Svpbmt "XPbmtUC"
riscv: alternatives: support auipc+load pair
riscv: apply page table attribute bits for XPbmtUC
riscv: select RISCV_ISA_XPBMTUC in STARFIVE and ESWIN SoC
riscv: dts: starfive: jh7110: activate XPbmtUC
[TESTING-ONLY] riscv: dts: eswin: eic7700: activate XPbmtUC
arch/riscv/Kconfig | 12 ++++++++++++
arch/riscv/Kconfig.socs | 2 ++
arch/riscv/boot/dts/eswin/eic7700.dtsi | 1 +
arch/riscv/boot/dts/starfive/jh7110.dtsi | 1 +
arch/riscv/include/asm/errata_list.h | 17 +++++++++++++++--
arch/riscv/include/asm/hwcap.h | 1 +
arch/riscv/include/asm/insn.h | 8 ++++++++
arch/riscv/include/asm/pgtable-64.h | 17 ++++++++++++++++-
arch/riscv/kernel/alternative.c | 11 ++++++-----
arch/riscv/kernel/cpufeature.c | 8 ++++++++
arch/riscv/mm/pgtable.c | 7 +++++++
11 files changed, 77 insertions(+), 8 deletions(-)
--
2.34.1
On platforms that doesn't support Svpbmt or XTheadMae, SoC vendors
sometimes map the system memory twice in physical address space, one
as cached, and the other as uncached. Through the uncached window,
device drivers will be able to map DMA buffer for noncoherent devices.
Such setup is usually found in SoC with pre-Svpbmt Sifive cores.
Make use of such feature by modeling it as "XPbmtUC", a customized
version of Svpbmt, where a single bit in PTE is used for UC control.
There's no IO bit with such scheme, as it's assumed that the PMA
(usually hard-wired on these SoCs) will properly convey the strongly-
ordered, non-idempotent attribute of the MMIO region.
The enablement of such position of "XPbmtUC" is controlled by the
device-tree property "riscv,xpbmt-uncache-bit".
Example:
Starfive JH7110 (Sifive U74):
[0x0, 0x40000000) Low MMIO
[0x40000000, 0x2_40000000) Cached Mem
[0x4_40000000, 0x6_40000000) Uncached Mem UC+
[0x9_00000000, 0x9_d0000000) High MMIO
Device-tree:
riscv,xpbmt-uncache-bit = <32>;
Use PTE bit 32 (PPN bit 34) as UC (uncache) control to perfectly
match the memory map of the SoC.
ESWIN EIC770X (Sifive U84/P550):
[0x0, 0x20000000) Core Internal
[0x20000000, 0x40000000) Core Internal (Die 1)
[0x40000000, 0x60000000) Low MMIO
[0x60000000, 0x80000000) Low MMIO (Die 1)
[0x80000000, 0x10_80000000) Cached Mem
[0x20_00000000, 0x30_00000000) Cached Mem (Die 1)
[0x80_00000000, 0xa0_00000000) High MMIO
[0xa0_00000000, 0xc0_00000000) High MMIO (Die 1)
[0xc0_00000000, 0xd0_00000000) Uncached Mem
[0xe0_00000000, 0xf0_00000000) Uncached Mem (Die 1)
EIC770X is not directly compatible to this model, as the uncached
regions are offsetted, and the offset is different among the Dies
in the dual-die version (EIC7702). so we expect the firmware to
provide a thin layer of hypervisor to transparently re-map:
[0x80000000, 0x10_80000000) Cached Mem
[0x20_00000000, 0x30_00000000) Cached Mem (Die 1)
[0xc0_00000000, 0xd0_00000000) Uncached Mem <----------.
[0xe0_00000000, 0xf0_00000000) Uncached Mem (Die 1) <--+--.
[0x100_80000000, 0x110_80000000) Mem UC+ ----------------' |
[0x120_00000000, 0x130_00000000) Mem UC+ (Die 1) -----------'
With that, the firmware/bootloader can set the following at boot:
riscv,xpbmt-uncache-bit = <38>;
Signed-off-by: Bo Gan <redacted>
---
arch/riscv/Kconfig | 12 ++++++++++++
arch/riscv/include/asm/hwcap.h | 1 +
arch/riscv/include/asm/pgtable-64.h | 8 ++++++++
arch/riscv/kernel/cpufeature.c | 8 ++++++++
arch/riscv/mm/pgtable.c | 7 +++++++
5 files changed, 36 insertions(+)
Hey,
Gonna offer some feedback on the detail of what's been done in this
series, without providing any commentary on whether this is the correct
approach to take.
On Fri, Mar 13, 2026 at 01:44:02AM -0700, Bo Gan wrote:
On platforms that doesn't support Svpbmt or XTheadMae, SoC vendors
sometimes map the system memory twice in physical address space, one
as cached, and the other as uncached. Through the uncached window,
device drivers will be able to map DMA buffer for noncoherent devices.
Such setup is usually found in SoC with pre-Svpbmt Sifive cores.
Make use of such feature by modeling it as "XPbmtUC", a customized
version of Svpbmt, where a single bit in PTE is used for UC control.
There's no IO bit with such scheme, as it's assumed that the PMA
(usually hard-wired on these SoCs) will properly convey the strongly-
ordered, non-idempotent attribute of the MMIO region.
The enablement of such position of "XPbmtUC" is controlled by the
device-tree property "riscv,xpbmt-uncache-bit".
Firstly, the naming generally I take some exception to. If this is some
fake vendor extension for linux purposes, it needs to have "xlinux" in
it, like our xlinuxenvcfg does. It should also be consistent, don't use
"xpmbtuc" and "xpbmt-uncache-bit", pick one and stick to it.
Athough, I think I disagree fundamentally with this property, as it seems
to me like "software configuration" that shouldn't be permitted in
devicetree. Maybe I am misunderstanding, but the numbers you chose are
convenient, not set in stone by the specific hardware, right?
I'd be much more comfortable with adding xlinuxwhatever to
riscv,isa-extensions, to signal that a soc supports this stuff than with
a property for the bit itself. I suppose that bit information could then
come from a LUT in the vendor extensions, that a validate callback could
check (via root compatible) before enabling. There's not a super neat
way to do that at the moment though I don't think, code currently
expects that vendor extensions are in a different "namespace" to
standard ones, and this would blur the lines because it's not from a
specific vendor, nor is it a standard extension.
I guess, it could be done by keeping it as a standard number, but then
it's a bit trickier to neatly access the LUT while keeping it split
apart.
I know this means having to modify the kernel if there's a new device,
but I'm inclined to say "deal with it" because they could've done
something standard and opted not to.
Could also argue that this should be shoved into a sifive specific
thing, but I don't expect that they're the only ones with devices like
this that could benefit.
quoted hunk
Example:
Starfive JH7110 (Sifive U74):
[0x0, 0x40000000) Low MMIO
[0x40000000, 0x2_40000000) Cached Mem
[0x4_40000000, 0x6_40000000) Uncached Mem UC+
[0x9_00000000, 0x9_d0000000) High MMIO
Device-tree:
riscv,xpbmt-uncache-bit = <32>;
Use PTE bit 32 (PPN bit 34) as UC (uncache) control to perfectly
match the memory map of the SoC.
ESWIN EIC770X (Sifive U84/P550):
[0x0, 0x20000000) Core Internal
[0x20000000, 0x40000000) Core Internal (Die 1)
[0x40000000, 0x60000000) Low MMIO
[0x60000000, 0x80000000) Low MMIO (Die 1)
[0x80000000, 0x10_80000000) Cached Mem
[0x20_00000000, 0x30_00000000) Cached Mem (Die 1)
[0x80_00000000, 0xa0_00000000) High MMIO
[0xa0_00000000, 0xc0_00000000) High MMIO (Die 1)
[0xc0_00000000, 0xd0_00000000) Uncached Mem
[0xe0_00000000, 0xf0_00000000) Uncached Mem (Die 1)
EIC770X is not directly compatible to this model, as the uncached
regions are offsetted, and the offset is different among the Dies
in the dual-die version (EIC7702). so we expect the firmware to
provide a thin layer of hypervisor to transparently re-map:
[0x80000000, 0x10_80000000) Cached Mem
[0x20_00000000, 0x30_00000000) Cached Mem (Die 1)
[0xc0_00000000, 0xd0_00000000) Uncached Mem <----------.
[0xe0_00000000, 0xf0_00000000) Uncached Mem (Die 1) <--+--.
[0x100_80000000, 0x110_80000000) Mem UC+ ----------------' |
[0x120_00000000, 0x130_00000000) Mem UC+ (Die 1) -----------'
With that, the firmware/bootloader can set the following at boot:
riscv,xpbmt-uncache-bit = <38>;
Signed-off-by: Bo Gan <redacted>
---
arch/riscv/Kconfig | 12 ++++++++++++
arch/riscv/include/asm/hwcap.h | 1 +
arch/riscv/include/asm/pgtable-64.h | 8 ++++++++
arch/riscv/kernel/cpufeature.c | 8 ++++++++
arch/riscv/mm/pgtable.c | 7 +++++++
5 files changed, 36 insertions(+)
Hi Conor,
Thanks so much for the prompt review. See inline.
On 3/13/26 06:24, Conor Dooley wrote:
Hey,
Gonna offer some feedback on the detail of what's been done in this
series, without providing any commentary on whether this is the correct
approach to take.
On Fri, Mar 13, 2026 at 01:44:02AM -0700, Bo Gan wrote:
quoted
On platforms that doesn't support Svpbmt or XTheadMae, SoC vendors
sometimes map the system memory twice in physical address space, one
as cached, and the other as uncached. Through the uncached window,
device drivers will be able to map DMA buffer for noncoherent devices.
Such setup is usually found in SoC with pre-Svpbmt Sifive cores.
Make use of such feature by modeling it as "XPbmtUC", a customized
version of Svpbmt, where a single bit in PTE is used for UC control.
There's no IO bit with such scheme, as it's assumed that the PMA
(usually hard-wired on these SoCs) will properly convey the strongly-
ordered, non-idempotent attribute of the MMIO region.
The enablement of such position of "XPbmtUC" is controlled by the
device-tree property "riscv,xpbmt-uncache-bit".
Firstly, the naming generally I take some exception to. If this is some
fake vendor extension for linux purposes, it needs to have "xlinux" in
it, like our xlinuxenvcfg does. It should also be consistent, don't use
"xpmbtuc" and "xpbmt-uncache-bit", pick one and stick to it.
Makes sense. I can certainly change that to be conformant.
Athough, I think I disagree fundamentally with this property, as it seems
to me like "software configuration" that shouldn't be permitted in
devicetree. Maybe I am misunderstanding, but the numbers you chose are
convenient, not set in stone by the specific hardware, right?
For JH7110, the bit 32 (PPN bit 34) matches exactly with the HW. Meaning
toggling this bit would re-map the page to the uncached window, which
matches perfectly with the synthetic UC bit in the scheme.
For EIC770X, the bit 38 (PPN bit 40) is hand picked to be able to map all
physical memory space (40 bit), while making it very easy for the thin-
hypervisor, which can utilize Sv39x4 (41 bit) page scheme in G-stage.
I also considered the sbi call approach, where the kernel can query for
the support and position of the uncache bit. The thing is that JH7110
can just hard-code the bit without any changes to firmware, and I want
to have a consistent way for both SoC, thus the device-tree approach, to
let the EIC770X firmware/bootloader adding the property to dt at runtime.
Any better ideas?
I'd be much more comfortable with adding xlinuxwhatever to
riscv,isa-extensions, to signal that a soc supports this stuff than with
a property for the bit itself. I suppose that bit information could then
come from a LUT in the vendor extensions, that a validate callback could
check (via root compatible) before enabling. There's not a super neat
way to do that at the moment though I don't think, code currently
expects that vendor extensions are in a different "namespace" to
standard ones, and this would blur the lines because it's not from a
specific vendor, nor is it a standard extension.
I guess, it could be done by keeping it as a standard number, but then
it's a bit trickier to neatly access the LUT while keeping it split
apart.
I know this means having to modify the kernel if there's a new device,
but I'm inclined to say "deal with it" because they could've done
something standard and opted not to.
Could also argue that this should be shoved into a sifive specific
thing, but I don't expect that they're the only ones with devices like
this that could benefit.
I've thought about riscv,isa-extensions. The issue with that is that it's
a per-CPU thing, but I'm adding a global extension, and I don't want to
pollute the isa-extension string. Thus, I followed Samuel's approach --
He uses "riscv,physical-memory-regions" in the root node.
quoted
Example:
Starfive JH7110 (Sifive U74):
[0x0, 0x40000000) Low MMIO
[0x40000000, 0x2_40000000) Cached Mem
[0x4_40000000, 0x6_40000000) Uncached Mem UC+
[0x9_00000000, 0x9_d0000000) High MMIO
Device-tree:
riscv,xpbmt-uncache-bit = <32>;
Use PTE bit 32 (PPN bit 34) as UC (uncache) control to perfectly
match the memory map of the SoC.
ESWIN EIC770X (Sifive U84/P550):
[0x0, 0x20000000) Core Internal
[0x20000000, 0x40000000) Core Internal (Die 1)
[0x40000000, 0x60000000) Low MMIO
[0x60000000, 0x80000000) Low MMIO (Die 1)
[0x80000000, 0x10_80000000) Cached Mem
[0x20_00000000, 0x30_00000000) Cached Mem (Die 1)
[0x80_00000000, 0xa0_00000000) High MMIO
[0xa0_00000000, 0xc0_00000000) High MMIO (Die 1)
[0xc0_00000000, 0xd0_00000000) Uncached Mem
[0xe0_00000000, 0xf0_00000000) Uncached Mem (Die 1)
EIC770X is not directly compatible to this model, as the uncached
regions are offsetted, and the offset is different among the Dies
in the dual-die version (EIC7702). so we expect the firmware to
provide a thin layer of hypervisor to transparently re-map:
[0x80000000, 0x10_80000000) Cached Mem
[0x20_00000000, 0x30_00000000) Cached Mem (Die 1)
[0xc0_00000000, 0xd0_00000000) Uncached Mem <----------.
[0xe0_00000000, 0xf0_00000000) Uncached Mem (Die 1) <--+--.
[0x100_80000000, 0x110_80000000) Mem UC+ ----------------' |
[0x120_00000000, 0x130_00000000) Mem UC+ (Die 1) -----------'
With that, the firmware/bootloader can set the following at boot:
riscv,xpbmt-uncache-bit = <38>;
Signed-off-by: Bo Gan <redacted>
---
arch/riscv/Kconfig | 12 ++++++++++++
arch/riscv/include/asm/hwcap.h | 1 +
arch/riscv/include/asm/pgtable-64.h | 8 ++++++++
arch/riscv/kernel/cpufeature.c | 8 ++++++++
arch/riscv/mm/pgtable.c | 7 +++++++
5 files changed, 36 insertions(+)
On Fri, Mar 13, 2026 at 02:33:16PM -0700, Bo Gan wrote:
Hi Conor,
Thanks so much for the prompt review. See inline.
On 3/13/26 06:24, Conor Dooley wrote:
quoted
Hey,
Gonna offer some feedback on the detail of what's been done in this
series, without providing any commentary on whether this is the correct
approach to take.
On Fri, Mar 13, 2026 at 01:44:02AM -0700, Bo Gan wrote:
quoted
On platforms that doesn't support Svpbmt or XTheadMae, SoC vendors
sometimes map the system memory twice in physical address space, one
as cached, and the other as uncached. Through the uncached window,
device drivers will be able to map DMA buffer for noncoherent devices.
Such setup is usually found in SoC with pre-Svpbmt Sifive cores.
Make use of such feature by modeling it as "XPbmtUC", a customized
version of Svpbmt, where a single bit in PTE is used for UC control.
There's no IO bit with such scheme, as it's assumed that the PMA
(usually hard-wired on these SoCs) will properly convey the strongly-
ordered, non-idempotent attribute of the MMIO region.
The enablement of such position of "XPbmtUC" is controlled by the
device-tree property "riscv,xpbmt-uncache-bit".
Firstly, the naming generally I take some exception to. If this is some
fake vendor extension for linux purposes, it needs to have "xlinux" in
it, like our xlinuxenvcfg does. It should also be consistent, don't use
"xpmbtuc" and "xpbmt-uncache-bit", pick one and stick to it.
Makes sense. I can certainly change that to be conformant.
quoted
Athough, I think I disagree fundamentally with this property, as it seems
to me like "software configuration" that shouldn't be permitted in
devicetree. Maybe I am misunderstanding, but the numbers you chose are
convenient, not set in stone by the specific hardware, right?
For JH7110, the bit 32 (PPN bit 34) matches exactly with the HW. Meaning
toggling this bit would re-map the page to the uncached window, which
matches perfectly with the synthetic UC bit in the scheme.
What does "matches exactly with the hardware" mean? AFAICT, you picked
it because it was the best value, but you could also have picked another
less optimal value?
For EIC770X, the bit 38 (PPN bit 40) is hand picked to be able to map all
physical memory space (40 bit), while making it very easy for the thin-
hypervisor, which can utilize Sv39x4 (41 bit) page scheme in G-stage.
I also considered the sbi call approach, where the kernel can query for
the support and position of the uncache bit. The thing is that JH7110
can just hard-code the bit without any changes to firmware, and I want
to have a consistent way for both SoC, thus the device-tree approach, to
let the EIC770X firmware/bootloader adding the property to dt at runtime.
Any better ideas?
Is the only thing that's variable on your eic770x platform whether or
not the bit is enabled? Or are you looking to vary the bit depending on
the specific platform?
quoted
I'd be much more comfortable with adding xlinuxwhatever to
riscv,isa-extensions, to signal that a soc supports this stuff than with
a property for the bit itself. I suppose that bit information could then
come from a LUT in the vendor extensions, that a validate callback could
check (via root compatible) before enabling. There's not a super neat
way to do that at the moment though I don't think, code currently
expects that vendor extensions are in a different "namespace" to
standard ones, and this would blur the lines because it's not from a
specific vendor, nor is it a standard extension.
I guess, it could be done by keeping it as a standard number, but then
it's a bit trickier to neatly access the LUT while keeping it split
apart.
I know this means having to modify the kernel if there's a new device,
but I'm inclined to say "deal with it" because they could've done
something standard and opted not to.
Could also argue that this should be shoved into a sifive specific
thing, but I don't expect that they're the only ones with devices like
this that could benefit.
I've thought about riscv,isa-extensions. The issue with that is that it's
a per-CPU thing, but I'm adding a global extension, and I don't want to
Most of the extensions in that string are effectively global. There's no
need to worry about "polluting" it.
pollute the isa-extension string. Thus, I followed Samuel's approach --
He uses "riscv,physical-memory-regions" in the root node.
On Fri, Mar 13, 2026 at 02:33:16PM -0700, Bo Gan wrote:
quoted
Hi Conor,
Thanks so much for the prompt review. See inline.
On 3/13/26 06:24, Conor Dooley wrote:
quoted
Hey,
Gonna offer some feedback on the detail of what's been done in this
series, without providing any commentary on whether this is the correct
approach to take.
On Fri, Mar 13, 2026 at 01:44:02AM -0700, Bo Gan wrote:
quoted
On platforms that doesn't support Svpbmt or XTheadMae, SoC vendors
sometimes map the system memory twice in physical address space, one
as cached, and the other as uncached. Through the uncached window,
device drivers will be able to map DMA buffer for noncoherent devices.
Such setup is usually found in SoC with pre-Svpbmt Sifive cores.
Make use of such feature by modeling it as "XPbmtUC", a customized
version of Svpbmt, where a single bit in PTE is used for UC control.
There's no IO bit with such scheme, as it's assumed that the PMA
(usually hard-wired on these SoCs) will properly convey the strongly-
ordered, non-idempotent attribute of the MMIO region.
The enablement of such position of "XPbmtUC" is controlled by the
device-tree property "riscv,xpbmt-uncache-bit".
Firstly, the naming generally I take some exception to. If this is some
fake vendor extension for linux purposes, it needs to have "xlinux" in
it, like our xlinuxenvcfg does. It should also be consistent, don't use
"xpmbtuc" and "xpbmt-uncache-bit", pick one and stick to it.
Makes sense. I can certainly change that to be conformant.
quoted
Athough, I think I disagree fundamentally with this property, as it seems
to me like "software configuration" that shouldn't be permitted in
devicetree. Maybe I am misunderstanding, but the numbers you chose are
convenient, not set in stone by the specific hardware, right?
For JH7110, the bit 32 (PPN bit 34) matches exactly with the HW. Meaning
toggling this bit would re-map the page to the uncached window, which
matches perfectly with the synthetic UC bit in the scheme.
What does "matches exactly with the hardware" mean? AFAICT, you picked
it because it was the best value, but you could also have picked another
less optimal value?
quoted
For EIC770X, the bit 38 (PPN bit 40) is hand picked to be able to map all
physical memory space (40 bit), while making it very easy for the thin-
hypervisor, which can utilize Sv39x4 (41 bit) page scheme in G-stage.
I also considered the sbi call approach, where the kernel can query for
the support and position of the uncache bit. The thing is that JH7110
can just hard-code the bit without any changes to firmware, and I want
to have a consistent way for both SoC, thus the device-tree approach, to
let the EIC770X firmware/bootloader adding the property to dt at runtime.
Any better ideas?
Is the only thing that's variable on your eic770x platform whether or
not the bit is enabled? Or are you looking to vary the bit depending on
the specific platform?
It'll be "fixed" for eic770x if a thin-hypervisor re-mapping is enabled
underneath. It just so happens that the physical address space is 40 bits
(ignoring the 40bit+ upper uncached region for interleaved memory, which
we don't need when the "xpbmt-uc" is enabled anyway), and the hypervisor
can use Sv39x4 (also 41bit) to re-map everything.
The variation comes with different SoCs, JH7110 vs. EIC770X. I'd like to
make it a variable, to make a unified kernel binary boot on all SoCs, so
I need to fix the alternative logic for PC-relative instructions to read
from a global variable "xpbmtuc_bit/mask". Also I want to avoid adding
too many branches to the alternative macro.
quoted
quoted
I'd be much more comfortable with adding xlinuxwhatever to
riscv,isa-extensions, to signal that a soc supports this stuff than with
a property for the bit itself. I suppose that bit information could then
come from a LUT in the vendor extensions, that a validate callback could
check (via root compatible) before enabling. There's not a super neat
way to do that at the moment though I don't think, code currently
expects that vendor extensions are in a different "namespace" to
standard ones, and this would blur the lines because it's not from a
specific vendor, nor is it a standard extension.
I guess, it could be done by keeping it as a standard number, but then
it's a bit trickier to neatly access the LUT while keeping it split
apart.
I know this means having to modify the kernel if there's a new device,
but I'm inclined to say "deal with it" because they could've done
something standard and opted not to.
Could also argue that this should be shoved into a sifive specific
thing, but I don't expect that they're the only ones with devices like
this that could benefit.
I've thought about riscv,isa-extensions. The issue with that is that it's
a per-CPU thing, but I'm adding a global extension, and I don't want to
Most of the extensions in that string are effectively global. There's no
need to worry about "polluting" it.
Got it. So I can use something like "xlinuxpbmtuc38" in isa-string? (until
someone comes up with a better naming. Naming things is hard...)
quoted
pollute the isa-extension string. Thus, I followed Samuel's approach --
He uses "riscv,physical-memory-regions" in the root node.
On Fri, Mar 13, 2026 at 05:29:53PM -0700, Bo Gan wrote:
Hi Conor,
On 3/13/26 16:55, Conor Dooley wrote:
quoted
On Fri, Mar 13, 2026 at 02:33:16PM -0700, Bo Gan wrote:
quoted
Hi Conor,
Thanks so much for the prompt review. See inline.
On 3/13/26 06:24, Conor Dooley wrote:
quoted
Hey,
Gonna offer some feedback on the detail of what's been done in this
series, without providing any commentary on whether this is the correct
approach to take.
On Fri, Mar 13, 2026 at 01:44:02AM -0700, Bo Gan wrote:
quoted
On platforms that doesn't support Svpbmt or XTheadMae, SoC vendors
sometimes map the system memory twice in physical address space, one
as cached, and the other as uncached. Through the uncached window,
device drivers will be able to map DMA buffer for noncoherent devices.
Such setup is usually found in SoC with pre-Svpbmt Sifive cores.
Make use of such feature by modeling it as "XPbmtUC", a customized
version of Svpbmt, where a single bit in PTE is used for UC control.
There's no IO bit with such scheme, as it's assumed that the PMA
(usually hard-wired on these SoCs) will properly convey the strongly-
ordered, non-idempotent attribute of the MMIO region.
The enablement of such position of "XPbmtUC" is controlled by the
device-tree property "riscv,xpbmt-uncache-bit".
Firstly, the naming generally I take some exception to. If this is some
fake vendor extension for linux purposes, it needs to have "xlinux" in
it, like our xlinuxenvcfg does. It should also be consistent, don't use
"xpmbtuc" and "xpbmt-uncache-bit", pick one and stick to it.
Makes sense. I can certainly change that to be conformant.
quoted
Athough, I think I disagree fundamentally with this property, as it seems
to me like "software configuration" that shouldn't be permitted in
devicetree. Maybe I am misunderstanding, but the numbers you chose are
convenient, not set in stone by the specific hardware, right?
For JH7110, the bit 32 (PPN bit 34) matches exactly with the HW. Meaning
toggling this bit would re-map the page to the uncached window, which
matches perfectly with the synthetic UC bit in the scheme.
What does "matches exactly with the hardware" mean? AFAICT, you picked
it because it was the best value, but you could also have picked another
less optimal value?
quoted
For EIC770X, the bit 38 (PPN bit 40) is hand picked to be able to map all
physical memory space (40 bit), while making it very easy for the thin-
hypervisor, which can utilize Sv39x4 (41 bit) page scheme in G-stage.
I also considered the sbi call approach, where the kernel can query for
the support and position of the uncache bit. The thing is that JH7110
can just hard-code the bit without any changes to firmware, and I want
to have a consistent way for both SoC, thus the device-tree approach, to
let the EIC770X firmware/bootloader adding the property to dt at runtime.
Any better ideas?
Is the only thing that's variable on your eic770x platform whether or
not the bit is enabled? Or are you looking to vary the bit depending on
the specific platform?
It'll be "fixed" for eic770x if a thin-hypervisor re-mapping is enabled
underneath. It just so happens that the physical address space is 40 bits
(ignoring the 40bit+ upper uncached region for interleaved memory, which
we don't need when the "xpbmt-uc" is enabled anyway), and the hypervisor
can use Sv39x4 (also 41bit) to re-map everything.
The variation comes with different SoCs, JH7110 vs. EIC770X. I'd like to
make it a variable, to make a unified kernel binary boot on all SoCs, so
FWIW, I have no interest in things that are not multiplatform-safe, so
anything I've been suggesting has been with that in mind. When I was
talking about not conveying the bit via DT, but storing the value in the
kernel, I was still considering that the values would be stored for
specific soc compatibles.
To be honest, I'm not completely dead-set opposed to a property that has
the bit positioning, but any property being added for what is
effectively an erratum needs to pass a high bar when the info could be
gathered in another way. That the eic7700 one depends on firmware for
what the bit may be is points in your favour, since firmware variability
is part of what dt is there to do. The jh7110 is points against, since
it could be fished out of the errata handling code.
I need to fix the alternative logic for PC-relative instructions to read
from a global variable "xpbmtuc_bit/mask". Also I want to avoid adding
too many branches to the alternative macro.
quoted
quoted
quoted
I'd be much more comfortable with adding xlinuxwhatever to
riscv,isa-extensions, to signal that a soc supports this stuff than with
a property for the bit itself. I suppose that bit information could then
come from a LUT in the vendor extensions, that a validate callback could
check (via root compatible) before enabling. There's not a super neat
way to do that at the moment though I don't think, code currently
expects that vendor extensions are in a different "namespace" to
standard ones, and this would blur the lines because it's not from a
specific vendor, nor is it a standard extension.
I guess, it could be done by keeping it as a standard number, but then
it's a bit trickier to neatly access the LUT while keeping it split
apart.
I know this means having to modify the kernel if there's a new device,
but I'm inclined to say "deal with it" because they could've done
something standard and opted not to.
Could also argue that this should be shoved into a sifive specific
thing, but I don't expect that they're the only ones with devices like
this that could benefit.
I've thought about riscv,isa-extensions. The issue with that is that it's
a per-CPU thing, but I'm adding a global extension, and I don't want to
Most of the extensions in that string are effectively global. There's no
need to worry about "polluting" it.
Got it. So I can use something like "xlinuxpbmtuc38" in isa-string? (until
someone comes up with a better naming. Naming things is hard...)
I don't think the encoding of the bit should be in the name, otherwise
we'd need to many different variations, if using riscv,isa-extensions is
the approach that ends up being used.
quoted
quoted
pollute the isa-extension string. Thus, I followed Samuel's approach --
He uses "riscv,physical-memory-regions" in the root node.
On Fri, Mar 13, 2026 at 05:29:53PM -0700, Bo Gan wrote:
quoted
Hi Conor,
On 3/13/26 16:55, Conor Dooley wrote:
quoted
On Fri, Mar 13, 2026 at 02:33:16PM -0700, Bo Gan wrote:
quoted
Hi Conor,
Thanks so much for the prompt review. See inline.
On 3/13/26 06:24, Conor Dooley wrote:
quoted
Hey,
Gonna offer some feedback on the detail of what's been done in this
series, without providing any commentary on whether this is the correct
approach to take.
On Fri, Mar 13, 2026 at 01:44:02AM -0700, Bo Gan wrote:
quoted
On platforms that doesn't support Svpbmt or XTheadMae, SoC vendors
sometimes map the system memory twice in physical address space, one
as cached, and the other as uncached. Through the uncached window,
device drivers will be able to map DMA buffer for noncoherent devices.
Such setup is usually found in SoC with pre-Svpbmt Sifive cores.
Make use of such feature by modeling it as "XPbmtUC", a customized
version of Svpbmt, where a single bit in PTE is used for UC control.
There's no IO bit with such scheme, as it's assumed that the PMA
(usually hard-wired on these SoCs) will properly convey the strongly-
ordered, non-idempotent attribute of the MMIO region.
The enablement of such position of "XPbmtUC" is controlled by the
device-tree property "riscv,xpbmt-uncache-bit".
Firstly, the naming generally I take some exception to. If this is some
fake vendor extension for linux purposes, it needs to have "xlinux" in
it, like our xlinuxenvcfg does. It should also be consistent, don't use
"xpmbtuc" and "xpbmt-uncache-bit", pick one and stick to it.
Makes sense. I can certainly change that to be conformant.
quoted
Athough, I think I disagree fundamentally with this property, as it seems
to me like "software configuration" that shouldn't be permitted in
devicetree. Maybe I am misunderstanding, but the numbers you chose are
convenient, not set in stone by the specific hardware, right?
For JH7110, the bit 32 (PPN bit 34) matches exactly with the HW. Meaning
toggling this bit would re-map the page to the uncached window, which
matches perfectly with the synthetic UC bit in the scheme.
What does "matches exactly with the hardware" mean? AFAICT, you picked
it because it was the best value, but you could also have picked another
less optimal value?
quoted
For EIC770X, the bit 38 (PPN bit 40) is hand picked to be able to map all
physical memory space (40 bit), while making it very easy for the thin-
hypervisor, which can utilize Sv39x4 (41 bit) page scheme in G-stage.
I also considered the sbi call approach, where the kernel can query for
the support and position of the uncache bit. The thing is that JH7110
can just hard-code the bit without any changes to firmware, and I want
to have a consistent way for both SoC, thus the device-tree approach, to
let the EIC770X firmware/bootloader adding the property to dt at runtime.
Any better ideas?
Is the only thing that's variable on your eic770x platform whether or
not the bit is enabled? Or are you looking to vary the bit depending on
the specific platform?
It'll be "fixed" for eic770x if a thin-hypervisor re-mapping is enabled
underneath. It just so happens that the physical address space is 40 bits
(ignoring the 40bit+ upper uncached region for interleaved memory, which
we don't need when the "xpbmt-uc" is enabled anyway), and the hypervisor
can use Sv39x4 (also 41bit) to re-map everything.
The variation comes with different SoCs, JH7110 vs. EIC770X. I'd like to
make it a variable, to make a unified kernel binary boot on all SoCs, so
FWIW, I have no interest in things that are not multiplatform-safe, so
anything I've been suggesting has been with that in mind. When I was
talking about not conveying the bit via DT, but storing the value in the
kernel, I was still considering that the values would be stored for
specific soc compatibles.
To be honest, I'm not completely dead-set opposed to a property that has
the bit positioning, but any property being added for what is
effectively an erratum needs to pass a high bar when the info could be
gathered in another way. That the eic7700 one depends on firmware for
what the bit may be is points in your favour, since firmware variability
is part of what dt is there to do. The jh7110 is points against, since
it could be fished out of the errata handling code.
Even for JH7110, I don't think it can be handled through the errata. It
describes the errata of the core (if I'm not mistaken), and there can be
other SoCs using the same core with the same archid/impid, but maps the
peripherals differently, and the UC bit position doesn't apply there. I
think you are probably looking for "SoC level errata" handling. It's not
there AFAIK. Hence I guess both SoC cases point in favor of the dt prop?
quoted
I need to fix the alternative logic for PC-relative instructions to read
from a global variable "xpbmtuc_bit/mask". Also I want to avoid adding
too many branches to the alternative macro.
quoted
quoted
quoted
I'd be much more comfortable with adding xlinuxwhatever to
riscv,isa-extensions, to signal that a soc supports this stuff than with
a property for the bit itself. I suppose that bit information could then
come from a LUT in the vendor extensions, that a validate callback could
check (via root compatible) before enabling. There's not a super neat
way to do that at the moment though I don't think, code currently
expects that vendor extensions are in a different "namespace" to
standard ones, and this would blur the lines because it's not from a
specific vendor, nor is it a standard extension.
I guess, it could be done by keeping it as a standard number, but then
it's a bit trickier to neatly access the LUT while keeping it split
apart.
I know this means having to modify the kernel if there's a new device,
but I'm inclined to say "deal with it" because they could've done
something standard and opted not to.
Could also argue that this should be shoved into a sifive specific
thing, but I don't expect that they're the only ones with devices like
this that could benefit.
I've thought about riscv,isa-extensions. The issue with that is that it's
a per-CPU thing, but I'm adding a global extension, and I don't want to
Most of the extensions in that string are effectively global. There's no
need to worry about "polluting" it.
Got it. So I can use something like "xlinuxpbmtuc38" in isa-string? (until
someone comes up with a better naming. Naming things is hard...)
I don't think the encoding of the bit should be in the name, otherwise
we'd need to many different variations, if using riscv,isa-extensions is
the approach that ends up being used.
quoted
quoted
quoted
pollute the isa-extension string. Thus, I followed Samuel's approach --
He uses "riscv,physical-memory-regions" in the root node.
On Fri, Mar 13, 2026 at 10:06:42PM -0700, Bo Gan wrote:
quoted
To be honest, I'm not completely dead-set opposed to a property that has
the bit positioning, but any property being added for what is
effectively an erratum needs to pass a high bar when the info could be
gathered in another way. That the eic7700 one depends on firmware for
what the bit may be is points in your favour, since firmware variability
is part of what dt is there to do. The jh7110 is points against, since
it could be fished out of the errata handling code.
Even for JH7110, I don't think it can be handled through the errata. It
describes the errata of the core (if I'm not mistaken), and there can be
other SoCs using the same core with the same archid/impid, but maps the
peripherals differently, and the UC bit position doesn't apply there. I
think you are probably looking for "SoC level errata" handling. It's not
there AFAIK. Hence I guess both SoC cases point in favor of the dt prop?
I dunno, nothing wrong with checking the devicetree during the errata
"probe" code. Checks are not limited to imp/arch ids, can do ecalls etc
etc in there too, so looking at the root compatible would be possible.
Either way, if people like what you've done here generally (because
coming up with our own use of PTE bits could be controversial), and a
custom property of some sort is to be used, you need to provide a good
justification of why it is needed in the commit messages because you're
setting a precedent of being the first "extension" conjured up to suit
linux that would need that kind of functionality.
Need to demonstrate that it describes an aspect of the hardware, and
isn't being conjured up to configure software to use one out of several
possible values, that it may even be able to determine heuristically
from information already provided in the devicetree (like the root
compatible or a completely described memory node).
On Fri, Mar 13, 2026 at 10:06:42PM -0700, Bo Gan wrote:
quoted
quoted
To be honest, I'm not completely dead-set opposed to a property that has
the bit positioning, but any property being added for what is
effectively an erratum needs to pass a high bar when the info could be
gathered in another way. That the eic7700 one depends on firmware for
what the bit may be is points in your favour, since firmware variability
is part of what dt is there to do. The jh7110 is points against, since
it could be fished out of the errata handling code.
Even for JH7110, I don't think it can be handled through the errata. It
describes the errata of the core (if I'm not mistaken), and there can be
other SoCs using the same core with the same archid/impid, but maps the
peripherals differently, and the UC bit position doesn't apply there. I
think you are probably looking for "SoC level errata" handling. It's not
there AFAIK. Hence I guess both SoC cases point in favor of the dt prop?
I dunno, nothing wrong with checking the devicetree during the errata
"probe" code. Checks are not limited to imp/arch ids, can do ecalls etc
etc in there too, so looking at the root compatible would be possible.
Either way, if people like what you've done here generally (because
coming up with our own use of PTE bits could be controversial), and a
custom property of some sort is to be used, you need to provide a good
justification of why it is needed in the commit messages because you're
setting a precedent of being the first "extension" conjured up to suit
linux that would need that kind of functionality.
Need to demonstrate that it describes an aspect of the hardware, and
isn't being conjured up to configure software to use one out of several
possible values, that it may even be able to determine heuristically
from information already provided in the devicetree (like the root
compatible or a completely described memory node).
Got your point. I've moved away from DT and "extension" in v2 and made
it a sifive "errata", given that mapping memory twice through front/sys
port is more of a sifive concept, not something generic to other core
vendors. The DT is kept untouched, and the detection logic is done by
a LUT and sbi ecalls if not predefined. Link:
https://lore.kernel.org/linux-riscv/20260316060328.1173634-1-ganboing@gmail.com
Bo
@@ -262,6 +268,8 @@ __RISCV_INSN_FUNCS(c_ebreak, RVC_MASK_C_EBREAK, RVC_MATCH_C_EBREAK)__RISCV_INSN_FUNCS(ebreak,RVG_MASK_EBREAK,RVG_MATCH_EBREAK)__RISCV_INSN_FUNCS(sret,RVG_MASK_SRET,RVG_MATCH_SRET)__RISCV_INSN_FUNCS(fence,RVG_MASK_FENCE,RVG_MATCH_FENCE);+__RISCV_INSN_FUNCS(load,RVG_MASK_LOAD,RVG_MATCH_LOAD);+__RISCV_INSN_FUNCS(store,RVG_MASK_STORE,RVG_MATCH_STORE);/* special case to catch _any_ system instruction */static__always_inlineboolriscv_insn_is_system(u32code)
@@ -123,14 +123,15 @@ void riscv_alternative_fix_offsets(void *alt_ptr, unsigned int len,if(riscv_insn_is_auipc(insn)&&i<num_insn-1){u32insn2=riscv_instruction_at(alt_ptr+(i+1)*sizeof(u32));-if(!riscv_insn_is_jalr(insn2))+if(!riscv_insn_is_jalr(insn2)&&+!riscv_insn_is_load(insn2))continue;-/* if instruction pair is a call, it will use the ra register */-if(RV_EXTRACT_RD_REG(insn)!=1)+if(RV_EXTRACT_RD_REG(insn)!=RV_EXTRACT_RS1_REG(insn2))continue;-riscv_alternative_fix_auipc_jalr(alt_ptr+i*sizeof(u32),+/* insn2 use rd of insn as rs1, patch it */+riscv_alternative_fix_auipc_pair(alt_ptr+i*sizeof(u32),insn,insn2,patch_offset);i++;}
Apply the UC bit like Svpbmt and THEAD_MAE does. Also changed the
_PAGE_PFN_MASK definition to exclude the UC bit, as it's position
is now determined at runtime, and can be part of PPN.
Signed-off-by: Bo Gan <redacted>
---
arch/riscv/include/asm/errata_list.h | 17 +++++++++++++++--
arch/riscv/include/asm/pgtable-64.h | 9 ++++++++-
2 files changed, 23 insertions(+), 3 deletions(-)
On Fri, Mar 13, 2026 at 01:44:04AM -0700, Bo Gan wrote:
Apply the UC bit like Svpbmt and THEAD_MAE does. Also changed the
_PAGE_PFN_MASK definition to exclude the UC bit, as it's position
is now determined at runtime, and can be part of PPN.
Signed-off-by: Bo Gan <redacted>
This should be squashed with the patch adding detection and the Kconfig
option.
On Fri, Mar 13, 2026 at 01:44:04AM -0700, Bo Gan wrote:
quoted
Apply the UC bit like Svpbmt and THEAD_MAE does. Also changed the
_PAGE_PFN_MASK definition to exclude the UC bit, as it's position
is now determined at runtime, and can be part of PPN.
Signed-off-by: Bo Gan <redacted>
This should be squashed with the patch adding detection and the Kconfig
option.
Enable the XPbmtUC feature for Starfive and ESWIN SoC
Signed-off-by: Bo Gan <redacted>
---
arch/riscv/Kconfig.socs | 2 ++
1 file changed, 2 insertions(+)
On Fri, Mar 13, 2026 at 01:44:05AM -0700, Bo Gan wrote:
quoted hunk
Enable the XPbmtUC feature for Starfive and ESWIN SoC
Signed-off-by: Bo Gan <redacted>
---
arch/riscv/Kconfig.socs | 2 ++
1 file changed, 2 insertions(+)
On Fri, Mar 13, 2026 at 01:44:05AM -0700, Bo Gan wrote:
quoted
Enable the XPbmtUC feature for Starfive and ESWIN SoC
Signed-off-by: Bo Gan <redacted>
---
arch/riscv/Kconfig.socs | 2 ++
1 file changed, 2 insertions(+)
Set riscv,xpbmt-uncache-bit to 32 to match SoC memory map:
[0x0, 0x40000000) Low MMIO
[0x40000000, 0x2_40000000) Cached Mem
[0x4_40000000, 0x6_40000000) Uncached Mem UC+
[0x9_00000000, 0x9_d0000000) High MMIO
Signed-off-by: Bo Gan <redacted>
---
arch/riscv/boot/dts/starfive/jh7110.dtsi | 1 +
1 file changed, 1 insertion(+)
On Fri, Mar 13, 2026 at 01:44:06AM -0700, Bo Gan wrote:
Set riscv,xpbmt-uncache-bit to 32 to match SoC memory map:
[0x0, 0x40000000) Low MMIO
[0x40000000, 0x2_40000000) Cached Mem
[0x4_40000000, 0x6_40000000) Uncached Mem UC+
[0x9_00000000, 0x9_d0000000) High MMIO
Signed-off-by: Bo Gan <redacted>
What I want know is how this whole setup interacts with the existing
support that we have for these devices?
Samuel's patchetset removed from the devicetree all of the nodes related
to having two mappings of the same memory, and modified the existing
erratum to only be required for older devicetrees.
You've not removed them, only added a new property. The non-coherent
peripherals on jh7110 already work prior to this patchset, is there not
going to be funky behaviour with both of these things operating in
parallel?
On Fri, Mar 13, 2026 at 01:44:06AM -0700, Bo Gan wrote:
quoted
Set riscv,xpbmt-uncache-bit to 32 to match SoC memory map:
[0x0, 0x40000000) Low MMIO
[0x40000000, 0x2_40000000) Cached Mem
[0x4_40000000, 0x6_40000000) Uncached Mem UC+
[0x9_00000000, 0x9_d0000000) High MMIO
Signed-off-by: Bo Gan <redacted>
What I want know is how this whole setup interacts with the existing
support that we have for these devices?
Samuel's patchetset removed from the devicetree all of the nodes related
to having two mappings of the same memory, and modified the existing
erratum to only be required for older devicetrees.
You've not removed them, only added a new property. The non-coherent
peripherals on jh7110 already work prior to this patchset, is there not
going to be funky behaviour with both of these things operating in
parallel?
I just want to clarify that Samuel's change is not touching JH7110, but
*JH7100*. They are very similar chips, can can confuse people sometimes,
but JH7110 evolved to put more devices such as gmac/sdio/usb/pcie through
the front port to make them cache coherent. The left over noncoherent
device are dealt by the driver through cache flushes, I believe. That's
why we have out-of-box support for JH7110 even without Samuel's patch. On
the older JH7100, none of the peripheral is DMA coherent, so we either
use ERRATA_STARFIVE_JH7100, making kernel NONPORTABLE or Samuel's patch.
Refer to this pdf for a detailed comparison:
https://github.com/starfive-tech/JH7100_Docs/blob/main/JH7100%20Cache%20Coherence%20V1.0.pdf
My patch doesn't touch anything JH7100. It's an enhancement to JH7110 (the
newer chip). The issue is that even though basic peripherals are coherent,
the video subsystem, GPU/VPU/ISP... is still not. (See page 6 in that pdf)
Hence technically, we still need to deal with the same issue on JH7110.
E.g., part of the reason we can't have GPU driver working is because we
can't map a DMA page as uncached in userspace. There's no cache flush insn
in userspace, either. This change tries to solve the issue in a simple yet
elegant way.
There's no other changes to the device-tree required, just adding the prop
for JH7110, and things should just work. As for JH7100 (the older chip), I
don't plan to support it, as the cached/uncached region are offsetted by
0xf80000000, so it can't be modeled by a single UC bit (w/o hypervisor
re-mapping it underneath, and there's no H for JH71x0) The chip was also
superseded very quickly after the release of JH7110. I don't imagine too
many users demanding support.
On Fri, Mar 13, 2026 at 02:59:44PM -0700, Bo Gan wrote:
Hi Conor,
On 3/13/26 06:48, Conor Dooley wrote:
quoted
On Fri, Mar 13, 2026 at 01:44:06AM -0700, Bo Gan wrote:
quoted
Set riscv,xpbmt-uncache-bit to 32 to match SoC memory map:
[0x0, 0x40000000) Low MMIO
[0x40000000, 0x2_40000000) Cached Mem
[0x4_40000000, 0x6_40000000) Uncached Mem UC+
[0x9_00000000, 0x9_d0000000) High MMIO
Signed-off-by: Bo Gan <redacted>
What I want know is how this whole setup interacts with the existing
support that we have for these devices?
Samuel's patchetset removed from the devicetree all of the nodes related
to having two mappings of the same memory, and modified the existing
erratum to only be required for older devicetrees.
You've not removed them, only added a new property. The non-coherent
peripherals on jh7110 already work prior to this patchset, is there not
going to be funky behaviour with both of these things operating in
parallel?
I just want to clarify that Samuel's change is not touching JH7110, but
*JH7100*. They are very similar chips, can can confuse people sometimes,
but JH7110 evolved to put more devices such as gmac/sdio/usb/pcie through
the front port to make them cache coherent. The left over noncoherent
Believe it or not, I know that! I just misread the filename ;)
On Fri, Mar 13, 2026 at 01:44:01AM -0700, Bo Gan wrote:
Starfive JH7110 and ESWIN EIC770X both have non cache-coherent
peripherals. On JH7110[1], GPU/VOUT/VPU/ISP are routed to the sys port,
making them not cache-coherent. On EIC770X, all peripherals are routed
to the sys port, and none is cache-coherent. To make drivers work on
such platforms, the standard solution is to use Svpbmt and map the DMA
buffer as uncacheable. However, neither SoC supports Svpbmt. Instead,
they map the system memory twice, as cached and uncached. The uncached
alias implicitly applies the uncacheable PMA. To support such platform,
a special form of Svpbmt, namely "XPbmtUC" is introduced in this patch.
It's a synthetical PTE format where a single bit (UC) is controlling
the cacheability and the bit position can be configured at runtime. It
is intended to model the physical memory aliasing with minimal effort.
On JH7110, it aligns perfectly with the HW, as the aliased UC region
happens to be offsetted by 2^34. Thus, configuring the XPbmtUC with
bit=32 (PPN is shifted by 2) is all that needs to be done.
On EIC770X, the aliased UC region is put to a awkward offset, and given
there can be 2 NUMA node (dual-die) with 2 separate memory regions and
their UC alias counterpart, we instead ask the firmware to provide a
thin-layer hypervisor to re-arrange the memory map. The XPbmtUC will be
enabled with bit=38, thus map all UC pages to 2^40 (the upper-half of
2^41), and the underlaying hypervisor will re-map the 2^40+ addresses
to the appropriate UC alias regions. (See description in PATCH 1/6)
We chose bit 38 (PPN bit 40) to make the 2-stage translation efficient.
Hypervisor can utilize Sv39x4 G-stage scheme, and map all pages as 1GB
huge page, consuming only the first-level page table (16KB total), and
several TLB entries. In practice, it's the firmware/bootloader that
configures XPbmtUC through device-tree, based on firmware capabilities,
and skip the enablement on stock firmware. This is tested on Hifive
Premier P550 with the modified OpenSBI[2]. It runs the host Linux in VS
mode, and provide the aforementioned remapping. The performance penalty
(if not running KVM in Linux) is minimal, as the CPU is never switched
to HS mode. A very slight, unavoidable, slow down is with the external
interrupt delivery. Due to the lack of AIA in EIC770X, all device irq
now needs to trap to M mode first, before forwarding to VS mode. The
overhead of running KVM in such setup is yet unknown, and may well be
noticeable, as all HS-qualified instructions will trap to M mode, and
there's also the extra cost of flushing G/VS-stage TLBs. I'm analyzing
it in parallel.
I'm aware there's an ongoing series that Samuel sent for physical
memory aliases. I haven't been following too closely, but if you're
worried about it touching to many areas, I hope my series can shed some
light on the problem. My change is very minimal and local, also fairly
easy to remove if we later decide deprecating it down the road.
[1] https://github.com/starfive-tech/JH7100_Docs/blob/main/JH7100%20Cache%20Coherence%20V1.0.pdf
[2] https://github.com/ganboing/opensbi/tree/eic77x-vspt-physalias-wip
On Fri, Mar 13, 2026 at 01:44:01AM -0700, Bo Gan wrote:
quoted
Starfive JH7110 and ESWIN EIC770X both have non cache-coherent
peripherals. On JH7110[1], GPU/VOUT/VPU/ISP are routed to the sys port,
making them not cache-coherent. On EIC770X, all peripherals are routed
to the sys port, and none is cache-coherent. To make drivers work on
such platforms, the standard solution is to use Svpbmt and map the DMA
buffer as uncacheable. However, neither SoC supports Svpbmt. Instead,
they map the system memory twice, as cached and uncached. The uncached
alias implicitly applies the uncacheable PMA. To support such platform,
a special form of Svpbmt, namely "XPbmtUC" is introduced in this patch.
It's a synthetical PTE format where a single bit (UC) is controlling
the cacheability and the bit position can be configured at runtime. It
is intended to model the physical memory aliasing with minimal effort.
On JH7110, it aligns perfectly with the HW, as the aliased UC region
happens to be offsetted by 2^34. Thus, configuring the XPbmtUC with
bit=32 (PPN is shifted by 2) is all that needs to be done.
On EIC770X, the aliased UC region is put to a awkward offset, and given
there can be 2 NUMA node (dual-die) with 2 separate memory regions and
their UC alias counterpart, we instead ask the firmware to provide a
thin-layer hypervisor to re-arrange the memory map. The XPbmtUC will be
enabled with bit=38, thus map all UC pages to 2^40 (the upper-half of
2^41), and the underlaying hypervisor will re-map the 2^40+ addresses
to the appropriate UC alias regions. (See description in PATCH 1/6)
We chose bit 38 (PPN bit 40) to make the 2-stage translation efficient.
Hypervisor can utilize Sv39x4 G-stage scheme, and map all pages as 1GB
huge page, consuming only the first-level page table (16KB total), and
several TLB entries. In practice, it's the firmware/bootloader that
configures XPbmtUC through device-tree, based on firmware capabilities,
and skip the enablement on stock firmware. This is tested on Hifive
Premier P550 with the modified OpenSBI[2]. It runs the host Linux in VS
mode, and provide the aforementioned remapping. The performance penalty
(if not running KVM in Linux) is minimal, as the CPU is never switched
to HS mode. A very slight, unavoidable, slow down is with the external
interrupt delivery. Due to the lack of AIA in EIC770X, all device irq
now needs to trap to M mode first, before forwarding to VS mode. The
overhead of running KVM in such setup is yet unknown, and may well be
noticeable, as all HS-qualified instructions will trap to M mode, and
there's also the extra cost of flushing G/VS-stage TLBs. I'm analyzing
it in parallel.
I'm aware there's an ongoing series that Samuel sent for physical
memory aliases. I haven't been following too closely, but if you're
worried about it touching to many areas, I hope my series can shed some
light on the problem. My change is very minimal and local, also fairly
easy to remove if we later decide deprecating it down the road.
[1] https://github.com/starfive-tech/JH7100_Docs/blob/main/JH7100%20Cache%20Coherence%20V1.0.pdf
[2] https://github.com/ganboing/opensbi/tree/eic77x-vspt-physalias-wip
Thanks, Conor. I don't mean to step ahead of Samuel, just want to find a
middle ground that's easier to maintain from kernel perspective. I know
riscv HW is evolving rapidly, and we just don't want to add way too many
workarounds for each SoC. Hence I decided to move the re-mapping logic to
firmware/hypervisor to keep the kernel clean. If you'd like, use my v6.19
tree for testing on Hifive P550 if you have one:
https://github.com/ganboing/linux-eic77/tree/ganboing-xpbmt-uc-v1-eic77-clk-v15
I'm also sanity checking on my JH7110.
Bo