On Mon, Jun 7, 2021 at 10:16 AM Nick Kossifidis [off-list ref] wrote:

Στις 2021-06-07 03:04, Guo Ren έγραψε:

quoted

On Mon, Jun 7, 2021 at 2:14 AM Nick Kossifidis [off-list ref]
wrote:

quoted

Στις 2021-05-20 04:45, Guo Ren έγραψε:

quoted

On Wed, May 19, 2021 at 2:53 PM Christoph Hellwig [off-list ref] wrote:

quoted

On Tue, May 18, 2021 at 11:44:35PM -0700, Drew Fustini wrote:

quoted

This patch series looks like it might be useful for the StarFive JH7100
[1] [2] too as it has peripherals on a non-coherent interconnect. GMAC,
USB and SDIO require that the L2 cache must be manually flushed after
DMA operations if the data is intended to be shared with U74 cores [2].

Not too much, given that the SiFive lineage CPUs have an uncached
window, that is a totally different way to allocate uncached memory.

It's a very big MIPS smell. What's the attribute of the uncached
window? (uncached + strong-order/ uncached + weak, most vendors still
use AXI interconnect, how to deal with a bufferable attribute?) In
fact, customers' drivers use different ways to deal with DMA memory in
non-coherent SOC. Most riscv SOC vendors are from ARM, so giving them
the same way in DMA memory is a smart choice. So using PTE attributes
is more suitable.

See:
https://github.com/riscv/virtual-memory/blob/main/specs/611-virtual-memory-diff.pdf
4.4.1
The draft supports custom attribute bits in PTE.

Not only it doesn't support custom attributes on PTEs:

"Bits63–54 are reserved for future standard use and must be zeroed by
software for forward compatibility."

It also goes further to say that:

"if any of these bits are set, a page-fault exception is raised"

In RISC-V VM TG, A C-bit discussion is raised. So it's a comm idea to
support it.

The C-bit was recently dropped, there is a new proposal for Page Based
Memory Attributes (PBMT) that we can work on / push for.

C-bit still needs discussion, we shouldn't drop it directly.

quoted

Let Linux support custom PTE attributes won't get any side effect in
practice.

IMO:
We needn't waste a bit in PTE, but the custom idea in PTE reserved
bits is necessary. Because Allwinner D1 needs custom PTE bits in
reserved bits to work around.
So I recommend just remove the "C" bit in PTE, but allow vendors to
define their own PTE attributes in reserved bits. I've found a way to
compact different PTE attributes of different vendors during the Linux
boot stage. That means we still could use One Image for all vendors in
Linux

The spec is clear, those attributes are for standard use only, not for
custom/platform use. It's one thing to implement custom CMOs where the
ISA doesn't have anything for it and doesn't prevent you to do so (so
you are not violating anything, it's just a custom extension), and we
can hide them behind SBI calls etc, and another to violate the current
Privilege Spec by using bits on PTEs that are reserved for standard use
only. The intentions of the VM TG are clear, not only those bits are
reserved but if software uses them the hw will result a page fault in
future revisions of the spec. What's the idea here, to support
non-compliant implementations on mainline ?

Raise a page fault won't solve anything. We still need access to the
page with proper performance.

I'm sure you have a good
idea on how to make this work, but as long as it violates the spec it
can't go in IMHO.

We need PTEs to provide a non-coherency solution, and only CMO
instructions are not enough. We can't modify so many Linux drivers to
fit it.
From Linux non-coherency view, we need:
 - Non-cache + Strong Order PTE attributes to deal with drivers' DMA descriptors
 - Non-cache + weak order to deal with framebuffer drivers
 - CMO dma_sync to sync cache with DMA devices
 - Userspace icache_sync solution, which prevents calls to S-mode with
IPI fence.i. (Necessary to JIT java scenarios.)

All above are not in spec, but the real chips are done.
(Actually, these have been talked about for more than five years, we
still haven't the uniform idea.)

The idea of C-bit is really important for us which prevents our chips
violates the spec.

-- 
Best Regards
 Guo Ren

ML: https://lore.kernel.org/linux-csky/