Re: [PATCH v2 0/8] Cache coherency management subsystem

On July 10, 2025 12:11:13 PM PDT, dan.j.williams@intel.com wrote:

H. Peter Anvin wrote:
[..]

quoted

quoted

quoted

quoted

In the near term though, current CXL platforms that do not support
device-initiated-invalidate still need coarse cache management for that
original infrequent provisioning events. Folks that want to go further
and attempt frequent DCD events with WBINVD get to keep all the pieces.

I would strongly prefer those pieces to include WARNs and or worse.

That is fair. It is not productive for the CXL subsystem to sit back and
hope that people notice the destructive side-effects of wbinvd and hope
that leads to device changes.

This discussion has me reconsidering that yes, it would indeed be better
to clflushopt loop over potentially terabytes on all CPUs. That should
only be suffered rarely for the provisioning case, and for the DCD case
the potential add/remove events should be more manageable.

drm already has drm_clflush_pages() for bulk cache management, CXL
should just align on that approach.

Let's not be flippant; looping over terabytes could take *hours*. But those are hours during which the system is alive, and only one CPU needs to be looping.

Do not all CPUs need to perform the invalidation for L1 copies of the
line?

Not trying to be flippant, but if wbinvd is only a one-shot per Peter's
proposed policy and the system experiences another CXL reconfiguration
event, then looping is the only option or fail the memory plug event.

quoted

The other question is: what happens if memory is unplugged and then a
cache line evicted? I'm guessing that existing memory hotplug
solutions simply drop the writeback, since the OS knows there is no
valid memory there, and so any cached data is inherently worthless.

Right, the expectation is that unplug is always coordinated and that
surprise unplug is unsupported / might lead to system instability.

CLFLUSH goes through the cache coherency protocol and is therefore system wide, which WBINVD is not.