Re: [PATCH 7/8] membarrier: Remove arm (32) support for SYNC_CORE

On Wed, Jun 16, 2021 at 02:22:27PM +0100, Russell King wrote:

On Wed, Jun 16, 2021 at 01:10:58PM +0200, Peter Zijlstra wrote:

quoted

On Wed, Jun 16, 2021 at 11:34:46AM +0100, Russell King (Oracle) wrote:

quoted

On Wed, Jun 16, 2021 at 12:20:06PM +0200, Peter Zijlstra wrote:

quoted

On Wed, Jun 16, 2021 at 12:16:27PM +0200, Peter Zijlstra wrote:

quoted

On Tue, Jun 15, 2021 at 08:21:12PM -0700, Andy Lutomirski wrote:

quoted

On arm32, the only way to safely flush icache from usermode is to call
cacheflush(2).  This also handles any required pipeline flushes, so
membarrier's SYNC_CORE feature is useless on arm.  Remove it.

So SYNC_CORE is there to help an architecture that needs to do something
per CPU. If I$ invalidation is broadcast and I$ invalidation also
triggers the flush of any uarch caches derived from it (if there are
any).

Incomplete sentence there: + then we don't need SYNC_CORE.

quoted

Now arm_syscall() NR(cacheflush) seems to do flush_icache_user_range(),
which, if I read things right, end up in arch/arm/mm/*.S, but that
doesn't consider cache_ops_need_broadcast().

Will suggests that perhaps ARM 11MPCore might need this due to their I$
flush maybe not being broadcast

If it leaves other cores with incoherent I cache, then that's already
a problem for SMP cores, since there could be no guarantee that the
modifications made by one core will be visible to some other core that
ends up running that code - and there is little option for userspace to
work around that except by pinning the thread making the modifications
and subsequently executing the code to a core.

That's where SYNC_CORE can help. Or you make sys_cacheflush() do a
system wide IPI.

If it's a problem, then it needs fixing. sys_cacheflush() is used to
implement GCC's __builtin___clear_cache(). I'm not sure who added this
to gcc.

I'm surprised that it works. I guess it's just luck that the thread
doing the code writing doesn't migrate before the sys_cacheflush() call.

quoted

quoted

The same is also true of flush_icache_range() - which is used when
loading a kernel module. In the case Will is referring to, these alias
to the same code.

Yes, cache_ops_need_broadcast() seems to be missing in more places.

Likely only in places where we care about I/D coherency - as the data
cache is required to be PIPT on these SMP platforms.

We had similar issue with the cache maintenance for DMA. The hack we
employed (in cache.S) is relying on the MESI protocol internals and
forcing a read/write for ownership before the D-cache maintenance.
Luckily ARM11MPCore doesn't do speculative data loads to trigger some
migration back.

The simpler fix for flush_icache_range() is to disable preemption, read
a word in a cacheline to force any dirty lines on another CPU to be
evicted and then issue the D-cache maintenance (for those cache lines
which are still dirty on the current CPU).

It's a hack that only works on ARM11MPCore. Newer MP cores are saner.

-- 
Catalin

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel