Re: [PATCH v2 7/7] pmem, dax: have direct_access use __pmem annotation

On Sat, 2015-08-15 at 08:44 -0700, Dan Williams wrote:

On Sat, Aug 15, 2015 at 2:19 AM, Christoph Hellwig [off-list ref] wrote:

quoted

On Thu, Aug 13, 2015 at 10:51:11AM -0600, Ross Zwisler wrote:

quoted

Update the annotation for the kaddr pointer returned by direct_access()
so that it is a __pmem pointer.  This is consistent with the PMEM driver
and with how this direct_access() pointer is used in the DAX code.

IFF we stick to the __pmem annotations this looks good.

That beeing said I start to really dislike them.  We don't special
accesors to read/write from pmem, we just need to explicitly commit
it if we want to make it persistent.  So I really don't see the need
to treat it special and require all the force casts to and from the
attribute.

I'm not going to put up much of a fight if it's really getting in the way....

That said, while we don't need special accessors we do need guarantees
that anything that has written to a persistent memory address has done
so in a way that wmb_pmem() is able to flush it.  It's more of a "I've
audited this code path for wmb_pmem() compatibility so use this api to
write to pmem."

Perhaps a better way to statically check for missed flushes might be
to have acquire_pmem_for_write() + release() annotations and the final
release does a wmb_pmem(), but as far as I can tell the sparse
acquire/release annotations don't stack.

FWIW I've been on the fence about the __pmem annotations, but my current
thought is that we really do need a way of saying that stores to these
pointers need special care for wmb_pmem() to do its thing and that __pmem does
a reasonably good job of that.  If we can figure out a cooler way, such as the
write() + release() flow Dan is talking about, great.  But I think we need
something to keep us from making errors by storing to PMEM pointers and
leaving data in the processor cache.