On Thu, Aug 24, 2017 at 11:00 PM, Dan Williams [off-list ref] wrote:

On Thu, Aug 24, 2017 at 9:13 AM, Christoph Hellwig [off-list ref] wrote:

quoted

I'm still very unhappy about the get/set flag state.  What is the
reason you can't use/extend leases? (take a look at the fcntl
man page and look for Leases).  A variant of the concept is what
the pNFS block server uses.

So I think leases could potentially be extended to replace the inode
flag. A MAP_DIRECT operation would take out a lease that is broken by
break_layouts(). However, like the pNFS case the lease break would
need to held off while any DMA might be in-flight. We can use an
elevated page count as that indication as ZONE_DEVICE pages only ever
have an elevated page count in response to get_user_pages().

However, I think the only practical difference is turning an immediate
ETXTBSY response that S_IOMAP_SEALED provides into an indefinite
blocking wait for break_layouts() to complete. Can pNFS run
break_layouts() in bounded time?

As far I can see a lease and S_IOMAP_SEALED have the same DMA
cancelling problem, so a lease is not better in that regard. Absent an
overlaying protocol like pNFS, I think S_IOMAP_SEALED is cleaner
because it fails incompatible operations outright rather than stalls
them in break_layouts(). Were their other benefits to a lease over an
inode flag that you had in mind for this case where the protocol is
userspace defined? Maybe I'm thinking too small on the ways a lease
might be extended.

At a minimum I can at least use a new lease type as an indication of
when to bail out an block-map operation with ETXTBSY, and reuse the
lease security model. That way we at least start to converge the
in-kernel lease machinery for pinning blocks with this userspace
mechanism.