Re: [net-next][PATCH v2 1/2] rds: handle unsupported rdma request to fs dax memory
From: Jason Gunthorpe <jgg@ziepe.ca>
Date: 2019-05-10 19:20:50
On Fri, May 10, 2019 at 11:58:42AM -0700, santosh.shilimkar@oracle.com wrote:
On 5/10/19 11:07 AM, Jason Gunthorpe wrote:quoted
On Fri, May 10, 2019 at 11:02:35AM -0700, santosh.shilimkar@oracle.com wrote:quoted
On 5/10/19 10:55 AM, Jason Gunthorpe wrote:quoted
On Fri, May 10, 2019 at 09:11:24AM -0700, Santosh Shilimkar wrote:quoted
On 5/10/2019 5:54 AM, Jason Gunthorpe wrote:quoted
On Mon, Apr 29, 2019 at 04:37:19PM -0700, Santosh Shilimkar wrote:quoted
From: Hans Westgaard Ry <redacted> RDS doesn't support RDMA on memory apertures that require On Demand Paging (ODP), such as FS DAX memory. User applications can try to use RDS to perform RDMA over such memories and since it doesn't report any failure, it can lead to unexpected issues like memory corruption when a couple of out of sync file system operations like ftruncate etc. are performed.This comment doesn't make any sense..quoted
The patch adds a check so that such an attempt to RDMA to/from memory apertures requiring ODP will fail. Reviewed-by: Håkon Bugge <redacted> Reviewed-tested-by: Zhu Yanjun [off-list ref] Signed-off-by: Hans Westgaard Ry <redacted> Signed-off-by: Santosh Shilimkar <redacted> net/rds/rdma.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-)[...]quoted
Why would you need to detect FS DAX memory? GUP users are not supposed to care. GUP is supposed to work just 'fine' - other than the usual bugs we have with GUP and any FS backed memory.Am not saying there is any issue with GUP. Let me try to explain the issue first. You are aware of various discussions about doing DMA or RDMA on FS DAX memory. e.g [1] [2] [3] One of the proposal to do safely RDMA on FS DAX memory is/was ODP
It is not about safety. ODP is required in all places that would have used gup_longterm because ODP avoids th gup_longterm entirely.
Currently RDS doesn't have support for ODP MR registration and hence we don't want user application to do RDMA using fastreg/fmr on FS DAX memory which isn't safe.
No, it is safe. The only issue is you need to determine if this use of GUP is longterm or short term. Longterm means userspace is in control of how long the GUP lasts, short term means the kernel is in control. ie posting a fastreg, sending the data, then un-GUP'ing on completion is a short term GUP and it is fine on any type of memory. So if it is a long term pin then it needs to be corrected and the only thing the comment needs to explain is that it is a long term pin. Jason