-----Original Message-----
From: Jason Gunthorpe [mailto:jgg@ziepe.ca]
Sent: Tuesday, February 9, 2021 10:30 AM
To: Song Bao Hua (Barry Song) <redacted>
Cc: David Hildenbrand <redacted>; Wangzhou (B)
[off-list ref]; linux-kernel@vger.kernel.org;
iommu@lists.linux-foundation.org; linux-mm@kvack.org;
linux-arm-kernel@lists.infradead.org; linux-api@vger.kernel.org; Andrew
Morton [off-list ref]; Alexander Viro [off-list ref];
gregkh@linuxfoundation.org; kevin.tian@intel.com; jean-philippe@linaro.org;
eric.auger@redhat.com; Liguozhu (Kenneth) [off-list ref];
zhangfei.gao@linaro.org; chensihang (A) [off-list ref]
Subject: Re: [RFC PATCH v3 1/2] mempinfd: Add new syscall to provide memory
pin

On Mon, Feb 08, 2021 at 08:35:31PM +0000, Song Bao Hua (Barry Song) wrote:

quoted

quoted

From: Jason Gunthorpe [mailto:jgg@ziepe.ca]
Sent: Tuesday, February 9, 2021 7:34 AM
To: David Hildenbrand <redacted>
Cc: Wangzhou (B) <wangzhou1@hisilicon.com>; linux-kernel@vger.kernel.org;
iommu@lists.linux-foundation.org; linux-mm@kvack.org;
linux-arm-kernel@lists.infradead.org; linux-api@vger.kernel.org; Andrew
Morton [off-list ref]; Alexander Viro

[off-list ref];

quoted

quoted

gregkh@linuxfoundation.org; Song Bao Hua (Barry Song)
[off-list ref]; kevin.tian@intel.com;
jean-philippe@linaro.org; eric.auger@redhat.com; Liguozhu (Kenneth)
[off-list ref]; zhangfei.gao@linaro.org; chensihang (A)
[off-list ref]
Subject: Re: [RFC PATCH v3 1/2] mempinfd: Add new syscall to provide memory
pin

On Mon, Feb 08, 2021 at 09:14:28AM +0100, David Hildenbrand wrote:

quoted

People are constantly struggling with the effects of long term pinnings
under user space control, like we already have with vfio and RDMA.

And here we are, adding yet another, easier way to mess with core MM in

the

quoted

quoted

quoted

same way. This feels like a step backwards to me.

Yes, this seems like a very poor candidate to be a system call in this
format. Much too narrow, poorly specified, and possibly security
implications to allow any process whatsoever to pin memory.

I keep encouraging people to explore a standard shared SVA interface
that can cover all these topics (and no, uaccel is not that
interface), that seems much more natural.

I still haven't seen an explanation why DMA is so special here,
migration and so forth jitter the CPU too, environments that care
about jitter have to turn this stuff off.

This paper has a good explanation:
https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=7482091

mainly because page fault can go directly to the CPU and we have
many CPUs. But IO Page Faults go a different way, thus mean much
higher latency 3-80x slower than page fault:
events in hardware queue -> Interrupts -> cpu processing page fault
-> return events to iommu/device -> continue I/O.

The justifications for this was migration scenarios and migration is
short. If you take a fault on what you are migrating only then does it
slow down the CPU.

I agree this can slow down CPU, but not as much as IO page fault.

On the other hand, wouldn't it be the benefit of hardware accelerators
to have a lower and more stable latency zip/encryption than CPU?

Are you also working with HW where the IOMMU becomes invalidated after
a migration and doesn't reload?

ie not true SVA but the sort of emulated SVA we see in a lot of
places?

Yes. It is true SVA not emulated SVA.

It would be much better to work improve that to have closer sync with the
CPU page table than to use pinning.

Absolutely I agree improving IOPF and making IOPF catch up with the 
performance of page fault is the best way. but it would take much
long time to optimize both HW and SW. While waiting for them to
mature, probably some way which can minimize IOPF should be used to
take the responsivity.

Jason

Thanks
Barry

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel