On Fri, Sep 14, 2018 at 06:50:55AM +0000, Tian, Kevin wrote:

Date: Fri, 14 Sep 2018 06:50:55 +0000
From: "Tian, Kevin" <kevin.tian@intel.com>
To: Jerome Glisse <redacted>, Kenneth Lee <redacted>
CC: Kenneth Lee <redacted>, Alex Williamson
 [off-list ref], Herbert Xu [off-list ref],
 "kvm@vger.kernel.org" [off-list ref], Jonathan Corbet
 [off-list ref], Greg Kroah-Hartman [off-list ref], Zaibo
 Xu [off-list ref], "linux-doc@vger.kernel.org"
 [off-list ref], "Kumar, Sanjay K" [off-list ref],
 Hao Fang [off-list ref], "linux-kernel@vger.kernel.org"
 [off-list ref], "linuxarm@huawei.com"
 [off-list ref], "iommu@lists.linux-foundation.org"
 [off-list ref], "David S . Miller"
 [off-list ref], "linux-crypto@vger.kernel.org"
 [off-list ref], Zhou Wang [off-list ref],
 Philippe Ombredanne [off-list ref], Thomas Gleixner
 [off-list ref], Joerg Roedel [off-list ref],
 "linux-accelerators@lists.ozlabs.org"
 [off-list ref], Lu Baolu [off-list ref]
Subject: RE: [RFCv2 PATCH 0/7] A General Accelerator Framework, WarpDrive
Message-ID: [ref]

quoted

From: Jerome Glisse
Sent: Thursday, September 13, 2018 10:52 PM

[...]
 > AFAIK, on x86 and PPC at least, all PCIE devices are in the same group

quoted

by default at boot or at least all devices behind the same bridge.

the group thing reflects physical hierarchy limitation, not changed
cross boot. Please note iommu group defines the minimal isolation
boundary - all devices within same group must be attached to the
same iommu domain or address space, because physically IOMMU
cannot differentiate DMAs out of those devices. devices behind
legacy PCI-X bridge is one example. other examples include devices
behind a PCIe switch port which doesn't support ACS thus cannot
route p2p transaction to IOMMU. If talking about typical PCIe 
endpoint (with upstreaming ports all supporting ACS), you'll get
one device per group.

One iommu group today is attached to only one iommu domain.
In the future one group may attach to multiple domains, as the
aux domain concept being discussed in another thread.

quoted

Maybe they are kernel option to avoid that and userspace init program
can definitly re-arrange that base on sysadmin policy).

I don't think there is such option, as it may break isolation model
enabled by IOMMU.

[...]

quoted

quoted

quoted

That is why i am being pedantic :) on making sure there is good reasons
to do what you do inside VFIO. I do believe that we want a common

frame-

quoted

quoted

work like the one you are proposing but i do not believe it should be
part of VFIO given the baggages it comes with and that are not relevant
to the use cases for this kind of devices.

The purpose of VFIO is clear - the kernel portal for granting generic 
device resource (mmio, irq, etc.) to user space. VFIO doesn't care
what exactly a resource is used for (queue, cmd reg, etc.). If really
pursuing VFIO path is necessary, maybe such common framework
should lay down in user space, which gets all granted resource from
kernel driver thru VFIO and then provides accelerator services to 
other processes?

Yes. I think this is exactly what WarpDrive is now doing. This patch is just let
the type1 driver use parent IOMMU for mdev.

Thanks
Kevin

-- 
			-Kenneth(Hisilicon)

================================================================================
本邮件及其附件含有华为公司的保密信息，仅限于发送给上面地址中列出的个人或群组。禁
止任何其他人以任何形式使用（包括但不限于全部或部分地泄露、复制、或散发）本邮件中
的信息。如果您错收了本邮件，请您立即电话或邮件通知发件人并删除本邮件！
This e-mail and its attachments contain confidential information from HUAWEI,
which is intended only for the person or entity whose address is listed above.
Any use of the 
information contained herein in any way (including, but not limited to, total or
partial disclosure, reproduction, or dissemination) by persons other than the
intended 
recipient(s) is prohibited. If you receive this e-mail in error, please notify
the sender by phone or email immediately and delete it!