Re: [patch 21/32] NTB/msi: Convert to msi_on_each_desc()

On 12/1/2021 11:41 AM, Thomas Gleixner wrote:

Dave,

please trim your replies.

On Wed, Dec 01 2021 at 09:28, Dave Jiang wrote:

quoted

On 12/1/2021 3:16 AM, Thomas Gleixner wrote:

quoted

Jason,

CC+ IOMMU folks

On Tue, Nov 30 2021 at 20:17, Jason Gunthorpe wrote:

quoted

On Tue, Nov 30, 2021 at 10:23:16PM +0100, Thomas Gleixner wrote:

Though I fear there is also a use case for MSI-X and IMS tied to the
same device. That network card you are talking about might end up using
MSI-X for a control block and then IMS for the actual network queues
when it is used as physical function device as a whole, but that's
conceptually a different case.

Hi Thomas. This is actually the IDXD usage for a mediated device passed
to a guest kernel when we plumb the pass through of IMS to the guest
rather than doing previous implementation of having a MSIX vector on
guest backed by IMS.

Which makes a lot of sense.

quoted

The control block for the mediated device is emulated and therefore an
emulated MSIX vector will be surfaced as vector 0. However the queues
will backed by IMS vectors. So we end up needing MSIX and IMS coexist
running on the guest kernel for the same device.

Why? What's wrong with using straight MSI-X for all of them?

The hardware implementation does not have enough MSIX vectors for 
guests. There are only 9 MSIX vectors total (8 for queues) and 2048 IMS 
vectors. So if we are to do MSI-X for all of them, then we need to do 
the IMS backed MSIX scheme rather than passthrough IMS to guests.

Thanks,

         tglx