On Sun, 2023-01-15 at 14:12 +0000, David Woodhouse wrote:

On Wed, 2020-08-26 at 13:16 +0200, Thomas Gleixner wrote:

quoted

From: Thomas Gleixner <redacted>

To allow utilizing the irq domain pointer in struct device it is necessary
to make XEN/MSI irq domain compatible.

While the right solution would be to truly convert XEN to irq domains, this
is an exercise which is not possible for mere mortals with limited XENology.

Provide a plain irqdomain wrapper around XEN. While this is blatant
violation of the irqdomain design, it's the only solution for a XEN igorant
person to make progress on the issue which triggered this change.

Signed-off-by: Thomas Gleixner <redacted>
Acked-by: Juergen Gross <jgross@suse.com>

I think it broke MSI-X support, because xen_pci_msi_domain_info is
lacking a .flags = MSI_FLAGS_PCI_MSIX?

Hm, I think it only actually *broke* with commit 99f3d27976 ("PCI/MSI:
Reject MSI-X early") from November last year. So 6.1 was OK and we have
time to fix it in 6.2.

Confirmed on real Xen at least that a Fedora 37 install with a 6.0.7
kernel works fine, then on upgrading to Rawhide's 6.2-rc3 it dies with 

[   41.498694] ena 0000:00:03.0 (unnamed net_device) (uninitialized): Failed to enable MSI-X. irq_cnt -524
[   41.498705] ena 0000:00:03.0: Can not reserve msix vectors
[   41.498712] ena 0000:00:03.0: Failed to enable and set the admin interrupts

quoted

---
Note: This is completely untested, but it compiles so it must be perfect.

I'm working on making it simple for you to test that, by hosting Xen
HVM guests natively in qemu (under KVM¹). 

But I'm absolutely not going to try hacking on both guest and host side
at the same time when I'm trying to ensure compatibility — that way
lies madness.

So for now I'm going to test qemu with older kernels, and maybe someone
(Jürgen}? can test MSI-X to PIRQ support under real Xen?) FWIW if I add
the missing MSI_FLAGS_PCI_MSIX flag then under my qemu I get:

 38:       3180          0  xen-pirq    -msi-x     ens4-rx-0
 39:          0       3610  xen-pirq    -msi-x     ens4-tx-0
 40:          1          0  xen-pirq    -msi-x     ens4

But without the flags I get:

[    8.464212] e1000e 0000:00:04.0 ens4: Failed to initialize MSI interrupts.  Falling back to legacy interrupts.

¹ https://lore.kernel.org/qemu-devel/20230110122042.1562155-1-dwmw2@infradead.org/ (local)