Alexey, Fred:

On Fri, 2021-07-23 at 15:34 +1000, Alexey Kardashevskiy wrote:

On 22/07/2021 01:04, Frederic Barrat wrote:

quoted

On 21/07/2021 05:32, Alexey Kardashevskiy wrote:

quoted

quoted

quoted

+        struct iommu_table *newtbl;
+        int i;
+
+        for (i = 0; i < ARRAY_SIZE(pci->phb->mem_resources);
i++) {
+            const unsigned long mask = IORESOURCE_MEM_64 | 
IORESOURCE_MEM;
+
+            /* Look for MMIO32 */
+            if ((pci->phb->mem_resources[i].flags & mask) ==
IORESOURCE_MEM)
+                break;
+        }
+
+        if (i == ARRAY_SIZE(pci->phb->mem_resources))
+            goto out_del_list;

So we exit and do nothing if there's no MMIO32 bar?
Isn't the intent just to figure out the MMIO32 area to reserve
it 
when init'ing the table? In which case we could default to 0,0

I'm actually not clear why we are reserving this area on
pseries.

If we do not reserve it, then the iommu code will allocate DMA
pages 
from there and these addresses are MMIO32 from the kernel pov at 
least. I saw crashes when (I think) a device tried DMAing to the
top 
2GB of the bus space which happened to be a some other device's
BAR.

hmmm... then figuring out the correct range needs more work. We
could 
have more than one MMIO32 bar. And they don't have to be adjacent. 

They all have to be within the MMIO32 window of a PHB and we reserve
the 
entire window here.

quoted

I 
don't see that we are reserving any range on the initial table
though 
(on pseries).

True, we did not need to, as the hypervisor always took care of DMA
and 
MMIO32 regions to not overlap.

And in this series we do not (strictly speaking) need this either as 
phyp never allocates more than one window dynamically and that only 
window is always the second one starting from 0x800.0000.0000.0000.
It 
is probably my mistake that KVM allows a new window to start from 0 -
PAPR did not prohibit this explicitly.

And for the KVM case, we do not need to remove the default window as
KVM 
can pretty much always allocate as many TCE as the VM wants. But we 
still allow removing the default window and creating a huge one
instead 
at 0x0 as this way we can allow 1:1 for every single PCI device even
if 
it only allows 48 (or similar but less than 64bit) DMA. Hope this
makes 
sense. Thanks,

Thank you for this discussion, I got to learn a lot!

If I got this, no further change will be necessary, is that correct?

I am testing a v6, and I intend to send it soon.