-----Original Message-----
From: Robin Murphy <robin.murphy@arm.com>
Date: Tue, Jun 28, 2016 at 11:18 AM
Subject: [RFC 1/2] iommu/dma: Restrict IOVAs to physical memory layout
To: iommu at lists.linux-foundation.org, linux-arm-kernel at lists.infradead.org
Certain peripherals may be bestowed with knowledge of the physical
memory map of the system in which they live, and refuse to handle
addresses that they do not think are memory, which causes issues when
remapping to arbitrary IOVAs. Sidestep the issue by restricting IOVA
domains to only allocate addresses within ranges which match the
physical memory layout.
Signed-off-by: Robin Murphy <robin.murphy@arm.com>
---
Posting this as an RFC because it's something I've been having to use
on Juno for all the PCI IOMMU development - it's pretty horrible, but I
can't easily think of a nicer solution...
Maybe I'm not getting the implications of this looking at the patch
in isolation, but how will this impact systems that have devices
limited to 32-bit addressing?
In our memory map we have physical memory regions at:
0x00_8000_0000
0x80_8000_0000
Will devices with a 32-bit DMA mask still get 32-bit IOVAs?
Thanks,
Stuart
On 01/07/16 17:03, Stuart Yoder wrote:
quoted
-----Original Message-----
From: Robin Murphy <robin.murphy@arm.com>
Date: Tue, Jun 28, 2016 at 11:18 AM
Subject: [RFC 1/2] iommu/dma: Restrict IOVAs to physical memory layout
To: iommu at lists.linux-foundation.org, linux-arm-kernel at lists.infradead.org
Certain peripherals may be bestowed with knowledge of the physical
memory map of the system in which they live, and refuse to handle
addresses that they do not think are memory, which causes issues when
remapping to arbitrary IOVAs. Sidestep the issue by restricting IOVA
domains to only allocate addresses within ranges which match the
physical memory layout.
Signed-off-by: Robin Murphy <robin.murphy@arm.com>
---
Posting this as an RFC because it's something I've been having to use
on Juno for all the PCI IOMMU development - it's pretty horrible, but I
can't easily think of a nicer solution...
Maybe I'm not getting the implications of this looking at the patch
in isolation, but how will this impact systems that have devices
limited to 32-bit addressing?
In our memory map we have physical memory regions at:
0x00_8000_0000
0x80_8000_0000
Will devices with a 32-bit DMA mask still get 32-bit IOVAs?
Assuming there's some free IOVA space between 0x80000000 and 0xffffffff,
yes, otherwise it gets nothing ;) This has no effect on the allocation
behaviour in general, it just makes sure that within that behaviour, we
avoid allocating any address that doesn't look "real". The primary issue
is with 64-bit DMA masks - since it's a top-down allocator, you
typically end up with the poor device issuing its first DMA transaction
to 0xfffffffffffff000 which on Juno a) gets silently eaten by the root
complex because it doesn't match any window in the PCI-AXI translation
table, or b) goes wrong anyway because it's beyond the input address
range of the SMMU (and there's something not quite right WRT
truncation/sign-extension which I've not looked into closely and am
semi-deliberately also sweeping under the rug thanks to the simpler
hardware issue...)
As I say, it's hideous, but I can't see what else to do.
Robin.
Thanks,
Stuart
On 01/07/16 17:15, Robin Murphy wrote:
On 01/07/16 17:03, Stuart Yoder wrote:
quoted
quoted
-----Original Message-----
From: Robin Murphy <robin.murphy@arm.com>
Date: Tue, Jun 28, 2016 at 11:18 AM
Subject: [RFC 1/2] iommu/dma: Restrict IOVAs to physical memory layout
To: iommu at lists.linux-foundation.org, linux-arm-kernel at lists.infradead.org
Certain peripherals may be bestowed with knowledge of the physical
memory map of the system in which they live, and refuse to handle
addresses that they do not think are memory, which causes issues when
remapping to arbitrary IOVAs. Sidestep the issue by restricting IOVA
domains to only allocate addresses within ranges which match the
physical memory layout.
Signed-off-by: Robin Murphy <robin.murphy@arm.com>
---
Posting this as an RFC because it's something I've been having to use
on Juno for all the PCI IOMMU development - it's pretty horrible, but I
can't easily think of a nicer solution...
Maybe I'm not getting the implications of this looking at the patch
in isolation, but how will this impact systems that have devices
limited to 32-bit addressing?
In our memory map we have physical memory regions at:
0x00_8000_0000
0x80_8000_0000
Will devices with a 32-bit DMA mask still get 32-bit IOVAs?
Assuming there's some free IOVA space between 0x80000000 and 0xffffffff,
yes, otherwise it gets nothing ;) This has no effect on the allocation
behaviour in general, it just makes sure that within that behaviour, we
avoid allocating any address that doesn't look "real". The primary issue
is with 64-bit DMA masks - since it's a top-down allocator, you
typically end up with the poor device issuing its first DMA transaction
to 0xfffffffffffff000 which on Juno a) gets silently eaten by the root
complex because it doesn't match any window in the PCI-AXI translation
table, or b) goes wrong anyway because it's beyond the input address
range of the SMMU (and there's something not quite right WRT
truncation/sign-extension which I've not looked into closely and am
semi-deliberately also sweeping under the rug thanks to the simpler
hardware issue...)
As I say, it's hideous, but I can't see what else to do.
Urgh, thinking some more, this is OK on Juno and LS2085 only because
there *is* some RAM below 4GB to begin with. On something like Seattle
where it's all high up, 32-bit peripherals will be as screwed as if the
IOMMU wasn't there :(
How on Earth do we work out which devices can only DMA to arbitrary
portions of their addressable range, and which are unrestricted? I guess
the reasonable answer is to use "dma-ranges" on the PCI RC to describe
the valid regions, but then we have to propagate that to the domain
somehow, and only after we've taught Linux how to handle multiple
dma-ranges entries at all (currently we'll just ignore everything other
than the last one). As for ACPI...
In short; oh dear.
Robin.
quoted
Thanks,
Stuart
_______________________________________________
iommu mailing list
iommu at lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
-----Original Message-----
From: Robin Murphy [mailto:robin.murphy at arm.com]
Sent: Friday, July 01, 2016 12:40 PM
To: Stuart Yoder <redacted>
Cc: iommu at lists.linux-foundation.org; linux-arm-kernel at lists.infradead.org
Subject: Re: [RFC 1/2] iommu/dma: Restrict IOVAs to physical memory layout
On 01/07/16 17:15, Robin Murphy wrote:
quoted
On 01/07/16 17:03, Stuart Yoder wrote:
quoted
quoted
-----Original Message-----
From: Robin Murphy <robin.murphy@arm.com>
Date: Tue, Jun 28, 2016 at 11:18 AM
Subject: [RFC 1/2] iommu/dma: Restrict IOVAs to physical memory layout
To: iommu at lists.linux-foundation.org, linux-arm-kernel at lists.infradead.org
Certain peripherals may be bestowed with knowledge of the physical
memory map of the system in which they live, and refuse to handle
addresses that they do not think are memory, which causes issues when
remapping to arbitrary IOVAs. Sidestep the issue by restricting IOVA
domains to only allocate addresses within ranges which match the
physical memory layout.
Signed-off-by: Robin Murphy <robin.murphy@arm.com>
---
Posting this as an RFC because it's something I've been having to use
on Juno for all the PCI IOMMU development - it's pretty horrible, but I
can't easily think of a nicer solution...
Maybe I'm not getting the implications of this looking at the patch
in isolation, but how will this impact systems that have devices
limited to 32-bit addressing?
In our memory map we have physical memory regions at:
0x00_8000_0000
0x80_8000_0000
Will devices with a 32-bit DMA mask still get 32-bit IOVAs?
Assuming there's some free IOVA space between 0x80000000 and 0xffffffff,
yes, otherwise it gets nothing ;) This has no effect on the allocation
behaviour in general, it just makes sure that within that behaviour, we
avoid allocating any address that doesn't look "real". The primary issue
is with 64-bit DMA masks - since it's a top-down allocator, you
typically end up with the poor device issuing its first DMA transaction
to 0xfffffffffffff000 which on Juno a) gets silently eaten by the root
complex because it doesn't match any window in the PCI-AXI translation
table, or b) goes wrong anyway because it's beyond the input address
range of the SMMU (and there's something not quite right WRT
truncation/sign-extension which I've not looked into closely and am
semi-deliberately also sweeping under the rug thanks to the simpler
hardware issue...)
As I say, it's hideous, but I can't see what else to do.
Urgh, thinking some more, this is OK on Juno and LS2085 only because
there *is* some RAM below 4GB to begin with. On something like Seattle
where it's all high up, 32-bit peripherals will be as screwed as if the
IOMMU wasn't there :(
Can the "Restrict IOVAs to physical memory layout" be a quirk type
property on the SMMU node for hardware that has this issue?
Then it is conditional and seems to only be needed by the Juno
platform, for now.
Stuart