On Wednesday 27 April 2011, Catalin Marinas wrote:
quoted
quoted
It's not broken since we moved to using Normal non-cacheable memory
for the coherent DMA buffers (as long as you flush the cacheable alias
before using the buffer, as we already do). The ARM ARM currently says
unpredictable for such situations but this is being clarified in
future updates and the Normal non-cacheable vs cacheable aliases can
be used (given correct cache maintenance before using the buffer).
Thanks for that information, I believe a number of people in the
previous discussions were relying on the information from the
documentation. Are you sure that this is not only correct for the
cores made by ARM ltd but also for the other implementations that
may have relied on documentation?
It is a clarification in the ARM ARM so it covers all the cores made by
architecture licensees, not just ARM Ltd. It basically makes the
"unpredictable" part more predictable to allow certain types of aliases
(e.g. Strongly Ordered vs Normal memory would still be disallowed).
All the current implementations are safe with Normal memory aliases
(cacheable vs non-cacheable) but of course, there may be some
performance benefits in not having any alias.
A lot of the discussions we are about to have in Budapest will be
around solving the problem of having only valid combinations of
mappings, so we really need to have a clear statement in specification
form about what is actually valid.
Would it be possible to have an updated version of the relevant
section of the ARM ARM by next week so we can use that as the
base for our discussions?
Arnd
On Fri, 29 Apr 2011 17:35:23 +1000
Benjamin Herrenschmidt [off-list ref] wrote:
quoted
I've been doing some thinking over the years on how we could extend that
functionality to other architectures. The reason we need those is
because some x86 processors (early AMDs and, I think VIA c3) dislike
multiple mappings of the same pages with conflicting caching attributes.
What we really want to be able to do is to unmap pages from the linear
kernel map, to avoid having to transition the linear kernel map every
time we change other mappings.
The reason we need to do this in the first place is that AGP and modern
GPUs has a fast mode where snooping is turned off.
Right. Unfortunately, unmapping pages from the linear mapping is
precisely what I cannot give you on powerpc :-(
This is due to our tendency to map it using the largest page size
available. That translates to things like:
- On hash based ppc64, I use 16M pages. I can't "break them up" due to
the limitation of the processor of having a single page size per segment
(and we use 1T segments nowadays). I could break the whole thing down to
4K but that would very seriously affect system performances.
- On embedded, I map it using 1G pages. I suppose I could break it up
since it's SW loaded but here too, system performance would suffer. In
addition, we rely on ppc32 embedded to have the first 768M of the linear
mapping and on ppc64 embedded, the first 1G, mapped using bolted TLB
entries, which we can really only do using very large entries
(respectively 256M and 1G) that can't be broken up.
So you need to make sure whatever APIs you come up with will work on
architectures where memory -has- to be cachable and coherent and you
cannot play with the linear mapping. But that won't help with our
non-coherent embedded systems :-(
You must be making it sound worse than it really is, otherwise how
would an embedded platform like the above deal with a display engine
that needed a large, contiguous chunk of uncached memory for the
display buffer? If the CPU is actively speculating into it and
overwriting blits etc it would never work... Or do you do such
reservations up front at 1G granularity??
Right. We should still shoot HW designers who give up coherency for the
sake of 3D benchmarks. It's insanely stupid.
Ah if it were that simple. :) There are big costs to implementing full
coherency for all your devices, as you well know, so it's just not a
question of benchmark optimization.
--
Jesse Barnes, Intel Open Source Technology Center
On Fri, 29 Apr 2011 08:59:58 +0100
Russell King - ARM Linux [off-list ref] wrote:
On Fri, Apr 29, 2011 at 07:50:12AM +0200, Thomas Hellstrom wrote:
quoted
However, we should be able to construct a completely generic api around
these operations, and for architectures that don't support them we need
to determine
a) Whether we want to support them anyway (IIRC the problem with PPC is
that the linear kernel map has huge tlb entries that are very
inefficient to break up?)
That same issue applies to ARM too - you'd need to stop the entire
machine, rewrite all processes page tables, flush tlbs, and only
then restart. Otherwise there's the possibility of ending up with
conflicting types of TLB entries, and I'm not sure what the effect
of having two matching TLB entries for the same address would be.
Right, I don't think anyone wants to see this sort of thing happen with
any frequency. So either a large, uncached region can be set up a boot
time for allocations, or infrequent, large requests and conversions can
be made on demand, with memory being freed back to the main, coherent
pool under pressure.
quoted
b) Whether they are needed at all on the particular architecture. The
Intel x86 spec is, (according to AMD), supposed to forbid conflicting
caching attributes, but the Intel graphics guys use them for GEM. PPC
appears not to need it.
Some versions of the architecture manual say that having multiple
mappings with differing attributes is unpredictable.
Yes, there's a bit of abuse going on there. We've received a guarantee
that if the CPU speculates a line into the cache, as long as it's not
modified through the cacheable mapping the CPU won't write it back to
memory; it'll discard the line as needed instead (iirc AMD CPUs will
actually write back clean lines, so GEM wouldn't work the same way
there).
But even with GEM, there is a large performance penalty for having to
allocate a new buffer object the first time. Even though we don't have
to change mappings by stopping the machine etc, we still have to flush
out everything from the CPU relating to the object (since some lines
may be dirty), and then flush the memory controller buffers before
accessing it through the uncached mapping. So at least currently,
we're all in the same boat when it comes to new object allocations:
they will be expensive unless you already have some uncached mappings
you can re-use.
--
Jesse Barnes, Intel Open Source Technology Center
On Friday, 29 April 2011, Arnd Bergmann [off-list ref] wrote:
On Wednesday 27 April 2011, Catalin Marinas wrote:
quoted
quoted
quoted
It's not broken since we moved to using Normal non-cacheable memory
for the coherent DMA buffers (as long as you flush the cacheable alias
before using the buffer, as we already do). The ARM ARM currently says
unpredictable for such situations but this is being clarified in
future updates and the Normal non-cacheable vs cacheable aliases can
be used (given correct cache maintenance before using the buffer).
Thanks for that information, I believe a number of people in the
previous discussions were relying on the information from the
documentation. Are you sure that this is not only correct for the
cores made by ARM ltd but also for the other implementations that
may have relied on documentation?
It is a clarification in the ARM ARM so it covers all the cores made by
architecture licensees, not just ARM Ltd. It basically makes the
"unpredictable" part more predictable to allow certain types of aliases
(e.g. Strongly Ordered vs Normal memory would still be disallowed).
All the current implementations are safe with Normal memory aliases
(cacheable vs non-cacheable) but of course, there may be some
performance benefits in not having any alias.
A lot of the discussions we are about to have in Budapest will be
around solving the problem of having only valid combinations of
mappings, so we really need to have a clear statement in specification
form about what is actually valid.
Would it be possible to have an updated version of the relevant
section of the ARM ARM by next week so we can use that as the
base for our discussions?
I'll ask the architecture people here in ARM and get back to you
(there is holiday until Tuesday next week in the UK).
--
Catalin
On Friday 29 April 2011 18:32:09 Jesse Barnes wrote:
On Fri, 29 Apr 2011 08:59:58 +0100
Russell King - ARM Linux [off-list ref] wrote:
quoted
On Fri, Apr 29, 2011 at 07:50:12AM +0200, Thomas Hellstrom wrote:
quoted
However, we should be able to construct a completely generic api around
these operations, and for architectures that don't support them we need
to determine
a) Whether we want to support them anyway (IIRC the problem with PPC is
that the linear kernel map has huge tlb entries that are very
inefficient to break up?)
That same issue applies to ARM too - you'd need to stop the entire
machine, rewrite all processes page tables, flush tlbs, and only
then restart. Otherwise there's the possibility of ending up with
conflicting types of TLB entries, and I'm not sure what the effect
of having two matching TLB entries for the same address would be.
Right, I don't think anyone wants to see this sort of thing happen with
any frequency. So either a large, uncached region can be set up a boot
time for allocations, or infrequent, large requests and conversions can
be made on demand, with memory being freed back to the main, coherent
pool under pressure.
I'd like to first have an official confirmation from the CPU designers
if there is actually a problem with mapping a single page both cacheable
and noncacheable. Based on what Catalin said, it's probably allowed and
the current spec is just being more paranoid than it needs to be. Also,
KyongHo Cho said that it might only be relevant for pages that are mapped
executable.
If that is the case, we can probably work around this by turning the entire
linear mapping (except for the kernel binary) into nonexecutable mode,
if we don't do that already.
This is desirable for security purposes anyway.
Arnd
From: Russell King - ARM Linux <hidden> Date: 2011-04-29 22:16:34
On Fri, Apr 29, 2011 at 08:29:50PM +0200, Arnd Bergmann wrote:
On Friday 29 April 2011 18:32:09 Jesse Barnes wrote:
quoted
On Fri, 29 Apr 2011 08:59:58 +0100
Russell King - ARM Linux [off-list ref] wrote:
quoted
On Fri, Apr 29, 2011 at 07:50:12AM +0200, Thomas Hellstrom wrote:
quoted
However, we should be able to construct a completely generic api around
these operations, and for architectures that don't support them we need
to determine
a) Whether we want to support them anyway (IIRC the problem with PPC is
that the linear kernel map has huge tlb entries that are very
inefficient to break up?)
That same issue applies to ARM too - you'd need to stop the entire
machine, rewrite all processes page tables, flush tlbs, and only
then restart. Otherwise there's the possibility of ending up with
conflicting types of TLB entries, and I'm not sure what the effect
of having two matching TLB entries for the same address would be.
Right, I don't think anyone wants to see this sort of thing happen with
any frequency. So either a large, uncached region can be set up a boot
time for allocations, or infrequent, large requests and conversions can
be made on demand, with memory being freed back to the main, coherent
pool under pressure.
I'd like to first have an official confirmation from the CPU designers
if there is actually a problem with mapping a single page both cacheable
and noncacheable.
Everytime this gets discussed, someone says that because they don't
believe what I say. OMAP folk confirmed it last time around.
I'm getting tired of this. I'm going to give up with answering any
further Linux questions until next week and I'll delete my entire
mailbox this weekend as I really can't be bothered to catch up with all
the crap that's happened over easter. I'm really getting pissed off at
all the shite crap that's flying around at the moment that I'm really
starting to not care one ounce about Linux, either on ARM or on this
utterly shite and broken x86 hardware.
Let ARM rot in mainline. I really don't care anymore.
From: Benjamin Herrenschmidt <benh@kernel.crashing.org> Date: 2011-04-29 22:38:01
On Fri, 2011-04-29 at 20:29 +0200, Arnd Bergmann wrote:
If that is the case, we can probably work around this by turning the
entire
linear mapping (except for the kernel binary) into nonexecutable mode,
if we don't do that already.
This is desirable for security purposes anyway
You'd still have an "edge" problem if you use large pages for the linear
mapping, you can't obviously make part of the kernel text NX and you'd
have to make sure you 'exclude' from those GPU allocations whatever
overlaps with your last executable large page.
In a way, it's a similar problem I have with bolted memory on BookE
where I can't restrict GPU allocations to memory that isn't bolted :-)
Cheers,
Ben.
From: Benjamin Herrenschmidt <benh@kernel.crashing.org> Date: 2011-04-29 22:47:15
On Fri, 2011-04-29 at 09:27 -0700, Jesse Barnes wrote:
You must be making it sound worse than it really is, otherwise how
would an embedded platform like the above deal with a display engine
that needed a large, contiguous chunk of uncached memory for the
display buffer? If the CPU is actively speculating into it and
overwriting blits etc it would never work... Or do you do such
reservations up front at 1G granularity??
Such embedded platforms have not been used with GPUs so far and our only
implementation of 64-bit BookE is fortunately also completely cache
coherent :-)
The good thing on ppc is that so far there is no new design coming from
us or FSL that isn't cache coherent. The bad thing is that people seem
to still try to pump out things using old 44x which isn't and somewhat
seem to also want to use GPUs on them :-)
The 44x is a case where I have a small (64 entries) SW loaded TLB and I
bolt the first 768M of the linear mapping (lowmem) using 3x256M entries.
What "saves" it is that it's also an ancient design with essentially a
busted prefetch engine that will thus cope with aliases as long as we
don't explicitely access the cached and non-cached aliases
simultaneously.
The nasty cases I have never really dealt with properly are the Apple
machines and their non coherent AGP. Those processors were really not
designed with the idea that one would do non-coherent DMA, especially
the 970 (G5) and our Linux code really don't like it.
Things tend to "work" with DRI 1 because we allocate the AGP memory once
in one big chunk (it's pages but they are allocated together and thus
tend to be contiguous) so the possible issues with prefetch are so rare,
I think we end up being lucky. With DRI 2 dynamically mapping things
in/out, we have a bigger problem and I don't know how to solve it other
than forcing the DRM to allocate graphic objects in reserved areas of
memory made of 16M pools that I unmap from the linear mapping.... (since
I use 16M pages to map the linear mapping).
For ppc32 laptops it's even worse as I use 256MB BATs (block address
translation, kind of special registers to create large static mappings)
to map the linear mapping, which brings me back to the 44x case to some
extent. I can't really do without at the moment, at the very least I
require the kernel text / data / bss to be covered by BATs.
quoted
Right. We should still shoot HW designers who give up coherency for the
sake of 3D benchmarks. It's insanely stupid.
Ah if it were that simple. :) There are big costs to implementing full
coherency for all your devices, as you well know, so it's just not a
question of benchmark optimization.
But it -is- that simple.
You do have to deal with coherency anyways for your PHB unless you start
advocating that we should make everything else non coherent as well. So
you have the logic. Just make your GPU operate on the same protocol.
It's really only a perf tradeoff I believe. And a bad one.
Cheers,
Ben.
From: Benjamin Herrenschmidt <benh@kernel.crashing.org> Date: 2011-04-29 22:51:23
On Fri, 2011-04-29 at 12:55 +0200, Thomas Hellstrom wrote:
On 04/29/2011 09:35 AM, Benjamin Herrenschmidt wrote:
quoted
We have problems with AGP and macs, we chose to mostly ignore them and
things have been working so-so ... with the old DRM. With DRI2 being
much more aggressive at mapping/unmapping things, things became a lot
less stable and it could be in part related to that. IE. Aliases are
similarily forbidden but we create them anyways.
Do you have any idea how other OS's solve this AGP issue on Macs?
Using a fixed pool of write-combined pages?
Write-combine is a different business, it's a matter of not mapping with
the G bit, but no, the way MacOS works I think is that they don't
actually use large pages at all, and I don't even think they have a
linear mapping of all memory. On the other hand they are slow :-)
quoted
quoted
c) If neither of the above applies, we might be able to either use
explicit cache flushes (which will require a TTM cache sync API), or
require the device to use snooping mode. The architecture may also
perhaps have a pool of write-combined pages that we can use. This should
be indicated by defines in the api header.
Right. We should still shoot HW designers who give up coherency for the
sake of 3D benchmarks. It's insanely stupid.
I agree. From a driver writer's perspective having the GPU always
snooping the system pages would be a dream. On the GPUs that do support
snooping that I have looked at, its internal MMU usually support both
modes, but the snooping mode is way slower (we're talking 50-70% or so
slower texturing operations), and often buggy causing crashes or scanout
timing issues since system designers apparently don't really count on it
being used. I've found it usable for device-to-system memory blits.
In addition memcpy to device is usually way faster if the destination is
write-combined. Probably due to cache thrashing effects.
Possibly. It's a matter of the HW folks actually spending some time to
make it work properly. It can be done :-) It's just that they don't
bother. Look at the perfs one can get out of fully coherent PCIe
nowadays, largely enough for a simple scanout :-)
Cheers,
Ben.
From: Benjamin Herrenschmidt <benh@kernel.crashing.org> Date: 2011-04-29 22:52:16
On Fri, 2011-04-29 at 12:56 +0100, Alan Cox wrote:
quoted
I believe that the PC graphics cards that have noncoherent DMA mappings
are all of the unified memory (integrated into the northbridge) kind,
so they are not on the same host bridge as all regular PCI devices,
even if they appear as a PCI device.
The AGP GART is not coherent on a lot of systems - not necessarily
unified memory though, it can be a plug in AGP card too.
The GART is basically an IOMMU (and indeed in the later AMD case used
exactly as that)
Right. Actually there's also the ability for PCIe devices to set a "no
snoop" bit on transactions and thus behave in a non-coherent manner.
Hopefully most sane PHBs ignore that bit ...
Cheers,
Ben.
From: Benjamin Herrenschmidt <benh@kernel.crashing.org> Date: 2011-04-29 22:53:55
On Fri, 2011-04-29 at 14:06 +0200, Thomas Hellstrom wrote:
I think Jerome has mentioned at one point that the Radeon graphics
cards support non-coherent mappings.
If the card is PCI/PCI-X/PCIe then coherency is not its business, it's
the business of the host bridge. However, on PCIe at least, the card can
indeed set a "no snoop" attribute on DMA transactions to request "no
coherency". At least the systems have the latitude to just ignore that
bit (like we do on all ppc afaik) :-)
Fwiw, the PowerVR SGX MMU also supports this mode of operation,
although it being functional I guess depends on the system
implementation.
Right, it's not a GPU thing, it's really a system design thing.
Cheers,
Ben.
From: Benjamin Herrenschmidt <benh@kernel.crashing.org> Date: 2011-04-29 22:55:22
On Fri, 2011-04-29 at 09:34 -0400, Jerome Glisse wrote:
Radeon memory controller can do non snooped pci transaction, as far as
i have tested most of the x86 pci bridge don't try to be coherent then
ie they don't analyze pci dma and ask for cpu flush they just perform
the request (and i guess it's what all bridge will do), so it endup
being noncoherent. I haven't done any benchmark of how faster it's for
the GPU when it's not snooping but i guess it can give 50% boost as it
likely drastictly reduce pci transaction overhead.
I am talking here about device that you plug into any pci or pcie
slot, so it's not igp integrated into northbridge or into the cpu.
Right, the card has nothing to do with the snooping process, it's purely
a feature of the bridge, based on a flag optionally set by the card. As
I said earlier, bridges have the freedom to ignore it, which we do on
ppc, so that's a non issue.
Cheers,
Ben.
On Sat, 30 Apr 2011 08:46:54 +1000
Benjamin Herrenschmidt [off-list ref] wrote:
quoted
Ah if it were that simple. :) There are big costs to implementing full
coherency for all your devices, as you well know, so it's just not a
question of benchmark optimization.
But it -is- that simple.
You do have to deal with coherency anyways for your PHB unless you start
advocating that we should make everything else non coherent as well. So
you have the logic. Just make your GPU operate on the same protocol.
It's really only a perf tradeoff I believe. And a bad one.
Ok so I was the one oversimplifying. :) Yes, it's definitely doable to
make a cache coherent PHB, and is awfully nice from a perf and
programming perspective.
But as you say, to make a high performance one for things like gfx, or
even to handle things like atomic ops, adds a lot of expense (in the
case of graphics, a whole lot unless you can integrate with the CPU,
and even then display can be tough to deal with).
I don't see even good coherent implementations being good enough for
high perf graphics in the near term (though at least on relatively high
power designs like Sandy Bridge we're getting close) so we'll have to
solve the uncached and simultaneous mapping issue both for today's
hardware and the near future.
--
Jesse Barnes, Intel Open Source Technology Center
On Wednesday 27 April 2011 12:43:16 Arnd Bergmann wrote:
On Wednesday 27 April 2011, Catalin Marinas wrote:
quoted
On 21 April 2011 20:29, Arnd Bergmann [off-list ref] wrote:
quoted
I think the recent discussions on linaro-mm-sig and the BoF last week
at ELC have been quite productive, and at least my understanding
of the missing pieces has improved quite a bit. This is a list of
things that I think need to be done in the kernel. Please complain
if any of these still seem controversial:
1. Fix the arm version of dma_alloc_coherent. It's in use today and
is broken on modern CPUs because it results in both cached and
uncached mappings. Rebecca suggested different approaches how to
get there.
It's not broken since we moved to using Normal non-cacheable memory
for the coherent DMA buffers (as long as you flush the cacheable alias
before using the buffer, as we already do). The ARM ARM currently says
unpredictable for such situations but this is being clarified in
future updates and the Normal non-cacheable vs cacheable aliases can
be used (given correct cache maintenance before using the buffer).
Thanks for that information, I believe a number of people in the
previous discussions were relying on the information from the
documentation. Are you sure that this is not only correct for the
cores made by ARM ltd but also for the other implementations that
may have relied on documentation?
As I mentioned before, there are other architectures, where having
conflicting cache settings in TLB entries for the same pysical page
immediately checkstops the CPU, and I guess that this was also allowed
by the current version of the ARM ARM.
quoted
quoted
2. Implement dma_alloc_noncoherent on ARM. Marek pointed out
that this is needed, and it currently is not implemented, with
an outdated comment explaining why it used to not be possible
to do it.
As Russell pointed out, there are 4 main combinations with iommu and
some coherency support (i.e. being able to snoop the CPU caches). But
in an SoC you can have different devices with different iommu and
coherency configurations. Some of them may even be able to see the L2
cache but not the L1 (in which case it would help if we can get an
inner non-cacheable outer cacheable mapping).
Anyway, we end up with different DMA ops per device via dev_archdata.
Having different DMA ops per device was the solution that I was suggesting
with dma_mapping_common.h, but Russell pointed out that it may not be
the best option.
The alternative would be to have just one set of dma_mapping functions
as we do today, but to extend the functions to also cover the iommu
case, for instance (example, don't take literally):
static inline dma_addr_t dma_map_single(struct device *dev, void *cpu_addr,
size_t size, enum dma_data_direction dir)
{
dma_addr_t ret;
#ifdef CONFIG_DMABOUNCE
if (dev->archdata.dmabounce)
return dmabounce_map_single(dev, cpu_addr, size, dir);
#endif
#ifdef CONFIG_IOMMU
if (dev->archdata.iommu)
ret = iommu_map_single(dev, cpu_addr, size, dir);
else
#endif
I wish it was that simple.
The OMAP4 ISS (Imaging Subsystem) has no IOMMU, but it can use the OMAP4 DMM
(Dynamic Memory Manager) which acts as a memory remapper. Basically (if my
understanding is correct), the ISS is configured to read/write from/to
physical addresses. If those physical addresses are in the DMM address range,
the DMM translates the accesses to physical accesses, acting as an IOMMU.
The ISS can thus write to physically contiguous memory directly, or to
scattered physical pages through the DMM. Whether an IOMMU (or, to be correct
in this case, the IOMMU-like DMM) needs to handle the DMA is a per-buffer
decision, not a per-device decision.
dma_addr = virt_to_dma(dev, ptr);
dma_sync_single_for_device(dev, dma_addr, size, dir);
}
This would not even conflict with having a common implementation
for iommu based dma_map_ops -- we would just call the iommu functions
directly when needed rather than having an indirect function call.
I wish it was that simple.
The OMAP4 ISS (Imaging Subsystem) has no IOMMU, but it can use the OMAP4 DMM
(Dynamic Memory Manager) which acts as a memory remapper. Basically (if my
understanding is correct), the ISS is configured to read/write from/to
physical addresses. If those physical addresses are in the DMM address range,
the DMM translates the accesses to physical accesses, acting as an IOMMU.
The ISS can thus write to physically contiguous memory directly, or to
scattered physical pages through the DMM. Whether an IOMMU (or, to be correct
in this case, the IOMMU-like DMM) needs to handle the DMA is a per-buffer
decision, not a per-device decision.
This doesn't sound too unusual for IOMMU implementations. A lot of time
you can access e.g. low memory using a direct mapping but you need the
IOMMU code for highmem. I've also seen a machine where a linear mapping
exists for all the memory in strict ordering, while you can use relaxed
DMA ordering when you go through the IOMMU address range. If we manage
to come up with a common dma-mapping API implementation for all IOMMUs,
it certainly needs to handle that case as well.
Arnd