On Tue, 2008-01-29 at 15:36 -0600, Olof Johansson wrote:

On Wed, Jan 30, 2008 at 08:18:15AM +1100, Benjamin Herrenschmidt wrote:

quoted

On Wed, 2008-01-30 at 02:13 +1100, Michael Ellerman wrote:

quoted

On Tue, 2008-01-29 at 09:15 -0600, Olof Johansson wrote:

quoted

On Wed, Jan 30, 2008 at 01:14:03AM +1100, Michael Ellerman wrote:

quoted

For example a machine with 4GB of memory would end up with the normal
IOMMU window from 0-2GB and the fixed mapping window from 2GB to 6GB. In
this case a 64-bit device wishing to DMA to 1GB would be told to DMA to
3GB, plus any offset required by firmware. The firmware offset is encoded
in the "dma-ranges" property.

Shouldn't the fixed mapping be between 4G and 8G (and the offset for 1G
is at 5G), to account for the MMIO range at 2-4G?

I don't think so, ie. it works setup like that, but I'm not entirely
sure why. Presumably the 2-4GB for MMIO is only for cycles heading out
of the CPU.

No no no... it's because on the PCI segment, it's all offset up
remember ?

Basically, the PCI host bridge on these has 2 interesting windows for
us:

0....2G			-> This goes up to memory @0 (via a couple of 
                           layers)

0x80*....0xF*		-> This goes untranslated to the PLB5 which
  			   drops the top bits and does some other 
                           manipulations, which allows to access, among
                           others the full 32GB of the cell inbound 
                           range.

The MMIO region of 2...4G is on the PCI (outbound from the Cell is yet
another range of addresses with different constraints but that ends up
generating cycles between 2 and 4G on the PCI segment).

If we had set the direct mapped region so that it uses 2G...N on PCI, we
would indeed be toast. But instead, the addresses for direct DMA that we
hand out to devices are in the 0x80* region and go hit the cell
directly, they never match MMIO.

Yeah, ok. That makes more sense. Thanks for the clarification.

Right, that's the firmware offset I mentioned in the changelog - 2am is
not a good time to think about these things.

Michael, btw, I wonder if it would make sense to duplicate the patch
description at the top of the file as well, since it'll be lost in the
change log for people who don't go back and read history, and having
the intentions documented in the file could be a good idea.

Yeah that probably makes sense. I've got most of the fixed mapping code
in a block, so I'll put a comment above that section. New patch coming.

cheers

-- 
Michael Ellerman
OzLabs, IBM Australia Development Lab

wwweb: http://michael.ellerman.id.au
phone: +61 2 6212 1183 (tie line 70 21183)

We do not inherit the earth from our ancestors,
we borrow it from our children. - S.M.A.R.T Person