Hi Thierry,

On Mon, Jul 14, 2014 at 07:44:53AM +0100, Thierry Reding wrote:

On Sun, Jul 13, 2014 at 10:43:41AM +0100, Will Deacon wrote:

quoted

My plan for the ARM SMMU driver is:

  (1) Change ->probe() to walk the device-tree looking for all masters with
      phandles back to the SMMU instance being probed

You and Rob mentioned this several times and I don't understand why the
SMMU needs to know all masters up front. Is this necessary because it
needs to program all registers at .probe() time and they can't be
reprogrammed subsequently? Or is this just some kind of optimization?

It's an optimization to reduce resource usage in the IOMMU, but one that
is *required* by certain platforms (e.g. Olav mentioned his 43-ID master
and Calxeda had something similar).

Basically, in order to perform an ->attach() for a master to a domain, we
need complete knowledge of the system so that we can avoid accidentally
attaching other masters to the same domain. The programming is done using
a form of StreamID wildcarding, so at this point we would need to have
parsed the entire DT to ensure our wildcard doesn't match other masters.

quoted

  (2) For each master, extract the Stream IDs and add them to the internal
      SMMU driver data structures (an rbtree per SMMU instance). For
      hotpluggable buses, we'll need a way for the bus controller to
      reserve a range of IDs -- this will likely be a later extension to
      the binding.

  (3) When we get an ->add() call, warn if it's a device we haven't seen
      and reject the addition.

It seems to me like this would be the logical place to parse stream IDs.

We could do that only if we were guaranteed to have an ->add() call for
*every* master before an ->attach() call for *any* master. I don't think
that is necessarily true.

You could for example have a case where some device tree contains a node
for which no driver will ever be loaded (for example because it hasn't
been built-in, or the device is never used and the module is therefore
never loaded). That's a situation that you cannot determine by simply
walking the device tree in the IOMMU's .probe().

Why not? If we're simply searching for phandles to the IOMMU, why does it
matter whether a driver is bound to the master?

I've always thought about IOMMU masters much in the same way as other
types of resources, such as memory or interrupts. In the rest of the
kernel we do carefully try to postpone allocation of these resources
until they are required, specifically so we don't waste resources when
they're unused.

See above, they are all required the moment anybody tries an ->attach().

That's also one of the reasons why I think associating an IOMMU with the
bus type is bad. Currently if an IOMMU driver thinks it should enable
translation for a given device, then there's no way for that device's
driver to opt out again. There may be reasons (performance, hardware
bugs, ...) for the driver to decide against using the IOMMU for
translation, but there's currently no way to do that if the IOMMU driver
disagrees.

Yes, we need a way to associate an IOMMU with a bus instance, but I think
that's a separate topic, no?

Will