RE: [PATCH 3/5][v2] fsl-rio: Add two ports and rapidio message units support

On Tue, Oct 25, 2011 at 6:40 AM, Liu Gang-B34182 [off-list ref]
wrote:

quoted

... skip ...

quoted

+
+      DBELL_TID(dmsg),
+      DBELL_INF(dmsg));
+     break;
+    }
    }
   }

Do we need to check for matching DBELL_TID and mport destID here and
scan only doorbell list attached to the right port? Otherwise this

may

quoted

call service routine associated with doorbell on a wrong port.
[Liu Gang-B34182] Do you mean to match DBELL_TID and mport DevID?

=20
Yes.
=20

quoted

Usually this is a reliable method, but for the rapidio module of
powerpc, will encounter some problem. We set the port's DevID by the
register "Base Device ID CSR" defined in Rapidio Specification.

The

quoted

rapidio module of powerpc can support two ports but have only one

the

quoted

Base Device ID CSR. So these two ports will have the same DevID.
Although there are two registers "Alternate Device ID CSR" to

separate

quoted

deviceIDs for each port, they are specific registers of the

freescale

quoted

rapidio and can't be accessed by getting the extended

feature

quoted

space block offset. For this doobell issue, in order to call a right
service routine, perhaps we should ensure that different ports in
different "res->start and res->end" configurations.

=20
This gives us an issue that has to be solved at some point.
Splitting doorbell resources may be a way in this case but should be
considered in context of RIO network with different endpoints and
therefore it will be some device-specific quirk.
=20
Probably the best approach in this case is to keep doorbell handler as
it is now (for purpose of this patchset) and address doorbell resource
assignment during enumeration/discovery. At least this has to be well
commented.
[Liu Gang-B34182] I think this is a very complex issue, and could be
very difficult to be resolved if we can't separate DevIDs for each
port.
Based on the current architecture of the RIO driver, the doorbell
resource will be assigned by the application. For example, RIONET will
assign the doorbell resource like this:
	if ((rc =3D rio_request_inb_dbell(rnet->mport,
					(void *)ndev,
					RIONET_DOORBELL_JOIN,
					RIONET_DOORBELL_LEAVE,
					rionet_dbell_event)) < 0)
		goto out;
The res->start and res->end were defined by RIONET to
RIONET_DOORBELL_JOIN (0x1000) and RIONET_DOORBELL_LEAVE (0x1001). And
RIONET will send a doorbell
package like this:
	rio_send_doorbell(peer->rdev, RIONET_DOORBELL_JOIN);
When the destination port of this doorbell package has been assigned
the same res->start and res->end, it can work well. But when we try to
address doorbell resource assignment during enumeration/discovery, and
give the different doorbell resource to different port, how an

endpoint

to get the destination port resource it wants to send to a doorbell
package?
=20
In fact, I also encountered some other issues due to the two ports
sharing one CSR. For example the "Host base device ID lock command and
status register" and "Port General control command and status
register", these caused some issues when enumeration/discovery, and
very difficult to be resolved
based on current RIO architecture.

We need to adjust enumeration/discovery to deal with multi-port
endpoints.
As per RIO spec multi-port device has only one Base Device ID CSR but at
least offers a way to properly identify such capability.
Unfortunately p4080 controller follows older RIO 1.2 spec and does not
implement the multiport bit and CSR 0x14. This will force us to have a
quirk for devices with specific Device Identity CAR values.

One of possible ways for discovery may be ignoring value in reg.0x60 and
use destID set in the routing table of the attached switch (assuming
that
endpoint is capable to accept packets with destIDs other than one
in reg.0x60). But this alone does not guarantee that destIDs assigned to
two different ports will not match. This may force us to consider
additional measures of identification (which may be easier to implement
if 16-bit destIDs are used).

Anyway, we are not in position to resolve this immediately and probably
the best what we can do now is to keep multi-port endpoint specifics
in mind for future extensions.=20

I am slowly addressing multiple port support issues in
enumeration/discovery prompted by introduction of tsi721 PCIe-to-SRIO
bridge
and will keep multi-port endpoints in mind (this is different from
multiple
tsi721's on single board).=20

Regards,
=20
Alex.