On Fri, Nov 05, 2021 at 04:24:24PM +0800, Dongdong Liu wrote:

On 2021/11/4 0:02, Bjorn Helgaas wrote:

quoted

But it does remind me that if the RC doesn't support 10-bit tags, but
we use sysfs to enable 10-bit tags for a reqester that intends to use
P2PDMA to a peer that *does* support them, I don't think there's
any check in the DMA API that prevents the driver from setting up DMA
to the RC in addition to the peer.

Current we use sysfs to enable/disable 10-bit tags for a requester also
depend on the RP support 10-bit tag completer, so it will be ok.

Ah, OK.  So we can never *enable* 10-bit tags unless the Root Port
supports them.

I misunderstood the purpose of this file.  When the Root Port doesn't
support 10-bit tags, we won't enable them during enumeration.  I
though the point was that if we want to do P2PDMA to a peer that
*does* support them, we could use this file to enable them.

But my understanding was wrong -- the real purpose of the file is to
*disable* 10-bit tags for the case when a P2PDMA peer doesn't support
them.

It does support enabling 10-bit tags as well, but that's only because
we need a way to get back to the default "enabled during enumeration"
state without having to reboot.

We might be able to highlight this a little more in the commit log.

quoted

10-bit tag support appeared in the spec four years ago (PCIe r4.0, in
September, 2017).  Surely there is production hardware that supports
this and could demonstrate a benefit from this.

I found the below introduction about "Number of tags needed to achieve
maximum throughput for PCIe 4.0 and PCIe 5.0 links"
https://www.synopsys.com/designware-ip/technical-bulletin/accelerating-32gtps-pcie5-designs.html

It seems pretty clear.

Yes, that's a start.  But we don't really need a white paper to tell
us that more outstanding transactions is better.  That's obvious.  But
this adds risk, and if we can't demonstrate a tangible, measurable
benefit, there's no point in doing it.

Bjorn