Re: [RFC net-next 0/6] psp: use virt cookie as Rx steering hint

Jakub Kicinski wrote:

On Sun, 23 Aug 2026 13:48:05 -0400 Willem de Bruijn wrote:

quoted

Jakub Kicinski wrote:

quoted

This PoC series uses a field of the PSP header intended for tunnels
to auto-steer Rx traffic. Various attempts have been made at trying
to get Rx traffic to land close to the core where the application runs.
By default RSS picks the Rx queue based on the flow hash.
I'm not going to cover all previous solutions in detail but broadly
- we have RFS in SW which looks on which CPU Tx happens and backlogs
Rx packets there, it is quite efficient. aRFS is built on top
of RFS but tries to program flows into the NIC. Some NICs have
a "cache" and try to automatically remember the flow to queue
association.

All those solutions are entirely local to the receiver.
Ideally we would want the solution to look something like
TCP timestamp option - we send an opaque cookie to the peer,
and the peer echoes it back to us. Our NIC can steer based
on that echoed cookie.  

Another option is to reverse RSS entropy. This requires knowledge of
the RSS secret.

Especially with PSP, the outer UDP source port is defined as flow
hash, so can be used for this.

Have the receiver compute a 4-tuple hash such that it knows the RSS
block will select the intended queue. The only free variable here in
general is the source port. Then communicate this preferred source
port to the sender.

Yes, we toyed with this a little. Communicating the hash to remote
is far less trivial. The thing that made me switch to the PSP idea

No need to communicate the hash, just the source port.

is that modern NICs can RSS on the flow label. Which is nice for
non-steering capable clients, and it conflicts with pre-computing.
Maybe the benefit is not big enough to matter.

Do you have any practical experience deploying reverse RSS?
Maybe I shied away from it too quickly..

Definitely implemented and it is quite straightforward. Using the
probing approach that Cosmin mentioned. No fancy algebraic solution,
though that would be preferable if has faster convergence.

Not sure whether we ever actually deployed it.
 

quoted

quoted

This patch set implements exactly that using the optional PSP
Virtualization Cookie field. The PSP standard doesn't have much
to say about this field:

 Virtualization Cookie - 64b
 An optional field, present if and only if V is set.
 It may contain a Virtual Network Identifier (VNI) or other data,
 as defined by the implementation.  

If using the PSP option space, no need to (ab)use the VC:

"When the Hdr Ext Len is greater than 1, a Virtualization Cookie
and/or other header extension fields may be present.  The presence of
a Virtualization Cookie is determined by the state of the V bit.  For
example, given Hdr Ext Len of 3, when V bit is set, there is an 8B VC
after the IV, and another 8B extension header after the VC.  The
format of other header extension fields is determined by the
applications and is opaque to the PSP hardware. "

Right now all use besides VC is opaque to the device.

For use-cases that should be interoperable across vendors, we should
probably define a standard option space, with well defined options.

In a manner that is cheap to parse at high rate, so no arbitrary order
variable length headers.

TBH I don't see the need for this at all. VC is well defined, and fed
into TCAMs. IMHO classifying the use of the VC as VNI vs steering tag
can be left entirely to SW / association state. The non-V options
should remain completely opaque to the HW IMHO.

The spec should have been more precise than "It may contain a Virtual
Network Identifier (VNI) or other data, as defined by the
implementation.". That second part is a pretty big loophole.

VNI is not clearly defined. A minimal interpretation is that no two
VNIs must have the same value. Yet thay may definitely have the same
queue mapping.

On TCAMs: is the assumption that the VC cookie today already, if
present, is used to steer among VFs? And therefore it can also be
used to select among queues? But this is a different TCAM mask, using
only 16b, and as a direct queue id.