-----Original Message-----
From: Yongseok Koh
Sent: Tuesday, August 28, 2018 10:44 PM
To: Dekel Peled <redacted>
Cc: dev <redacted>; Shahaf Shuler <redacted>; Ori Kam
[off-list ref]; Andrew Rybchenko [off-list ref];
Yigit, Ferruh [off-list ref]; Thomas Monjalon
[off-list ref]; Ananyev, Konstantin
[off-list ref]; Adrien Mazarguil
[off-list ref]; Olivier Matz [off-list ref];
Alex Rosenbaum [off-list ref]
Subject: Re: [RFC v2] ethdev: support metadata as flow rule criteria

quoted

On Aug 26, 2018, at 7:09 AM, Dekel Peled [off-list ref] wrote:

Current implementation of rte_flow allows match pattern of flow rule,
based on packet data or header fields.
This limits the application use of match patterns.

For example, consider a vswitch application which controls a set of
VMs, connected with virtio, in a fabric with overlay of VXLAN.
Several VMs can have the same inner tuple, while the outer tuple is
different and controlled by the vswitch (encap action).
For the vswtich to be able to offload the rule to the NIC, it must use
a unique match criteria, independent from the inner tuple, to perform
the encap action.

This RFC adds support for additional metadata to use as match pattern.
The metadata is an opaque item, fully controlled by the application.

The use of metadata is relevant for egress rules only.
It can be set in the flow rule using the RTE_FLOW_ITEM_META.

In order to avoid change in mbuf API, exisitng field mbuf.hash.fdir.hi
will be used to carry the metadata item. This field is used only in
ingress packets, so using it for egress metadata will not cause conflicts.

Application should set the packet metdata in the mbuf dedicated field,
and set the PKT_TX_METADATA flag in the mbuf->ol_flags.
The NIC will use the packet metadata as match criteria for relevant
flow rules.

For example, to do an encap action depending on the VM id, the
application needs to configure 'match on metadata' rte_flow rule with
VM id as metadata, along with desired encap action.
When preparing an egress data packet, application will set VM id data
in mbuf dedicated field, and set PKT_TX_METADATA flag.

PMD will send data packets to NIC, with VM id as metadata.
Egress flow on NIC will match metadata as done with other criteria.
Upon match on metadata (VM id) the appropriate encap action will be
performed.

This RFC introduces metadata item type for rte_flow
RTE_FLOW_ITEM_META, along with corresponding struct

rte_flow_item_meta

quoted

and ol_flag PKT_TX_METADATA.

Comments are welcome.

Signed-off-by: Dekel Peled <redacted>
---
v2: Use existing field in mbuf for metadata item, as suggested, instead
   of adding a new field.
   Metadata item size adjusted to 32 bits.
---
doc/guides/prog_guide/rte_flow.rst | 21 +++++++++++++++++++++
lib/librte_ethdev/rte_flow.c       |  1 +
lib/librte_ethdev/rte_flow.h       | 25 +++++++++++++++++++++++++
lib/librte_mbuf/rte_mbuf.h         | 13 +++++++++++++
4 files changed, 60 insertions(+)

diff --git a/doc/guides/prog_guide/rte_flow.rst

b/doc/guides/prog_guide/rte_flow.rst
index b305a72..560e45a 100644

--- a/doc/guides/prog_guide/rte_flow.rst
+++ b/doc/guides/prog_guide/rte_flow.rst

@@ -1191,6 +1191,27 @@ Normally preceded by any of:
- `Item: ICMP6_ND_NS`_
- `Item: ICMP6_ND_OPT`_

+Item: ``META``
+^^^^^^^^^^^^^^
+
+Matches an application specific 32 bit metadata item.
+
+- Default ``mask`` matches any 32 bit value.
+
+.. _table_rte_flow_item_meta:
+
+.. table:: META
+
+   +----------+----------+---------------------------+
+   | Field    | Subfield | Value                     |
+   +==========+==========+===========================+
+   | ``spec`` | ``data`` | 32 bit metadata value     |
+   +----------+--------------------------------------+
+   | ``last`` | ``data`` | upper range value         |
+   +----------+----------+---------------------------+
+   | ``mask`` | ``data`` | zeroed to match any value |
+   +----------+----------+---------------------------+
+

Actions
~~~~~~~

diff --git a/lib/librte_ethdev/rte_flow.c

b/lib/librte_ethdev/rte_flow.c index cff4b52..54e5ef8 100644

--- a/lib/librte_ethdev/rte_flow.c
+++ b/lib/librte_ethdev/rte_flow.c

@@ -66,6 +66,7 @@ struct rte_flow_desc_data {

		     sizeof(struct rte_flow_item_icmp6_nd_opt_sla_eth)),
	MK_FLOW_ITEM(ICMP6_ND_OPT_TLA_ETH,
		     sizeof(struct rte_flow_item_icmp6_nd_opt_tla_eth)),
+	MK_FLOW_ITEM(META, sizeof(struct rte_flow_item_meta)),
};

/** Generate flow_action[] entry. */

diff --git a/lib/librte_ethdev/rte_flow.h

b/lib/librte_ethdev/rte_flow.h index f8ba71c..eba3cc4 100644

--- a/lib/librte_ethdev/rte_flow.h
+++ b/lib/librte_ethdev/rte_flow.h

@@ -413,6 +413,15 @@ enum rte_flow_item_type {

	 * See struct rte_flow_item_mark.
	 */
	RTE_FLOW_ITEM_TYPE_MARK,
+
+	/**
+	 * [META]
+	 *
+	 * Matches a metadata value specified in mbuf metadata field.
+	 *
+	 * See struct rte_flow_item_meta.
+	 */
+	RTE_FLOW_ITEM_TYPE_META,
};

/**

@@ -849,6 +858,22 @@ struct rte_flow_item_gre { #endif

/**
+ * RTE_FLOW_ITEM_TYPE_META.
+ *
+ * Matches a specified metadata value.
+ */
+struct rte_flow_item_meta {
+	uint32_t data;
+};
+
+/** Default mask for RTE_FLOW_ITEM_TYPE_META. */ #ifndef

__cplusplus

quoted

+static const struct rte_flow_item_meta rte_flow_item_meta_mask = {
+	.data = RTE_BE32(UINT32_MAX),
+};
+#endif
+
+/**
 * RTE_FLOW_ITEM_TYPE_FUZZY
 *
 * Fuzzy pattern match, expect faster than default.

diff --git a/lib/librte_mbuf/rte_mbuf.h b/lib/librte_mbuf/rte_mbuf.h
index 9ce5d76..77c1552 100644
--- a/lib/librte_mbuf/rte_mbuf.h
+++ b/lib/librte_mbuf/rte_mbuf.h

@@ -182,6 +182,11 @@

/* add new TX flags here */

/**
+ * This flag indicates that the metadata field in the mbuf is in use.
+ */
+#define PKT_TX_METADATA		(1ULL << 41)
+
+/**
 * UDP Fragmentation Offload flag. This flag is used for enabling UDP
 * fragmentation in SW or in HW. When use UFO, mbuf->tso_segsz is used
 * to store the MSS of UDP fragments.

@@ -526,6 +531,14 @@ struct rte_mbuf {

			uint32_t hi;
			/**< First 4 flexible bytes or FD ID, dependent on
			     PKT_RX_FDIR_* flag in ol_flags. */
+
+			/**
+			 * Above item has optional use on egress:
+			 * Application specific metadata value
+			 * for flow rule match.
+			 * Valid if PKT_TX_METADATA is set.
+			 */
+

Hi Dekel,

I don't think we have reached to a conclusion?? I remember there were
three options.
1) add a new 64bit field
2) use userdata/udata64
3) use hash

I still prefer 1) but if people here think that more fields will have to be added
in the near feature then 2) would be my next preference. But, if we just
have some unclear anxiety (like the depletion of IPv4 address :-), 1) would
still be good.

But, 3) is my least preference as a Rx mbuf still can have both flow ID and
metadata.

We still need more input/discussion.

Option 1 was not favored in discussions so far, see RFC email chain.
Option 2 is unwanted since there may be applications using userdata/udata64.
Currently we see use of metadata in tx only, hence option 3 is preferred.

Thanks,
Yongseok

quoted

		} fdir;           /**< Filter identifier if FDIR enabled */
		struct {
			uint32_t lo;
--
1.8.3.1