[PATCH 0/5] handle seq no overflow in IPsec offload

This series enables application to set the sequence number soft limit
for IPsec offload. In inline IPsec offload, as the sequence number
(maintained by PMD/device) reaches the specified soft limit, the PMD
would raise an "IPSEC_EVENT". This event would have some metadata,
which would be used by the application to identify the SA on which the
sequence number overflow is about to happen.

Anoob Joseph (5):
  lib/ethdev: support for inline IPsec events
  lib/security: add ESN soft limit in conf
  lib/security: extend userdata for IPsec events
  examples/ipsec-secgw: handle ESN soft limit event
  app/testpmd: support for IPsec event

 app/test-pmd/parameters.c                 |  2 ++
 app/test-pmd/testpmd.c                    |  2 ++
 examples/ipsec-secgw/ipsec-secgw.c        | 56 +++++++++++++++++++++++++++++++
 examples/ipsec-secgw/ipsec.c              | 10 ++++--
 examples/ipsec-secgw/ipsec.h              |  2 ++
 lib/librte_ether/rte_ethdev.h             | 22 ++++++++++++
 lib/librte_security/rte_security.h        | 16 +++++----
 lib/librte_security/rte_security_driver.h |  6 ++--
 8 files changed, 104 insertions(+), 12 deletions(-)

-- 
2.7.4

Adding support for IPsec events in rte_eth_event framework. In inline
IPsec offload, the per packet protocol defined variables, like ESN,
would be managed by PMD. In such cases, PMD would need IPsec events
to notify application about various conditions like, ESN overflow.

Signed-off-by: Anoob Joseph <redacted>
---
 lib/librte_ether/rte_ethdev.h | 22 ++++++++++++++++++++++
 1 file changed, 22 insertions(+)

diff --git a/lib/librte_ether/rte_ethdev.h b/lib/librte_ether/rte_ethdev.h
index 0361533..4e4e18d 100644
--- a/lib/librte_ether/rte_ethdev.h
+++ b/lib/librte_ether/rte_ethdev.h

@@ -2438,6 +2438,27 @@ int
 rte_eth_tx_done_cleanup(uint16_t port_id, uint16_t queue_id, uint32_t free_cnt);
 
 /**
+ * Subtypes for IPsec offload events raised by eth device.
+ */
+enum rte_eth_event_ipsec_subtype {
+	RTE_ETH_EVENT_IPSEC_ESN_OVERFLOW,
+			/** Sequence number overflow in security offload */
+	RTE_ETH_EVENT_IPSEC_MAX
+			/** Max value of this enum */
+};
+
+/**
+ * Descriptor for IPsec event. Used by eth dev to send extra information of the
+ * event.
+ */
+struct rte_eth_event_ipsec_desc {
+	enum rte_eth_event_ipsec_subtype stype;
+			/** Type of IPsec event */
+	uint64_t md;
+			/** Event specific metadata */
+};
+
+/**
  * The eth device event type for interrupt, and maybe others in the future.
  */
 enum rte_eth_event_type {

@@ -2448,6 +2469,7 @@ enum rte_eth_event_type {
 	RTE_ETH_EVENT_INTR_RESET,
 			/**< reset interrupt event, sent to VF on PF reset */
 	RTE_ETH_EVENT_VF_MBOX,  /**< message from the VF received by PF */
+	RTE_ETH_EVENT_IPSEC,    /**< IPsec offload related event */
 	RTE_ETH_EVENT_MACSEC,   /**< MACsec offload related event */
 	RTE_ETH_EVENT_INTR_RMV, /**< device removal event */
 	RTE_ETH_EVENT_NEW,      /**< port is probed */

-- 
2.7.4

Adding ESN soft limit in conf. This will be used in case of protocol
offload. Per SA, application could specify for what ESN the security
device need to notify application. In case of eth dev(inline protocol),
rte_eth_event framework would raise an IPsec event.

Signed-off-by: Anoob Joseph <redacted>
---
 lib/librte_security/rte_security.h | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/lib/librte_security/rte_security.h b/lib/librte_security/rte_security.h
index c75c121..a71ff6f 100644
--- a/lib/librte_security/rte_security.h
+++ b/lib/librte_security/rte_security.h

@@ -222,6 +222,8 @@ struct rte_security_ipsec_xform {
 	/**< IPsec SA Mode - transport/tunnel */
 	struct rte_security_ipsec_tunnel_param tunnel;
 	/**< Tunnel parameters, NULL for transport mode */
+	uint64_t esn_soft_limit;
+	/**< ESN for which the overflow event need to be raised by eth dev */
 };
 
 /**

-- 
2.7.4

Extending 'userdata' to be used for IPsec events too.

IPsec events would have some metadata which would uniquely identify the
security session for which the event is raised. But application would
need some construct which it can understand. The 'userdata' solves a
similar problem for inline processed inbound traffic. Updating the
documentation to extend the usage of 'userdata'.

Signed-off-by: Anoob Joseph <redacted>
---
 lib/librte_security/rte_security.h        | 14 ++++++++------
 lib/librte_security/rte_security_driver.h |  6 +++---
 2 files changed, 11 insertions(+), 9 deletions(-)

diff --git a/lib/librte_security/rte_security.h b/lib/librte_security/rte_security.h
index a71ff6f..e8b5888 100644
--- a/lib/librte_security/rte_security.h
+++ b/lib/librte_security/rte_security.h

@@ -364,15 +364,17 @@ rte_security_set_pkt_metadata(struct rte_security_ctx *instance,
 			      struct rte_mbuf *mb, void *params);
 
 /**
- * Get userdata associated with the security session which processed the
- * packet. This userdata would be registered while creating the session, and
- * application can use this to identify the SA etc. Device-specific metadata
- * in the mbuf would be used for this.
+ * Get userdata associated with the security session. Device specific metadata
+ * provided would be used to uniquely identify the security session being
+ * referred to. This userdata would be registered while creating the session,
+ * and application can use this to identify the SA etc.
  *
- * This is valid only for inline processed ingress packets.
+ * Device specific metadata would be set in mbuf for inline processed inbound
+ * packets. In addition, the same metadata would be set for IPsec events
+ * reported by rte_eth_event framework.
  *
  * @param   instance	security instance
- * @param   md		device-specific metadata set in mbuf
+ * @param   md		device-specific metadata
  *
  * @return
  *  - On success, userdata

diff --git a/lib/librte_security/rte_security_driver.h b/lib/librte_security/rte_security_driver.h
index 4623904..0583f88 100644
--- a/lib/librte_security/rte_security_driver.h
+++ b/lib/librte_security/rte_security_driver.h

@@ -134,9 +134,9 @@ typedef int (*security_set_pkt_metadata_t)(void *device,
 		void *params);
 
 /**
- * Get application specific userdata associated with the security session which
- * processed the packet. This would be retrieved using the metadata obtained
- * from packet.
+ * Get application specific userdata associated with the security session.
+ * Device specific metadata provided would be used to uniquely identify
+ * the security session being referred to.
  *
  * @param	device		Crypto/eth device pointer
  * @param	md		Metadata

-- 
2.7.4

For inline protocol processing, the PMD/device is required to maintain
the ESN. But the application is required to monitor ESN overflow to
initiate SA expiry.

For such cases, application would set the ESN soft limit. An IPsec event
would be raised by rte_eth_event framework, when ESN hits the soft limit
set by the application.

Signed-off-by: Anoob Joseph <redacted>
---
 examples/ipsec-secgw/ipsec-secgw.c | 56 ++++++++++++++++++++++++++++++++++++++
 examples/ipsec-secgw/ipsec.c       | 10 +++++--
 examples/ipsec-secgw/ipsec.h       |  2 ++
 3 files changed, 65 insertions(+), 3 deletions(-)

diff --git a/examples/ipsec-secgw/ipsec-secgw.c b/examples/ipsec-secgw/ipsec-secgw.c
index 3a8562e..5726fd3 100644
--- a/examples/ipsec-secgw/ipsec-secgw.c
+++ b/examples/ipsec-secgw/ipsec-secgw.c

@@ -40,6 +40,7 @@
 #include <rte_hash.h>
 #include <rte_jhash.h>
 #include <rte_cryptodev.h>
+#include <rte_security.h>
 
 #include "ipsec.h"
 #include "parser.h"

@@ -1640,6 +1641,58 @@ pool_init(struct socket_ctx *ctx, int32_t socket_id, uint32_t nb_mbuf)
 		printf("Allocated mbuf pool on socket %d\n", socket_id);
 }
 
+static inline int
+inline_ipsec_event_esn_overflow(struct rte_security_ctx *ctx, uint64_t md)
+{
+	struct ipsec_sa *sa;
+
+	/* For inline protocol processing, the metadata in the event will
+	 * uniquely identify the security session which raised the event.
+	 * Application would then need the userdata it had registered with the
+	 * security session to process the event.
+	 */
+
+	sa = (struct ipsec_sa *)rte_security_get_userdata(ctx, md);
+
+	if (sa == NULL) {
+		/* userdata could not be retrieved */
+		return -1;
+	}
+
+	/* Sequence number over flow. SA need to be re-established */
+	RTE_SET_USED(sa);
+	return 0;
+}
+
+static int
+inline_ipsec_event_callback(uint16_t port_id, enum rte_eth_event_type type,
+		 void *param, void *ret_param)
+{
+	struct rte_eth_event_ipsec_desc *event_desc = NULL;
+	struct rte_security_ctx *ctx = (struct rte_security_ctx *)
+					rte_eth_dev_get_sec_ctx(port_id);
+
+	RTE_SET_USED(param);
+
+	if (type != RTE_ETH_EVENT_IPSEC)
+		return -1;
+
+	event_desc = ret_param;
+	if (event_desc == NULL) {
+		printf("Event descriptor not set\n");
+		return -1;
+	}
+
+	if (event_desc->stype == RTE_ETH_EVENT_IPSEC_ESN_OVERFLOW)
+		return inline_ipsec_event_esn_overflow(ctx, event_desc->md);
+	else if (event_desc->stype >= RTE_ETH_EVENT_IPSEC_MAX) {
+		printf("Invalid IPsec event reported\n");
+		return -1;
+	}
+
+	return -1;
+}
+
 int32_t
 main(int32_t argc, char **argv)
 {

@@ -1727,6 +1780,9 @@ main(int32_t argc, char **argv)
 		 */
 		if (promiscuous_on)
 			rte_eth_promiscuous_enable(portid);
+
+		rte_eth_dev_callback_register(portid,
+			RTE_ETH_EVENT_IPSEC, inline_ipsec_event_callback, NULL);
 	}
 
 	check_all_ports_link_status(nb_ports, enabled_port_mask);

diff --git a/examples/ipsec-secgw/ipsec.c b/examples/ipsec-secgw/ipsec.c
index 5fb5bc1..acdd189 100644
--- a/examples/ipsec-secgw/ipsec.c
+++ b/examples/ipsec-secgw/ipsec.c

@@ -36,6 +36,7 @@ set_ipsec_conf(struct ipsec_sa *sa, struct rte_security_ipsec_xform *ipsec)
 		}
 		/* TODO support for Transport and IPV6 tunnel */
 	}
+	ipsec->esn_soft_limit = IPSEC_OFFLOAD_ESN_SOFTLIMIT;
 }
 
 static inline int

@@ -270,11 +271,14 @@ create_session(struct ipsec_ctx *ipsec_ctx, struct ipsec_sa *sa)
 			 * the packet is received, this userdata will be
 			 * retrieved using the metadata from the packet.
 			 *
-			 * This is required only for inbound SAs.
+			 * The PMD is expected to set similar metadata for other
+			 * operations, like rte_eth_event, which are tied to
+			 * security session. In such cases, the userdata could
+			 * be obtained to uniquely identify the security
+			 * parameters denoted.
 			 */
 
-			if (sa->direction == RTE_SECURITY_IPSEC_SA_DIR_INGRESS)
-				sess_conf.userdata = (void *) sa;
+			sess_conf.userdata = (void *) sa;
 
 			sa->sec_session = rte_security_session_create(ctx,
 					&sess_conf, ipsec_ctx->session_pool);

diff --git a/examples/ipsec-secgw/ipsec.h b/examples/ipsec-secgw/ipsec.h
index 6059f6c..c1450f6 100644
--- a/examples/ipsec-secgw/ipsec.h
+++ b/examples/ipsec-secgw/ipsec.h

@@ -21,6 +21,8 @@
 
 #define MAX_DIGEST_SIZE 32 /* Bytes -- 256 bits */
 
+#define IPSEC_OFFLOAD_ESN_SOFTLIMIT 0xffffff00
+
 #define IV_OFFSET		(sizeof(struct rte_crypto_op) + \
 				sizeof(struct rte_crypto_sym_op))
 

-- 
2.7.4

Adding support for IPsec event

Signed-off-by: Anoob Joseph <redacted>
---
 app/test-pmd/parameters.c | 2 ++
 app/test-pmd/testpmd.c    | 2 ++
 2 files changed, 4 insertions(+)

diff --git a/app/test-pmd/parameters.c b/app/test-pmd/parameters.c
index 97d22b8..7ea882f 100644
--- a/app/test-pmd/parameters.c
+++ b/app/test-pmd/parameters.c

@@ -512,6 +512,8 @@ parse_event_printing_config(const char *optarg, int enable)
 		mask = UINT32_C(1) << RTE_ETH_EVENT_INTR_RESET;
 	else if (!strcmp(optarg, "vf_mbox"))
 		mask = UINT32_C(1) << RTE_ETH_EVENT_VF_MBOX;
+	else if (!strcmp(optarg, "ipsec"))
+		mask = UINT32_C(1) << RTE_ETH_EVENT_IPSEC;
 	else if (!strcmp(optarg, "macsec"))
 		mask = UINT32_C(1) << RTE_ETH_EVENT_MACSEC;
 	else if (!strcmp(optarg, "intr_rmv"))

diff --git a/app/test-pmd/testpmd.c b/app/test-pmd/testpmd.c
index 4c0e258..32fb8b1 100644
--- a/app/test-pmd/testpmd.c
+++ b/app/test-pmd/testpmd.c

@@ -292,6 +292,7 @@ uint32_t event_print_mask = (UINT32_C(1) << RTE_ETH_EVENT_UNKNOWN) |
 			    (UINT32_C(1) << RTE_ETH_EVENT_INTR_LSC) |
 			    (UINT32_C(1) << RTE_ETH_EVENT_QUEUE_STATE) |
 			    (UINT32_C(1) << RTE_ETH_EVENT_INTR_RESET) |
+			    (UINT32_C(1) << RTE_ETH_EVENT_IPSEC) |
 			    (UINT32_C(1) << RTE_ETH_EVENT_MACSEC) |
 			    (UINT32_C(1) << RTE_ETH_EVENT_INTR_RMV);
 

@@ -2024,6 +2025,7 @@ eth_event_callback(portid_t port_id, enum rte_eth_event_type type, void *param,
 		[RTE_ETH_EVENT_QUEUE_STATE] = "Queue state",
 		[RTE_ETH_EVENT_INTR_RESET] = "Interrupt reset",
 		[RTE_ETH_EVENT_VF_MBOX] = "VF Mbox",
+		[RTE_ETH_EVENT_IPSEC] = "IPsec",
 		[RTE_ETH_EVENT_MACSEC] = "MACsec",
 		[RTE_ETH_EVENT_INTR_RMV] = "device removal",
 		[RTE_ETH_EVENT_NEW] = "device probed",

-- 
2.7.4

I would add some more events to the list (to make it look less like a very specific case implementation): crypto/auth failed and undefined/unspecified being the most obvious.
Apart from this, the patchset looks fine.

Hi Radu,

Please see inline.

Thanks,
Anoob

On 26/02/18 15:05, Nicolau, Radu wrote:

quoted

-----Original Message-----
From: Anoob Joseph [mailto:anoob.joseph@caviumnetworks.com]
Sent: Wednesday, February 21, 2018 5:37 AM
To: Akhil Goyal <redacted>; Doherty, Declan
[off-list ref]; Nicolau, Radu [off-list ref]
Cc: Anoob Joseph <redacted>; Jerin Jacob
[off-list ref]; Narayana Prasad
[off-list ref]; Nelio Laranjeiro
[off-list ref]; dev@dpdk.org
Subject: [PATCH 1/5] lib/ethdev: support for inline IPsec events

Adding support for IPsec events in rte_eth_event framework. In inline IPsec
offload, the per packet protocol defined variables, like ESN, would be
managed by PMD. In such cases, PMD would need IPsec events to notify
application about various conditions like, ESN overflow.

Signed-off-by: Anoob Joseph <redacted>
---
  lib/librte_ether/rte_ethdev.h | 22 ++++++++++++++++++++++
  1 file changed, 22 insertions(+)

diff --git a/lib/librte_ether/rte_ethdev.h b/lib/librte_ether/rte_ethdev.h
index 0361533..4e4e18d 100644
--- a/lib/librte_ether/rte_ethdev.h
+++ b/lib/librte_ether/rte_ethdev.h

@@ -2438,6 +2438,27 @@ int
  rte_eth_tx_done_cleanup(uint16_t port_id, uint16_t queue_id, uint32_t

free_cnt);

  /**
+ * Subtypes for IPsec offload events raised by eth device.
+ */
+enum rte_eth_event_ipsec_subtype {
+	RTE_ETH_EVENT_IPSEC_ESN_OVERFLOW,
+			/** Sequence number overflow in security offload */
+	RTE_ETH_EVENT_IPSEC_MAX
+			/** Max value of this enum */
+};

I would add some more events to the list (to make it look less like a very specific case implementation): crypto/auth failed and undefined/unspecified being the most obvious.
Apart from this, the patchset looks fine.

Understood your point. But crypto/auth failed would be per packet, 
right? How are we handling such error cases presently? Just want to make 
sure we are not adding two error reporting mechanisms.

quoted

+
+/**
+ * Descriptor for IPsec event. Used by eth dev to send extra
+information of the
+ * event.
+ */
+struct rte_eth_event_ipsec_desc {
+	enum rte_eth_event_ipsec_subtype stype;
+			/** Type of IPsec event */
+	uint64_t md;
+			/** Event specific metadata */
+};
+
+/**
   * The eth device event type for interrupt, and maybe others in the future.
   */
  enum rte_eth_event_type {

@@ -2448,6 +2469,7 @@ enum rte_eth_event_type {
  	RTE_ETH_EVENT_INTR_RESET,
  			/**< reset interrupt event, sent to VF on PF reset */
  	RTE_ETH_EVENT_VF_MBOX,  /**< message from the VF received by

PF */
+	RTE_ETH_EVENT_IPSEC,    /**< IPsec offload related event */
  	RTE_ETH_EVENT_MACSEC,   /**< MACsec offload related event */
  	RTE_ETH_EVENT_INTR_RMV, /**< device removal event */
  	RTE_ETH_EVENT_NEW,      /**< port is probed */
--
2.7.4

-----Original Message-----
From: Anoob Joseph [mailto:Anoob.Joseph@caviumnetworks.com]
Sent: Tuesday, February 27, 2018 6:57 AM
To: Nicolau, Radu <redacted>; Akhil Goyal
[off-list ref]; Doherty, Declan [off-list ref]
Cc: Jerin Jacob <redacted>; Narayana Prasad
[off-list ref]; Nelio Laranjeiro
[off-list ref]; dev@dpdk.org
Subject: Re: [PATCH 1/5] lib/ethdev: support for inline IPsec events

Hi Radu,

Please see inline.

Thanks,
Anoob

On 26/02/18 15:05, Nicolau, Radu wrote:

quoted

quoted

-----Original Message-----
From: Anoob Joseph [mailto:anoob.joseph@caviumnetworks.com]
Sent: Wednesday, February 21, 2018 5:37 AM
To: Akhil Goyal <redacted>; Doherty, Declan
[off-list ref]; Nicolau, Radu [off-list ref]
Cc: Anoob Joseph <redacted>; Jerin Jacob
[off-list ref]; Narayana Prasad
[off-list ref]; Nelio Laranjeiro
[off-list ref]; dev@dpdk.org
Subject: [PATCH 1/5] lib/ethdev: support for inline IPsec events

Adding support for IPsec events in rte_eth_event framework. In inline
IPsec offload, the per packet protocol defined variables, like ESN,
would be managed by PMD. In such cases, PMD would need IPsec events
to notify application about various conditions like, ESN overflow.

Signed-off-by: Anoob Joseph <redacted>
---
  lib/librte_ether/rte_ethdev.h | 22 ++++++++++++++++++++++
  1 file changed, 22 insertions(+)

diff --git a/lib/librte_ether/rte_ethdev.h

b/lib/librte_ether/rte_ethdev.h index 0361533..4e4e18d 100644

--- a/lib/librte_ether/rte_ethdev.h
+++ b/lib/librte_ether/rte_ethdev.h

@@ -2438,6 +2438,27 @@ int
  rte_eth_tx_done_cleanup(uint16_t port_id, uint16_t queue_id,

uint32_t free_cnt);

  /**
+ * Subtypes for IPsec offload events raised by eth device.
+ */
+enum rte_eth_event_ipsec_subtype {
+	RTE_ETH_EVENT_IPSEC_ESN_OVERFLOW,
+			/** Sequence number overflow in security offload */
+	RTE_ETH_EVENT_IPSEC_MAX
+			/** Max value of this enum */
+};

I would add some more events to the list (to make it look less like a very

specific case implementation): crypto/auth failed and undefined/unspecified
being the most obvious.

quoted

Apart from this, the patchset looks fine.

Understood your point. But crypto/auth failed would be per packet, right?
How are we handling such error cases presently? Just want to make sure we
are not adding two error reporting mechanisms.

The only reason for my suggestion was to keep the API as flexible and generic as possible.
For the inline crypto on ixgbe we only flag the mbuf with the security error flag, but no extra info is added. I guess we can have a ipsec crypto error event with a list of failed mbufs or similar. In any case, it's just a suggestion.

quoted

quoted

+
+/**
+ * Descriptor for IPsec event. Used by eth dev to send extra
+information of the
+ * event.
+ */
+struct rte_eth_event_ipsec_desc {
+	enum rte_eth_event_ipsec_subtype stype;
+			/** Type of IPsec event */
+	uint64_t md;
+			/** Event specific metadata */
+};
+
+/**
   * The eth device event type for interrupt, and maybe others in the

future.

quoted

quoted

   */
  enum rte_eth_event_type {

@@ -2448,6 +2469,7 @@ enum rte_eth_event_type {
  	RTE_ETH_EVENT_INTR_RESET,
  			/**< reset interrupt event, sent to VF on PF reset */
  	RTE_ETH_EVENT_VF_MBOX,  /**< message from the VF received by

PF */

quoted

quoted

+	RTE_ETH_EVENT_IPSEC,    /**< IPsec offload related event */
  	RTE_ETH_EVENT_MACSEC,   /**< MACsec offload related event */
  	RTE_ETH_EVENT_INTR_RMV, /**< device removal event */
  	RTE_ETH_EVENT_NEW,      /**< port is probed */
--
2.7.4

Hi Radu,

Please see inline.

Thanks,

Anoob


On 27/02/18 15:49, Nicolau, Radu wrote:

quoted

-----Original Message-----
From: Anoob Joseph [mailto:Anoob.Joseph@caviumnetworks.com]
Sent: Tuesday, February 27, 2018 6:57 AM
To: Nicolau, Radu <redacted>; Akhil Goyal
[off-list ref]; Doherty, Declan [off-list ref]
Cc: Jerin Jacob <redacted>; Narayana Prasad
[off-list ref]; Nelio Laranjeiro
[off-list ref]; dev@dpdk.org
Subject: Re: [PATCH 1/5] lib/ethdev: support for inline IPsec events

Hi Radu,

Please see inline.

Thanks,
Anoob

On 26/02/18 15:05, Nicolau, Radu wrote:

quoted

quoted

-----Original Message-----
From: Anoob Joseph [mailto:anoob.joseph@caviumnetworks.com]
Sent: Wednesday, February 21, 2018 5:37 AM
To: Akhil Goyal <redacted>; Doherty, Declan
[off-list ref]; Nicolau, Radu [off-list ref]
Cc: Anoob Joseph <redacted>; Jerin Jacob
[off-list ref]; Narayana Prasad
[off-list ref]; Nelio Laranjeiro
[off-list ref]; dev@dpdk.org
Subject: [PATCH 1/5] lib/ethdev: support for inline IPsec events

Adding support for IPsec events in rte_eth_event framework. In inline
IPsec offload, the per packet protocol defined variables, like ESN,
would be managed by PMD. In such cases, PMD would need IPsec events
to notify application about various conditions like, ESN overflow.

Signed-off-by: Anoob Joseph <redacted>
---
   lib/librte_ether/rte_ethdev.h | 22 ++++++++++++++++++++++
   1 file changed, 22 insertions(+)

diff --git a/lib/librte_ether/rte_ethdev.h

b/lib/librte_ether/rte_ethdev.h index 0361533..4e4e18d 100644

--- a/lib/librte_ether/rte_ethdev.h
+++ b/lib/librte_ether/rte_ethdev.h

@@ -2438,6 +2438,27 @@ int
   rte_eth_tx_done_cleanup(uint16_t port_id, uint16_t queue_id,

uint32_t free_cnt);

   /**
+ * Subtypes for IPsec offload events raised by eth device.
+ */
+enum rte_eth_event_ipsec_subtype {
+	RTE_ETH_EVENT_IPSEC_ESN_OVERFLOW,
+			/** Sequence number overflow in security offload */
+	RTE_ETH_EVENT_IPSEC_MAX
+			/** Max value of this enum */
+};

I would add some more events to the list (to make it look less like a very

specific case implementation): crypto/auth failed and undefined/unspecified
being the most obvious.

quoted

Apart from this, the patchset looks fine.

Understood your point. But crypto/auth failed would be per packet, right?
How are we handling such error cases presently? Just want to make sure we
are not adding two error reporting mechanisms.

The only reason for my suggestion was to keep the API as flexible and generic as possible.

I agree to your suggestion.

For the inline crypto on ixgbe we only flag the mbuf with the security error flag, but no extra info is added. I guess we can have a ipsec crypto error event with a list of failed mbufs or similar. In any case, it's just a suggestion.

Do you think having a crypto error with failed mbufs would be useful? If 
yes, I can add that. While considering other SA specific events, there 
could be two other such events that we may need to consider.
1) Byte expiry of SA [1]
2) Time expiry of SA [1]

Shall I add these events? Or do we need to make that a separate patch? 
Considering that it would need an entry in conf for actually of any use.

[1] https://tools.ietf.org/html/rfc4301#page-37

quoted

quoted

quoted

+
+/**
+ * Descriptor for IPsec event. Used by eth dev to send extra
+information of the
+ * event.
+ */
+struct rte_eth_event_ipsec_desc {
+	enum rte_eth_event_ipsec_subtype stype;
+			/** Type of IPsec event */
+	uint64_t md;
+			/** Event specific metadata */
+};
+
+/**
    * The eth device event type for interrupt, and maybe others in the

future.

quoted

quoted

    */
   enum rte_eth_event_type {

@@ -2448,6 +2469,7 @@ enum rte_eth_event_type {
   	RTE_ETH_EVENT_INTR_RESET,
   			/**< reset interrupt event, sent to VF on PF reset */
   	RTE_ETH_EVENT_VF_MBOX,  /**< message from the VF received by

PF */

quoted

quoted

+	RTE_ETH_EVENT_IPSEC,    /**< IPsec offload related event */
   	RTE_ETH_EVENT_MACSEC,   /**< MACsec offload related event */
   	RTE_ETH_EVENT_INTR_RMV, /**< device removal event */
   	RTE_ETH_EVENT_NEW,      /**< port is probed */
--
2.7.4

Hi,

-----Original Message-----
From: Anoob Joseph [mailto:Anoob.Joseph@caviumnetworks.com]
Sent: Tuesday, February 27, 2018 11:32 AM
To: Nicolau, Radu <redacted>; Akhil Goyal
[off-list ref]; Doherty, Declan [off-list ref]
Cc: Jerin Jacob <redacted>; Narayana Prasad
[off-list ref]; Nelio Laranjeiro
[off-list ref]; dev@dpdk.org
Subject: Re: [PATCH 1/5] lib/ethdev: support for inline IPsec events

Hi Radu,

Please see inline.

Thanks,

Anoob


On 27/02/18 15:49, Nicolau, Radu wrote:

quoted

quoted

-----Original Message-----
From: Anoob Joseph [mailto:Anoob.Joseph@caviumnetworks.com]
Sent: Tuesday, February 27, 2018 6:57 AM
To: Nicolau, Radu <redacted>; Akhil Goyal
[off-list ref]; Doherty, Declan [off-list ref]
Cc: Jerin Jacob <redacted>; Narayana Prasad
[off-list ref]; Nelio Laranjeiro
[off-list ref]; dev@dpdk.org
Subject: Re: [PATCH 1/5] lib/ethdev: support for inline IPsec events

Hi Radu,

Please see inline.

Thanks,
Anoob

On 26/02/18 15:05, Nicolau, Radu wrote:

quoted

quoted

-----Original Message-----
From: Anoob Joseph [mailto:anoob.joseph@caviumnetworks.com]
Sent: Wednesday, February 21, 2018 5:37 AM
To: Akhil Goyal <redacted>; Doherty, Declan
[off-list ref]; Nicolau, Radu [off-list ref]
Cc: Anoob Joseph <redacted>; Jerin Jacob
[off-list ref]; Narayana Prasad
[off-list ref]; Nelio Laranjeiro
[off-list ref]; dev@dpdk.org
Subject: [PATCH 1/5] lib/ethdev: support for inline IPsec events

Adding support for IPsec events in rte_eth_event framework. In
inline IPsec offload, the per packet protocol defined variables,
like ESN, would be managed by PMD. In such cases, PMD would need
IPsec events to notify application about various conditions like, ESN

overflow.

quoted

quoted

quoted

quoted

Signed-off-by: Anoob Joseph <redacted>
---
   lib/librte_ether/rte_ethdev.h | 22 ++++++++++++++++++++++
   1 file changed, 22 insertions(+)

diff --git a/lib/librte_ether/rte_ethdev.h

b/lib/librte_ether/rte_ethdev.h index 0361533..4e4e18d 100644

--- a/lib/librte_ether/rte_ethdev.h
+++ b/lib/librte_ether/rte_ethdev.h

@@ -2438,6 +2438,27 @@ int
   rte_eth_tx_done_cleanup(uint16_t port_id, uint16_t queue_id,

uint32_t free_cnt);

   /**
+ * Subtypes for IPsec offload events raised by eth device.
+ */
+enum rte_eth_event_ipsec_subtype {
+	RTE_ETH_EVENT_IPSEC_ESN_OVERFLOW,
+			/** Sequence number overflow in security offload */
+	RTE_ETH_EVENT_IPSEC_MAX
+			/** Max value of this enum */
+};

I would add some more events to the list (to make it look less like
a very

specific case implementation): crypto/auth failed and
undefined/unspecified being the most obvious.

quoted

Apart from this, the patchset looks fine.

Understood your point. But crypto/auth failed would be per packet, right?
How are we handling such error cases presently? Just want to make
sure we are not adding two error reporting mechanisms.

The only reason for my suggestion was to keep the API as flexible and

generic as possible.
I agree to your suggestion.

quoted

For the inline crypto on ixgbe we only flag the mbuf with the security error

flag, but no extra info is added. I guess we can have a ipsec crypto error
event with a list of failed mbufs or similar. In any case, it's just a suggestion.
Do you think having a crypto error with failed mbufs would be useful? If yes, I
can add that. While considering other SA specific events, there could be two
other such events that we may need to consider.
1) Byte expiry of SA [1]
2) Time expiry of SA [1]

You can add the flags even if we don't provide support for them in the sample app.

Shall I add these events? Or do we need to make that a separate patch?
Considering that it would need an entry in conf for actually of any use.

[1] https://tools.ietf.org/html/rfc4301#page-37

quoted

quoted

quoted

quoted

+
+/**
+ * Descriptor for IPsec event. Used by eth dev to send extra
+information of the
+ * event.
+ */
+struct rte_eth_event_ipsec_desc {
+	enum rte_eth_event_ipsec_subtype stype;
+			/** Type of IPsec event */
+	uint64_t md;
+			/** Event specific metadata */
+};
+
+/**
    * The eth device event type for interrupt, and maybe others in
the

future.

quoted

quoted

    */
   enum rte_eth_event_type {

@@ -2448,6 +2469,7 @@ enum rte_eth_event_type {
   	RTE_ETH_EVENT_INTR_RESET,
   			/**< reset interrupt event, sent to VF on PF reset */
   	RTE_ETH_EVENT_VF_MBOX,  /**< message from the VF received by

PF */

quoted

quoted

+	RTE_ETH_EVENT_IPSEC,    /**< IPsec offload related event */
   	RTE_ETH_EVENT_MACSEC,   /**< MACsec offload related event */
   	RTE_ETH_EVENT_INTR_RMV, /**< device removal event */
   	RTE_ETH_EVENT_NEW,      /**< port is probed */
--
2.7.4

This series enables application to set the sequence number soft limit
for IPsec offload. In inline IPsec offload, as the sequence number
(maintained by PMD/device) reaches the specified soft limit, the PMD
would raise an "IPSEC_EVENT". This event would have some metadata,
which would be used by the application to identify the SA on which the
sequence number overflow is about to happen.

Anoob Joseph (5):
  lib/ethdev: support for inline IPsec events
  lib/security: add ESN soft limit in conf
  lib/security: extend userdata for IPsec events
  examples/ipsec-secgw: handle ESN soft limit event
  app/testpmd: support for IPsec event

 app/test-pmd/parameters.c                 |  2 ++
 app/test-pmd/testpmd.c                    |  2 ++
 examples/ipsec-secgw/ipsec-secgw.c        | 56 +++++++++++++++++++++++++++++++
 examples/ipsec-secgw/ipsec.c              | 10 ++++--
 examples/ipsec-secgw/ipsec.h              |  2 ++
 lib/librte_ether/rte_ethdev.h             | 28 ++++++++++++++++
 lib/librte_security/rte_security.h        | 16 +++++----
 lib/librte_security/rte_security_driver.h |  6 ++--
 8 files changed, 110 insertions(+), 12 deletions(-)

-- 
2.7.4

Adding support for IPsec events in rte_eth_event framework. In inline
IPsec offload, the per packet protocol defined variables, like ESN,
would be managed by PMD. In such cases, PMD would need IPsec events
to notify application about various conditions like, ESN overflow.

Signed-off-by: Anoob Joseph <redacted>
---
v2:
* Added time expiry & byte expiry IPsec events in the enum

 lib/librte_ether/rte_ethdev.h | 28 ++++++++++++++++++++++++++++
 1 file changed, 28 insertions(+)

diff --git a/lib/librte_ether/rte_ethdev.h b/lib/librte_ether/rte_ethdev.h
index 0361533..96b2aa0 100644
--- a/lib/librte_ether/rte_ethdev.h
+++ b/lib/librte_ether/rte_ethdev.h

@@ -2438,6 +2438,33 @@ int
 rte_eth_tx_done_cleanup(uint16_t port_id, uint16_t queue_id, uint32_t free_cnt);
 
 /**
+ * Subtypes for IPsec offload events raised by eth device.
+ */
+enum rte_eth_event_ipsec_subtype {
+	RTE_ETH_EVENT_IPSEC_UNKNOWN = 0,
+			/** Unknown event type */
+	RTE_ETH_EVENT_IPSEC_ESN_OVERFLOW,
+			/** Sequence number overflow in security offload */
+	RTE_ETH_EVENT_IPSEC_SA_TIME_EXPIRY,
+			/** Soft time expiry of SA */
+	RTE_ETH_EVENT_IPSEC_SA_BYTE_EXPIRY,
+			/** Soft byte expiry of SA */
+	RTE_ETH_EVENT_IPSEC_MAX
+			/** Max value of this enum */
+};
+
+/**
+ * Descriptor for IPsec event. Used by eth dev to send extra information of the
+ * event.
+ */
+struct rte_eth_event_ipsec_desc {
+	enum rte_eth_event_ipsec_subtype stype;
+			/** Type of IPsec event */
+	uint64_t md;
+			/** Event specific metadata */
+};
+
+/**
  * The eth device event type for interrupt, and maybe others in the future.
  */
 enum rte_eth_event_type {

@@ -2448,6 +2475,7 @@ enum rte_eth_event_type {
 	RTE_ETH_EVENT_INTR_RESET,
 			/**< reset interrupt event, sent to VF on PF reset */
 	RTE_ETH_EVENT_VF_MBOX,  /**< message from the VF received by PF */
+	RTE_ETH_EVENT_IPSEC,    /**< IPsec offload related event */
 	RTE_ETH_EVENT_MACSEC,   /**< MACsec offload related event */
 	RTE_ETH_EVENT_INTR_RMV, /**< device removal event */
 	RTE_ETH_EVENT_NEW,      /**< port is probed */

-- 
2.7.4

Adding ESN soft limit in conf. This will be used in case of protocol
offload. Per SA, application could specify for what ESN the security
device need to notify application. In case of eth dev(inline protocol),
rte_eth_event framework would raise an IPsec event.

Signed-off-by: Anoob Joseph <redacted>
---
v2:
* No change

 lib/librte_security/rte_security.h | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/lib/librte_security/rte_security.h b/lib/librte_security/rte_security.h
index c75c121..a71ff6f 100644
--- a/lib/librte_security/rte_security.h
+++ b/lib/librte_security/rte_security.h

@@ -222,6 +222,8 @@ struct rte_security_ipsec_xform {
 	/**< IPsec SA Mode - transport/tunnel */
 	struct rte_security_ipsec_tunnel_param tunnel;
 	/**< Tunnel parameters, NULL for transport mode */
+	uint64_t esn_soft_limit;
+	/**< ESN for which the overflow event need to be raised by eth dev */
 };
 
 /**

-- 
2.7.4

Extending 'userdata' to be used for IPsec events too.

IPsec events would have some metadata which would uniquely identify the
security session for which the event is raised. But application would
need some construct which it can understand. The 'userdata' solves a
similar problem for inline processed inbound traffic. Updating the
documentation to extend the usage of 'userdata'.

Signed-off-by: Anoob Joseph <redacted>
---
v2:
* No change

 lib/librte_security/rte_security.h        | 14 ++++++++------
 lib/librte_security/rte_security_driver.h |  6 +++---
 2 files changed, 11 insertions(+), 9 deletions(-)

diff --git a/lib/librte_security/rte_security.h b/lib/librte_security/rte_security.h
index a71ff6f..e8b5888 100644
--- a/lib/librte_security/rte_security.h
+++ b/lib/librte_security/rte_security.h

@@ -364,15 +364,17 @@ rte_security_set_pkt_metadata(struct rte_security_ctx *instance,
 			      struct rte_mbuf *mb, void *params);
 
 /**
- * Get userdata associated with the security session which processed the
- * packet. This userdata would be registered while creating the session, and
- * application can use this to identify the SA etc. Device-specific metadata
- * in the mbuf would be used for this.
+ * Get userdata associated with the security session. Device specific metadata
+ * provided would be used to uniquely identify the security session being
+ * referred to. This userdata would be registered while creating the session,
+ * and application can use this to identify the SA etc.
  *
- * This is valid only for inline processed ingress packets.
+ * Device specific metadata would be set in mbuf for inline processed inbound
+ * packets. In addition, the same metadata would be set for IPsec events
+ * reported by rte_eth_event framework.
  *
  * @param   instance	security instance
- * @param   md		device-specific metadata set in mbuf
+ * @param   md		device-specific metadata
  *
  * @return
  *  - On success, userdata

diff --git a/lib/librte_security/rte_security_driver.h b/lib/librte_security/rte_security_driver.h
index 4623904..0583f88 100644
--- a/lib/librte_security/rte_security_driver.h
+++ b/lib/librte_security/rte_security_driver.h

@@ -134,9 +134,9 @@ typedef int (*security_set_pkt_metadata_t)(void *device,
 		void *params);
 
 /**
- * Get application specific userdata associated with the security session which
- * processed the packet. This would be retrieved using the metadata obtained
- * from packet.
+ * Get application specific userdata associated with the security session.
+ * Device specific metadata provided would be used to uniquely identify
+ * the security session being referred to.
  *
  * @param	device		Crypto/eth device pointer
  * @param	md		Metadata

-- 
2.7.4

For inline protocol processing, the PMD/device is required to maintain
the ESN. But the application is required to monitor ESN overflow to
initiate SA expiry.

For such cases, application would set the ESN soft limit. An IPsec event
would be raised by rte_eth_event framework, when ESN hits the soft limit
set by the application.

Signed-off-by: Anoob Joseph <redacted>
---
v2:
* No change

 examples/ipsec-secgw/ipsec-secgw.c | 56 ++++++++++++++++++++++++++++++++++++++
 examples/ipsec-secgw/ipsec.c       | 10 +++++--
 examples/ipsec-secgw/ipsec.h       |  2 ++
 3 files changed, 65 insertions(+), 3 deletions(-)

diff --git a/examples/ipsec-secgw/ipsec-secgw.c b/examples/ipsec-secgw/ipsec-secgw.c
index 3a8562e..5726fd3 100644
--- a/examples/ipsec-secgw/ipsec-secgw.c
+++ b/examples/ipsec-secgw/ipsec-secgw.c

@@ -40,6 +40,7 @@
 #include <rte_hash.h>
 #include <rte_jhash.h>
 #include <rte_cryptodev.h>
+#include <rte_security.h>
 
 #include "ipsec.h"
 #include "parser.h"

@@ -1640,6 +1641,58 @@ pool_init(struct socket_ctx *ctx, int32_t socket_id, uint32_t nb_mbuf)
 		printf("Allocated mbuf pool on socket %d\n", socket_id);
 }
 
+static inline int
+inline_ipsec_event_esn_overflow(struct rte_security_ctx *ctx, uint64_t md)
+{
+	struct ipsec_sa *sa;
+
+	/* For inline protocol processing, the metadata in the event will
+	 * uniquely identify the security session which raised the event.
+	 * Application would then need the userdata it had registered with the
+	 * security session to process the event.
+	 */
+
+	sa = (struct ipsec_sa *)rte_security_get_userdata(ctx, md);
+
+	if (sa == NULL) {
+		/* userdata could not be retrieved */
+		return -1;
+	}
+
+	/* Sequence number over flow. SA need to be re-established */
+	RTE_SET_USED(sa);
+	return 0;
+}
+
+static int
+inline_ipsec_event_callback(uint16_t port_id, enum rte_eth_event_type type,
+		 void *param, void *ret_param)
+{
+	struct rte_eth_event_ipsec_desc *event_desc = NULL;
+	struct rte_security_ctx *ctx = (struct rte_security_ctx *)
+					rte_eth_dev_get_sec_ctx(port_id);
+
+	RTE_SET_USED(param);
+
+	if (type != RTE_ETH_EVENT_IPSEC)
+		return -1;
+
+	event_desc = ret_param;
+	if (event_desc == NULL) {
+		printf("Event descriptor not set\n");
+		return -1;
+	}
+
+	if (event_desc->stype == RTE_ETH_EVENT_IPSEC_ESN_OVERFLOW)
+		return inline_ipsec_event_esn_overflow(ctx, event_desc->md);
+	else if (event_desc->stype >= RTE_ETH_EVENT_IPSEC_MAX) {
+		printf("Invalid IPsec event reported\n");
+		return -1;
+	}
+
+	return -1;
+}
+
 int32_t
 main(int32_t argc, char **argv)
 {

@@ -1727,6 +1780,9 @@ main(int32_t argc, char **argv)
 		 */
 		if (promiscuous_on)
 			rte_eth_promiscuous_enable(portid);
+
+		rte_eth_dev_callback_register(portid,
+			RTE_ETH_EVENT_IPSEC, inline_ipsec_event_callback, NULL);
 	}
 
 	check_all_ports_link_status(nb_ports, enabled_port_mask);

diff --git a/examples/ipsec-secgw/ipsec.c b/examples/ipsec-secgw/ipsec.c
index 5fb5bc1..acdd189 100644
--- a/examples/ipsec-secgw/ipsec.c
+++ b/examples/ipsec-secgw/ipsec.c

@@ -36,6 +36,7 @@ set_ipsec_conf(struct ipsec_sa *sa, struct rte_security_ipsec_xform *ipsec)
 		}
 		/* TODO support for Transport and IPV6 tunnel */
 	}
+	ipsec->esn_soft_limit = IPSEC_OFFLOAD_ESN_SOFTLIMIT;
 }
 
 static inline int

@@ -270,11 +271,14 @@ create_session(struct ipsec_ctx *ipsec_ctx, struct ipsec_sa *sa)
 			 * the packet is received, this userdata will be
 			 * retrieved using the metadata from the packet.
 			 *
-			 * This is required only for inbound SAs.
+			 * The PMD is expected to set similar metadata for other
+			 * operations, like rte_eth_event, which are tied to
+			 * security session. In such cases, the userdata could
+			 * be obtained to uniquely identify the security
+			 * parameters denoted.
 			 */
 
-			if (sa->direction == RTE_SECURITY_IPSEC_SA_DIR_INGRESS)
-				sess_conf.userdata = (void *) sa;
+			sess_conf.userdata = (void *) sa;
 
 			sa->sec_session = rte_security_session_create(ctx,
 					&sess_conf, ipsec_ctx->session_pool);

diff --git a/examples/ipsec-secgw/ipsec.h b/examples/ipsec-secgw/ipsec.h
index 6059f6c..c1450f6 100644
--- a/examples/ipsec-secgw/ipsec.h
+++ b/examples/ipsec-secgw/ipsec.h

@@ -21,6 +21,8 @@
 
 #define MAX_DIGEST_SIZE 32 /* Bytes -- 256 bits */
 
+#define IPSEC_OFFLOAD_ESN_SOFTLIMIT 0xffffff00
+
 #define IV_OFFSET		(sizeof(struct rte_crypto_op) + \
 				sizeof(struct rte_crypto_sym_op))
 

-- 
2.7.4

Adding support for IPsec event

Signed-off-by: Anoob Joseph <redacted>
---
v2:
* No change

 app/test-pmd/parameters.c | 2 ++
 app/test-pmd/testpmd.c    | 2 ++
 2 files changed, 4 insertions(+)

diff --git a/app/test-pmd/parameters.c b/app/test-pmd/parameters.c
index 97d22b8..7ea882f 100644
--- a/app/test-pmd/parameters.c
+++ b/app/test-pmd/parameters.c

@@ -512,6 +512,8 @@ parse_event_printing_config(const char *optarg, int enable)
 		mask = UINT32_C(1) << RTE_ETH_EVENT_INTR_RESET;
 	else if (!strcmp(optarg, "vf_mbox"))
 		mask = UINT32_C(1) << RTE_ETH_EVENT_VF_MBOX;
+	else if (!strcmp(optarg, "ipsec"))
+		mask = UINT32_C(1) << RTE_ETH_EVENT_IPSEC;
 	else if (!strcmp(optarg, "macsec"))
 		mask = UINT32_C(1) << RTE_ETH_EVENT_MACSEC;
 	else if (!strcmp(optarg, "intr_rmv"))

diff --git a/app/test-pmd/testpmd.c b/app/test-pmd/testpmd.c
index 4c0e258..32fb8b1 100644
--- a/app/test-pmd/testpmd.c
+++ b/app/test-pmd/testpmd.c

@@ -292,6 +292,7 @@ uint32_t event_print_mask = (UINT32_C(1) << RTE_ETH_EVENT_UNKNOWN) |
 			    (UINT32_C(1) << RTE_ETH_EVENT_INTR_LSC) |
 			    (UINT32_C(1) << RTE_ETH_EVENT_QUEUE_STATE) |
 			    (UINT32_C(1) << RTE_ETH_EVENT_INTR_RESET) |
+			    (UINT32_C(1) << RTE_ETH_EVENT_IPSEC) |
 			    (UINT32_C(1) << RTE_ETH_EVENT_MACSEC) |
 			    (UINT32_C(1) << RTE_ETH_EVENT_INTR_RMV);
 

@@ -2024,6 +2025,7 @@ eth_event_callback(portid_t port_id, enum rte_eth_event_type type, void *param,
 		[RTE_ETH_EVENT_QUEUE_STATE] = "Queue state",
 		[RTE_ETH_EVENT_INTR_RESET] = "Interrupt reset",
 		[RTE_ETH_EVENT_VF_MBOX] = "VF Mbox",
+		[RTE_ETH_EVENT_IPSEC] = "IPsec",
 		[RTE_ETH_EVENT_MACSEC] = "MACsec",
 		[RTE_ETH_EVENT_INTR_RMV] = "device removal",
 		[RTE_ETH_EVENT_NEW] = "device probed",

-- 
2.7.4

Hi Akhil, Radu,

Can you review the patch set and share your comments?

Thanks,
Anoob

On 01/03/18 14:51, Anoob Joseph wrote:

This series enables application to set the sequence number soft limit
for IPsec offload. In inline IPsec offload, as the sequence number
(maintained by PMD/device) reaches the specified soft limit, the PMD
would raise an "IPSEC_EVENT". This event would have some metadata,
which would be used by the application to identify the SA on which the
sequence number overflow is about to happen.

Anoob Joseph (5):
   lib/ethdev: support for inline IPsec events
   lib/security: add ESN soft limit in conf
   lib/security: extend userdata for IPsec events
   examples/ipsec-secgw: handle ESN soft limit event
   app/testpmd: support for IPsec event

  app/test-pmd/parameters.c                 |  2 ++
  app/test-pmd/testpmd.c                    |  2 ++
  examples/ipsec-secgw/ipsec-secgw.c        | 56 +++++++++++++++++++++++++++++++
  examples/ipsec-secgw/ipsec.c              | 10 ++++--
  examples/ipsec-secgw/ipsec.h              |  2 ++
  lib/librte_ether/rte_ethdev.h             | 28 ++++++++++++++++
  lib/librte_security/rte_security.h        | 16 +++++----
  lib/librte_security/rte_security_driver.h |  6 ++--
  8 files changed, 110 insertions(+), 12 deletions(-)

Hi Anoob,

Just a minor comment.
On 3/1/2018 2:51 PM, Anoob Joseph wrote:

eth/crypto dev

  };
  
  /**

Hi Anoob,

On 3/1/2018 2:51 PM, Anoob Joseph wrote:

With this patchset, application will be able to get notification if the 
error has occurred. But it is not re-configuring the SA.
Do you intend to add the same?

Hi Akhil,

Will revise the patch with the mentioned change.

Thanks,
Anoob

On 13/03/18 17:49, Akhil Goyal wrote:

Hi Anoob,

Just a minor comment.
On 3/1/2018 2:51 PM, Anoob Joseph wrote:

quoted

Adding ESN soft limit in conf. This will be used in case of protocol
offload. Per SA, application could specify for what ESN the security
device need to notify application. In case of eth dev(inline protocol),
rte_eth_event framework would raise an IPsec event.

Signed-off-by: Anoob Joseph <redacted>
---
v2:
* No change

  lib/librte_security/rte_security.h | 2 ++
  1 file changed, 2 insertions(+)

diff --git a/lib/librte_security/rte_security.h 

b/lib/librte_security/rte_security.h
index c75c121..a71ff6f 100644

--- a/lib/librte_security/rte_security.h
+++ b/lib/librte_security/rte_security.h

@@ -222,6 +222,8 @@ struct rte_security_ipsec_xform {

      /**< IPsec SA Mode - transport/tunnel */
      struct rte_security_ipsec_tunnel_param tunnel;
      /**< Tunnel parameters, NULL for transport mode */
+    uint64_t esn_soft_limit;
+    /**< ESN for which the overflow event need to be raised by eth 
dev */

eth/crypto dev

quoted

  };
    /**

Hi Akhil,

Please see inline.

Thanks,
Anoob

On 13/03/18 17:54, Akhil Goyal wrote:

Hi Anoob,

On 3/1/2018 2:51 PM, Anoob Joseph wrote:

quoted

For inline protocol processing, the PMD/device is required to maintain
the ESN. But the application is required to monitor ESN overflow to
initiate SA expiry.

For such cases, application would set the ESN soft limit. An IPsec event
would be raised by rte_eth_event framework, when ESN hits the soft limit
set by the application.

Signed-off-by: Anoob Joseph <redacted>
---
v2:
* No change

  examples/ipsec-secgw/ipsec-secgw.c | 56 
++++++++++++++++++++++++++++++++++++++
  examples/ipsec-secgw/ipsec.c       | 10 +++++--
  examples/ipsec-secgw/ipsec.h       |  2 ++
  3 files changed, 65 insertions(+), 3 deletions(-)

diff --git a/examples/ipsec-secgw/ipsec-secgw.c 

b/examples/ipsec-secgw/ipsec-secgw.c
index 3a8562e..5726fd3 100644

--- a/examples/ipsec-secgw/ipsec-secgw.c
+++ b/examples/ipsec-secgw/ipsec-secgw.c

@@ -40,6 +40,7 @@

  #include <rte_hash.h>
  #include <rte_jhash.h>
  #include <rte_cryptodev.h>
+#include <rte_security.h>
    #include "ipsec.h"
  #include "parser.h"

@@ -1640,6 +1641,58 @@ pool_init(struct socket_ctx *ctx, int32_t 

socket_id, uint32_t nb_mbuf)
          printf("Allocated mbuf pool on socket %d\n", socket_id);
  }
  +static inline int
+inline_ipsec_event_esn_overflow(struct rte_security_ctx *ctx, 
uint64_t md)
+{
+    struct ipsec_sa *sa;
+
+    /* For inline protocol processing, the metadata in the event will
+     * uniquely identify the security session which raised the event.
+     * Application would then need the userdata it had registered 
with the
+     * security session to process the event.
+     */
+
+    sa = (struct ipsec_sa *)rte_security_get_userdata(ctx, md);
+
+    if (sa == NULL) {
+        /* userdata could not be retrieved */
+        return -1;
+    }
+
+    /* Sequence number over flow. SA need to be re-established */

With this patchset, application will be able to get notification if 
the error has occurred. But it is not re-configuring the SA.
Do you intend to add the same?

Ideally the application should initiate a SA renegotiation sequence 
(with IKE etc). But ipsec-secgw uses predetermined SAs, and so addition 
of SA renegotiation might not fit in with the current design. I was just 
adding this as a place holder for future expansion (and a model for real 
applications).

What are your thoughts on addition here? Similar handling would be 
needed for byte & time expiry as well, when that is added. May be we 
could just log the event and leave it be.

quoted

+    RTE_SET_USED(sa);
+    return 0;
+}
+
+static int
+inline_ipsec_event_callback(uint16_t port_id, enum 
rte_eth_event_type type,
+         void *param, void *ret_param)
+{
+    struct rte_eth_event_ipsec_desc *event_desc = NULL;
+    struct rte_security_ctx *ctx = (struct rte_security_ctx *)
+                    rte_eth_dev_get_sec_ctx(port_id);
+
+    RTE_SET_USED(param);
+
+    if (type != RTE_ETH_EVENT_IPSEC)
+        return -1;
+
+    event_desc = ret_param;
+    if (event_desc == NULL) {
+        printf("Event descriptor not set\n");
+        return -1;
+    }
+
+    if (event_desc->stype == RTE_ETH_EVENT_IPSEC_ESN_OVERFLOW)
+        return inline_ipsec_event_esn_overflow(ctx, event_desc->md);
+    else if (event_desc->stype >= RTE_ETH_EVENT_IPSEC_MAX) {
+        printf("Invalid IPsec event reported\n");
+        return -1;
+    }
+
+    return -1;
+}
+
  int32_t
  main(int32_t argc, char **argv)
  {

@@ -1727,6 +1780,9 @@ main(int32_t argc, char **argv)

           */
          if (promiscuous_on)
              rte_eth_promiscuous_enable(portid);
+
+        rte_eth_dev_callback_register(portid,
+            RTE_ETH_EVENT_IPSEC, inline_ipsec_event_callback, NULL);
      }
        check_all_ports_link_status(nb_ports, enabled_port_mask);

diff --git a/examples/ipsec-secgw/ipsec.c b/examples/ipsec-secgw/ipsec.c
index 5fb5bc1..acdd189 100644
--- a/examples/ipsec-secgw/ipsec.c
+++ b/examples/ipsec-secgw/ipsec.c

@@ -36,6 +36,7 @@ set_ipsec_conf(struct ipsec_sa *sa, struct 

rte_security_ipsec_xform *ipsec)
          }
          /* TODO support for Transport and IPV6 tunnel */
      }
+    ipsec->esn_soft_limit = IPSEC_OFFLOAD_ESN_SOFTLIMIT;
  }
    static inline int

@@ -270,11 +271,14 @@ create_session(struct ipsec_ctx *ipsec_ctx, 

struct ipsec_sa *sa)
               * the packet is received, this userdata will be
               * retrieved using the metadata from the packet.
               *
-             * This is required only for inbound SAs.
+             * The PMD is expected to set similar metadata for other
+             * operations, like rte_eth_event, which are tied to
+             * security session. In such cases, the userdata could
+             * be obtained to uniquely identify the security
+             * parameters denoted.
               */
  -            if (sa->direction == RTE_SECURITY_IPSEC_SA_DIR_INGRESS)
-                sess_conf.userdata = (void *) sa;
+            sess_conf.userdata = (void *) sa;
                sa->sec_session = rte_security_session_create(ctx,
                      &sess_conf, ipsec_ctx->session_pool);

diff --git a/examples/ipsec-secgw/ipsec.h b/examples/ipsec-secgw/ipsec.h
index 6059f6c..c1450f6 100644
--- a/examples/ipsec-secgw/ipsec.h
+++ b/examples/ipsec-secgw/ipsec.h

@@ -21,6 +21,8 @@

    #define MAX_DIGEST_SIZE 32 /* Bytes -- 256 bits */
  +#define IPSEC_OFFLOAD_ESN_SOFTLIMIT 0xffffff00
+
  #define IV_OFFSET        (sizeof(struct rte_crypto_op) + \
                  sizeof(struct rte_crypto_sym_op))

Hi Akhil,

If you are fine with the existing code, I'll send a revised patchset 
incorporating the comment change you had suggested for 3rd patch. Shall 
I proceed?

Thanks,
Anoob

On 14/03/18 11:36, Anoob Joseph wrote:

Hi Akhil,

Please see inline.

Thanks,
Anoob

On 13/03/18 17:54, Akhil Goyal wrote:

quoted

Hi Anoob,

On 3/1/2018 2:51 PM, Anoob Joseph wrote:

quoted

For inline protocol processing, the PMD/device is required to maintain
the ESN. But the application is required to monitor ESN overflow to
initiate SA expiry.

For such cases, application would set the ESN soft limit. An IPsec 
event
would be raised by rte_eth_event framework, when ESN hits the soft 
limit
set by the application.

Signed-off-by: Anoob Joseph <redacted>
---
v2:
* No change

  examples/ipsec-secgw/ipsec-secgw.c | 56 
++++++++++++++++++++++++++++++++++++++
  examples/ipsec-secgw/ipsec.c       | 10 +++++--
  examples/ipsec-secgw/ipsec.h       |  2 ++
  3 files changed, 65 insertions(+), 3 deletions(-)

diff --git a/examples/ipsec-secgw/ipsec-secgw.c 

b/examples/ipsec-secgw/ipsec-secgw.c
index 3a8562e..5726fd3 100644

--- a/examples/ipsec-secgw/ipsec-secgw.c
+++ b/examples/ipsec-secgw/ipsec-secgw.c

@@ -40,6 +40,7 @@

  #include <rte_hash.h>
  #include <rte_jhash.h>
  #include <rte_cryptodev.h>
+#include <rte_security.h>
    #include "ipsec.h"
  #include "parser.h"

@@ -1640,6 +1641,58 @@ pool_init(struct socket_ctx *ctx, int32_t 

socket_id, uint32_t nb_mbuf)
          printf("Allocated mbuf pool on socket %d\n", socket_id);
  }
  +static inline int
+inline_ipsec_event_esn_overflow(struct rte_security_ctx *ctx, 
uint64_t md)
+{
+    struct ipsec_sa *sa;
+
+    /* For inline protocol processing, the metadata in the event will
+     * uniquely identify the security session which raised the event.
+     * Application would then need the userdata it had registered 
with the
+     * security session to process the event.
+     */
+
+    sa = (struct ipsec_sa *)rte_security_get_userdata(ctx, md);
+
+    if (sa == NULL) {
+        /* userdata could not be retrieved */
+        return -1;
+    }
+
+    /* Sequence number over flow. SA need to be re-established */

With this patchset, application will be able to get notification if 
the error has occurred. But it is not re-configuring the SA.
Do you intend to add the same?

Ideally the application should initiate a SA renegotiation sequence 
(with IKE etc). But ipsec-secgw uses predetermined SAs, and so 
addition of SA renegotiation might not fit in with the current design. 
I was just adding this as a place holder for future expansion (and a 
model for real applications).

What are your thoughts on addition here? Similar handling would be 
needed for byte & time expiry as well, when that is added. May be we 
could just log the event and leave it be.

quoted

quoted

+    RTE_SET_USED(sa);
+    return 0;
+}
+
+static int
+inline_ipsec_event_callback(uint16_t port_id, enum 
rte_eth_event_type type,
+         void *param, void *ret_param)
+{
+    struct rte_eth_event_ipsec_desc *event_desc = NULL;
+    struct rte_security_ctx *ctx = (struct rte_security_ctx *)
+                    rte_eth_dev_get_sec_ctx(port_id);
+
+    RTE_SET_USED(param);
+
+    if (type != RTE_ETH_EVENT_IPSEC)
+        return -1;
+
+    event_desc = ret_param;
+    if (event_desc == NULL) {
+        printf("Event descriptor not set\n");
+        return -1;
+    }
+
+    if (event_desc->stype == RTE_ETH_EVENT_IPSEC_ESN_OVERFLOW)
+        return inline_ipsec_event_esn_overflow(ctx, event_desc->md);
+    else if (event_desc->stype >= RTE_ETH_EVENT_IPSEC_MAX) {
+        printf("Invalid IPsec event reported\n");
+        return -1;
+    }
+
+    return -1;
+}
+
  int32_t
  main(int32_t argc, char **argv)
  {

@@ -1727,6 +1780,9 @@ main(int32_t argc, char **argv)

           */
          if (promiscuous_on)
              rte_eth_promiscuous_enable(portid);
+
+        rte_eth_dev_callback_register(portid,
+            RTE_ETH_EVENT_IPSEC, inline_ipsec_event_callback, NULL);
      }
        check_all_ports_link_status(nb_ports, enabled_port_mask);

diff --git a/examples/ipsec-secgw/ipsec.c 

b/examples/ipsec-secgw/ipsec.c
index 5fb5bc1..acdd189 100644

--- a/examples/ipsec-secgw/ipsec.c
+++ b/examples/ipsec-secgw/ipsec.c

@@ -36,6 +36,7 @@ set_ipsec_conf(struct ipsec_sa *sa, struct 

rte_security_ipsec_xform *ipsec)
          }
          /* TODO support for Transport and IPV6 tunnel */
      }
+    ipsec->esn_soft_limit = IPSEC_OFFLOAD_ESN_SOFTLIMIT;
  }
    static inline int

@@ -270,11 +271,14 @@ create_session(struct ipsec_ctx *ipsec_ctx, 

struct ipsec_sa *sa)
               * the packet is received, this userdata will be
               * retrieved using the metadata from the packet.
               *
-             * This is required only for inbound SAs.
+             * The PMD is expected to set similar metadata for other
+             * operations, like rte_eth_event, which are tied to
+             * security session. In such cases, the userdata could
+             * be obtained to uniquely identify the security
+             * parameters denoted.
               */
  -            if (sa->direction == RTE_SECURITY_IPSEC_SA_DIR_INGRESS)
-                sess_conf.userdata = (void *) sa;
+            sess_conf.userdata = (void *) sa;
                sa->sec_session = rte_security_session_create(ctx,
                      &sess_conf, ipsec_ctx->session_pool);

diff --git a/examples/ipsec-secgw/ipsec.h 

b/examples/ipsec-secgw/ipsec.h
index 6059f6c..c1450f6 100644

--- a/examples/ipsec-secgw/ipsec.h
+++ b/examples/ipsec-secgw/ipsec.h

@@ -21,6 +21,8 @@

    #define MAX_DIGEST_SIZE 32 /* Bytes -- 256 bits */
  +#define IPSEC_OFFLOAD_ESN_SOFTLIMIT 0xffffff00
+
  #define IV_OFFSET        (sizeof(struct rte_crypto_op) + \
                  sizeof(struct rte_crypto_sym_op))

Hi Anoob,

On 3/21/2018 10:50 AM, Anoob Joseph wrote:

Hi Akhil,

If you are fine with the existing code, I'll send a revised patchset 
incorporating the comment change you had suggested for 3rd patch. Shall 
I proceed?

Thanks,
Anoob

Yes you can send the patchset with existing code.
BTW we are open for an approach to add sa rediscovery in the application 
in future.

Thanks,
Akhil

On 14/03/18 11:36, Anoob Joseph wrote:

quoted

Hi Akhil,

Please see inline.

Thanks,
Anoob

On 13/03/18 17:54, Akhil Goyal wrote:

quoted

Hi Anoob,

On 3/1/2018 2:51 PM, Anoob Joseph wrote:

quoted

For inline protocol processing, the PMD/device is required to maintain
the ESN. But the application is required to monitor ESN overflow to
initiate SA expiry.

For such cases, application would set the ESN soft limit. An IPsec 
event
would be raised by rte_eth_event framework, when ESN hits the soft 
limit
set by the application.

Signed-off-by: Anoob Joseph <redacted>
---
v2:
* No change

  examples/ipsec-secgw/ipsec-secgw.c | 56 
++++++++++++++++++++++++++++++++++++++
  examples/ipsec-secgw/ipsec.c       | 10 +++++--
  examples/ipsec-secgw/ipsec.h       |  2 ++
  3 files changed, 65 insertions(+), 3 deletions(-)

diff --git a/examples/ipsec-secgw/ipsec-secgw.c 

b/examples/ipsec-secgw/ipsec-secgw.c
index 3a8562e..5726fd3 100644

--- a/examples/ipsec-secgw/ipsec-secgw.c
+++ b/examples/ipsec-secgw/ipsec-secgw.c

@@ -40,6 +40,7 @@

  #include <rte_hash.h>
  #include <rte_jhash.h>
  #include <rte_cryptodev.h>
+#include <rte_security.h>
    #include "ipsec.h"
  #include "parser.h"

@@ -1640,6 +1641,58 @@ pool_init(struct socket_ctx *ctx, int32_t 

socket_id, uint32_t nb_mbuf)
          printf("Allocated mbuf pool on socket %d\n", socket_id);
  }
  +static inline int
+inline_ipsec_event_esn_overflow(struct rte_security_ctx *ctx, 
uint64_t md)
+{
+    struct ipsec_sa *sa;
+
+    /* For inline protocol processing, the metadata in the event will
+     * uniquely identify the security session which raised the event.
+     * Application would then need the userdata it had registered 
with the
+     * security session to process the event.
+     */
+
+    sa = (struct ipsec_sa *)rte_security_get_userdata(ctx, md);
+
+    if (sa == NULL) {
+        /* userdata could not be retrieved */
+        return -1;
+    }
+
+    /* Sequence number over flow. SA need to be re-established */

With this patchset, application will be able to get notification if 
the error has occurred. But it is not re-configuring the SA.
Do you intend to add the same?

Ideally the application should initiate a SA renegotiation sequence 
(with IKE etc). But ipsec-secgw uses predetermined SAs, and so 
addition of SA renegotiation might not fit in with the current design. 
I was just adding this as a place holder for future expansion (and a 
model for real applications).

What are your thoughts on addition here? Similar handling would be 
needed for byte & time expiry as well, when that is added. May be we 
could just log the event and leave it be.

quoted

quoted

+    RTE_SET_USED(sa);
+    return 0;
+}
+
+static int
+inline_ipsec_event_callback(uint16_t port_id, enum 
rte_eth_event_type type,
+         void *param, void *ret_param)
+{
+    struct rte_eth_event_ipsec_desc *event_desc = NULL;
+    struct rte_security_ctx *ctx = (struct rte_security_ctx *)
+                    rte_eth_dev_get_sec_ctx(port_id);
+
+    RTE_SET_USED(param);
+
+    if (type != RTE_ETH_EVENT_IPSEC)
+        return -1;
+
+    event_desc = ret_param;
+    if (event_desc == NULL) {
+        printf("Event descriptor not set\n");
+        return -1;
+    }
+
+    if (event_desc->stype == RTE_ETH_EVENT_IPSEC_ESN_OVERFLOW)
+        return inline_ipsec_event_esn_overflow(ctx, event_desc->md);
+    else if (event_desc->stype >= RTE_ETH_EVENT_IPSEC_MAX) {
+        printf("Invalid IPsec event reported\n");
+        return -1;
+    }
+
+    return -1;
+}
+
  int32_t
  main(int32_t argc, char **argv)
  {

@@ -1727,6 +1780,9 @@ main(int32_t argc, char **argv)

           */
          if (promiscuous_on)
              rte_eth_promiscuous_enable(portid);
+
+        rte_eth_dev_callback_register(portid,
+            RTE_ETH_EVENT_IPSEC, inline_ipsec_event_callback, NULL);
      }
        check_all_ports_link_status(nb_ports, enabled_port_mask);

diff --git a/examples/ipsec-secgw/ipsec.c 

b/examples/ipsec-secgw/ipsec.c
index 5fb5bc1..acdd189 100644

--- a/examples/ipsec-secgw/ipsec.c
+++ b/examples/ipsec-secgw/ipsec.c

@@ -36,6 +36,7 @@ set_ipsec_conf(struct ipsec_sa *sa, struct 

rte_security_ipsec_xform *ipsec)
          }
          /* TODO support for Transport and IPV6 tunnel */
      }
+    ipsec->esn_soft_limit = IPSEC_OFFLOAD_ESN_SOFTLIMIT;
  }
    static inline int

@@ -270,11 +271,14 @@ create_session(struct ipsec_ctx *ipsec_ctx, 

struct ipsec_sa *sa)
               * the packet is received, this userdata will be
               * retrieved using the metadata from the packet.
               *
-             * This is required only for inbound SAs.
+             * The PMD is expected to set similar metadata for other
+             * operations, like rte_eth_event, which are tied to
+             * security session. In such cases, the userdata could
+             * be obtained to uniquely identify the security
+             * parameters denoted.
               */
  -            if (sa->direction == RTE_SECURITY_IPSEC_SA_DIR_INGRESS)
-                sess_conf.userdata = (void *) sa;
+            sess_conf.userdata = (void *) sa;
                sa->sec_session = rte_security_session_create(ctx,
                      &sess_conf, ipsec_ctx->session_pool);

diff --git a/examples/ipsec-secgw/ipsec.h 

b/examples/ipsec-secgw/ipsec.h
index 6059f6c..c1450f6 100644

--- a/examples/ipsec-secgw/ipsec.h
+++ b/examples/ipsec-secgw/ipsec.h

@@ -21,6 +21,8 @@

    #define MAX_DIGEST_SIZE 32 /* Bytes -- 256 bits */
  +#define IPSEC_OFFLOAD_ESN_SOFTLIMIT 0xffffff00
+
  #define IV_OFFSET        (sizeof(struct rte_crypto_op) + \
                  sizeof(struct rte_crypto_sym_op))

This series enables application to set the sequence number soft limit
for IPsec offload. In inline IPsec offload, as the sequence number
(maintained by PMD/device) reaches the specified soft limit, the PMD
would raise an "IPSEC_EVENT". This event would have some metadata,
which would be used by the application to identify the SA on which the
sequence number overflow is about to happen.

Anoob Joseph (5):
  lib/ethdev: support for inline IPsec events
  lib/security: add ESN soft limit in conf
  lib/security: extend userdata for IPsec events
  examples/ipsec-secgw: handle ESN soft limit event
  app/testpmd: support for IPsec event

 app/test-pmd/parameters.c                 |  2 ++
 app/test-pmd/testpmd.c                    |  2 ++
 examples/ipsec-secgw/ipsec-secgw.c        | 56 +++++++++++++++++++++++++++++++
 examples/ipsec-secgw/ipsec.c              | 10 ++++--
 examples/ipsec-secgw/ipsec.h              |  2 ++
 lib/librte_ether/rte_ethdev.h             | 28 ++++++++++++++++
 lib/librte_security/rte_security.h        | 16 +++++----
 lib/librte_security/rte_security_driver.h |  6 ++--
 8 files changed, 110 insertions(+), 12 deletions(-)

-- 
2.7.4

Adding support for IPsec events in rte_eth_event framework. In inline
IPsec offload, the per packet protocol defined variables, like ESN,
would be managed by PMD. In such cases, PMD would need IPsec events
to notify application about various conditions like, ESN overflow.

Signed-off-by: Anoob Joseph <redacted>
---
v3:
* No change

v2:
* Added time expiry & byte expiry IPsec events in the enum

 lib/librte_ether/rte_ethdev.h | 28 ++++++++++++++++++++++++++++
 1 file changed, 28 insertions(+)

diff --git a/lib/librte_ether/rte_ethdev.h b/lib/librte_ether/rte_ethdev.h
index 0361533..96b2aa0 100644
--- a/lib/librte_ether/rte_ethdev.h
+++ b/lib/librte_ether/rte_ethdev.h

@@ -2438,6 +2438,33 @@ int
 rte_eth_tx_done_cleanup(uint16_t port_id, uint16_t queue_id, uint32_t free_cnt);
 
 /**
+ * Subtypes for IPsec offload events raised by eth device.
+ */
+enum rte_eth_event_ipsec_subtype {
+	RTE_ETH_EVENT_IPSEC_UNKNOWN = 0,
+			/** Unknown event type */
+	RTE_ETH_EVENT_IPSEC_ESN_OVERFLOW,
+			/** Sequence number overflow in security offload */
+	RTE_ETH_EVENT_IPSEC_SA_TIME_EXPIRY,
+			/** Soft time expiry of SA */
+	RTE_ETH_EVENT_IPSEC_SA_BYTE_EXPIRY,
+			/** Soft byte expiry of SA */
+	RTE_ETH_EVENT_IPSEC_MAX
+			/** Max value of this enum */
+};
+
+/**
+ * Descriptor for IPsec event. Used by eth dev to send extra information of the
+ * event.
+ */
+struct rte_eth_event_ipsec_desc {
+	enum rte_eth_event_ipsec_subtype stype;
+			/** Type of IPsec event */
+	uint64_t md;
+			/** Event specific metadata */
+};
+
+/**
  * The eth device event type for interrupt, and maybe others in the future.
  */
 enum rte_eth_event_type {

@@ -2448,6 +2475,7 @@ enum rte_eth_event_type {
 	RTE_ETH_EVENT_INTR_RESET,
 			/**< reset interrupt event, sent to VF on PF reset */
 	RTE_ETH_EVENT_VF_MBOX,  /**< message from the VF received by PF */
+	RTE_ETH_EVENT_IPSEC,    /**< IPsec offload related event */
 	RTE_ETH_EVENT_MACSEC,   /**< MACsec offload related event */
 	RTE_ETH_EVENT_INTR_RMV, /**< device removal event */
 	RTE_ETH_EVENT_NEW,      /**< port is probed */

-- 
2.7.4

Adding ESN soft limit in conf. This will be used in case of protocol
offload. Per SA, application could specify for what ESN the security
device need to notify application. In case of eth dev(inline protocol),
rte_eth_event framework would raise an IPsec event.

Signed-off-by: Anoob Joseph <redacted>
---
v3:
* Minor change in the comment

v2:
* No change

 lib/librte_security/rte_security.h | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/lib/librte_security/rte_security.h b/lib/librte_security/rte_security.h
index c75c121..ca1e912 100644
--- a/lib/librte_security/rte_security.h
+++ b/lib/librte_security/rte_security.h

@@ -222,6 +222,8 @@ struct rte_security_ipsec_xform {
 	/**< IPsec SA Mode - transport/tunnel */
 	struct rte_security_ipsec_tunnel_param tunnel;
 	/**< Tunnel parameters, NULL for transport mode */
+	uint64_t esn_soft_limit;
+	/**< ESN for which the overflow event need to be raised */
 };
 
 /**

-- 
2.7.4

Extending 'userdata' to be used for IPsec events too.

IPsec events would have some metadata which would uniquely identify the
security session for which the event is raised. But application would
need some construct which it can understand. The 'userdata' solves a
similar problem for inline processed inbound traffic. Updating the
documentation to extend the usage of 'userdata'.

Signed-off-by: Anoob Joseph <redacted>
---
v3:
* No change

v2:
* No change

 lib/librte_security/rte_security.h        | 14 ++++++++------
 lib/librte_security/rte_security_driver.h |  6 +++---
 2 files changed, 11 insertions(+), 9 deletions(-)

diff --git a/lib/librte_security/rte_security.h b/lib/librte_security/rte_security.h
index ca1e912..afa2861 100644
--- a/lib/librte_security/rte_security.h
+++ b/lib/librte_security/rte_security.h

@@ -364,15 +364,17 @@ rte_security_set_pkt_metadata(struct rte_security_ctx *instance,
 			      struct rte_mbuf *mb, void *params);
 
 /**
- * Get userdata associated with the security session which processed the
- * packet. This userdata would be registered while creating the session, and
- * application can use this to identify the SA etc. Device-specific metadata
- * in the mbuf would be used for this.
+ * Get userdata associated with the security session. Device specific metadata
+ * provided would be used to uniquely identify the security session being
+ * referred to. This userdata would be registered while creating the session,
+ * and application can use this to identify the SA etc.
  *
- * This is valid only for inline processed ingress packets.
+ * Device specific metadata would be set in mbuf for inline processed inbound
+ * packets. In addition, the same metadata would be set for IPsec events
+ * reported by rte_eth_event framework.
  *
  * @param   instance	security instance
- * @param   md		device-specific metadata set in mbuf
+ * @param   md		device-specific metadata
  *
  * @return
  *  - On success, userdata

diff --git a/lib/librte_security/rte_security_driver.h b/lib/librte_security/rte_security_driver.h
index 4623904..0583f88 100644
--- a/lib/librte_security/rte_security_driver.h
+++ b/lib/librte_security/rte_security_driver.h

@@ -134,9 +134,9 @@ typedef int (*security_set_pkt_metadata_t)(void *device,
 		void *params);
 
 /**
- * Get application specific userdata associated with the security session which
- * processed the packet. This would be retrieved using the metadata obtained
- * from packet.
+ * Get application specific userdata associated with the security session.
+ * Device specific metadata provided would be used to uniquely identify
+ * the security session being referred to.
  *
  * @param	device		Crypto/eth device pointer
  * @param	md		Metadata

-- 
2.7.4

For inline protocol processing, the PMD/device is required to maintain
the ESN. But the application is required to monitor ESN overflow to
initiate SA expiry.

For such cases, application would set the ESN soft limit. An IPsec event
would be raised by rte_eth_event framework, when ESN hits the soft limit
set by the application.

Signed-off-by: Anoob Joseph <redacted>
---
v3:
* No change

v2:
* No change

 examples/ipsec-secgw/ipsec-secgw.c | 56 ++++++++++++++++++++++++++++++++++++++
 examples/ipsec-secgw/ipsec.c       | 10 +++++--
 examples/ipsec-secgw/ipsec.h       |  2 ++
 3 files changed, 65 insertions(+), 3 deletions(-)

diff --git a/examples/ipsec-secgw/ipsec-secgw.c b/examples/ipsec-secgw/ipsec-secgw.c
index 3a8562e..5726fd3 100644
--- a/examples/ipsec-secgw/ipsec-secgw.c
+++ b/examples/ipsec-secgw/ipsec-secgw.c

@@ -40,6 +40,7 @@
 #include <rte_hash.h>
 #include <rte_jhash.h>
 #include <rte_cryptodev.h>
+#include <rte_security.h>
 
 #include "ipsec.h"
 #include "parser.h"

@@ -1640,6 +1641,58 @@ pool_init(struct socket_ctx *ctx, int32_t socket_id, uint32_t nb_mbuf)
 		printf("Allocated mbuf pool on socket %d\n", socket_id);
 }
 
+static inline int
+inline_ipsec_event_esn_overflow(struct rte_security_ctx *ctx, uint64_t md)
+{
+	struct ipsec_sa *sa;
+
+	/* For inline protocol processing, the metadata in the event will
+	 * uniquely identify the security session which raised the event.
+	 * Application would then need the userdata it had registered with the
+	 * security session to process the event.
+	 */
+
+	sa = (struct ipsec_sa *)rte_security_get_userdata(ctx, md);
+
+	if (sa == NULL) {
+		/* userdata could not be retrieved */
+		return -1;
+	}
+
+	/* Sequence number over flow. SA need to be re-established */
+	RTE_SET_USED(sa);
+	return 0;
+}
+
+static int
+inline_ipsec_event_callback(uint16_t port_id, enum rte_eth_event_type type,
+		 void *param, void *ret_param)
+{
+	struct rte_eth_event_ipsec_desc *event_desc = NULL;
+	struct rte_security_ctx *ctx = (struct rte_security_ctx *)
+					rte_eth_dev_get_sec_ctx(port_id);
+
+	RTE_SET_USED(param);
+
+	if (type != RTE_ETH_EVENT_IPSEC)
+		return -1;
+
+	event_desc = ret_param;
+	if (event_desc == NULL) {
+		printf("Event descriptor not set\n");
+		return -1;
+	}
+
+	if (event_desc->stype == RTE_ETH_EVENT_IPSEC_ESN_OVERFLOW)
+		return inline_ipsec_event_esn_overflow(ctx, event_desc->md);
+	else if (event_desc->stype >= RTE_ETH_EVENT_IPSEC_MAX) {
+		printf("Invalid IPsec event reported\n");
+		return -1;
+	}
+
+	return -1;
+}
+
 int32_t
 main(int32_t argc, char **argv)
 {

@@ -1727,6 +1780,9 @@ main(int32_t argc, char **argv)
 		 */
 		if (promiscuous_on)
 			rte_eth_promiscuous_enable(portid);
+
+		rte_eth_dev_callback_register(portid,
+			RTE_ETH_EVENT_IPSEC, inline_ipsec_event_callback, NULL);
 	}
 
 	check_all_ports_link_status(nb_ports, enabled_port_mask);

diff --git a/examples/ipsec-secgw/ipsec.c b/examples/ipsec-secgw/ipsec.c
index 5fb5bc1..acdd189 100644
--- a/examples/ipsec-secgw/ipsec.c
+++ b/examples/ipsec-secgw/ipsec.c

@@ -36,6 +36,7 @@ set_ipsec_conf(struct ipsec_sa *sa, struct rte_security_ipsec_xform *ipsec)
 		}
 		/* TODO support for Transport and IPV6 tunnel */
 	}
+	ipsec->esn_soft_limit = IPSEC_OFFLOAD_ESN_SOFTLIMIT;
 }
 
 static inline int

@@ -270,11 +271,14 @@ create_session(struct ipsec_ctx *ipsec_ctx, struct ipsec_sa *sa)
 			 * the packet is received, this userdata will be
 			 * retrieved using the metadata from the packet.
 			 *
-			 * This is required only for inbound SAs.
+			 * The PMD is expected to set similar metadata for other
+			 * operations, like rte_eth_event, which are tied to
+			 * security session. In such cases, the userdata could
+			 * be obtained to uniquely identify the security
+			 * parameters denoted.
 			 */
 
-			if (sa->direction == RTE_SECURITY_IPSEC_SA_DIR_INGRESS)
-				sess_conf.userdata = (void *) sa;
+			sess_conf.userdata = (void *) sa;
 
 			sa->sec_session = rte_security_session_create(ctx,
 					&sess_conf, ipsec_ctx->session_pool);

diff --git a/examples/ipsec-secgw/ipsec.h b/examples/ipsec-secgw/ipsec.h
index 6059f6c..c1450f6 100644
--- a/examples/ipsec-secgw/ipsec.h
+++ b/examples/ipsec-secgw/ipsec.h

@@ -21,6 +21,8 @@
 
 #define MAX_DIGEST_SIZE 32 /* Bytes -- 256 bits */
 
+#define IPSEC_OFFLOAD_ESN_SOFTLIMIT 0xffffff00
+
 #define IV_OFFSET		(sizeof(struct rte_crypto_op) + \
 				sizeof(struct rte_crypto_sym_op))
 

-- 
2.7.4

Adding support for IPsec event

Signed-off-by: Anoob Joseph <redacted>
---
v3:
* No change

v2:
* No change

 app/test-pmd/parameters.c | 2 ++
 app/test-pmd/testpmd.c    | 2 ++
 2 files changed, 4 insertions(+)

diff --git a/app/test-pmd/parameters.c b/app/test-pmd/parameters.c
index 97d22b8..7ea882f 100644
--- a/app/test-pmd/parameters.c
+++ b/app/test-pmd/parameters.c

@@ -512,6 +512,8 @@ parse_event_printing_config(const char *optarg, int enable)
 		mask = UINT32_C(1) << RTE_ETH_EVENT_INTR_RESET;
 	else if (!strcmp(optarg, "vf_mbox"))
 		mask = UINT32_C(1) << RTE_ETH_EVENT_VF_MBOX;
+	else if (!strcmp(optarg, "ipsec"))
+		mask = UINT32_C(1) << RTE_ETH_EVENT_IPSEC;
 	else if (!strcmp(optarg, "macsec"))
 		mask = UINT32_C(1) << RTE_ETH_EVENT_MACSEC;
 	else if (!strcmp(optarg, "intr_rmv"))

diff --git a/app/test-pmd/testpmd.c b/app/test-pmd/testpmd.c
index 4c0e258..32fb8b1 100644
--- a/app/test-pmd/testpmd.c
+++ b/app/test-pmd/testpmd.c

@@ -292,6 +292,7 @@ uint32_t event_print_mask = (UINT32_C(1) << RTE_ETH_EVENT_UNKNOWN) |
 			    (UINT32_C(1) << RTE_ETH_EVENT_INTR_LSC) |
 			    (UINT32_C(1) << RTE_ETH_EVENT_QUEUE_STATE) |
 			    (UINT32_C(1) << RTE_ETH_EVENT_INTR_RESET) |
+			    (UINT32_C(1) << RTE_ETH_EVENT_IPSEC) |
 			    (UINT32_C(1) << RTE_ETH_EVENT_MACSEC) |
 			    (UINT32_C(1) << RTE_ETH_EVENT_INTR_RMV);
 

@@ -2024,6 +2025,7 @@ eth_event_callback(portid_t port_id, enum rte_eth_event_type type, void *param,
 		[RTE_ETH_EVENT_QUEUE_STATE] = "Queue state",
 		[RTE_ETH_EVENT_INTR_RESET] = "Interrupt reset",
 		[RTE_ETH_EVENT_VF_MBOX] = "VF Mbox",
+		[RTE_ETH_EVENT_IPSEC] = "IPsec",
 		[RTE_ETH_EVENT_MACSEC] = "MACsec",
 		[RTE_ETH_EVENT_INTR_RMV] = "device removal",
 		[RTE_ETH_EVENT_NEW] = "device probed",

-- 
2.7.4

On 3/21/2018 4:41 PM, Anoob Joseph wrote:

Adding support for IPsec events in rte_eth_event framework. In inline
IPsec offload, the per packet protocol defined variables, like ESN,
would be managed by PMD. In such cases, PMD would need IPsec events
to notify application about various conditions like, ESN overflow.

Signed-off-by: Anoob Joseph <redacted>
---
v3:
* No change

v2:
* Added time expiry & byte expiry IPsec events in the enum

  lib/librte_ether/rte_ethdev.h | 28 ++++++++++++++++++++++++++++
  1 file changed, 28 insertions(+)

Series Acked-by: Akhil Goyal [off-list ref]

Adding maintainers of testpmd & lib/ethdev

Thanks,
Anoob

On 21/03/18 16:41, Anoob Joseph wrote:

This series enables application to set the sequence number soft limit
for IPsec offload. In inline IPsec offload, as the sequence number
(maintained by PMD/device) reaches the specified soft limit, the PMD
would raise an "IPSEC_EVENT". This event would have some metadata,
which would be used by the application to identify the SA on which the
sequence number overflow is about to happen.

Anoob Joseph (5):
   lib/ethdev: support for inline IPsec events
   lib/security: add ESN soft limit in conf
   lib/security: extend userdata for IPsec events
   examples/ipsec-secgw: handle ESN soft limit event
   app/testpmd: support for IPsec event

  app/test-pmd/parameters.c                 |  2 ++
  app/test-pmd/testpmd.c                    |  2 ++
  examples/ipsec-secgw/ipsec-secgw.c        | 56 +++++++++++++++++++++++++++++++
  examples/ipsec-secgw/ipsec.c              | 10 ++++--
  examples/ipsec-secgw/ipsec.h              |  2 ++
  lib/librte_ether/rte_ethdev.h             | 28 ++++++++++++++++
  lib/librte_security/rte_security.h        | 16 +++++----
  lib/librte_security/rte_security_driver.h |  6 ++--
  8 files changed, 110 insertions(+), 12 deletions(-)

Adding maintainers of testpmd & lib/ethdev

Thanks,
Anoob
On 21/03/18 16:41, Anoob Joseph wrote: