[PATCH v2 net-next 0/7] qed*: RDMA and infrastructure for iWARP

STALE3405d SUPERSEDED

Revision v2 of 5 in this series.

6 messages, 4 authors, 2017-06-19 · open the first message on its own page

[PATCH v2 net-next 0/7] qed*: RDMA and infrastructure for iWARP

From: Yuval Mintz <hidden>
Date: 2017-06-18 11:50:28

This series focuses on RDMA in general with emphasis on required changes
toward adding iWARP support. The vast majority of the changes introduced
are in qed/qede, with a couple of small changes to qedr
[mentioned below].

The infrastructure changes:
 - Patch #1 adds the ability to pass PBL memory externally for a newly
created chain.
 - Patches #4, #5 rename qede_roce.[ch] into qede_rdma.[ch] + change
prefixes from _roce_ to _rdma_, as the API between qede and qedr is
agnostic to the variant of the RDMA protocol used. These patches also
touch qedr [basically to align it with the renaming, nothing more].
 - Patch #7 replaces the current SPQ async mechanism into serving
registered callbacks [before adding iWARP which would add another client
in need of this sort of functionallity].

The non-infrastrucutre changes:
 - Patches #2, #3 contain DCB-related changes to better align RDMA with
configured DCB.
 - Patch #6 contains a minor [mostly theoretical fix] to release flow.

Dave,

Please consider applying this series to `net-next'.

Thanks,
Yuval

Changes from previous versions
------------------------------
 - V2: Add several inclusion into qede_rdma.h to have proper declarations
   of all variable types used in it

Michal Kalderon (3):
  qed*: Rename qede_roce.[ch]
  qed*: Set rdma generic functions prefix
  qed: Wait for resources before FUNC_CLOSE
  qed: SPQ async callback registration

Yuval Mintz (4):
  qed: Chain support for external PBL
  qed: RoCE EDPM to honor PFC
  qed: Disable RoCE dpm when DCBx change occurs

 drivers/infiniband/hw/qedr/main.c                  |  10 +-
 drivers/infiniband/hw/qedr/qedr.h                  |   2 +-
 drivers/infiniband/hw/qedr/verbs.c                 |   6 +-
 drivers/net/ethernet/qlogic/qed/qed_dcbx.c         |  24 ++++
 drivers/net/ethernet/qlogic/qed/qed_dev.c          |  35 +++--
 drivers/net/ethernet/qlogic/qed/qed_dev_api.h      |   5 +-
 drivers/net/ethernet/qlogic/qed/qed_iscsi.c        |  30 ++++-
 drivers/net/ethernet/qlogic/qed/qed_ll2.c          |   6 +-
 drivers/net/ethernet/qlogic/qed/qed_reg_addr.h     |   6 +
 drivers/net/ethernet/qlogic/qed/qed_roce.c         |  87 ++++++++++---
 drivers/net/ethernet/qlogic/qed/qed_roce.h         |   9 +-
 drivers/net/ethernet/qlogic/qed/qed_sp.h           |  17 +++
 drivers/net/ethernet/qlogic/qed/qed_spq.c          |  60 +++++----
 drivers/net/ethernet/qlogic/qed/qed_sriov.c        |  16 ++-
 drivers/net/ethernet/qlogic/qed/qed_sriov.h        |  18 ---
 drivers/net/ethernet/qlogic/qede/Makefile          |   2 +-
 drivers/net/ethernet/qlogic/qede/qede.h            |   5 +-
 drivers/net/ethernet/qlogic/qede/qede_main.c       |  21 ++-
 .../qlogic/qede/{qede_roce.c => qede_rdma.c}       | 144 ++++++++++-----------
 include/linux/qed/qed_chain.h                      |   7 +
 include/linux/qed/qed_if.h                         |   3 +-
 include/linux/qed/{qede_roce.h => qede_rdma.h}     |  42 +++---
 22 files changed, 353 insertions(+), 202 deletions(-)
 rename drivers/net/ethernet/qlogic/qede/{qede_roce.c => qede_rdma.c} (59%)
 rename include/linux/qed/{qede_roce.h => qede_rdma.h} (65%)

-- 
2.9.4

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v2 net-next 2/7] qed: RoCE EDPM to honor PFC

From: Yuval Mintz <hidden>
Date: 2017-06-18 11:51:17

Configure device according to DCBx results so that EDPMs
made by RoCE would honor flow-control.

Signed-off-by: Yuval Mintz <redacted>
---
 drivers/net/ethernet/qlogic/qed/qed_dcbx.c     | 16 ++++++++++++++++
 drivers/net/ethernet/qlogic/qed/qed_reg_addr.h |  6 ++++++
 2 files changed, 22 insertions(+)
diff --git a/drivers/net/ethernet/qlogic/qed/qed_dcbx.c b/drivers/net/ethernet/qlogic/qed/qed_dcbx.c
index e2a62c0..15b516a 100644
--- a/drivers/net/ethernet/qlogic/qed/qed_dcbx.c
+++ b/drivers/net/ethernet/qlogic/qed/qed_dcbx.c
@@ -896,6 +896,22 @@ qed_dcbx_mib_update_event(struct qed_hwfn *p_hwfn,
 	}
 
 	qed_dcbx_get_params(p_hwfn, &p_hwfn->p_dcbx_info->get, type);
+
+	if (type == QED_DCBX_OPERATIONAL_MIB) {
+		struct qed_dcbx_results *p_data;
+		u16 val;
+
+		/* Configure in NIG which protocols support EDPM and should
+		 * honor PFC.
+		 */
+		p_data = &p_hwfn->p_dcbx_info->results;
+		val = (0x1 << p_data->arr[DCBX_PROTOCOL_ROCE].tc) |
+		      (0x1 << p_data->arr[DCBX_PROTOCOL_ROCE_V2].tc);
+		val <<= NIG_REG_TX_EDPM_CTRL_TX_EDPM_TC_EN_SHIFT;
+		val |= NIG_REG_TX_EDPM_CTRL_TX_EDPM_EN;
+		qed_wr(p_hwfn, p_ptt, NIG_REG_TX_EDPM_CTRL, val);
+	}
+
 	qed_dcbx_aen(p_hwfn, type);
 
 	return rc;
diff --git a/drivers/net/ethernet/qlogic/qed/qed_reg_addr.h b/drivers/net/ethernet/qlogic/qed/qed_reg_addr.h
index 7e4639c..0cdb433 100644
--- a/drivers/net/ethernet/qlogic/qed/qed_reg_addr.h
+++ b/drivers/net/ethernet/qlogic/qed/qed_reg_addr.h
@@ -1564,6 +1564,12 @@
 #define NIG_REG_TSGEN_FREECNT_UPDATE_K2 0x509008UL
 #define CNIG_REG_NIG_PORT0_CONF_K2 0x218200UL
 
+#define NIG_REG_TX_EDPM_CTRL 0x501f0cUL
+#define NIG_REG_TX_EDPM_CTRL_TX_EDPM_EN (0x1 << 0)
+#define NIG_REG_TX_EDPM_CTRL_TX_EDPM_EN_SHIFT 0
+#define NIG_REG_TX_EDPM_CTRL_TX_EDPM_TC_EN (0xff << 1)
+#define NIG_REG_TX_EDPM_CTRL_TX_EDPM_TC_EN_SHIFT 1
+
 #define PRS_REG_SEARCH_GFT 0x1f11bcUL
 #define PRS_REG_CM_HDR_GFT 0x1f11c8UL
 #define PRS_REG_GFT_CAM 0x1f1100UL
-- 
2.9.4

[PATCH v2 net-next 5/7] qed*: Set rdma generic functions prefix

From: Yuval Mintz <hidden>
Date: 2017-06-18 11:51:31

From: Michal Kalderon <redacted>

Rename the functions common to both iWARP and RoCE to have a prefix of
_rdma_ instead of _roce_.

Signed-off-by: Michal Kalderon <redacted>
Signed-off-by: Yuval Mintz <redacted>
---
 drivers/infiniband/hw/qedr/main.c            |   6 +-
 drivers/net/ethernet/qlogic/qede/qede.h      |   4 +-
 drivers/net/ethernet/qlogic/qede/qede_main.c |  12 +--
 drivers/net/ethernet/qlogic/qede/qede_rdma.c | 142 +++++++++++++--------------
 include/linux/qed/qede_rdma.h                |  37 +++----
 5 files changed, 101 insertions(+), 100 deletions(-)
diff --git a/drivers/infiniband/hw/qedr/main.c b/drivers/infiniband/hw/qedr/main.c
index 714eb0c..b5851fd 100644
--- a/drivers/infiniband/hw/qedr/main.c
+++ b/drivers/infiniband/hw/qedr/main.c
@@ -902,7 +902,7 @@ static void qedr_mac_address_change(struct qedr_dev *dev)
  * initialization done before RoCE driver notifies
  * event to stack.
  */
-static void qedr_notify(struct qedr_dev *dev, enum qede_roce_event event)
+static void qedr_notify(struct qedr_dev *dev, enum qede_rdma_event event)
 {
 	switch (event) {
 	case QEDE_UP:
@@ -931,12 +931,12 @@ static struct qedr_driver qedr_drv = {
 
 static int __init qedr_init_module(void)
 {
-	return qede_roce_register_driver(&qedr_drv);
+	return qede_rdma_register_driver(&qedr_drv);
 }
 
 static void __exit qedr_exit_module(void)
 {
-	qede_roce_unregister_driver(&qedr_drv);
+	qede_rdma_unregister_driver(&qedr_drv);
 }
 
 module_init(qedr_init_module);
diff --git a/drivers/net/ethernet/qlogic/qede/qede.h b/drivers/net/ethernet/qlogic/qede/qede.h
index 2d6b30c..4dfb238 100644
--- a/drivers/net/ethernet/qlogic/qede/qede.h
+++ b/drivers/net/ethernet/qlogic/qede/qede.h
@@ -154,8 +154,8 @@ struct qede_vlan {
 struct qede_rdma_dev {
 	struct qedr_dev *qedr_dev;
 	struct list_head entry;
-	struct list_head roce_event_list;
-	struct workqueue_struct *roce_wq;
+	struct list_head rdma_event_list;
+	struct workqueue_struct *rdma_wq;
 };
 
 struct qede_ptp;
diff --git a/drivers/net/ethernet/qlogic/qede/qede_main.c b/drivers/net/ethernet/qlogic/qede/qede_main.c
index e9eaa38..06ca13d 100644
--- a/drivers/net/ethernet/qlogic/qede/qede_main.c
+++ b/drivers/net/ethernet/qlogic/qede/qede_main.c
@@ -262,7 +262,7 @@ static int qede_netdev_event(struct notifier_block *this, unsigned long event,
 		break;
 	case NETDEV_CHANGEADDR:
 		edev = netdev_priv(ndev);
-		qede_roce_event_changeaddr(edev);
+		qede_rdma_event_changeaddr(edev);
 		break;
 	}
 
@@ -977,7 +977,7 @@ static int __qede_probe(struct pci_dev *pdev, u32 dp_module, u8 dp_level,
 
 	qede_init_ndev(edev);
 
-	rc = qede_roce_dev_add(edev);
+	rc = qede_rdma_dev_add(edev);
 	if (rc)
 		goto err3;
 
@@ -1013,7 +1013,7 @@ static int __qede_probe(struct pci_dev *pdev, u32 dp_module, u8 dp_level,
 	return 0;
 
 err4:
-	qede_roce_dev_remove(edev);
+	qede_rdma_dev_remove(edev);
 err3:
 	free_netdev(edev->ndev);
 err2:
@@ -1064,7 +1064,7 @@ static void __qede_remove(struct pci_dev *pdev, enum qede_remove_mode mode)
 
 	qede_ptp_disable(edev);
 
-	qede_roce_dev_remove(edev);
+	qede_rdma_dev_remove(edev);
 
 	edev->ops->common->set_power_state(cdev, PCI_D0);
 
@@ -1964,7 +1964,7 @@ static void qede_unload(struct qede_dev *edev, enum qede_unload_mode mode,
 
 	edev->state = QEDE_STATE_CLOSED;
 
-	qede_roce_dev_event_close(edev);
+	qede_rdma_dev_event_close(edev);
 
 	/* Close OS Tx */
 	netif_tx_disable(edev->ndev);
@@ -2069,7 +2069,7 @@ static int qede_load(struct qede_dev *edev, enum qede_load_mode mode,
 	link_params.link_up = true;
 	edev->ops->common->set_link(edev->cdev, &link_params);
 
-	qede_roce_dev_event_open(edev);
+	qede_rdma_dev_event_open(edev);
 
 	edev->state = QEDE_STATE_OPEN;
 
diff --git a/drivers/net/ethernet/qlogic/qede/qede_rdma.c b/drivers/net/ethernet/qlogic/qede/qede_rdma.c
index 9837ee2..50b142f 100644
--- a/drivers/net/ethernet/qlogic/qede/qede_rdma.c
+++ b/drivers/net/ethernet/qlogic/qede/qede_rdma.c
@@ -40,12 +40,12 @@ static struct qedr_driver *qedr_drv;
 static LIST_HEAD(qedr_dev_list);
 static DEFINE_MUTEX(qedr_dev_list_lock);
 
-bool qede_roce_supported(struct qede_dev *dev)
+bool qede_rdma_supported(struct qede_dev *dev)
 {
 	return dev->dev_info.common.rdma_supported;
 }
 
-static void _qede_roce_dev_add(struct qede_dev *edev)
+static void _qede_rdma_dev_add(struct qede_dev *edev)
 {
 	if (!qedr_drv)
 		return;
@@ -54,11 +54,11 @@ static void _qede_roce_dev_add(struct qede_dev *edev)
 						 edev->ndev);
 }
 
-static int qede_roce_create_wq(struct qede_dev *edev)
+static int qede_rdma_create_wq(struct qede_dev *edev)
 {
-	INIT_LIST_HEAD(&edev->rdma_info.roce_event_list);
-	edev->rdma_info.roce_wq = create_singlethread_workqueue("roce_wq");
-	if (!edev->rdma_info.roce_wq) {
+	INIT_LIST_HEAD(&edev->rdma_info.rdma_event_list);
+	edev->rdma_info.rdma_wq = create_singlethread_workqueue("rdma_wq");
+	if (!edev->rdma_info.rdma_wq) {
 		DP_NOTICE(edev, "qedr: Could not create workqueue\n");
 		return -ENOMEM;
 	}
@@ -66,14 +66,14 @@ static int qede_roce_create_wq(struct qede_dev *edev)
 	return 0;
 }
 
-static void qede_roce_cleanup_event(struct qede_dev *edev)
+static void qede_rdma_cleanup_event(struct qede_dev *edev)
 {
-	struct list_head *head = &edev->rdma_info.roce_event_list;
-	struct qede_roce_event_work *event_node;
+	struct list_head *head = &edev->rdma_info.rdma_event_list;
+	struct qede_rdma_event_work *event_node;
 
-	flush_workqueue(edev->rdma_info.roce_wq);
+	flush_workqueue(edev->rdma_info.rdma_wq);
 	while (!list_empty(head)) {
-		event_node = list_entry(head->next, struct qede_roce_event_work,
+		event_node = list_entry(head->next, struct qede_rdma_event_work,
 					list);
 		cancel_work_sync(&event_node->work);
 		list_del(&event_node->list);
@@ -81,85 +81,85 @@ static void qede_roce_cleanup_event(struct qede_dev *edev)
 	}
 }
 
-static void qede_roce_destroy_wq(struct qede_dev *edev)
+static void qede_rdma_destroy_wq(struct qede_dev *edev)
 {
-	qede_roce_cleanup_event(edev);
-	destroy_workqueue(edev->rdma_info.roce_wq);
+	qede_rdma_cleanup_event(edev);
+	destroy_workqueue(edev->rdma_info.rdma_wq);
 }
 
-int qede_roce_dev_add(struct qede_dev *edev)
+int qede_rdma_dev_add(struct qede_dev *edev)
 {
 	int rc = 0;
 
-	if (qede_roce_supported(edev)) {
-		rc = qede_roce_create_wq(edev);
+	if (qede_rdma_supported(edev)) {
+		rc = qede_rdma_create_wq(edev);
 		if (rc)
 			return rc;
 
 		INIT_LIST_HEAD(&edev->rdma_info.entry);
 		mutex_lock(&qedr_dev_list_lock);
 		list_add_tail(&edev->rdma_info.entry, &qedr_dev_list);
-		_qede_roce_dev_add(edev);
+		_qede_rdma_dev_add(edev);
 		mutex_unlock(&qedr_dev_list_lock);
 	}
 
 	return rc;
 }
 
-static void _qede_roce_dev_remove(struct qede_dev *edev)
+static void _qede_rdma_dev_remove(struct qede_dev *edev)
 {
 	if (qedr_drv && qedr_drv->remove && edev->rdma_info.qedr_dev)
 		qedr_drv->remove(edev->rdma_info.qedr_dev);
 	edev->rdma_info.qedr_dev = NULL;
 }
 
-void qede_roce_dev_remove(struct qede_dev *edev)
+void qede_rdma_dev_remove(struct qede_dev *edev)
 {
-	if (!qede_roce_supported(edev))
+	if (!qede_rdma_supported(edev))
 		return;
 
-	qede_roce_destroy_wq(edev);
+	qede_rdma_destroy_wq(edev);
 	mutex_lock(&qedr_dev_list_lock);
-	_qede_roce_dev_remove(edev);
+	_qede_rdma_dev_remove(edev);
 	list_del(&edev->rdma_info.entry);
 	mutex_unlock(&qedr_dev_list_lock);
 }
 
-static void _qede_roce_dev_open(struct qede_dev *edev)
+static void _qede_rdma_dev_open(struct qede_dev *edev)
 {
 	if (qedr_drv && edev->rdma_info.qedr_dev && qedr_drv->notify)
 		qedr_drv->notify(edev->rdma_info.qedr_dev, QEDE_UP);
 }
 
-static void qede_roce_dev_open(struct qede_dev *edev)
+static void qede_rdma_dev_open(struct qede_dev *edev)
 {
-	if (!qede_roce_supported(edev))
+	if (!qede_rdma_supported(edev))
 		return;
 
 	mutex_lock(&qedr_dev_list_lock);
-	_qede_roce_dev_open(edev);
+	_qede_rdma_dev_open(edev);
 	mutex_unlock(&qedr_dev_list_lock);
 }
 
-static void _qede_roce_dev_close(struct qede_dev *edev)
+static void _qede_rdma_dev_close(struct qede_dev *edev)
 {
 	if (qedr_drv && edev->rdma_info.qedr_dev && qedr_drv->notify)
 		qedr_drv->notify(edev->rdma_info.qedr_dev, QEDE_DOWN);
 }
 
-static void qede_roce_dev_close(struct qede_dev *edev)
+static void qede_rdma_dev_close(struct qede_dev *edev)
 {
-	if (!qede_roce_supported(edev))
+	if (!qede_rdma_supported(edev))
 		return;
 
 	mutex_lock(&qedr_dev_list_lock);
-	_qede_roce_dev_close(edev);
+	_qede_rdma_dev_close(edev);
 	mutex_unlock(&qedr_dev_list_lock);
 }
 
-static void qede_roce_dev_shutdown(struct qede_dev *edev)
+static void qede_rdma_dev_shutdown(struct qede_dev *edev)
 {
-	if (!qede_roce_supported(edev))
+	if (!qede_rdma_supported(edev))
 		return;
 
 	mutex_lock(&qedr_dev_list_lock);
@@ -168,7 +168,7 @@ static void qede_roce_dev_shutdown(struct qede_dev *edev)
 	mutex_unlock(&qedr_dev_list_lock);
 }
 
-int qede_roce_register_driver(struct qedr_driver *drv)
+int qede_rdma_register_driver(struct qedr_driver *drv)
 {
 	struct qede_dev *edev;
 	u8 qedr_counter = 0;
@@ -184,52 +184,52 @@ int qede_roce_register_driver(struct qedr_driver *drv)
 		struct net_device *ndev;
 
 		qedr_counter++;
-		_qede_roce_dev_add(edev);
+		_qede_rdma_dev_add(edev);
 		ndev = edev->ndev;
 		if (netif_running(ndev) && netif_oper_up(ndev))
-			_qede_roce_dev_open(edev);
+			_qede_rdma_dev_open(edev);
 	}
 	mutex_unlock(&qedr_dev_list_lock);
 
-	pr_notice("qedr: discovered and registered %d RoCE funcs\n",
+	pr_notice("qedr: discovered and registered %d RDMA funcs\n",
 		  qedr_counter);
 
 	return 0;
 }
-EXPORT_SYMBOL(qede_roce_register_driver);
+EXPORT_SYMBOL(qede_rdma_register_driver);
 
-void qede_roce_unregister_driver(struct qedr_driver *drv)
+void qede_rdma_unregister_driver(struct qedr_driver *drv)
 {
 	struct qede_dev *edev;
 
 	mutex_lock(&qedr_dev_list_lock);
 	list_for_each_entry(edev, &qedr_dev_list, rdma_info.entry) {
 		if (edev->rdma_info.qedr_dev)
-			_qede_roce_dev_remove(edev);
+			_qede_rdma_dev_remove(edev);
 	}
 	qedr_drv = NULL;
 	mutex_unlock(&qedr_dev_list_lock);
 }
-EXPORT_SYMBOL(qede_roce_unregister_driver);
+EXPORT_SYMBOL(qede_rdma_unregister_driver);
 
-static void qede_roce_changeaddr(struct qede_dev *edev)
+static void qede_rdma_changeaddr(struct qede_dev *edev)
 {
-	if (!qede_roce_supported(edev))
+	if (!qede_rdma_supported(edev))
 		return;
 
 	if (qedr_drv && edev->rdma_info.qedr_dev && qedr_drv->notify)
 		qedr_drv->notify(edev->rdma_info.qedr_dev, QEDE_CHANGE_ADDR);
 }
 
-static struct qede_roce_event_work *
-qede_roce_get_free_event_node(struct qede_dev *edev)
+static struct qede_rdma_event_work *
+qede_rdma_get_free_event_node(struct qede_dev *edev)
 {
-	struct qede_roce_event_work *event_node = NULL;
+	struct qede_rdma_event_work *event_node = NULL;
 	struct list_head *list_node = NULL;
 	bool found = false;
 
-	list_for_each(list_node, &edev->rdma_info.roce_event_list) {
-		event_node = list_entry(list_node, struct qede_roce_event_work,
+	list_for_each(list_node, &edev->rdma_info.rdma_event_list) {
+		event_node = list_entry(list_node, struct qede_rdma_event_work,
 					list);
 		if (!work_pending(&event_node->work)) {
 			found = true;
@@ -241,74 +241,74 @@ qede_roce_get_free_event_node(struct qede_dev *edev)
 		event_node = kzalloc(sizeof(*event_node), GFP_KERNEL);
 		if (!event_node) {
 			DP_NOTICE(edev,
-				  "qedr: Could not allocate memory for roce work\n");
+				  "qedr: Could not allocate memory for rdma work\n");
 			return NULL;
 		}
 		list_add_tail(&event_node->list,
-			      &edev->rdma_info.roce_event_list);
+			      &edev->rdma_info.rdma_event_list);
 	}
 
 	return event_node;
 }
 
-static void qede_roce_handle_event(struct work_struct *work)
+static void qede_rdma_handle_event(struct work_struct *work)
 {
-	struct qede_roce_event_work *event_node;
-	enum qede_roce_event event;
+	struct qede_rdma_event_work *event_node;
+	enum qede_rdma_event event;
 	struct qede_dev *edev;
 
-	event_node = container_of(work, struct qede_roce_event_work, work);
+	event_node = container_of(work, struct qede_rdma_event_work, work);
 	event = event_node->event;
 	edev = event_node->ptr;
 
 	switch (event) {
 	case QEDE_UP:
-		qede_roce_dev_open(edev);
+		qede_rdma_dev_open(edev);
 		break;
 	case QEDE_DOWN:
-		qede_roce_dev_close(edev);
+		qede_rdma_dev_close(edev);
 		break;
 	case QEDE_CLOSE:
-		qede_roce_dev_shutdown(edev);
+		qede_rdma_dev_shutdown(edev);
 		break;
 	case QEDE_CHANGE_ADDR:
-		qede_roce_changeaddr(edev);
+		qede_rdma_changeaddr(edev);
 		break;
 	default:
-		DP_NOTICE(edev, "Invalid roce event %d", event);
+		DP_NOTICE(edev, "Invalid rdma event %d", event);
 	}
 }
 
-static void qede_roce_add_event(struct qede_dev *edev,
-				enum qede_roce_event event)
+static void qede_rdma_add_event(struct qede_dev *edev,
+				enum qede_rdma_event event)
 {
-	struct qede_roce_event_work *event_node;
+	struct qede_rdma_event_work *event_node;
 
 	if (!edev->rdma_info.qedr_dev)
 		return;
 
-	event_node = qede_roce_get_free_event_node(edev);
+	event_node = qede_rdma_get_free_event_node(edev);
 	if (!event_node)
 		return;
 
 	event_node->event = event;
 	event_node->ptr = edev;
 
-	INIT_WORK(&event_node->work, qede_roce_handle_event);
-	queue_work(edev->rdma_info.roce_wq, &event_node->work);
+	INIT_WORK(&event_node->work, qede_rdma_handle_event);
+	queue_work(edev->rdma_info.rdma_wq, &event_node->work);
 }
 
-void qede_roce_dev_event_open(struct qede_dev *edev)
+void qede_rdma_dev_event_open(struct qede_dev *edev)
 {
-	qede_roce_add_event(edev, QEDE_UP);
+	qede_rdma_add_event(edev, QEDE_UP);
 }
 
-void qede_roce_dev_event_close(struct qede_dev *edev)
+void qede_rdma_dev_event_close(struct qede_dev *edev)
 {
-	qede_roce_add_event(edev, QEDE_DOWN);
+	qede_rdma_add_event(edev, QEDE_DOWN);
 }
 
-void qede_roce_event_changeaddr(struct qede_dev *edev)
+void qede_rdma_event_changeaddr(struct qede_dev *edev)
 {
-	qede_roce_add_event(edev, QEDE_CHANGE_ADDR);
+	qede_rdma_add_event(edev, QEDE_CHANGE_ADDR);
 }
diff --git a/include/linux/qed/qede_rdma.h b/include/linux/qed/qede_rdma.h
index a1a9b81..1348a16 100644
--- a/include/linux/qed/qede_rdma.h
+++ b/include/linux/qed/qede_rdma.h
@@ -41,18 +41,18 @@ struct qedr_dev;
 struct qed_dev;
 struct qede_dev;
 
-enum qede_roce_event {
+enum qede_rdma_event {
 	QEDE_UP,
 	QEDE_DOWN,
 	QEDE_CHANGE_ADDR,
 	QEDE_CLOSE
 };
 
-struct qede_roce_event_work {
+struct qede_rdma_event_work {
 	struct list_head list;
 	struct work_struct work;
 	void *ptr;
-	enum qede_roce_event event;
+	enum qede_rdma_event event;
 };
 
 struct qedr_driver {
@@ -62,32 +62,33 @@ struct qedr_driver {
 				struct net_device *);
 
 	void (*remove)(struct qedr_dev *);
-	void (*notify)(struct qedr_dev *, enum qede_roce_event);
+	void (*notify)(struct qedr_dev *, enum qede_rdma_event);
 };
 
-/* APIs for RoCE driver to register callback handlers,
+/* APIs for RDMA driver to register callback handlers,
  * which will be invoked when device is added, removed, ifup, ifdown
  */
-int qede_roce_register_driver(struct qedr_driver *drv);
-void qede_roce_unregister_driver(struct qedr_driver *drv);
+int qede_rdma_register_driver(struct qedr_driver *drv);
+void qede_rdma_unregister_driver(struct qedr_driver *drv);
 
-bool qede_roce_supported(struct qede_dev *dev);
+bool qede_rdma_supported(struct qede_dev *dev);
 
 #if IS_ENABLED(CONFIG_QED_RDMA)
-int qede_roce_dev_add(struct qede_dev *dev);
-void qede_roce_dev_event_open(struct qede_dev *dev);
-void qede_roce_dev_event_close(struct qede_dev *dev);
-void qede_roce_dev_remove(struct qede_dev *dev);
-void qede_roce_event_changeaddr(struct qede_dev *qedr);
+int qede_rdma_dev_add(struct qede_dev *dev);
+void qede_rdma_dev_event_open(struct qede_dev *dev);
+void qede_rdma_dev_event_close(struct qede_dev *dev);
+void qede_rdma_dev_remove(struct qede_dev *dev);
+void qede_rdma_event_changeaddr(struct qede_dev *edr);
+
 #else
-static inline int qede_roce_dev_add(struct qede_dev *dev)
+static inline int qede_rdma_dev_add(struct qede_dev *dev);
 {
 	return 0;
 }
 
-static inline void qede_roce_dev_event_open(struct qede_dev *dev) {}
-static inline void qede_roce_dev_event_close(struct qede_dev *dev) {}
-static inline void qede_roce_dev_remove(struct qede_dev *dev) {}
-static inline void qede_roce_event_changeaddr(struct qede_dev *qedr) {}
+static inline void qede_rdma_dev_event_open(struct qede_dev *dev) {}
+static inline void qede_rdma_dev_event_close(struct qede_dev *dev) {}
+static inline void qede_rdma_dev_remove(struct qede_dev *dev) {}
+static inline void qede_rdma_event_changeaddr(struct qede_dev *edr) {}
 #endif
 #endif
-- 
2.9.4

[PATCH v2 net-next 7/7] qed: SPQ async callback registration

From: Yuval Mintz <hidden>
Date: 2017-06-18 11:51:43

From: Michal Kalderon <redacted>

Whenever firmware indicates that there's an async indication it needs
to handle, there's a switch-case where the right functionality is called
based on function's personality and information.

Before iWARP is added [as yet another client], switch over the SPQ into
a callback-registered mechanism, allowing registration of the relevant
event-processing logic based on the function's personality. This allows
us to tidy the code by removing protocol-specifics from a common file.

Signed-off-by: Michal Kalderon <redacted>
Signed-off-by: Yuval Mintz <redacted>
---
 drivers/net/ethernet/qlogic/qed/qed_iscsi.c | 24 ++++++++++++-
 drivers/net/ethernet/qlogic/qed/qed_roce.c  | 16 ++++++---
 drivers/net/ethernet/qlogic/qed/qed_roce.h  |  6 ----
 drivers/net/ethernet/qlogic/qed/qed_sp.h    | 17 +++++++++
 drivers/net/ethernet/qlogic/qed/qed_spq.c   | 54 ++++++++++++++++-------------
 drivers/net/ethernet/qlogic/qed/qed_sriov.c | 16 +++++++--
 drivers/net/ethernet/qlogic/qed/qed_sriov.h | 18 ----------
 7 files changed, 96 insertions(+), 55 deletions(-)
diff --git a/drivers/net/ethernet/qlogic/qed/qed_iscsi.c b/drivers/net/ethernet/qlogic/qed/qed_iscsi.c
index 5a1ed05..813c77c 100644
--- a/drivers/net/ethernet/qlogic/qed/qed_iscsi.c
+++ b/drivers/net/ethernet/qlogic/qed/qed_iscsi.c
@@ -62,6 +62,22 @@
 #include "qed_sriov.h"
 #include "qed_reg_addr.h"
 
+static int
+qed_iscsi_async_event(struct qed_hwfn *p_hwfn,
+		      u8 fw_event_code,
+		      u16 echo, union event_ring_data *data, u8 fw_return_code)
+{
+	if (p_hwfn->p_iscsi_info->event_cb) {
+		struct qed_iscsi_info *p_iscsi = p_hwfn->p_iscsi_info;
+
+		return p_iscsi->event_cb(p_iscsi->event_context,
+					 fw_event_code, data);
+	} else {
+		DP_NOTICE(p_hwfn, "iSCSI async completion is not set\n");
+		return -EINVAL;
+	}
+}
+
 struct qed_iscsi_conn {
 	struct list_head list_entry;
 	bool free_on_delete;
@@ -265,6 +281,9 @@ qed_sp_iscsi_func_start(struct qed_hwfn *p_hwfn,
 	p_hwfn->p_iscsi_info->event_context = event_context;
 	p_hwfn->p_iscsi_info->event_cb = async_event_cb;
 
+	qed_spq_register_async_cb(p_hwfn, PROTOCOLID_ISCSI,
+				  qed_iscsi_async_event);
+
 	return qed_spq_post(p_hwfn, p_ent, NULL);
 }
 
@@ -631,7 +650,10 @@ static int qed_sp_iscsi_func_stop(struct qed_hwfn *p_hwfn,
 	p_ramrod = &p_ent->ramrod.iscsi_destroy;
 	p_ramrod->hdr.op_code = ISCSI_RAMROD_CMD_ID_DESTROY_FUNC;
 
-	return qed_spq_post(p_hwfn, p_ent, NULL);
+	rc = qed_spq_post(p_hwfn, p_ent, NULL);
+
+	qed_spq_unregister_async_cb(p_hwfn, PROTOCOLID_ISCSI);
+	return rc;
 }
 
 static void __iomem *qed_iscsi_get_db_addr(struct qed_hwfn *p_hwfn, u32 cid)
diff --git a/drivers/net/ethernet/qlogic/qed/qed_roce.c b/drivers/net/ethernet/qlogic/qed/qed_roce.c
index 7482905..673f80a 100644
--- a/drivers/net/ethernet/qlogic/qed/qed_roce.c
+++ b/drivers/net/ethernet/qlogic/qed/qed_roce.c
@@ -68,12 +68,14 @@
 
 static void qed_roce_free_real_icid(struct qed_hwfn *p_hwfn, u16 icid);
 
-void qed_roce_async_event(struct qed_hwfn *p_hwfn,
-			  u8 fw_event_code, union rdma_eqe_data *rdma_data)
+static int
+qed_roce_async_event(struct qed_hwfn *p_hwfn,
+		     u8 fw_event_code,
+		     u16 echo, union event_ring_data *data, u8 fw_return_code)
 {
 	if (fw_event_code == ROCE_ASYNC_EVENT_DESTROY_QP_DONE) {
 		u16 icid =
-		    (u16)le32_to_cpu(rdma_data->rdma_destroy_qp_data.cid);
+		    (u16)le32_to_cpu(data->rdma_data.rdma_destroy_qp_data.cid);
 
 		/* icid release in this async event can occur only if the icid
 		 * was offloaded to the FW. In case it wasn't offloaded this is
@@ -85,8 +87,10 @@ void qed_roce_async_event(struct qed_hwfn *p_hwfn,
 
 		events->affiliated_event(p_hwfn->p_rdma_info->events.context,
 					 fw_event_code,
-					 &rdma_data->async_handle);
+				     (void *)&data->rdma_data.async_handle);
 	}
+
+	return 0;
 }
 
 static int qed_rdma_bmap_alloc(struct qed_hwfn *p_hwfn,
@@ -686,6 +690,9 @@ static int qed_rdma_setup(struct qed_hwfn *p_hwfn,
 	if (rc)
 		return rc;
 
+	qed_spq_register_async_cb(p_hwfn, PROTOCOLID_ROCE,
+				  qed_roce_async_event);
+
 	return qed_rdma_start_fw(p_hwfn, params, p_ptt);
 }
 
@@ -706,6 +713,7 @@ void qed_roce_stop(struct qed_hwfn *p_hwfn)
 			break;
 		}
 	}
+	qed_spq_unregister_async_cb(p_hwfn, PROTOCOLID_ROCE);
 }
 
 static int qed_rdma_stop(void *rdma_cxt)
diff --git a/drivers/net/ethernet/qlogic/qed/qed_roce.h b/drivers/net/ethernet/qlogic/qed/qed_roce.h
index ddd7761..b178d99 100644
--- a/drivers/net/ethernet/qlogic/qed/qed_roce.h
+++ b/drivers/net/ethernet/qlogic/qed/qed_roce.h
@@ -169,16 +169,10 @@ struct qed_rdma_qp {
 #if IS_ENABLED(CONFIG_QED_RDMA)
 void qed_rdma_dpm_bar(struct qed_hwfn *p_hwfn, struct qed_ptt *p_ptt);
 void qed_roce_dpm_dcbx(struct qed_hwfn *p_hwfn, struct qed_ptt *p_ptt);
-void qed_roce_async_event(struct qed_hwfn *p_hwfn,
-			  u8 fw_event_code, union rdma_eqe_data *rdma_data);
 #else
 static inline void qed_rdma_dpm_bar(struct qed_hwfn *p_hwfn, struct qed_ptt *p_ptt) {}
 
 static inline void qed_roce_dpm_dcbx(struct qed_hwfn *p_hwfn,
 				     struct qed_ptt *p_ptt) {}
-
-static inline void qed_roce_async_event(struct qed_hwfn *p_hwfn,
-					u8 fw_event_code,
-					union rdma_eqe_data *rdma_data) {}
 #endif
 #endif
diff --git a/drivers/net/ethernet/qlogic/qed/qed_sp.h b/drivers/net/ethernet/qlogic/qed/qed_sp.h
index 00dd50f..56c95fb 100644
--- a/drivers/net/ethernet/qlogic/qed/qed_sp.h
+++ b/drivers/net/ethernet/qlogic/qed/qed_sp.h
@@ -174,6 +174,22 @@ struct qed_consq {
 	struct qed_chain chain;
 };
 
+typedef int
+(*qed_spq_async_comp_cb)(struct qed_hwfn *p_hwfn,
+			 u8 opcode,
+			 u16 echo,
+			 union event_ring_data *data,
+			 u8 fw_return_code);
+
+int
+qed_spq_register_async_cb(struct qed_hwfn *p_hwfn,
+			  enum protocol_type protocol_id,
+			  qed_spq_async_comp_cb cb);
+
+void
+qed_spq_unregister_async_cb(struct qed_hwfn *p_hwfn,
+			    enum protocol_type protocol_id);
+
 struct qed_spq {
 	spinlock_t		lock; /* SPQ lock */
 
@@ -203,6 +219,7 @@ struct qed_spq {
 	u32			comp_count;
 
 	u32			cid;
+	qed_spq_async_comp_cb async_comp_cb[MAX_PROTOCOL_TYPE];
 };
 
 /**
diff --git a/drivers/net/ethernet/qlogic/qed/qed_spq.c b/drivers/net/ethernet/qlogic/qed/qed_spq.c
index a971916..78954d2 100644
--- a/drivers/net/ethernet/qlogic/qed/qed_spq.c
+++ b/drivers/net/ethernet/qlogic/qed/qed_spq.c
@@ -302,32 +302,16 @@ static int
 qed_async_event_completion(struct qed_hwfn *p_hwfn,
 			   struct event_ring_entry *p_eqe)
 {
-	switch (p_eqe->protocol_id) {
-#if IS_ENABLED(CONFIG_QED_RDMA)
-	case PROTOCOLID_ROCE:
-		qed_roce_async_event(p_hwfn, p_eqe->opcode,
-				     &p_eqe->data.rdma_data);
-		return 0;
-#endif
-	case PROTOCOLID_COMMON:
-		return qed_sriov_eqe_event(p_hwfn,
-					   p_eqe->opcode,
-					   p_eqe->echo, &p_eqe->data);
-	case PROTOCOLID_ISCSI:
-		if (!IS_ENABLED(CONFIG_QED_ISCSI))
-			return -EINVAL;
+	qed_spq_async_comp_cb cb;
 
-		if (p_hwfn->p_iscsi_info->event_cb) {
-			struct qed_iscsi_info *p_iscsi = p_hwfn->p_iscsi_info;
+	if (!p_hwfn->p_spq || (p_eqe->protocol_id >= MAX_PROTOCOL_TYPE))
+		return -EINVAL;
 
-			return p_iscsi->event_cb(p_iscsi->event_context,
-						 p_eqe->opcode, &p_eqe->data);
-		} else {
-			DP_NOTICE(p_hwfn,
-				  "iSCSI async completion is not set\n");
-			return -EINVAL;
-		}
-	default:
+	cb = p_hwfn->p_spq->async_comp_cb[p_eqe->protocol_id];
+	if (cb) {
+		return cb(p_hwfn, p_eqe->opcode, p_eqe->echo,
+			  &p_eqe->data, p_eqe->fw_return_code);
+	} else {
 		DP_NOTICE(p_hwfn,
 			  "Unknown Async completion for protocol: %d\n",
 			  p_eqe->protocol_id);
@@ -335,6 +319,28 @@ qed_async_event_completion(struct qed_hwfn *p_hwfn,
 	}
 }
 
+int
+qed_spq_register_async_cb(struct qed_hwfn *p_hwfn,
+			  enum protocol_type protocol_id,
+			  qed_spq_async_comp_cb cb)
+{
+	if (!p_hwfn->p_spq || (protocol_id >= MAX_PROTOCOL_TYPE))
+		return -EINVAL;
+
+	p_hwfn->p_spq->async_comp_cb[protocol_id] = cb;
+	return 0;
+}
+
+void
+qed_spq_unregister_async_cb(struct qed_hwfn *p_hwfn,
+			    enum protocol_type protocol_id)
+{
+	if (!p_hwfn->p_spq || (protocol_id >= MAX_PROTOCOL_TYPE))
+		return;
+
+	p_hwfn->p_spq->async_comp_cb[protocol_id] = NULL;
+}
+
 /***************************************************************************
 * EQ API
 ***************************************************************************/
diff --git a/drivers/net/ethernet/qlogic/qed/qed_sriov.c b/drivers/net/ethernet/qlogic/qed/qed_sriov.c
index e39ad22..2cfd3bd 100644
--- a/drivers/net/ethernet/qlogic/qed/qed_sriov.c
+++ b/drivers/net/ethernet/qlogic/qed/qed_sriov.c
@@ -44,6 +44,11 @@
 #include "qed_sp.h"
 #include "qed_sriov.h"
 #include "qed_vf.h"
+static int qed_sriov_eqe_event(struct qed_hwfn *p_hwfn,
+			       u8 opcode,
+			       __le16 echo,
+			       union event_ring_data *data, u8 fw_return_code);
+
 
 static u8 qed_vf_calculate_legacy(struct qed_vf_info *p_vf)
 {
@@ -565,6 +570,9 @@ int qed_iov_alloc(struct qed_hwfn *p_hwfn)
 
 	p_hwfn->pf_iov_info = p_sriov;
 
+	qed_spq_register_async_cb(p_hwfn, PROTOCOLID_COMMON,
+				  qed_sriov_eqe_event);
+
 	return qed_iov_allocate_vfdb(p_hwfn);
 }
 
@@ -578,6 +586,8 @@ void qed_iov_setup(struct qed_hwfn *p_hwfn)
 
 void qed_iov_free(struct qed_hwfn *p_hwfn)
 {
+	qed_spq_unregister_async_cb(p_hwfn, PROTOCOLID_COMMON);
+
 	if (IS_PF_SRIOV_ALLOC(p_hwfn)) {
 		qed_iov_free_vfdb(p_hwfn);
 		kfree(p_hwfn->pf_iov_info);
@@ -3833,8 +3843,10 @@ static void qed_sriov_vfpf_malicious(struct qed_hwfn *p_hwfn,
 	}
 }
 
-int qed_sriov_eqe_event(struct qed_hwfn *p_hwfn,
-			u8 opcode, __le16 echo, union event_ring_data *data)
+static int qed_sriov_eqe_event(struct qed_hwfn *p_hwfn,
+			       u8 opcode,
+			       __le16 echo,
+			       union event_ring_data *data, u8 fw_return_code)
 {
 	switch (opcode) {
 	case COMMON_EVENT_VF_PF_CHANNEL:
diff --git a/drivers/net/ethernet/qlogic/qed/qed_sriov.h b/drivers/net/ethernet/qlogic/qed/qed_sriov.h
index 95f55ae..c2e44bc 100644
--- a/drivers/net/ethernet/qlogic/qed/qed_sriov.h
+++ b/drivers/net/ethernet/qlogic/qed/qed_sriov.h
@@ -344,17 +344,6 @@ void qed_iov_free(struct qed_hwfn *p_hwfn);
 void qed_iov_free_hw_info(struct qed_dev *cdev);
 
 /**
- * @brief qed_sriov_eqe_event - handle async sriov event arrived on eqe.
- *
- * @param p_hwfn
- * @param opcode
- * @param echo
- * @param data
- */
-int qed_sriov_eqe_event(struct qed_hwfn *p_hwfn,
-			u8 opcode, __le16 echo, union event_ring_data *data);
-
-/**
  * @brief Mark structs of vfs that have been FLR-ed.
  *
  * @param p_hwfn
@@ -418,13 +407,6 @@ static inline void qed_iov_free_hw_info(struct qed_dev *cdev)
 {
 }
 
-static inline int qed_sriov_eqe_event(struct qed_hwfn *p_hwfn,
-				      u8 opcode,
-				      __le16 echo, union event_ring_data *data)
-{
-	return -EINVAL;
-}
-
 static inline bool qed_iov_mark_vf_flr(struct qed_hwfn *p_hwfn,
 				       u32 *disabled_vfs)
 {
-- 
2.9.4

Re: [PATCH v2 net-next 0/7] qed*: RDMA and infrastructure for iWARP

From: Christoph Hellwig <hch@infradead.org>
Date: 2017-06-18 13:14:46

On Sun, Jun 18, 2017 at 02:50:28PM +0300, Yuval Mintz wrote:
This series focuses on RDMA in general with emphasis on required changes
toward adding iWARP support. The vast majority of the changes introduced
are in qed/qede, with a couple of small changes to qedr
[mentioned below].
Btw, can you explain us how you are going to expose RoCE vs iWarp?
Is it going to be a per-port setting?  Are you going to use different
ib_device structures or do you plan to differenciate at another level?

RE: [PATCH v2 net-next 0/7] qed*: RDMA and infrastructure for iWARP

From: Kalderon, Michal <hidden>
Date: 2017-06-19 06:28:56

From: Christoph Hellwig [mailto:hch@infradead.org]
Sent: Sunday, June 18, 2017 4:15 PM
 
On Sun, Jun 18, 2017 at 02:50:28PM +0300, Yuval Mintz wrote:
quoted
This series focuses on RDMA in general with emphasis on required
changes toward adding iWARP support. The vast majority of the changes
introduced are in qed/qede, with a couple of small changes to qedr
[mentioned below].
Btw, can you explain us how you are going to expose RoCE vs iWarp?
Is it going to be a per-port setting?  Are you going to use different ib_device
structures or do you plan to differenciate at another level?
Initial submission will be based on iWARP or RoCE being part of the device configuration ( using EFI-Hii), 
Later on we're considering adding devlink support for changing the device configuration.
This is not a port setting, pci devices sharing the same physical port can have different RDMA protocols.
Keyboard shortcuts
hback out one level
jnext message in thread
kprevious message in thread
ldrill in
Escclose help / fold thread tree
?toggle this help