Re: [RFC v2 00/10] HFI Virtual Network Interface Controller (VNIC)

On Tue, Jan 17, 2017 at 11:27:20AM -0800, Vishwanathapura, Niranjana wrote:

Thanks Jason for the valuable inputs.

Here is the new generic interface.

Overview:
Bottom driver defines net_device_ops. The upper driver can override it.
For example, upper driver can implement ndo_open() which calls bottom
driver's ndo_open() and also do some book keeping.


include/rdma/ib_verbs.h:

/* rdma netdev type - specifies protocol type */
enum rdma_netdev_t {
	RDMA_NETDEV_HFI_VNIC,
};

/* rdma netdev
 * For usecases where netstack interfacing is required.
 */
struct rdma_netdev {
	struct net_device *netdev;
	u8 port_num;

	/* client private data structure */
	void *clnt_priv;

	/* control functions */
	void (*set_id)(struct rdma_netdev *rn, int id);
	void (*set_state)(struct rdma_netdev *rn, int state);
};

struct ib_device {
	...
	...
	/* rdma netdev operations */
	struct net_device *(*alloc_rdma_netdev)(struct ib_device *device,
					u8 port_num,
					enum rdma_netdev_t type,
					const char *name,
					unsigned char name_assign_type,
					void (*setup)(struct net_device *));
	void (*free_rdma_netdev)(struct net_device *netdev);
};


hfi1 driver:

/* rdma netdev's private data structure */
struct hfi1_rdma_netdev {
	struct rdma_netdev  rn;		/* keep this first */
	/* hfi1's vnic private data follows */
};


include/rdma/opa_hfi.h:

/* Client's ndo operations use below function instead of netdev_priv() */
static inline void *hfi_vnic_priv(const struct net_device *dev)
{
	struct rdma_netdev *rn = netdev_priv(dev);

	return rn->clnt_priv;
}

/* Overrides rtnl_link_stats64 to include hfi_vnic stats.
 * ndo_get_stats64() can be used to get the stats
 */
struct hfi_vnic_stats {
	/* standard netdev statistics */
	struct rtnl_link_stats64  netstat;

	/* HFI VNIC statistics */
	u64  tx_mcastbcast;
	u64  tx_untagged;
	u64  tx_vlan;
	u64  tx_64_size;
	u64  tx_65_127;
	u64  tx_128_255;
	u64  tx_256_511;
	u64  tx_512_1023;
	u64  tx_1024_1518;
	u64  tx_1519_max;

	u64  rx_untagged;
	u64  rx_vlan;
	u64  rx_64_size;
	u64  rx_65_127;
	u64  rx_128_255;
	u64  rx_256_511;
	u64  rx_512_1023;
	u64  rx_1024_1518;
	u64  rx_1519_max;

	u64  rx_runt;
	u64  rx_oversize;
};

I have started working on porting hfi_vnic as per this new interface.
I will post RFC v3 later.
Posting the interface definition early for comments.

I wonder how many people will comment it without seeing usage example.

Thanks,
Niranjana

On Wed, Jan 18, 2017 at 07:43:54AM +0200, Leon Romanovsky wrote:

quoted

I have started working on porting hfi_vnic as per this new interface.
I will post RFC v3 later.
Posting the interface definition early for comments.

I wonder how many people will comment it without seeing usage example.

It is my hope that Mellanox will contribute to this and use it for
ipoib - we've have had conversations along these lines in the
past... Apparently mlx4/5 could run quite a bit faster if used like
this.

Jason