[PATCH v2 0/3] VSOCK: vsockmon virtual device to monitor AF_VSOCK sockets.

STALE3632d

9 messages, 4 authors, 2016-09-05 · open the first message on its own page

[PATCH v2 0/3] VSOCK: vsockmon virtual device to monitor AF_VSOCK sockets.

From: <hidden>
Date: 2016-08-13 10:22:02

From: Gerard Garcia <redacted>

This patch applies over the mst vhost git repository:
http://git.kernel.org/cgit/linux/kernel/git/mst/vhost.git

v2:
 * Clone skb before transmitting them to vsockmon.
 * Use consume_skb() instead of kfree_skb().
 * Pass skb lifetime responsibility to tap functions.
 * Remove t_hdr member from vsockmon header to avoid problems when/if it
    changes it size if more transports are supported.

This was already been sent as a RFC where several issues where fixed.
This is the summary of changes from the first RFC:

v2:
 * Do not clone skb, instead take ownership before transmitting.
 * Split tap functions from af_vsock.c.
 * Simplify vsockmon header to remove unnecessary padding and
    set little endian byte order.
 * Various simple fixes from the comments received to the first RFC.

Additionally, first pach version changes:
 * Add len field to the vsockmon header to ease parsing.
 * Pack vsockmon header.
 * Various simple fixes and styling.

Overview:

Virtual socket transports operate at kernel level therefore, there is no easy
way to see the traffic exchanged between virtual machines and hypervisors that
communicate using AF_VSOCK sockets. In addition, being able to see the control
messages exchanged by the transports may be useful for debugging and
optimization purposes. This patch adds a virtual device that may be used to see
the traffic exchanged between virtual machines and hypervisors through AF_VSOCK
sockets.

Its structure is based on the nlmon device and this version just targets the
virtio transport, but support for the VMCI transport can be easily implemented.
The vsockmon header contains a generic header and includes the header specific to
the transport. The generic header allows to follow an AF_VSOCK stream without
having to dig into the details of the transport while the transport header
gives more detail which may be useful for troubleshooting and debugging.

Testing:

To set up a vsockmon device:

ip link add type vsockmon
ip link set vsockmon0 up

The Wireshark development version (master branch) includes a vsock dissector
that is capable of parsing packets received through vsockmon. The dissector
needs to be manually selected.

Thanks to Stefan Hajnoczi for his help.

Gerard Garcia (3):
  VSOCK: Add vsockmon tap functions
  VSOCK: Add vsockmon device
  VSOCK: Add virtio vsock vsockmon hooks

 drivers/net/Kconfig           |   8 ++
 drivers/net/Makefile          |   1 +
 drivers/net/vsockmon.c        | 168 ++++++++++++++++++++++++++++++++++++++++++
 drivers/vhost/vsock.c         |  72 ++++++++++++++++++
 include/net/af_vsock.h        |  13 ++++
 include/uapi/linux/Kbuild     |   1 +
 include/uapi/linux/if_arp.h   |   1 +
 include/uapi/linux/vsockmon.h |  38 ++++++++++
 net/vmw_vsock/Makefile        |   2 +-
 net/vmw_vsock/af_vsock_tap.c  | 113 ++++++++++++++++++++++++++++
 10 files changed, 416 insertions(+), 1 deletion(-)
 create mode 100644 drivers/net/vsockmon.c
 create mode 100644 include/uapi/linux/vsockmon.h
 create mode 100644 net/vmw_vsock/af_vsock_tap.c

-- 
2.9.1

[PATCH v2 1/3] VSOCK: Add vsockmon tap functions

From: <hidden>
Date: 2016-08-13 10:22:03

From: Gerard Garcia <redacted>

Add tap functions that can be used by the vsock transports to
deliver packets to vsockmon virtual network devices.

Signed-off-by: Gerard Garcia <redacted>
---
 include/net/af_vsock.h       |  13 +++++
 include/uapi/linux/if_arp.h  |   1 +
 net/vmw_vsock/Makefile       |   2 +-
 net/vmw_vsock/af_vsock_tap.c | 113 +++++++++++++++++++++++++++++++++++++++++++
 4 files changed, 128 insertions(+), 1 deletion(-)
 create mode 100644 net/vmw_vsock/af_vsock_tap.c
diff --git a/include/net/af_vsock.h b/include/net/af_vsock.h
index f275896..f7c51b1 100644
--- a/include/net/af_vsock.h
+++ b/include/net/af_vsock.h
@@ -185,4 +185,17 @@ struct sock *vsock_find_connected_socket(struct sockaddr_vm *src,
 void vsock_remove_sock(struct vsock_sock *vsk);
 void vsock_for_each_connected_socket(void (*fn)(struct sock *sk));
 
+/**** TAP ****/
+
+struct vsock_tap {
+	struct net_device *dev;
+	struct module *module;
+	struct list_head list;
+};
+
+int vsock_init_tap(void);
+int vsock_add_tap(struct vsock_tap *vt);
+int vsock_remove_tap(struct vsock_tap *vt);
+void vsock_deliver_tap(struct sk_buff *skb);
+
 #endif /* __AF_VSOCK_H__ */
diff --git a/include/uapi/linux/if_arp.h b/include/uapi/linux/if_arp.h
index 4d024d7..cf73510 100644
--- a/include/uapi/linux/if_arp.h
+++ b/include/uapi/linux/if_arp.h
@@ -95,6 +95,7 @@
 #define ARPHRD_IP6GRE	823		/* GRE over IPv6		*/
 #define ARPHRD_NETLINK	824		/* Netlink header		*/
 #define ARPHRD_6LOWPAN	825		/* IPv6 over LoWPAN             */
+#define ARPHRD_VSOCKMON	826		/* Vsock monitor header		*/
 
 #define ARPHRD_VOID	  0xFFFF	/* Void type, nothing is known */
 #define ARPHRD_NONE	  0xFFFE	/* zero header length */
diff --git a/net/vmw_vsock/Makefile b/net/vmw_vsock/Makefile
index bc27c70..09fc2eb 100644
--- a/net/vmw_vsock/Makefile
+++ b/net/vmw_vsock/Makefile
@@ -3,7 +3,7 @@ obj-$(CONFIG_VMWARE_VMCI_VSOCKETS) += vmw_vsock_vmci_transport.o
 obj-$(CONFIG_VIRTIO_VSOCKETS) += vmw_vsock_virtio_transport.o
 obj-$(CONFIG_VIRTIO_VSOCKETS_COMMON) += vmw_vsock_virtio_transport_common.o
 
-vsock-y += af_vsock.o vsock_addr.o
+vsock-y += af_vsock.o af_vsock_tap.o vsock_addr.o
 
 vmw_vsock_vmci_transport-y += vmci_transport.o vmci_transport_notify.o \
 	vmci_transport_notify_qstate.o
diff --git a/net/vmw_vsock/af_vsock_tap.c b/net/vmw_vsock/af_vsock_tap.c
new file mode 100644
index 0000000..ff242b1
--- /dev/null
+++ b/net/vmw_vsock/af_vsock_tap.c
@@ -0,0 +1,113 @@
+/*
+ * Tap functions for AF_VSOCK sockets.
+ *
+ * Code based on net/netlink/af_netlink.c tap functions.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License
+ * as published by the Free Software Foundation; either version
+ * 2 of the License, or (at your option) any later version.
+ */
+
+#include <net/sock.h>
+#include <net/af_vsock.h>
+#include <linux/if_arp.h>
+
+static DEFINE_SPINLOCK(vsock_tap_lock);
+static struct list_head vsock_tap_all __read_mostly =
+				LIST_HEAD_INIT(vsock_tap_all);
+
+int vsock_add_tap(struct vsock_tap *vt) {
+	if (unlikely(vt->dev->type != ARPHRD_VSOCKMON))
+		return -EINVAL;
+
+	__module_get(vt->module);
+
+	spin_lock(&vsock_tap_lock);
+	list_add_rcu(&vt->list, &vsock_tap_all);
+	spin_unlock(&vsock_tap_lock);
+
+
+	return 0;
+}
+EXPORT_SYMBOL_GPL(vsock_add_tap);
+
+int __vsock_remove_tap(struct vsock_tap *vt) {
+	bool found = false;
+	struct vsock_tap *tmp;
+
+	spin_lock(&vsock_tap_lock);
+
+	list_for_each_entry(tmp, &vsock_tap_all, list) {
+		if (vt == tmp) {
+			list_del_rcu(&vt->list);
+			found = true;
+			goto out;
+		}
+	}
+
+	pr_warn("__vsock_remove_tap: %p not found\n", vt);
+out:
+	spin_unlock(&vsock_tap_lock);
+
+	if (found)
+		module_put(vt->module);
+
+	return found ? 0 : -ENODEV;
+}
+
+int vsock_remove_tap(struct vsock_tap *vt)
+{
+	int ret;
+
+	ret = __vsock_remove_tap(vt);
+	synchronize_net();
+
+	return ret;
+}
+EXPORT_SYMBOL_GPL(vsock_remove_tap);
+
+static int __vsock_deliver_tap_skb(struct sk_buff *skb,
+				     struct net_device *dev)
+{
+	int ret = 0;
+	struct sk_buff *nskb = skb_clone(skb, GFP_ATOMIC);
+
+	if (nskb) {
+		dev_hold(dev);
+
+		nskb->dev = dev;
+		ret = dev_queue_xmit(nskb);
+		if (unlikely(ret > 0))
+			ret = net_xmit_errno(ret);
+
+		dev_put(dev);
+	}
+
+	return ret;
+}
+
+static void __vsock_deliver_tap(struct sk_buff *skb)
+{
+	int ret;
+	struct vsock_tap *tmp;
+
+	list_for_each_entry_rcu(tmp, &vsock_tap_all, list) {
+		ret = __vsock_deliver_tap_skb(skb, tmp->dev);
+		if (unlikely(ret))
+			break;
+	}
+
+	consume_skb(skb);
+}
+
+void vsock_deliver_tap(struct sk_buff *skb)
+{
+	rcu_read_lock();
+
+	if (unlikely(!list_empty(&vsock_tap_all)))
+		__vsock_deliver_tap(skb);
+
+	rcu_read_unlock();
+}
+EXPORT_SYMBOL_GPL(vsock_deliver_tap);
-- 
2.9.1

[PATCH v2 2/3] VSOCK: Add vsockmon device

From: <hidden>
Date: 2016-08-13 10:22:05

From: Gerard Garcia <redacted>

Add vsockmon virtual network device that receives packets from the vsock
transports and exposes them to user space.

Based on the nlmon device.

Signed-off-by: Gerard Garcia <redacted>
---
 drivers/net/Kconfig           |   8 ++
 drivers/net/Makefile          |   1 +
 drivers/net/vsockmon.c        | 168 ++++++++++++++++++++++++++++++++++++++++++
 include/uapi/linux/Kbuild     |   1 +
 include/uapi/linux/vsockmon.h |  38 ++++++++++
 5 files changed, 216 insertions(+)
 create mode 100644 drivers/net/vsockmon.c
 create mode 100644 include/uapi/linux/vsockmon.h
diff --git a/drivers/net/Kconfig b/drivers/net/Kconfig
index 0c5415b..42c43b6 100644
--- a/drivers/net/Kconfig
+++ b/drivers/net/Kconfig
@@ -330,6 +330,14 @@ config NET_VRF
 	  This option enables the support for mapping interfaces into VRF's. The
 	  support enables VRF devices.
 
+config VSOCKMON
+    tristate "Virtual vsock monitoring device"
+    depends on VHOST_VSOCK
+    ---help---
+     This option enables a monitoring net device for vsock sockets. It is
+     mostly intended for developers or support to debug vsock issues. If
+     unsure, say N.
+
 endif # NET_CORE
 
 config SUNGEM_PHY
diff --git a/drivers/net/Makefile b/drivers/net/Makefile
index 7336cbd..e2188d4 100644
--- a/drivers/net/Makefile
+++ b/drivers/net/Makefile
@@ -28,6 +28,7 @@ obj-$(CONFIG_GENEVE) += geneve.o
 obj-$(CONFIG_GTP) += gtp.o
 obj-$(CONFIG_NLMON) += nlmon.o
 obj-$(CONFIG_NET_VRF) += vrf.o
+obj-$(CONFIG_VSOCKMON) += vsockmon.o
 
 #
 # Networking Drivers
diff --git a/drivers/net/vsockmon.c b/drivers/net/vsockmon.c
new file mode 100644
index 0000000..9ad4f0a
--- /dev/null
+++ b/drivers/net/vsockmon.c
@@ -0,0 +1,168 @@
+#include <linux/module.h>
+#include <linux/kernel.h>
+#include <linux/if_arp.h>
+#include <net/rtnetlink.h>
+#include <net/sock.h>
+#include <net/af_vsock.h>
+#include <uapi/linux/vsockmon.h>
+#include <linux/virtio_vsock.h>
+
+/* Virtio transport max packet size plus header */
+#define DEFAULT_MTU VIRTIO_VSOCK_MAX_PKT_BUF_SIZE + sizeof(struct af_vsockmon_hdr);
+
+struct pcpu_lstats {
+	u64 rx_packets;
+	u64 rx_bytes;
+	struct u64_stats_sync syncp;
+};
+
+static int vsockmon_dev_init(struct net_device *dev)
+{
+	dev->lstats = netdev_alloc_pcpu_stats(struct pcpu_lstats);
+	return dev->lstats == NULL ? -ENOMEM : 0;
+}
+
+static void vsockmon_dev_uninit(struct net_device *dev)
+{
+	free_percpu(dev->lstats);
+}
+
+struct vsockmon {
+	struct vsock_tap vt;
+};
+
+static int vsockmon_open(struct net_device *dev)
+{
+	struct vsockmon *vsockmon = netdev_priv(dev);
+
+	vsockmon->vt.dev = dev;
+	vsockmon->vt.module = THIS_MODULE;
+	return vsock_add_tap(&vsockmon->vt);
+}
+
+static int vsockmon_close(struct net_device *dev) {
+	struct vsockmon *vsockmon = netdev_priv(dev);
+
+	return vsock_remove_tap(&vsockmon->vt);
+}
+
+static netdev_tx_t vsockmon_xmit(struct sk_buff *skb, struct net_device *dev)
+{
+	int len = skb->len;
+	struct pcpu_lstats *stats = this_cpu_ptr(dev->lstats);
+
+	u64_stats_update_begin(&stats->syncp);
+	stats->rx_bytes += len;
+	stats->rx_packets++;
+	u64_stats_update_end(&stats->syncp);
+
+	dev_kfree_skb(skb);
+
+	return NETDEV_TX_OK;
+}
+
+static struct rtnl_link_stats64 *
+vsockmon_get_stats64(struct net_device *dev, struct rtnl_link_stats64 *stats)
+{
+	int i;
+	u64 bytes = 0, packets = 0;
+
+	for_each_possible_cpu(i) {
+		const struct pcpu_lstats *vstats;
+		u64 tbytes, tpackets;
+		unsigned int start;
+
+		vstats = per_cpu_ptr(dev->lstats, i);
+
+		do {
+			start = u64_stats_fetch_begin_irq(&vstats->syncp);
+			tbytes = vstats->rx_bytes;
+			tpackets = vstats->rx_packets;
+		} while (u64_stats_fetch_retry_irq(&vstats->syncp, start));
+
+		packets += tpackets;
+		bytes += tbytes;
+	}
+
+	stats->rx_packets = packets;
+	stats->tx_packets = 0;
+
+	stats->rx_bytes = bytes;
+	stats->tx_bytes = 0;
+
+	return stats;
+}
+
+static int vsockmon_is_valid_mtu(int new_mtu)
+{
+	return new_mtu >= (int) sizeof(struct af_vsockmon_hdr);
+}
+
+static int vsockmon_change_mtu(struct net_device *dev, int new_mtu)
+{
+	if (!vsockmon_is_valid_mtu(new_mtu))
+		return -EINVAL;
+
+	dev->mtu = new_mtu;
+	return 0;
+}
+
+static const struct net_device_ops vsockmon_ops = {
+	.ndo_init = vsockmon_dev_init,
+	.ndo_uninit = vsockmon_dev_uninit,
+	.ndo_open = vsockmon_open,
+	.ndo_stop = vsockmon_close,
+	.ndo_start_xmit = vsockmon_xmit,
+	.ndo_get_stats64 = vsockmon_get_stats64,
+	.ndo_change_mtu = vsockmon_change_mtu,
+};
+
+static u32 always_on(struct net_device *dev)
+{
+	return 1;
+}
+
+static const struct ethtool_ops vsockmon_ethtool_ops = {
+	.get_link = always_on,
+};
+
+static void vsockmon_setup(struct net_device *dev)
+{
+	dev->type = ARPHRD_VSOCKMON;
+	dev->priv_flags |= IFF_NO_QUEUE;
+
+	dev->netdev_ops	= &vsockmon_ops;
+	dev->ethtool_ops = &vsockmon_ethtool_ops;
+	dev->destructor	= free_netdev;
+
+	dev->features = NETIF_F_SG | NETIF_F_FRAGLIST |
+			NETIF_F_HIGHDMA | NETIF_F_LLTX;
+
+	dev->flags = IFF_NOARP;
+
+	dev->mtu = DEFAULT_MTU;
+}
+
+static struct rtnl_link_ops vsockmon_link_ops __read_mostly = {
+	.kind			= "vsockmon",
+	.priv_size		= sizeof(struct vsockmon),
+	.setup			= vsockmon_setup,
+};
+
+static __init int vsockmon_register(void)
+{
+	return rtnl_link_register(&vsockmon_link_ops);
+}
+
+static __exit void vsockmon_unregister(void)
+{
+	rtnl_link_unregister(&vsockmon_link_ops);
+}
+
+module_init(vsockmon_register);
+module_exit(vsockmon_unregister);
+
+MODULE_LICENSE("GPL v2");
+MODULE_AUTHOR("Gerard Garcia <ggarcia@deic.uab.cat>");
+MODULE_DESCRIPTION("Vsock monitoring device. Based on nlmon device.");
+MODULE_ALIAS_RTNL_LINK("vsockmon");
diff --git a/include/uapi/linux/Kbuild b/include/uapi/linux/Kbuild
index 185f8ea..ef7f878 100644
--- a/include/uapi/linux/Kbuild
+++ b/include/uapi/linux/Kbuild
@@ -456,6 +456,7 @@ header-y += virtio_scsi.h
 header-y += virtio_types.h
 header-y += virtio_vsock.h
 header-y += vm_sockets.h
+header-y += vsockmon.h
 header-y += vt.h
 header-y += vtpm_proxy.h
 header-y += wait.h
diff --git a/include/uapi/linux/vsockmon.h b/include/uapi/linux/vsockmon.h
new file mode 100644
index 0000000..0e9c54e
--- /dev/null
+++ b/include/uapi/linux/vsockmon.h
@@ -0,0 +1,38 @@
+#ifndef _UAPI_VSOCKMON_H
+#define _UAPI_VSOCKMON_H
+
+#include <linux/virtio_vsock.h>
+
+/*
+ * Structure of packets received through the vsockmon device.
+ *
+ * Note that after the vsockmon header comes the transport header (len bytes and
+ * type specified by t) and if the packet op is AF_VSOCK_OP_PAYLOAD then comes
+ * the payload.
+ */
+
+struct af_vsockmon_hdr {
+	__le64 src_cid;
+	__le64 dst_cid;
+	__le32 src_port;
+	__le32 dst_port;
+	__le16 op;			/* enum af_vsockmon_op */
+	__le16 t;			/* enum af_vosckmon_t */
+	__le16 len;			/* Transport header length */
+} __attribute__((packed));
+
+enum af_vsockmon_op {
+	AF_VSOCK_OP_UNKNOWN = 0,
+	AF_VSOCK_OP_CONNECT = 1,
+	AF_VSOCK_OP_DISCONNECT = 2,
+	AF_VSOCK_OP_CONTROL = 3,
+	AF_VSOCK_OP_PAYLOAD = 4,
+};
+
+enum af_vsockmon_t {
+	AF_VSOCK_T_UNKNOWN = 0,
+	AF_VSOCK_T_NO_INFO = 1,		/* No transport information */
+	AF_VSOCK_T_VIRTIO = 2,		/* Virtio transport header (struct virtio_vsock_hdr) */
+};
+
+#endif
-- 
2.9.1

[PATCH v2 3/3] VSOCK: Add virtio vsock vsockmon hooks

From: <hidden>
Date: 2016-08-13 10:22:07

From: Gerard Garcia <redacted>

Add hooks to the virtio transport host driver to deliver a copy of
the received and sent messages to all vsockmon virtual network devices.

Signed-off-by: Gerard Garcia <redacted>
---
 drivers/vhost/vsock.c | 72 +++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 72 insertions(+)
diff --git a/drivers/vhost/vsock.c b/drivers/vhost/vsock.c
index e3b30ea..4670c3c 100644
--- a/drivers/vhost/vsock.c
+++ b/drivers/vhost/vsock.c
@@ -15,8 +15,10 @@
 #include <net/sock.h>
 #include <linux/virtio_vsock.h>
 #include <linux/vhost.h>
+#include <linux/skbuff.h>
 
 #include <net/af_vsock.h>
+#include <uapi/linux/vsockmon.h>
 #include "vhost.h"
 
 #define VHOST_VSOCK_DEFAULT_HOST_CID	2
@@ -45,6 +47,68 @@ struct vhost_vsock {
 	u32 guest_cid;
 };
 
+static struct sk_buff *
+virtio_vsock_pkt_to_vsockmon_skb(struct virtio_vsock_pkt *pkt)
+{
+	struct sk_buff *skb;
+	struct af_vsockmon_hdr *hdr;
+	unsigned char *t_hdr, *payload;
+
+	u32 skb_len = sizeof(*hdr) + sizeof(pkt->hdr) +
+		pkt->len;
+
+	skb = alloc_skb(skb_len, GFP_ATOMIC);
+	if (!skb)
+		return NULL;
+
+	hdr = (struct af_vsockmon_hdr *) skb_put(skb, sizeof(*hdr));
+
+	hdr->src_cid = pkt->hdr.src_cid;
+	hdr->src_port = pkt->hdr.src_port;
+	hdr->dst_cid = pkt->hdr.dst_cid;
+	hdr->dst_port = pkt->hdr.dst_port;
+	hdr->t = cpu_to_le16(AF_VSOCK_T_VIRTIO);
+	hdr->len = cpu_to_le16(sizeof(pkt->hdr));
+
+	switch(cpu_to_le16(pkt->hdr.op)) {
+	case VIRTIO_VSOCK_OP_REQUEST:
+	case VIRTIO_VSOCK_OP_RESPONSE:
+		hdr->op = cpu_to_le16(AF_VSOCK_OP_CONNECT);
+		break;
+	case VIRTIO_VSOCK_OP_RST:
+	case VIRTIO_VSOCK_OP_SHUTDOWN:
+		hdr->op = cpu_to_le16(AF_VSOCK_OP_DISCONNECT);
+		break;
+	case VIRTIO_VSOCK_OP_RW:
+		hdr->op = cpu_to_le16(AF_VSOCK_OP_PAYLOAD);
+		break;
+	case VIRTIO_VSOCK_OP_CREDIT_UPDATE:
+	case VIRTIO_VSOCK_OP_CREDIT_REQUEST:
+		hdr->op = cpu_to_le16(AF_VSOCK_OP_CONTROL);
+		break;
+	default:
+		hdr->op = cpu_to_le16(AF_VSOCK_OP_UNKNOWN);
+		break;
+	}
+
+	t_hdr = skb_put(skb, sizeof(pkt->hdr));
+	memcpy(t_hdr, &pkt->hdr, sizeof(pkt->hdr));
+
+	if (pkt->len) {
+		payload = skb_put(skb, pkt->len);
+		memcpy(payload, pkt->buf, pkt->len);
+	}
+
+	return skb;
+}
+
+static void vsock_deliver_tap_pkt(struct virtio_vsock_pkt *pkt)
+{
+	struct sk_buff *skb = virtio_vsock_pkt_to_vsockmon_skb(pkt);
+	if (skb)
+		vsock_deliver_tap(skb);
+}
+
 static u32 vhost_transport_get_local_cid(void)
 {
 	return VHOST_VSOCK_DEFAULT_HOST_CID;
@@ -168,6 +232,11 @@ vhost_transport_do_send_pkt(struct vhost_vsock *vsock,
 				restart_tx = true;
 		}
 
+		/* Deliver to monitoring devices all correctly transmitted
+		 * packets.
+		 */
+		vsock_deliver_tap_pkt(pkt);
+
 		virtio_transport_free_pkt(pkt);
 	}
 	if (added)
@@ -338,6 +407,9 @@ static void vhost_vsock_handle_tx_kick(struct vhost_work *work)
 
 		len = pkt->len;
 
+		/* Deliver to monitoring devices all received packets */
+		vsock_deliver_tap_pkt(pkt);
+
 		/* Only accept correctly addressed packets */
 		if (le64_to_cpu(pkt->hdr.src_cid) == vsock->guest_cid)
 			virtio_transport_recv_pkt(pkt);
-- 
2.9.1

Re: [PATCH v2 2/3] VSOCK: Add vsockmon device

From: zhuyj <zyjzyj2000@gmail.com>
Date: 2016-08-13 11:38:36

+#define DEFAULT_MTU (VIRTIO_VSOCK_MAX_PKT_BUF_SIZE + sizeof(struct
af_vsockmon_hdr));

It is better.

On Sat, Aug 13, 2016 at 6:21 PM,  [off-list ref] wrote:
quoted hunk
From: Gerard Garcia <redacted>

Add vsockmon virtual network device that receives packets from the vsock
transports and exposes them to user space.

Based on the nlmon device.

Signed-off-by: Gerard Garcia <redacted>
---
 drivers/net/Kconfig           |   8 ++
 drivers/net/Makefile          |   1 +
 drivers/net/vsockmon.c        | 168 ++++++++++++++++++++++++++++++++++++++++++
 include/uapi/linux/Kbuild     |   1 +
 include/uapi/linux/vsockmon.h |  38 ++++++++++
 5 files changed, 216 insertions(+)
 create mode 100644 drivers/net/vsockmon.c
 create mode 100644 include/uapi/linux/vsockmon.h
diff --git a/drivers/net/Kconfig b/drivers/net/Kconfig
index 0c5415b..42c43b6 100644
--- a/drivers/net/Kconfig
+++ b/drivers/net/Kconfig
@@ -330,6 +330,14 @@ config NET_VRF
          This option enables the support for mapping interfaces into VRF's. The
          support enables VRF devices.

+config VSOCKMON
+    tristate "Virtual vsock monitoring device"
+    depends on VHOST_VSOCK
+    ---help---
+     This option enables a monitoring net device for vsock sockets. It is
+     mostly intended for developers or support to debug vsock issues. If
+     unsure, say N.
+
 endif # NET_CORE

 config SUNGEM_PHY
diff --git a/drivers/net/Makefile b/drivers/net/Makefile
index 7336cbd..e2188d4 100644
--- a/drivers/net/Makefile
+++ b/drivers/net/Makefile
@@ -28,6 +28,7 @@ obj-$(CONFIG_GENEVE) += geneve.o
 obj-$(CONFIG_GTP) += gtp.o
 obj-$(CONFIG_NLMON) += nlmon.o
 obj-$(CONFIG_NET_VRF) += vrf.o
+obj-$(CONFIG_VSOCKMON) += vsockmon.o

 #
 # Networking Drivers
diff --git a/drivers/net/vsockmon.c b/drivers/net/vsockmon.c
new file mode 100644
index 0000000..9ad4f0a
--- /dev/null
+++ b/drivers/net/vsockmon.c
@@ -0,0 +1,168 @@
+#include <linux/module.h>
+#include <linux/kernel.h>
+#include <linux/if_arp.h>
+#include <net/rtnetlink.h>
+#include <net/sock.h>
+#include <net/af_vsock.h>
+#include <uapi/linux/vsockmon.h>
+#include <linux/virtio_vsock.h>
+
+/* Virtio transport max packet size plus header */
+#define DEFAULT_MTU VIRTIO_VSOCK_MAX_PKT_BUF_SIZE + sizeof(struct af_vsockmon_hdr);
+
+struct pcpu_lstats {
+       u64 rx_packets;
+       u64 rx_bytes;
+       struct u64_stats_sync syncp;
+};
+
+static int vsockmon_dev_init(struct net_device *dev)
+{
+       dev->lstats = netdev_alloc_pcpu_stats(struct pcpu_lstats);
+       return dev->lstats == NULL ? -ENOMEM : 0;
+}
+
+static void vsockmon_dev_uninit(struct net_device *dev)
+{
+       free_percpu(dev->lstats);
+}
+
+struct vsockmon {
+       struct vsock_tap vt;
+};
+
+static int vsockmon_open(struct net_device *dev)
+{
+       struct vsockmon *vsockmon = netdev_priv(dev);
+
+       vsockmon->vt.dev = dev;
+       vsockmon->vt.module = THIS_MODULE;
+       return vsock_add_tap(&vsockmon->vt);
+}
+
+static int vsockmon_close(struct net_device *dev) {
+       struct vsockmon *vsockmon = netdev_priv(dev);
+
+       return vsock_remove_tap(&vsockmon->vt);
+}
+
+static netdev_tx_t vsockmon_xmit(struct sk_buff *skb, struct net_device *dev)
+{
+       int len = skb->len;
+       struct pcpu_lstats *stats = this_cpu_ptr(dev->lstats);
+
+       u64_stats_update_begin(&stats->syncp);
+       stats->rx_bytes += len;
+       stats->rx_packets++;
+       u64_stats_update_end(&stats->syncp);
+
+       dev_kfree_skb(skb);
+
+       return NETDEV_TX_OK;
+}
+
+static struct rtnl_link_stats64 *
+vsockmon_get_stats64(struct net_device *dev, struct rtnl_link_stats64 *stats)
+{
+       int i;
+       u64 bytes = 0, packets = 0;
+
+       for_each_possible_cpu(i) {
+               const struct pcpu_lstats *vstats;
+               u64 tbytes, tpackets;
+               unsigned int start;
+
+               vstats = per_cpu_ptr(dev->lstats, i);
+
+               do {
+                       start = u64_stats_fetch_begin_irq(&vstats->syncp);
+                       tbytes = vstats->rx_bytes;
+                       tpackets = vstats->rx_packets;
+               } while (u64_stats_fetch_retry_irq(&vstats->syncp, start));
+
+               packets += tpackets;
+               bytes += tbytes;
+       }
+
+       stats->rx_packets = packets;
+       stats->tx_packets = 0;
+
+       stats->rx_bytes = bytes;
+       stats->tx_bytes = 0;
+
+       return stats;
+}
+
+static int vsockmon_is_valid_mtu(int new_mtu)
+{
+       return new_mtu >= (int) sizeof(struct af_vsockmon_hdr);
+}
+
+static int vsockmon_change_mtu(struct net_device *dev, int new_mtu)
+{
+       if (!vsockmon_is_valid_mtu(new_mtu))
+               return -EINVAL;
+
+       dev->mtu = new_mtu;
+       return 0;
+}
+
+static const struct net_device_ops vsockmon_ops = {
+       .ndo_init = vsockmon_dev_init,
+       .ndo_uninit = vsockmon_dev_uninit,
+       .ndo_open = vsockmon_open,
+       .ndo_stop = vsockmon_close,
+       .ndo_start_xmit = vsockmon_xmit,
+       .ndo_get_stats64 = vsockmon_get_stats64,
+       .ndo_change_mtu = vsockmon_change_mtu,
+};
+
+static u32 always_on(struct net_device *dev)
+{
+       return 1;
+}
+
+static const struct ethtool_ops vsockmon_ethtool_ops = {
+       .get_link = always_on,
+};
+
+static void vsockmon_setup(struct net_device *dev)
+{
+       dev->type = ARPHRD_VSOCKMON;
+       dev->priv_flags |= IFF_NO_QUEUE;
+
+       dev->netdev_ops = &vsockmon_ops;
+       dev->ethtool_ops = &vsockmon_ethtool_ops;
+       dev->destructor = free_netdev;
+
+       dev->features = NETIF_F_SG | NETIF_F_FRAGLIST |
+                       NETIF_F_HIGHDMA | NETIF_F_LLTX;
+
+       dev->flags = IFF_NOARP;
+
+       dev->mtu = DEFAULT_MTU;
+}
+
+static struct rtnl_link_ops vsockmon_link_ops __read_mostly = {
+       .kind                   = "vsockmon",
+       .priv_size              = sizeof(struct vsockmon),
+       .setup                  = vsockmon_setup,
+};
+
+static __init int vsockmon_register(void)
+{
+       return rtnl_link_register(&vsockmon_link_ops);
+}
+
+static __exit void vsockmon_unregister(void)
+{
+       rtnl_link_unregister(&vsockmon_link_ops);
+}
+
+module_init(vsockmon_register);
+module_exit(vsockmon_unregister);
+
+MODULE_LICENSE("GPL v2");
+MODULE_AUTHOR("Gerard Garcia <ggarcia@deic.uab.cat>");
+MODULE_DESCRIPTION("Vsock monitoring device. Based on nlmon device.");
+MODULE_ALIAS_RTNL_LINK("vsockmon");
diff --git a/include/uapi/linux/Kbuild b/include/uapi/linux/Kbuild
index 185f8ea..ef7f878 100644
--- a/include/uapi/linux/Kbuild
+++ b/include/uapi/linux/Kbuild
@@ -456,6 +456,7 @@ header-y += virtio_scsi.h
 header-y += virtio_types.h
 header-y += virtio_vsock.h
 header-y += vm_sockets.h
+header-y += vsockmon.h
 header-y += vt.h
 header-y += vtpm_proxy.h
 header-y += wait.h
diff --git a/include/uapi/linux/vsockmon.h b/include/uapi/linux/vsockmon.h
new file mode 100644
index 0000000..0e9c54e
--- /dev/null
+++ b/include/uapi/linux/vsockmon.h
@@ -0,0 +1,38 @@
+#ifndef _UAPI_VSOCKMON_H
+#define _UAPI_VSOCKMON_H
+
+#include <linux/virtio_vsock.h>
+
+/*
+ * Structure of packets received through the vsockmon device.
+ *
+ * Note that after the vsockmon header comes the transport header (len bytes and
+ * type specified by t) and if the packet op is AF_VSOCK_OP_PAYLOAD then comes
+ * the payload.
+ */
+
+struct af_vsockmon_hdr {
+       __le64 src_cid;
+       __le64 dst_cid;
+       __le32 src_port;
+       __le32 dst_port;
+       __le16 op;                      /* enum af_vsockmon_op */
+       __le16 t;                       /* enum af_vosckmon_t */
+       __le16 len;                     /* Transport header length */
+} __attribute__((packed));
+
+enum af_vsockmon_op {
+       AF_VSOCK_OP_UNKNOWN = 0,
+       AF_VSOCK_OP_CONNECT = 1,
+       AF_VSOCK_OP_DISCONNECT = 2,
+       AF_VSOCK_OP_CONTROL = 3,
+       AF_VSOCK_OP_PAYLOAD = 4,
+};
+
+enum af_vsockmon_t {
+       AF_VSOCK_T_UNKNOWN = 0,
+       AF_VSOCK_T_NO_INFO = 1,         /* No transport information */
+       AF_VSOCK_T_VIRTIO = 2,          /* Virtio transport header (struct virtio_vsock_hdr) */
+};
+
+#endif
--
2.9.1

Re: [PATCH v2 0/3] VSOCK: vsockmon virtual device to monitor AF_VSOCK sockets.

From: "Michael S. Tsirkin" <mst@redhat.com>
Date: 2016-08-14 23:15:42

On Sat, Aug 13, 2016 at 12:21:51PM +0200, ggarcia@abra.uab.cat wrote:
From: Gerard Garcia <redacted>

This patch applies over the mst vhost git repository:
http://git.kernel.org/cgit/linux/kernel/git/mst/vhost.git
So I do like where this is going, but it gives me pause
that there's a global list of taps, where all sockets
seem to multicast to them all.

In particular, this won't play well with things
like containers.

As each socket is bound to a physical device, how about binding
the monitor there as well? Only sockets from this device
would do the forwarding, and only one monitor per
device would be supported.

In a sense this will make it more like macvtap than tap.

v2:
 * Clone skb before transmitting them to vsockmon.
 * Use consume_skb() instead of kfree_skb().
 * Pass skb lifetime responsibility to tap functions.
 * Remove t_hdr member from vsockmon header to avoid problems when/if it
    changes it size if more transports are supported.

This was already been sent as a RFC where several issues where fixed.
This is the summary of changes from the first RFC:

v2:
 * Do not clone skb, instead take ownership before transmitting.
 * Split tap functions from af_vsock.c.
 * Simplify vsockmon header to remove unnecessary padding and
    set little endian byte order.
 * Various simple fixes from the comments received to the first RFC.

Additionally, first pach version changes:
 * Add len field to the vsockmon header to ease parsing.
 * Pack vsockmon header.
 * Various simple fixes and styling.

Overview:

Virtual socket transports operate at kernel level therefore, there is no easy
way to see the traffic exchanged between virtual machines and hypervisors that
communicate using AF_VSOCK sockets. In addition, being able to see the control
messages exchanged by the transports may be useful for debugging and
optimization purposes. This patch adds a virtual device that may be used to see
the traffic exchanged between virtual machines and hypervisors through AF_VSOCK
sockets.

Its structure is based on the nlmon device and this version just targets the
virtio transport, but support for the VMCI transport can be easily implemented.
The vsockmon header contains a generic header and includes the header specific to
the transport. The generic header allows to follow an AF_VSOCK stream without
having to dig into the details of the transport while the transport header
gives more detail which may be useful for troubleshooting and debugging.

Testing:

To set up a vsockmon device:

ip link add type vsockmon
ip link set vsockmon0 up

The Wireshark development version (master branch) includes a vsock dissector
that is capable of parsing packets received through vsockmon. The dissector
needs to be manually selected.

Thanks to Stefan Hajnoczi for his help.

Gerard Garcia (3):
  VSOCK: Add vsockmon tap functions
  VSOCK: Add vsockmon device
  VSOCK: Add virtio vsock vsockmon hooks

 drivers/net/Kconfig           |   8 ++
 drivers/net/Makefile          |   1 +
 drivers/net/vsockmon.c        | 168 ++++++++++++++++++++++++++++++++++++++++++
 drivers/vhost/vsock.c         |  72 ++++++++++++++++++
 include/net/af_vsock.h        |  13 ++++
 include/uapi/linux/Kbuild     |   1 +
 include/uapi/linux/if_arp.h   |   1 +
 include/uapi/linux/vsockmon.h |  38 ++++++++++
 net/vmw_vsock/Makefile        |   2 +-
 net/vmw_vsock/af_vsock_tap.c  | 113 ++++++++++++++++++++++++++++
 10 files changed, 416 insertions(+), 1 deletion(-)
 create mode 100644 drivers/net/vsockmon.c
 create mode 100644 include/uapi/linux/vsockmon.h
 create mode 100644 net/vmw_vsock/af_vsock_tap.c

-- 
2.9.1

Re: [PATCH v2 0/3] VSOCK: vsockmon virtual device to monitor AF_VSOCK sockets.

From: Stefan Hajnoczi <stefanha@redhat.com>
Date: 2016-08-15 15:13:28

On Mon, Aug 15, 2016 at 02:15:38AM +0300, Michael S. Tsirkin wrote:
On Sat, Aug 13, 2016 at 12:21:51PM +0200, ggarcia@abra.uab.cat wrote:
quoted
From: Gerard Garcia <redacted>

This patch applies over the mst vhost git repository:
http://git.kernel.org/cgit/linux/kernel/git/mst/vhost.git
So I do like where this is going, but it gives me pause
that there's a global list of taps, where all sockets
seem to multicast to them all.

In particular, this won't play well with things
like containers.
vsock currently has no network namespace support.  I agree that the tap
instances should be per-namespace when we add namespace support.
As each socket is bound to a physical device, how about binding
the monitor there as well?
Sockets aren't bound to physical devices, they are bound globally in the
af_vsock.ko module.  The module currently doesn't allow multiple
instances (you cannot have multiple VMCI or virtio transports).
Only sockets from this device
would do the forwarding, and only one monitor per
device would be supported.

In a sense this will make it more like macvtap than tap.
Restricting the number of monitors could make userspace cumbersome.
Imagine two scripts that want to capture packets.  The two scripts have
no knowledge of each other and create their own vsockmon interfaces.  If
we restrict vsockmon to just 1 interface then users need to agree on
sharing just 1 vsockmon interface.  I don't think this is beneficial.

So I think this global list is acceptable until we introduce network
namespace support.  At that point it will become per-namespace.

Stefan

Re: [PATCH v2 0/3] VSOCK: vsockmon virtual device to monitor AF_VSOCK sockets.

From: Gerard Garcia <hidden>
Date: 2016-08-22 10:48:08

On 08/15/2016 05:13 PM, Stefan Hajnoczi wrote:
On Mon, Aug 15, 2016 at 02:15:38AM +0300, Michael S. Tsirkin wrote:
quoted
On Sat, Aug 13, 2016 at 12:21:51PM +0200, ggarcia@abra.uab.cat wrote:
quoted
From: Gerard Garcia <redacted>

This patch applies over the mst vhost git repository:
http://git.kernel.org/cgit/linux/kernel/git/mst/vhost.git
So I do like where this is going, but it gives me pause
that there's a global list of taps, where all sockets
seem to multicast to them all.

In particular, this won't play well with things
like containers.
vsock currently has no network namespace support.  I agree that the tap
instances should be per-namespace when we add namespace support.
quoted
As each socket is bound to a physical device, how about binding
the monitor there as well?
Sockets aren't bound to physical devices, they are bound globally in the
af_vsock.ko module.  The module currently doesn't allow multiple
instances (you cannot have multiple VMCI or virtio transports).
quoted
Only sockets from this device
would do the forwarding, and only one monitor per
device would be supported.

In a sense this will make it more like macvtap than tap.
Restricting the number of monitors could make userspace cumbersome.
Imagine two scripts that want to capture packets.  The two scripts have
no knowledge of each other and create their own vsockmon interfaces.  If
we restrict vsockmon to just 1 interface then users need to agree on
sharing just 1 vsockmon interface.  I don't think this is beneficial.

So I think this global list is acceptable until we introduce network
namespace support.  At that point it will become per-namespace.
Sorry, I was out last week.

I don't have much to add to what Stefan said. I agree that when vsock 
introduces namespace support it will be necessary to have monitors 
divided per-namespace. Right now, if only one af_vsock instance is 
allowed, I think it makes sense to have a global list of taps.

Gerard

Re: [PATCH v2 0/3] VSOCK: vsockmon virtual device to monitor AF_VSOCK sockets.

From: Gerard Garcia <hidden>
Date: 2016-09-05 10:22:04

On 08/22/2016 12:48 PM, Gerard Garcia wrote:
On 08/15/2016 05:13 PM, Stefan Hajnoczi wrote:
quoted
On Mon, Aug 15, 2016 at 02:15:38AM +0300, Michael S. Tsirkin wrote:
quoted
On Sat, Aug 13, 2016 at 12:21:51PM +0200, ggarcia@abra.uab.cat wrote:
quoted
From: Gerard Garcia <redacted>

This patch applies over the mst vhost git repository:
http://git.kernel.org/cgit/linux/kernel/git/mst/vhost.git
So I do like where this is going, but it gives me pause
that there's a global list of taps, where all sockets
seem to multicast to them all.

In particular, this won't play well with things
like containers.
vsock currently has no network namespace support.  I agree that the tap
instances should be per-namespace when we add namespace support.
quoted
As each socket is bound to a physical device, how about binding
the monitor there as well?
Sockets aren't bound to physical devices, they are bound globally in the
af_vsock.ko module.  The module currently doesn't allow multiple
instances (you cannot have multiple VMCI or virtio transports).
quoted
Only sockets from this device
would do the forwarding, and only one monitor per
device would be supported.

In a sense this will make it more like macvtap than tap.
Restricting the number of monitors could make userspace cumbersome.
Imagine two scripts that want to capture packets.  The two scripts have
no knowledge of each other and create their own vsockmon interfaces.  If
we restrict vsockmon to just 1 interface then users need to agree on
sharing just 1 vsockmon interface.  I don't think this is beneficial.

So I think this global list is acceptable until we introduce network
namespace support.  At that point it will become per-namespace.
Sorry, I was out last week.

I don't have much to add to what Stefan said. I agree that when vsock
introduces namespace support it will be necessary to have monitors
divided per-namespace. Right now, if only one af_vsock instance is
allowed, I think it makes sense to have a global list of taps.

Gerard
Sorry to insist Michael, but do you agree with us? or do you think it is 
better to follow a different approach?
Keyboard shortcuts
hback out one level
jnext message in thread
kprevious message in thread
ldrill in
Escclose help / fold thread tree
?toggle this help