On Thu, 27 Dec 2012 11:43:30 +0800
Wanlong Gao [off-list ref] wrote:

On 12/27/2012 11:28 AM, Jason Wang wrote:

quoted

On 12/26/2012 06:19 PM, Wanlong Gao wrote:

quoted

On 12/26/2012 06:06 PM, Jason Wang wrote:

quoted

On 12/26/2012 03:06 PM, Wanlong Gao wrote:

quoted

Add a cpu notifier to virtio-net, so that we can reset the
virtqueue affinity if the cpu hotplug happens. It improve
the performance through enabling or disabling the virtqueue
affinity after doing cpu hotplug.

Hi Wanlong:

Thanks for looking at this.

quoted

Cc: Rusty Russell <redacted>
Cc: "Michael S. Tsirkin" <mst@redhat.com>
Cc: Jason Wang <jasowang@redhat.com>
Cc: virtualization@lists.linux-foundation.org
Cc: netdev@vger.kernel.org
Signed-off-by: Wanlong Gao <redacted>
---
 drivers/net/virtio_net.c | 39 ++++++++++++++++++++++++++++++++++++++-
 1 file changed, 38 insertions(+), 1 deletion(-)

diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c
index a6fcf15..9710cf4 100644
--- a/drivers/net/virtio_net.c
+++ b/drivers/net/virtio_net.c

@@ -26,6 +26,7 @@
 #include <linux/scatterlist.h>
 #include <linux/if_vlan.h>
 #include <linux/slab.h>
+#include <linux/cpu.h>
 
 static int napi_weight = 128;
 module_param(napi_weight, int, 0444);

@@ -34,6 +35,8 @@ static bool csum = true, gso = true;
 module_param(csum, bool, 0444);
 module_param(gso, bool, 0444);
 
+static bool cpu_hotplug = false;
+
 /* FIXME: MTU in config. */
 #define MAX_PACKET_LEN (ETH_HLEN + VLAN_HLEN + ETH_DATA_LEN)
 #define GOOD_COPY_LEN	128

@@ -1041,6 +1044,26 @@ static void virtnet_set_affinity(struct virtnet_info *vi, bool set)
 		vi->affinity_hint_set = false;
 }
 
+static int virtnet_cpu_callback(struct notifier_block *nfb,
+			       unsigned long action, void *hcpu)
+{
+	switch(action) {
+	case CPU_ONLINE:
+	case CPU_ONLINE_FROZEN:
+	case CPU_DEAD:
+	case CPU_DEAD_FROZEN:
+		cpu_hotplug = true;
+		break;
+	default:
+		break;
+	}
+	return NOTIFY_OK;
+}
+
+static struct notifier_block virtnet_cpu_notifier = {
+	.notifier_call = virtnet_cpu_callback,
+};
+
 static void virtnet_get_ringparam(struct net_device *dev,
 				struct ethtool_ringparam *ring)
 {

@@ -1131,7 +1154,14 @@ static int virtnet_change_mtu(struct net_device *dev, int new_mtu)
  */
 static u16 virtnet_select_queue(struct net_device *dev, struct sk_buff *skb)
 {
-	int txq = skb_rx_queue_recorded(skb) ? skb_get_rx_queue(skb) :
+	int txq;
+
+	if (unlikely(cpu_hotplug == true)) {
+		virtnet_set_affinity(netdev_priv(dev), true);
+		cpu_hotplug = false;
+	}
+

Why don't you just do this in callback?

Callback can just give us a "hcpu", can't get the virtnet_info from callback. Am I missing something?

Well, I think you can just embed the notifier block into virtnet_info,
then use something like container_of in the callback to make the
notifier per device. This also solve the concern of Eric.

Yeah, thank you very much for your suggestion. I'll try it.

quoted

quoted

quoted

btw. Does qemu/kvm support cpu-hotplug now?

From http://www.linux-kvm.org/page/CPUHotPlug, I saw that qemu-kvm can support hotplug
but failed to merge to qemu.git, right?

Not sure, I just try latest qemu, it even does not have a cpu_set command.

Adding Igor to CC,

As I know, hotplug support is cleaned from qemu, and Igor want to rework it but not been completed?
I'm not sure about that, Igor, could you send out your tech-preview-patches?

CPU hot-add is supported by upstream now, hot-remove is not supported yet,
besides of qemu work it would require quite a work on kvm side as well.

Thanks,
Wanlong Gao

quoted

Thanks

quoted

Thanks,
Wanlong Gao