Re: [PATCH 1/3] vhost: replace vhost_workqueue with per-vhost kthread

On Mon, May 31, 2010 at 05:45:07PM +0200, Tejun Heo wrote:

Hello,

On 05/31/2010 05:22 PM, Michael S. Tsirkin wrote:

quoted

On Sun, May 30, 2010 at 10:24:01PM +0200, Tejun Heo wrote:

quoted

Replace vhost_workqueue with per-vhost kthread.  Other than callback
argument change from struct work_struct * to struct vhost_poll *,
there's no visible change to vhost_poll_*() interface.

I would prefer a substructure vhost_work, even just to make
the code easier to review and compare to workqueue.c.

Yeap, sure.

quoted

quoted

The problem is that I have no idea how to test this.

It's a 3 step process:

...

quoted

You should now be able to ping guest to host and back.
Use something like netperf to stress the connection.
Close qemu with kill -9 and unload module to test flushing code.

Thanks for the instruction.  I'll see if there's a way to do it
without building qemu myself on opensuse.

My guess is no, there was no stable qemu release with vhost net support
yet.  Building it is mostly configure/make/make install,
as far as I remember you only need devel versions of X libraries,
SDL and curses installed.

 But please feel free to go
ahead and test it.  It might just work!  :-)

quoted

quoted

+	if (poll) {
+		__set_current_state(TASK_RUNNING);
+		poll->fn(poll);
+		smp_wmb();	/* paired with rmb in vhost_poll_flush() */
+		poll->done_seq = poll->queue_seq;
+		wake_up_all(&poll->done);

quoted

This seems to add wakeups on data path, which uses spinlocks etc.
OTOH workqueue.c adds a special barrier entry which only does a
wakeup when needed.  Right?

Yeah, well, if it's a really hot path sure we can avoid wake_up_all()
in most cases.  Do you think that would be necessary?

My guess is yes. This is really very hot path in code, and we are
close to 100% CPU in some benchmarks.

quoted

quoted

-void vhost_cleanup(void)
-{
-	destroy_workqueue(vhost_workqueue);

I note that destroy_workqueue does a flush, kthread_stop
doesn't. Right? Sure we don't need to check nothing is in one of
the lists? Maybe add a BUG_ON?

There were a bunch of flushes before kthread_stop() and they seemed to
stop and flush everything.  Aren't they enough?

I was just asking, I'll need to review the code in depth.

We can definitely add
BUG_ON() after kthread_should_stop() check succeeds either way tho.

Thanks.

-- 
tejun