Hi!

On 11:40 Tue 15 Nov     , Kai Meyer wrote:

On 11/15/2011 11:13 AM, michi1 at michaelblizek.twilightparadox.com wrote:

...

quoted

You might want to take a look at wait queues (the kernel equivalent to pthread
"condidions"). Basically you instead of calling msleep(), you call
wait_event(). In the function which decrements numbios, you check whether it
is 0 and if so call wake_up().

...

That sounds very promising. When I read up on wait_event here:
lxr.linux.no/#linux+v2.6.32/include/linux/wait.h#L191

It sounds like it's basically doing the same thing. I would call it like so:

wait_event(wq, atomic_read(numbios) == 0);

Yes, you dol something like this.

To make sure I understand, this seems very much like what I'm doing, 
except I'm being woken up every time a bio finishes instead of being 
woken up once every millisecond. That is, I'm assuming I would use the 
same work queue for all my bios.

You are *not* woken up every time you a bio finishes. You are woken up every
time you call wake_up(). You could do something like:

if (atomic_dec_return(numbios) == 0)
	wake_up(wp);

During my testing, when I do a lot of disk I/O, I may potentially have 
hundreds of threads waiting on anywhere between 1 and 32 bios. Help me 
understand the sort of impact you think I might see between having 
hundreds waiting for a millisecond, and having hundreds get woken up 
each time a bio completes. It seems like it would be very helpful in low 
I/O scenarios, especially when there are fast disks involved. I'm 
concerned that during heavy I/O loads, I'll be doing a lot of 
atomic_reads, and I have the impression that atomic_read isn't the 
cheapest operation.

The wakeups might some some overhead. However, I would worry more about
scheduling overhead on smp systems than atomic_read performance.

	-Michi
-- 
programing a layer 3+4 network protocol for mesh networks
see http://michaelblizek.twilightparadox.com