On 01/17/2017 05:50 PM, Sagi Grimberg wrote:

quoted

    So it looks like we are super not efficient because most of the
    times we catch 1
    completion per interrupt and the whole point is that we need to find
    more! This fio
    is single threaded with QD=32 so I'd expect that we be somewhere in
    8-31 almost all
    the time... I also tried QD=1024, histogram is still the same.


It looks like it takes you longer to submit an I/O than to service an
interrupt,

Well, with irq-poll we do practically nothing in the interrupt handler,
only schedule irq-poll. Note that the latency measures are only from
the point the interrupt arrives and the point we actually service it
by polling for completions.

quoted

so increasing queue depth in the singe-threaded case doesn't
make much difference. You might want to try multiple threads per core
with QD, say, 32

This is how I ran, QD=32.

The one thing which I found _really_ curious is this:

  IO depths    : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=0.1%,

=64=100.0%

     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%,

=64=0.0%

     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%,

=64=0.1%

     issued    : total=r=7673377/w=0/d=0, short=r=0/w=0/d=0,
drop=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=256

(note the lines starting with 'submit' and 'complete').
They are _always_ 4, irrespective of the hardware and/or tests which I
run. Jens, what are these numbers supposed to mean?
Is this intended?
ATM the information content from those two lines is essentially 0,
seeing that the never change irrespective of the tests I'm doing.
(And which fio version I'm using ...)

Cheers,

Hannes
-- 
Dr. Hannes Reinecke		   Teamlead Storage & Networking
hare@suse.de			               +49 911 74053 688
SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg
GF: F. Imendörffer, J. Smithard, J. Guild, D. Upmanyu, G. Norton
HRB 21284 (AG Nürnberg)