On 20/1/21 9:54 pm, Michal Rostecki wrote:

On Wed, Jan 20, 2021 at 08:30:56PM +0800, Anand Jain wrote:

quoted

  I ran fio tests again, now with dstat in an another window. I don't
  notice any such stalls, the read numbers went continuous until fio
  finished. Could you please check with the below fio command, also
  could you please share your fio command options.

That's the fio config I used:

https://gitlab.com/vadorovsky/playground/-/blob/master/fio/btrfs-raid1-seqread.fio

The main differences seem to be:
- the number of jobs (I used the number of CPU threads)
- direct vs buffered I/O

quoted

fio \
--filename=/btrfs/largefile \
--directory=/btrfs \
--filesize=50G \
--size=50G \
--bs=64k \
--ioengine=libaio \
--rw=read \
--direct=1 \
--numjobs=1 \
--group_reporting \
--thread \
--name iops-test-job

  It is system specific?

With this command, dstat output looks good:

https://paste.opensuse.org/view/simple/93159623

So I think it might be specific to whether we test direct of buffered
I/O. Or to the number of jobs (single vs multiple jobs). Since the most
of I/O on production environments is usually buffered, I think we should
test with direct=0 too.

  It explains the stall for now, so it might be reading from the cache
  so there was actually no IO to the device.

  Agreed it must be tested with the most common type of IO. But as the
  cache comes into play I was a bit reluctant.

Thanks, Anand

Cheers,
Michal