Re: [PATCH RFC] Btrfs: improve multi-thread buffer read

On 07/11/2012 08:31 PM, Josef Bacik wrote:

On Tue, Jul 10, 2012 at 07:57:55PM -0600, Liu Bo wrote:

quoted

On 07/11/2012 02:58 AM, Josef Bacik wrote:

quoted

On Tue, Jul 10, 2012 at 05:27:59AM -0600, Liu Bo wrote:

quoted

While testing with my buffer read fio jobs[1], I find that btrfs does not
perform well enough.

Here is a scenario in fio jobs:

We have 4 threads, "t1 t2 t3 t4", starting to buffer read a same file,
and all of them will race on add_to_page_cache_lru(), and if one thread
successfully puts its page into the page cache, it takes the responsibility
to read the page's data.

And what's more, reading a page needs a period of time to finish, in which
other threads can slide in and process rest pages:

     t1          t2          t3          t4
   add Page1
   read Page1  add Page2
     |         read Page2  add Page3
     |            |        read Page3  add Page4
     |            |           |        read Page4
-----|------------|-----------|-----------|--------
     v            v           v           v
    bio          bio         bio         bio

Now we have four bios, each of which holds only one page since we need to
maintain consecutive pages in bio.  Thus, we can end up with far more bios
than we need.

Here we're going to
a) delay the real read-page section and
b) try to put more pages into page cache.

With that said, we can make each bio hold more pages and reduce the number
of bios we need.

Here is some numbers taken from fio results:
         w/o patch                 w patch
       -------------  --------  ---------------
READ:    745MB/s        +32%       987MB/s

Um, I have this in btrfs-next

Btrfs: use large extent range for read and its endio

that seems to do the same thing, did you not want to do that anymore?  Thanks,

I'm still hard working on that patchset. :)

Although the patchset is well worthy of testing, it is not good enough for btrfs upstream.

While doing some tuning work on it, I realized that I could make this improvement without
the help of rwlock extent state stuff, so I made this smaller and cleaner patch for upstream
so that we could gain some performance here first.

So do you want me to drop the rwlock stuff and take this instead?  Take a look
at whats in btrfs-next and tell me what I should drop.  Thanks,

Yes Josef, please take this patch instead and drop the following:

Btrfs: use radix tree for checksum
Btrfs: merge adjacent states as much as possible
Btrfs: use large extent range for read and its endio
Btrfs: apply rwlock for extent state

When I finish fixing all the noticed bugs on hands, I'll send a new rebased version to btrfs-next for you. :)

thanks,
liubo

Josef